        Derek Atkins <derek@ihtfp.com>
                  2004-01-12

Work in progress

Background

The GNOME Druid infrastructure wants to force you to be UI-driven. What this means is that the druid is started and lives in gtk_main() while the user responds to the displayed Druid page. When the user clicks a button (Next, Back, Help, or something inside the Druid page) it performs some operation and then returns control to the GUI. For example, a user clicking "Next" forces a callback based on that specific page. That callback can (and should) set the next druid page before returning control to the GUI.

If we were to make the importer backend-driven instead of GUI-driven we would necessarily require nested gtk_main()'s. The reason is that the backend was originally executed from GUI callback, so what should happen is that the initialization sets up the import process and then returns control so we dont have a nested gtk_main().

Nesting gtk_main() can result in crashes. If the originating window is destroyed then when control returns we've jumped back into invalid data space. This has happened on numerous occasions within the gnucash code over time and has caused numerous bugs. For example, the 'Save while saving crashes gnucash' bug was due to a nested gtk_main caused by the progress-bar. Consequently, the best way to avoid this problem is not to introduce this problem and avoid nested gtk_main() whenever possible. This means the importer should be GUI-driven, not backend-driven.

The Problem

For a generic importer druid, we want to create a single druid (or set of druid components) that all importers can use. Moreover, this framework should be UI-widget independent. The problem is that different importer backends have different requirements for the UI. For example, the QIF importer needs to map QIF Categories to GnuCash Accounts, but OFX or HBCI have no such mapping requirement.

Another issue is that some sub-processes of the importing process require multiple druid pages. If this sub-process is repeatable, it means the druid needs to be able to jump back to the beginning of the sub-process. For example, the process to choose files to import should allow users to import multiple files at one time.

Moreover, even when a backend may sometimes require access to particular druid sub-process, it may need to skip that sub-process sometimes. For example, the QIF importer may have an ambiguity in the date format for a file, requiring the user to choose the actual date format. However if the imported file is not ambiguous this sub-process of the druid can be skipped.

All of this means the druid framework should be able to rotate across a subset of the pages for a sub-process or skip pages for a sub-process based on the requirements of the backend.

In addition the framework should allow a global setting to enable or disable "documentation pages" (c.f. the Show QIF Documentation preference). Each sub-process can have a set of doc pages available which can be displayed (or not) based on a user preference.

The Druid Framework

In order to refrain from pulling Gnome and GTK into the backend implementations, we need a GUI-independent UI framework for Druids. The framework is broken into Providers and the Druid Builder. A Provider supplies a set of druid pages and the appropriate callbacks for the backend to retrieve the user's data. The Druid Builder is the process the backend uses to combine the various Providers into the ultimate druid and set up all the callbacks to properly hook the druid and backend together.

Each provider implements the Provider API and Provider Registrar API and registers itself with the druid provider registry. The Provider Registrar API defines the minimal set of functions and methods to instantiate a provider and place a set of one or more pages into a druid instance. The Provider API is used to set up the callbacks to hook that provider into the backend.

In addition to the standard Provider API, each provider must define a private API for use with the backend. Each provider is going to interact with the backend differently, so there is no way to define a common API for this interaction. On the other hand, the backend already knows a priori which providers it needs to use, so it can know the provider-dependent API and use that interaction.

The druid provider registry allows the Druid Builder to combine the providers in the requested order when a backend asks to build a druid. It uses the Provider Registrar API to instantiate a Provider and hook it into the Druid, and then uses the Provider API to connect the Provider to the backend using the data provided by the backend.

This leaves the importer backend blissfully unaware of the actual Druid GUI/toolkit implementation (i.e. it doesn't need to know that the Druid is actually a GnomeDruid – it could be some other UI toolkit). The backend calls the Druid Builder to put together the druid with the appropriate providers and supplies the provider-specific callback information necessary to hook into the druid.

In order to initiate an import for a particular backend, the GUI calls into the "start import" routine which builds an import context, builds the import druid, and then returns control let the GUI run. This means the import backend should be completely callback-based, including the cleanup code in case the user interrupts the import.

Druid Sub-Process Providers

In order to abstract the Druid from the various importer backends, the import process is broken into a set of sub-processes. Each sub-process is implemented as a Provider in the Druid framework. The Provider implements a set of Druid pages that can be added to the running druid and provides a set of callbacks to supply that input to the import backend. For example, one sub-process could be "select the file(s) for import" and another is "choose the date/number format".

The interface between the Provider and the Builder is obviously toolkit-specific because the builder need to piece together the actual druid pages (e.g. GnomeDruidPageStandard). The druid builder requests an instance of a provider (and its pages) and supplies the provider with the backend callbacks requests. The provider instance creates the druid pages and connects the passed-in callback data so it can call the backend appropriately.

Each provider necessarily requires its own callback interface (because each provider needs to supply different data to the backend in different ways). This is implemented by subclassing the basic callback storage type. Because the importer backend knows the providers being used, it can provide the required callback storage type when it builds the druid.

When a user fills in a druid page and clicks on "Next" the druid will call the next-page callback and supply the provided data (as defined in the particular provider callback API). The backend then acts on the callback data, sets the next page in the druid, and returns control. Similar operations occur when the user clicks "Back", a back-page callback, or any other callbacks required by the specific provider.

Putting the Druid Together

In order to build the druid, the import backend builds the list of providers and passes that list to the Druid Builder. The Builder creates the base druid and then pulls the druid pages from each provider and inserts them into the druid. Finally, the builder displays the druid and starts the process.

Linking into the Backend

When supplying the list of providers to the builder, the backend also provides a set of callbacks. Since the backend knows what providers it wants, it can set up the appropriate callbacks. Each callback, when called, passes in the user-input data in the callback function. The callback function should process the data, set the next druid page, and then return.

Core GncDruid Types

GncDruidPage

Opaque (toolkit-specific) type: a druid page. Used to pass an opaque object from the provider, through the backend, to the druid system (e.g. in order to set the druid page).

GncDruidCB

Base type of a druid callback. Minimum information is the backend context.

Members:

gpointer backend_ctx;

GncDruid

The context object of a druid. This object contains all the necessary data to maintain the druid and reference all it's data.

Members:

void set_page(
GncDruid,
GncDruidPage);

Set the current page of the druid to the GncDruidPage.

GncDruidProvider current_provider;
GncDruidProvider (*next_provider)(GncDruid);
GncDruidProvider (*prev_provider)(GncDruid);

GncDruidProviderDesc

The Druid Provider Description base class. This defines the minimal information to name a provider and provide the interface required to connect the provider to the backend. Each provider description implementation should subclass this type. The backend should use the subclasses.

Members:

const gchar *name;
gboolean (*provider_needed)(GncDruidCB);
gboolean (*next_cb)(GncDruid, GncDruidCB);
gboolean (*prev_cb)(GncDruid, GncDruidCB);

GncDruidProvider

An instance of a Druid Provider. Still toolkit-independent (a toolkit-specific subclass actually implements the functions necessary for the builder) this interface allows the backend to interface to the Provider to walk through the provider pages.

Members:

GncDruidPage (*first_page)(GncDruidProvider);
GncDruidPage (*next_page)(GncDruidProvider);
GncDruidPage (*prev_page)(GncDruidProvider);

The Druid Builder API

GncDruid gnc_druid_build(
GList *providers,
gpointer backend_ctx,
void (*end)(gpointer))

Build a druid using the supplied list of providers descriptions (GncDruidProviderDesc). The provider list also contains all the callback information necessary to hook the backend into the druid. The backend_ctx and end() parameter are used to end the session in the case of the user clicking "cancel". It cleans up the backend context.

The Basic (internal) Provider APIs

GncDruidProvider gnc_provider_get_instance(
GncDruid druid_ctx,
GncDruidProviderDesc description,
gpointer backend_ctx)

Obtain an instance of a Druid Provider based on the Provider Description. This is used by the druid builder to obtain a Provider Instance given the Provider Description.

void gnc_provider_register(
const gchar* name,
gnc_provider_get_instance);

Register a Provider Implementation of the provided name. Provide a creation function that gnc_provider_get_instance() can use to obtain a fully initialized provider object.

Specific GncDruidProviderDesc APIs

Each provider needs to create its own subclass of GncDruidProviderDesc that defines its specific interface to the backend. The following sections show some example interfaces. In addition to the ProviderDesc object, each provider can also provide a subclassed GncDruidCB object for use in the callbacks.

GncDruidProviderDescChooseFmt

A provider that allows the user to choose a format in the case of ambiguous input.

Members:

void (*get_ambiguity)(
gpointer be_ctx,
GncImportFormat* choices,
GncImportFormat* last_choice);

GncDruidChooseFmtCB

Members:

GncImportFormat choice;

GncDruidProviderDescSelectFile

A provider that allows the user to select the file(s) to import.

Members:

gboolean multi_file;
gchar* last_directory;
GList * (*get_files)(gpointer be_ctx);
const gchar* (*get_filename)(GncImportFile file);
void (*remove_file)(gpointer be_ctx, GncImportFile file)

GncDruidSelectFileCB

Members:

gchar* filename;