gnucash/libgnucash/doc/generic-druid-framework.txt

/** \page druidframework Generic (Import) Druid Framework

			Derek Atkins <derek@ihtfp.com>
			          2004-01-12

			       Work in progress

\section druidbackground Background

The GNOME Druid infrastructure wants to force you to be UI-driven.
What this means is that the druid is started and lives in gtk_main()
while the user responds to the displayed Druid page.  When the user
clicks a button (Next, Back, Help, or something inside the Druid page)
it performs some operation and then returns control to the GUI.  For
example, a user clicking "Next" forces a callback based on that
specific page.  That callback can (and should) set the next druid page
before returning control to the GUI.

If we were to make the importer backend-driven instead of GUI-driven
we would necessarily require nested gtk_main()'s.  The reason is
that the backend was originally executed from GUI callback, so what
should happen is that the initialization sets up the import process
and then returns control so we dont have a nested gtk_main().

Nesting gtk_main() can result in crashes.  If the originating window
is destroyed then when control returns we've jumped back into invalid
data space.  This has happened on numerous occasions within the
gnucash code over time and has caused numerous bugs.  For example, the
'Save while saving crashes gnucash' bug was due to a nested gtk_main
caused by the progress-bar.  Consequently, the best way to avoid this
problem is not to introduce this problem and avoid nested gtk_main()
whenever possible.  This means the importer should be GUI-driven, not
backend-driven.

\section druidproblem The Problem

For a generic importer druid, we want to create a single druid (or set
of druid components) that all importers can use.  Moreover, this
framework should be UI-widget independent.  The problem is that
different importer backends have different requirements for the UI.
For example, the QIF importer needs to map QIF Categories to GnuCash
Accounts, but OFX or HBCI have no such mapping requirement.

Another issue is that some sub-processes of the importing process
require multiple druid pages.  If this sub-process is repeatable, it
means the druid needs to be able to jump back to the beginning of the
sub-process.  For example, the process to choose files to import
should allow users to import multiple files at one time.

Moreover, even when a backend may sometimes require access to
particular druid sub-process, it may need to skip that sub-process
sometimes.  For example, the QIF importer may have an ambiguity in the
date format for a file, requiring the user to choose the actual date
format.  However if the imported file is not ambiguous this
sub-process of the druid can be skipped.

All of this means the druid framework should be able to rotate across
a subset of the pages for a sub-process or skip pages for a
sub-process based on the requirements of the backend.

In addition the framework should allow a global setting to enable or
disable "documentation pages" (c.f. the Show QIF Documentation
preference).  Each sub-process can have a set of doc pages available
which can be displayed (or not) based on a user preference.

\section druidmain The Druid Framework

In order to refrain from pulling Gnome and GTK into the backend
implementations, we need a GUI-independent UI framework for Druids.
The framework is broken into Providers and the Druid Builder.  A
Provider supplies a set of druid pages and the appropriate callbacks
for the backend to retrieve the user's data.  The Druid Builder is the
process the backend uses to combine the various Providers into the
ultimate druid and set up all the callbacks to properly hook the druid
and backend together.

Each provider implements the Provider API and Provider Registrar API
and registers itself with the druid provider registry.  The Provider
Registrar API defines the minimal set of functions and methods to
instantiate a provider and place a set of one or more pages into a
druid instance.  The Provider API is used to set up the callbacks to
hook that provider into the backend.

In addition to the standard Provider API, each provider must define a
private API for use with the backend.  Each provider is going to
interact with the backend differently, so there is no way to define a
common API for this interaction.  On the other hand, the backend
already knows a priori which providers it needs to use, so it can know
the provider-dependent API and use that interaction.

The druid provider registry allows the Druid Builder to combine the
providers in the requested order when a backend asks to build a druid.
It uses the Provider Registrar API to instantiate a Provider and hook
it into the Druid, and then uses the Provider API to connect the
Provider to the backend using the data provided by the backend.

This leaves the importer backend blissfully unaware of the actual
Druid GUI/toolkit implementation (i.e. it doesn't need to know that
the Druid is actually a GnomeDruid -- it could be some other UI
toolkit).  The backend calls the Druid Builder to put together the
druid with the appropriate providers and supplies the
provider-specific callback information necessary to hook into the
druid.

In order to initiate an import for a particular backend, the GUI calls
into the "start import" routine which builds an import context, builds
the import druid, and then returns control let the GUI run.  This
means the import backend should be completely callback-based,
including the cleanup code in case the user interrupts the import.

\section druidprocess Druid Sub-Process Providers 

In order to abstract the Druid from the various importer backends, the
import process is broken into a set of sub-processes.  Each
sub-process is implemented as a Provider in the Druid framework.  The
Provider implements a set of Druid pages that can be added to the
running druid and provides a set of callbacks to supply that input to
the import backend.  For example, one sub-process could be "select the
file(s) for import" and another is "choose the date/number format".

The interface between the Provider and the Builder is obviously
toolkit-specific because the builder need to piece together the actual
druid pages (e.g. GnomeDruidPageStandard).  The druid builder requests
an instance of a provider (and its pages) and supplies the provider
with the backend callbacks requests.  The provider instance creates
the druid pages and connects the passed-in callback data so it can
call the backend appropriately.

Each provider necessarily requires its own callback interface (because
each provider needs to supply different data to the backend in
different ways).  This is implemented by subclassing the basic
callback storage type.  Because the importer backend knows the
providers being used, it can provide the required callback storage
type when it builds the druid.

When a user fills in a druid page and clicks on "Next" the druid will
call the next-page callback and supply the provided data (as defined
in the particular provider callback API).  The backend then acts on
the callback data, sets the next page in the druid, and returns
control.  Similar operations occur when the user clicks "Back", a
back-page callback, or any other callbacks required by the specific
provider.

\section druidtogether Putting the Druid Together

In order to build the druid, the import backend builds the list of
providers and passes that list to the Druid Builder.  The Builder
creates the base druid and then pulls the druid pages from each
provider and inserts them into the druid.  Finally, the builder
displays the druid and starts the process.

\section druidbackend Linking into the Backend

When supplying the list of providers to the builder, the backend also
provides a set of callbacks.  Since the backend knows what providers
it wants, it can set up the appropriate callbacks.  Each callback,
when called, passes in the user-input data in the callback function.
The callback function should process the data, set the next druid
page, and then return.

\section druidcore Core GncDruid Types

\subsection druidpage GncDruidPage

  Opaque (toolkit-specific) type:  a druid page.  Used to pass an
  opaque object from the provider, through the backend, to the druid
  system (e.g. in order to set the druid page).

\subsection druidcallback GncDruidCB

  Base type of a druid callback.  Minimum information is the backend context.

  Members:

  gpointer backend_ctx;

\subsection druidobject GncDruid

  The context object of a druid.  This object contains all the
  necessary data to maintain the druid and reference all it's data.

  Members:

  void set_page(\n
		GncDruid,\n
		GncDruidPage);

    Set the current page of the druid to the GncDruidPage.

  GncDruidProvider current_provider;\n
  GncDruidProvider (*next_provider)(GncDruid);\n
  GncDruidProvider (*prev_provider)(GncDruid);

\subsection druidproviderdesc GncDruidProviderDesc

  The Druid Provider Description base class.  This defines the minimal
  information to name a provider and provide the interface required to
  connect the provider to the backend.  Each provider description
  implementation should subclass this type.  The backend should use
  the subclasses.

  Members:

  const gchar *name;\n
  gboolean (*provider_needed)(GncDruidCB);\n
  gboolean (*next_cb)(GncDruid, GncDruidCB);\n
  gboolean (*prev_cb)(GncDruid, GncDruidCB);

\subsection druidprovider GncDruidProvider

  An instance of a Druid Provider.  Still toolkit-independent (a
  toolkit-specific subclass actually implements the functions
  necessary for the builder) this interface allows the backend to
  interface to the Provider to walk through the provider pages.

  Members:

  GncDruidPage (*first_page)(GncDruidProvider);\n
  GncDruidPage (*next_page)(GncDruidProvider);\n
  GncDruidPage (*prev_page)(GncDruidProvider);

\section druidapi The Druid Builder API

GncDruid gnc_druid_build(\n
		GList *providers,\n
		gpointer backend_ctx,\n
		void (*end)(gpointer))

  Build a druid using the supplied list of providers descriptions
  (GncDruidProviderDesc).  The provider list also contains all the
  callback information necessary to hook the backend into the druid.
  The backend_ctx and end() parameter are used to end the session in
  the case of the user clicking "cancel".  It cleans up the backend
  context.

\section druidinternal The Basic (internal) Provider APIs

GncDruidProvider gnc_provider_get_instance(\n
		GncDruid druid_ctx,\n
		GncDruidProviderDesc description,\n
		gpointer backend_ctx)

  Obtain an instance of a Druid Provider based on the Provider
  Description.  This is used by the druid builder to obtain a Provider
  Instance given the Provider Description.

void gnc_provider_register(\n
		const gchar* name,\n
		gnc_provider_get_instance);

  Register a Provider Implementation of the provided name.  Provide a
  creation function that gnc_provider_get_instance() can use to obtain
  a fully initialized provider object.

\section druidproviderapi Specific GncDruidProviderDesc APIs

Each provider needs to create its own subclass of GncDruidProviderDesc
that defines its specific interface to the backend.  The following
sections show some example interfaces.  In addition to the
ProviderDesc object, each provider can also provide a subclassed
GncDruidCB object for use in the callbacks.

  GncDruidProviderDescChooseFmt

    A provider that allows the user to choose a format in the case of
    ambiguous input.

    Members:

    void (*get_ambiguity)(\n
		gpointer be_ctx,\n
		GncImportFormat* choices,\n
		GncImportFormat* last_choice);

  GncDruidChooseFmtCB

    Members:

    GncImportFormat choice;


  GncDruidProviderDescSelectFile

    A provider that allows the user to select the file(s) to import.

    Members:

    gboolean multi_file;\n
    gchar* last_directory;\n
    GList * (*get_files)(gpointer be_ctx);\n
    const gchar* (*get_filename)(GncImportFile file);\n
    void (*remove_file)(gpointer be_ctx, GncImportFile file)

  GncDruidSelectFileCB

    Members:

    gchar* filename;

*/