add version numbers: these allow multi-user updates to be distinguished

properly.


git-svn-id: svn+ssh://svn.gnucash.org/repo/gnucash/trunk@3670 57a11ea4-9604-0410-9ed3-97b8803252fd

src/engine/sql/Makefile.am

@@ -2,7 +2,7 @@
 # Build the postgres backend as its own loadable shared object.
 
 lib_LTLIBRARIES = libgnc_postgres.la
-libgnc_postgres_la_LDFLAGS = -version-info 5:3:5
+libgnc_postgres_la_LDFLAGS = -version-info 6:0:6
 
 
 libgnc_postgres_la_SOURCES = \

src/engine/sql/README

@@ -50,12 +50,6 @@ To Be Done
 ----------
 Core bugs/features that still need work:
 
--- bug/feature: the 'save as' semantics are ... unexpected, if 
-   the database already exists and has data in it ... 
-
--- bug: group sync doesn't pull in newer data from the db ...
-   (related to the 'save as' above)
-
 -- allow user to enter URL in GUI dialog, get GUI to remember the URL 
 
 -- Implement GUI to ask user for username/password to log onto the
@@ -65,21 +59,47 @@ Core bugs/features that still need work:
    mods to gnc-book to keep it happy about lock files & such.
    (coord with rlb on gnc-book redesign)
 
+-- review (actually, design & architect) the communications error
+   handling policy.  For example, CopyToEngine() will leave
+   the backend in a disabled state if a communication error occurs;
+   there will be other debiliting conditions if the backend disappears,
+   leaving the engine in a possibly confused state.
+
+-- during sync, detect and report conflicting edits to accounts 
+   and transactions.  See the notes for pgendSync() for details
+   as to what this is about.  For the first pass, this is not a 
+   serious issue; its a 'nice to have' thing.
+
 
 To Be Done, Part II
 -------------------
 This list only affects the multi-user and advanced/optional features.
 
+-- saving as single-user, then re-opening in multi-user mains missing
+   checkpoints, and thus no balances.
 
--- implement account commit edit (actually, the check&rollback part)
+-- checkpoint ending balance is showing up as starting balance
-   (need to check the account version number beofer the commit happens)
 
--- use version numbers for accounts, commodities, splits & transactions,
+-- transaction rollbck is 'incorrect'; sort of ?? since we should
-   as this will provide a far more efficient 'compare' for 
+   roll back to what's in the db, and not the old stored transaction ...
-   user changes.
+   but the way the register works, it hides this problem ...
+
+-- store account balances in database.  This will be tricky ...
+   -- split query gets hard ... 
+   -- use postgres aggregates ?? 
 
 -- provide support for more query types in gncquery.c
 
+-- implement account rollback (i.e. of other user has modified the
+   account, we need to do something to merge thier work into ours...)
+
+-- fix caching in the face of lost contact to the backend.  If the
+   backend can't contact its server, then we should just save up caches,
+   and then when contact with backend re-established, we should spit
+   them out.  The pgendSync routine should more or less dothe trick;
+   note, however, that the file format needs to save the version number
+   ...
+
 -- Implement logging history in the SQL server.  i.e. save the old
    copies of stuff in log tables.  Make the username part of the 
    logging scheme.
@@ -93,40 +113,32 @@ This list only affects the multi-user and advanced/optional features.
    tries to write to something they're not allowed to write to,
    then they should be bounced back.
 
--- versioning verification in backend. The desired semantic for updates
+-- Review versioning verification in backend. The desired semantic for 
-   should be like CVS: multiple nearly-simultaneous writers are allowed;
+   updates should be like CVS: multiple nearly-simultaneous writers 
-   however, only one wins, and others are kicked back.  The losers know
+   are allowed; however, only one wins, and others are kicked back. 
-   themselves because they are trying to update info of the wrong
+   The losers know themselves because they are trying to update info 
-   version. But right now, version is not done everywhere, nor is it
+   of the wrong version. 
-   done uniformly:
+   -- pgend_transaction_commit does it correctly; but the gui doesn't
-   -- pgend_transaction_commit does it correctly.
+      report a rollback.
-   -- pgTransactionSync  does not, it clobbers.
+   -- pgTransactionSync() is broken, but its not used anywher.
-   -- pgend_account_commit clobbers.
+   -- pgend_account_commit checks version but doesn't rollback.
-   -- pgendAccountGroupSync is unfinished.
+      (nor does the dui report a rollback.
+   -- pgendSync does the right thing, except that it doesn't
+      detect any version conflicts, nor does it notify the user
+      of such conflicts.
+
    I'm not sure how critical this all is; with a small number of users
    it shouldn't be a problem.  With a huge number of users, each editing
    the same transaction (unlikely!?) then there is risk of clobbered
-   data, but so what?
+   data, but so what?  ersioning is at least partly a people-management
-
+   problem.  Anyway, what's there now should be pretty good & should
--- fix rollback bug in GUI.  If backend tells engine to rollback,
+   work for now.  Except its mostly untested.
-   (e.g. on a new transaction), the GUI still shows traces, instead
-   of kicking back.
-   This is a symptom of a lack of kick-back detection in GUI.
--- finish implementing pgendAccountGroupSync
-
--- store account balances in database.  This will be tricky ...
-   -- split query gets hard ... 
-   -- use postgres aggregates ?? 
-
--- review multiuser operation for correctness 
-
--- fix caching in the face of lost contact to the backend.  If the
-   backend can't contact its server, then we should just save up caches,
-   and then when contact with backend re-established, we should spit
-   them out.  But right now, this is broken.  In particular, 
-   the use of xaccGrouparkSaved screws up some status bits ...
 
 -- review & match up against docs at 
    http://www.lupercalia.net/gnc-db/
 
+-- use version numbers for commodities.  Right now, multi-user
+   updates of commodities are not detected (this seem OK for
+   now, since this is a rare occurrance, right ???)
+
 

src/engine/sql/checkpoint.c

@@ -260,9 +260,12 @@ pgendAccountGetCheckpoint (PGBackend *be, Checkpoint *chk)
             PQclear (result);
             return;
          }
-         chk->balance = atoll(DB_GET_VAL("balance", j));
+         if (0 < nrows )
-         chk->cleared_balance = atoll(DB_GET_VAL("cleared_balance", j));
+         {
-         chk->reconciled_balance = atoll(DB_GET_VAL("reconciled_balance", j));
+            chk->balance = atoll(DB_GET_VAL("balance", j));
+            chk->cleared_balance = atoll(DB_GET_VAL("cleared_balance", j));
+            chk->reconciled_balance = atoll(DB_GET_VAL("reconciled_balance", j));
+         }
       }
 
       PQclear (result);

src/engine/sql/design.txt

@@ -2,7 +2,7 @@
 SQL Tables
 ----------
 These mostly parallel the data structures in the gnc engine.
-See gnc-init.txt for more info.
+See table-create.sql for more info.
 
 
 Session Table, Session Modes
@@ -20,10 +20,10 @@ functioning.
    to debug, and has reasonable demands on the database for small
    datasets.
 
-   This mode is guarenteed to clobber any sort of changes made by
+   This mode is guaranteed to clobber any sort of changes made by
    other users (in the same way that multiple file writers clobber
    each other).  Thus, this mode is mutually exclusive of any other,
-   and te engine prevents more than one concurrent user.  
+   and the engine prevents more than one concurrent user.  
 
 
 -- "Single User Update Mode" -- 
@@ -34,18 +34,18 @@ functioning.
    of a crash.
 
 -- "Multi-User Polled" -- 
-   Multiple users are assumed, Gnucash polls the database to detect 
+   Multiple users are assumed, GnuCash polls the database to detect 
    changes in the data.  Partially Implemented. 
 
 
 -- "Multi-User Events" --
-   Gnucash uses the SQL LISTEN/NOTIFY async message delivery routines.
+   GnuCash uses the SQL LISTEN/NOTIFY async message delivery routines.
    Not implemented.
 
 Safety Lockout
 --------------
 There is a safety lockout that prevents a single-user mode database from
-being ccessed by another user, and from having a multi-user mode 
+being accessed by another user, and from having a multi-user mode 
 database from being accessed by a single-user-mode client.
 
 
@@ -62,7 +62,7 @@ If you want incremental deletion, then use the 'Single Update' mode.
 
 m4 macros
 ---------
-Some of the code is auto-gen'ed from m4 macros.  This mopstly just
+Some of the code is auto-gen'ed from m4 macros.  This mostly just
 simplifies some rather repetitive, cut-n-paste code that's identical
 from function to function.  If you can think of a better way ...
 
@@ -75,14 +75,14 @@ sql corruption.
 KVP frames
 ----------
 Storage of KVP values in the sql database is treated more or less as
-described in the main kvp docs.  The hierarchical structure is converted
+described in the main KVP docs.  The hierarchical structure is converted
 into 'paths' by concatenating key names, and using / as the separator.
 (Thus, paths look like file-paths).  The root of each frame is
 associated with a guid (and thus, a url kvp://12341234/some/kvp/keys, 
 where 12341234 is the guid).
 
 The implementation caches the paths, associating a 32-bit inode number
-with each path.  Caching is done because the same path names will recurr 
+with each path.  Caching is done because the same path names will recur 
 frequently for different guids (e.g. /reconcile-info/last-date will
 occur in most accounts). 
 
@@ -95,3 +95,17 @@ the type.  Note that the binary type and the glist type are not currently
 implemented.  The glist type could be implemented, as long as the glist
 only stored strings ... The binary type could be implemented with blobs.
 
+Version Numbers
+---------------
+Both the Account structure, and the Transaction structure, have version
+numbers in them.  These are used to compare the sql and the engine
+contents, and update the one or the other as appropriate.  Version
+numbers would not be necessary for single-user access, but are important
+for multi-user access, where several engines must be kept in sync with
+the database contents.   An alternative to version numbers might have
+been the date of the last update.  However, version numbers are better
+than dates in the case where the engines reside on machines whose clocks
+are not closely synchronized.  (e.g. which may happen if the machines
+are not using NTP for time synchronization; or, e.g. if one machine failed 
+to have daylight-savings time set correctly: its transactions would be 
+an hour newer/older than the others, leading to bad updates).

src/engine/sql/putil.h

@@ -231,13 +231,15 @@ gpointer pgendGetResults (PGBackend *be,
 }
 
 /* Compare the date of last modification. 
- * This is a special date comp to make the m4 macros simpler.
+ * This is a special date comp to 
+ * (1) make the m4 macros simpler, and
+ * (2) avoid needless updates 
  */
 #define COMP_NOW(sqlname,fun,ndiffs) { 	 			\
     Timespec eng_time = xaccTransRetDateEnteredTS(ptr);		\
     Timespec sql_time = gnc_iso8601_to_timespec_local(		\
                      DB_GET_VAL(sqlname,0)); 			\
-    if (eng_time.tv_sec != sql_time.tv_sec) {			\
+    if (eng_time.tv_sec > sql_time.tv_sec) {			\
        char buff[80];						\
        gnc_timespec_to_iso8601_buff(eng_time, buff);		\
        PINFO("mis-match: %s sql='%s' eng=%s", sqlname, 		\

src/engine/sql/table-create.sql

@@ -47,7 +47,8 @@ CREATE TABLE gncAccount (
 	description 	TEXT,
 	notes	 	TEXT,
 	type		TEXT NOT NULL,
-	commodity	TEXT NOT NULL CHECK (commodity <>'')
+	commodity	TEXT NOT NULL CHECK (commodity <>''),
+	version		INT4 NOT NULL
 );
 
 -- CREATE INDEX gncAccount_pg_idx ON gncAccount (parentGuid);
@@ -56,12 +57,13 @@ CREATE TABLE gncAccount (
 -- hack alert -- add kvp frames ??
 
 CREATE TABLE gncTransaction (
-	transGuid		CHAR(32) PRIMARY KEY,
+	transGuid	CHAR(32) PRIMARY KEY,
-	date_entered	 	DATETIME DEFAULT 'NOW',
+	date_entered 	DATETIME DEFAULT 'NOW',
-	date_posted	 	DATETIME,
+	date_posted 	DATETIME,
-	num			TEXT,
+	num		TEXT,
-	description		TEXT,
+	description	TEXT,
-        currency                TEXT NOT NULL CHECK (currency <> '')
+        currency	TEXT NOT NULL CHECK (currency <> ''),
+	version		INT4 NOT NULL
 );
 
 CREATE INDEX gncTransaction_posted_idx ON gncTransaction (date_posted);

src/engine/sql/table.m4

@@ -12,6 +12,7 @@ define(`account', `gncAccount, Account, Account,
        notes,          , char *, xaccAccountGetNotes(ptr),
        type,           , char *, xaccAccountTypeEnumAsString(xaccAccountGetType(ptr)),
        commodity,      , char *, gnc_commodity_get_unique_name(xaccAccountGetCommodity(ptr)),
+       version,        , int32,  xaccAccountGetVersion(ptr),
        parentGUID,     , GUID *, xaccAccountGetGUID(xaccAccountGetParentAccount(ptr)),
        accountGUID, KEY, GUID *, xaccAccountGetGUID(ptr),
        ')
@@ -44,6 +45,7 @@ define(`transaction', `gncTransaction, Transaction, Transaction,
        currency,       , commod,   gnc_commodity_get_unique_name(xaccTransGetCurrency(ptr)),
        date_entered,   , now,      "NOW",
        date_posted,    , Timespec, xaccTransRetDatePostedTS(ptr),
+       version,        , int32,    xaccTransGetVersion(ptr),
        transGUID,   KEY, GUID *,   xaccTransGetGUID(ptr),
        ')