Not quite true, it just takes a really long time for a large database.
The underlying problem is gnc_file_do_save_as reloads the data to make
sure that the save-as saves everything. On the SQL backend that
triggers a scrub. The scrub itseld doesn't take long, but every
transaction commit was logged in the transaction log and did a refresh
of the registers. So:
* Suspend logging while doing the scrub.
* Suspend UI refreshes and QOF events while reloading the data.
Scrubbing during SQL load can't commit the changes from the scrub
because the backend's m_loading member is true so disable scrubbing
during database load and edit/commit all transactions again after
loading is complete.
For wrong value type when retrieving a value from the SQL results row.
Profiling showed that most of the SQL load time was spent in handling
these exceptions, and using std::optional instead produced a > 11x
speedup (10 seconds vs. 115 seconds) when loading a large file.
to mysql database.
It's more general, any save-as to a different backend type, i.e.
xml->sql or sql->xml, left the previous book locked.
Ensure that the session is ended and the book unlocked in the
backend's destructor. This also fixes a memory leak in the SQL
backend because the GncSqlConnection wasn't being deleted.
when database already exists.
Crashed because of dereferencing GncSqlBackend::m_conn when it's nullptr
after the DB create failed. Protect all such dereferences.
There are a very few left that need deeper study, but this gets
rid of most of the noise. For the most part it's just getting rid of
extra variables or removing an assignment that is always
replaced later but before any reads of the variable. A few are
discarded result variables.
That is let the percentage increase gradually. The current granularity is still
very rough, but at least it gives an indication of getting closer to fully
loading the data. The previous configuration on the other hand only suggested
something was happening but with no indication where in the load process
gnucash was.
Only string values should be quoted in queries; in particular NULL
isn't a string value and must not be quoted.
Note that this is a less than perfect solution because it doesn't use
the Database's quoting function and so doesn't escape quotes, linefeeds,
or carriage returns inside the string. That's because the SQL generating
logic is independent of the connection class and can't easily get to it.
For each subclass, getting rid of GNC_SQL_OBJECT_BACKEND_VERSION which
was a bit misguided.
Also remove the bogus test the skipped loading a table if its version
didn't match GNC_SQL_OBJECT_BACKEND_VERSION which was even more misguided.
It is split into
- /libgnucash (for the non-gui bits)
- /gnucash (for the gui)
- /common (misc source files used by both)
- /bindings (currently only holds python bindings)
This is the first step in restructuring the code. It will need much
more fine tuning later on.
