(gnc:define-report
'version 3
'name "Test Report Template4"
'invalid-field-name 'x
'report-guid "xxx")
gnc:define-report and other constructors will not crash when fieldname
is invalid.
* add comments describing args and return val
* rename vars to better describe them
* exported gnc:cmdline-template-export and gnc:cmdline-get-report-id
handle ambiguous reportnames by returning #f
export code was tested using a custom saved-report, needing to call
the parent template export code. this could not work when a parent
template export code was called. enable path to call parent or custom
export-code.
there was bug in budget.scm whereby budget report would crash with
unreversed-budget featured book. the 'note' argument was missing when
calling disp-cols.
Extracting functions query_imported_transaction_accounts and perform_matching
and eliminating the early creation and passing of several local
variables.
I'm getting this error:
/home/user/src/gnucash/gnucash/import-export/import-main-matcher.c: In
function âgnc_gen_trans_list_show_allâ:
/home/user/src/gnucash/gnucash/import-export/import-main-matcher.c:1716:28:
error: âimport_trans_accountâ may be used uninitialized in this
function [-Werror=maybe-uninitialized]
per_account_list = g_hash_table_lookup (lists_per_accounts,
import_trans_account);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/user/src/gnucash/gnucash/import-export/import-main-matcher.c:1676:14:
note: âimport_trans_accountâ was declared here
Account *import_trans_account;
Looks like the variable name was incorrect.
until the very end of the import (OK or Cancel), because account commits trigger very lengthy balance
computations. For this, I call xaccAccountBeginEdit on all the accounts involved in the import,
keeping a list of them so BeginEdit is called only once. At the end of the import, commit is called
on all the accounts in the list. Note that when the user selects a target account for an imported
transaction, xaccAccountBeginEdit is called on the target account, and it is added to the list.
Another area of improvement is avoiding re-checking all register transactions to verify whether
a given imported transaction has already been matched. Instead, a hash table of split online IDs
is computed once (per account), and verified for each incoming transactions.
Finally, the list of register transactions that are potential matches for the imported ones is
further pruned ahead of time to only keep transactions that do not have an online ID. This avoid
the repeated checks that were previously happening in the match-score loop.
With this, importing 6000 transactions into a 6000 split account becomes fairly fast (a few seconds
on my slowish machine).
There are still slow areas: If you select all 6000 imported transactions and assign a destination
account to all of them, the process is impossibly sluggish because of repeated path operations
(selections, freeing) in the tree view.
If you do not specify a target account for any of the 6000 imported transactions, an "imbalance"
account is used, but the xaccAccountBeginEdit mechanism isn't applied to it by the new code, so
each imported transaction will trigger a commit, and therefore a slow balance recomputation.
Remove use of xaccTransGetSplit
I now save them into a temporary list. A single query is done for all imported transactions
and the resulting register splits are put into a hash table of lists with accounts
as key, which will speed things up when multiple accounts are found.
My tests of large imports on large accounts seems to ate that most of the time is spent
verifying whether the imported transactions has already been imported, then computing
the balance repeatedly for each imported transaction (!) when the user clicks OK to
add all the transactions to the account. So there's still room for improvement here!
The previous ofx import code performed one query for each imported transactions, which
was quite slow. The change consists of gathering all ofx transactions before doing the
query. The query must be wider to search for all matching accounts (in case the imported
transactions come from different accounts) and an enlarge date range (according to the
earliest and latest imported transaction). The rest of the code is identical to what was
done before. The final query is performed just before the matching dialog is displayed.
Rearranged gnc_price_cell_modify_verify to remove code duplication.
Also added removal of any thousands separators as if they were left
in the wrong place gnc_basic_cell_set_value_internal would raise
an error dialog and fail to accept the value.
This is a simple solution to the conflict problem. It does not guarantee an optimal match (i.e. a match with
the best combined match score) but it removes conflicts and finds an acceptable solution. The idea is simple:
- Start with the first imported transaction,
- Find all other imported transations that conflict with it (i.e., want to match the same register transaction).
- Of all these conflicting transactions, find the one with the best match to the register transaction
and set it as the winner: it keeps its match, and all other transactions go to their next favorite match.
- If any conflict happened, start back from the top as new conflicts can arise when imported transactions
lose their top match and go to their second best one (if any).
This loop is guaranteed to terminate because every time we go to the top, we remove at least one match
from the sum of all matches.
A bit of cleanup, a few questions are unresolved
Bug 797730 - Transaction matching can match multiple imported transactions to the same existing one
This is a simple solution to the conflict problem. It does not guarantee an optimal match (i.e. a match with
the best combined match score) but it removes conflicts and finds an acceptable solution. The idea is simple:
- Start with the first imported transaction,
- Find all other imported transations that conflict with it (i.e., want to match the same register transaction).
- Of all these conflicting transactions, find the one with the best match to the register transaction
and set it as the winner: it keeps its match, and all other transactions go to their next favorite match.
- If any conflict happened, start back from the top as new conflicts can arise when imported transactions
lose their top match and go to their second best one (if any).
This loop is guaranteed to terminate because every time we go to the top, we remove at least one match
from the sum of all matches.
