That is let the percentage increase gradually. The current granularity is still
very rough, but at least it gives an indication of getting closer to fully
loading the data. The previous configuration on the other hand only suggested
something was happening but with no indication where in the load process
gnucash was.
This prevents calling xaccAccountRecomputeBalanceInCurrency on each split that gets added,
which was exponentially increasing load times. On a huge test book the
load time dropped from 53 minutes to 1m20s.
The core issue was that the delete visitor was never called because its parameter
type (char *) didn't match the boost::variant type (const char *).
Fixing the visitor's parameter type also require a const_cast
back to char * because that's what g_free takes as argument.
The rest of this commit is merely fixing KvpValue instantiations that
tried to create a char* KvpValue from a stack based const string instead
of a heap allocated one. That would bomb out on calling the
delete visitor.
Splits were not marked for deletion if the transaction is read-only
and the account is not marked for deletion yet. The net result is
that split will not be freed later on.
However xaccSplitDestroy is also called from a Transaction's do_destroy.
At that point accounts are not necessarily marked for deletion yet (like
is the case when a datafile is closed). This turned out to be a problem
for invoice post transactions (which are also read only) and hence
would cause memory to leak.
Always use absolute paths for configured directories (BINDIR etc.)
Abstract out the guts of gnc_gbr_find_foo_dir for foo in lib, bin, and data.
etc requires special handling because of the way it's treated if prefix
begins with /opt.
Always fall back on the configured directory if binreloc is disabled and
no default is passed in.
Casting a char* to a struct containing a uint32_t is not universally safe
due to alignment constraints on reads on some platforms. Copy our possibly
unaligned source data into an aligned area of memory to avoid SIGBUS on
armhf.
Reported by vorlonofportland in PR#403. This commit the John's optimized
version of Vorlon's proposed fix.
- add -Wno-deprecated-declarations to CXX_FLAGS as well. This was
reported by vorlonofportland in PR#401 to become necessary for glib 2.58
as that has deprecated g_type_class_add_private which appears in our
c++ code.
- change -Wno-deprecated-register into -Wregister. The former appeared to
be a clang dialect and alias for the latter (see
https://github.com/Barro/compiler-warnings for an overview of clang
and gcc warnings). It was moved to global CXX_FLAGS as it can only be
added for g++.
The KVP value for the qof_book_get_num_days_autoreadonly was being
called many times so it makes sense to cache it in the book to avoid
the KVP lookup.
Each was used exactly once and simply wrapped a function call.
Also replace static function time_parse_failure that just returned a
constant with the constant.
This report had renamed "Action" to "Charge Type". For consistency,
let's rename it back, and add migration path in options.scm for
saved-options. 2 fewer strings for translation.
test_suite_gncInvoice sets up the test suite. It's not part of the test
runtime, so stack variables in it have gone out of scope by the time the
tests are actually run. Making invoiceData static makes it permanent so
it exists at runtime.
With the options dialogue open when Gnucash is closed this error is
logged in the trace file, gnc_close_gui_component() component not found.
This is fixed by reversing the components list found for session so the
options dialogue is closed before the report window.
Anticipating that some users might prefer to see exact prices,
add a preference to General>Numbers to configure whether prices
are rounded to decimals or are displayed as exact fractions.
When printing numbers convert them to a new decimal denominator with
rounding if the passed-in print info specifies that they should be
forced and rounded.
Make the default price settings forced and rounded.
Pass the price currency to gnc_default_price_print_info and
use the currency's fraction * 100 to determine the round-to
denominator and the number of decimal places to display.
First, save isn't necessary if the book is dirty, so don't... but that
means that the book has to be marked dirty after a session swap. No more
laziness.
Second, regardless of the outcome of inner_main_add_price_quotes the
session must be destroyed to remove the lock.
A couple of cleanups in QofSessionImpl::save as well: Rewrote the
descriptive comment to reflect how it really works when the backend has
gotten disconnected and removed the superfluous qof_book_set_backend
with the backend that we'd *just gotten from the book*.
Change all instances of bugzilla.gnome.org to bugs.gnucash.org, reflecting
our migration to a self-hosted bug tracker.
Inform the Translation Project Coordinator at release that this affects
translatable strings and that all message catalogs have been updated.
I think this crash is triggered because the 'account' variable
defaults to the first available AR account. If there's no AR account
it becomes null, and querying null's default book leads to segfault.
I guess I can fix segfault too by fixing gnc_account_get_book.
In addition to not begining to edit already-loaded transactions,
don't try to load splits that are already loaded. It shouldn't
be possible to load a transaction without also loading its splits.
Change some plain string literals to std::string constants, which helps
avoiding typos and also saves some string constructors/destructors
in the KVP lookup. Nevertheless the functions in Account.cpp do not
contribute that much to the overall UI speed, but whatever.
The underlying problem was that the vendor object remained in infant state
That confused the backend code so it used an sql INSERT statement instead
of an UPDATE statement to write back the changes. As the object already
existed in the db this would fail.
The fix is to ensure the object doesn't remain in infant state during
sql loading. See the bug report for a more detailed explanation.
The function qof_book_use_split_action_for_num_field gets called quite a
lot in each register display refresh (due to sorting all splits from
Split.x's xaccSplitOrder function), but it always used to use a KVP
lookup, which is rather expensive compared to accessing a gboolean member
variable.
To get rid of this cost, I had to remove the KVP lookup in this
simple-looking function. The pattern is this: A gboolean cache variable is
introduced, along with an isvalid flag. The lookup makes the expensive
KVP lookup once, then caches the value. The GObject property mechanism
offers a callback for when the setter was called, which is used to mark
the cached value as invalid. A parallel setter method (here:
qof_book_set_option) also just marks the cache as invalid. This covers
all setters, and the getters will use the cached value except for their
first invocation.
The NUM_FIELD_SOURCE feature was introduced in 2012 by the very large
commit 7cdd7372 and apparently its costs never were a problem
until the KVP lookup became more costly due to the std::vector
construction and destruction.
Turns out that the on-the-fly conversion from const char* (the KVP_OPTION_PATH
constants) to std::string with their immediate deletion afterwards is
a quite costly operation. Avoiding this is surprisingly easy: Just keep
local std::string objects at hand, and they don't have to be created
and deleted anymore.
The more optimized solution might be to turn the std::vector<std::string>
into a std::vector<GQuark>, but this commit at least improves the picture for now.
An extra XaccTransBeginEdit, never committed, for transactions that
the backend tried to load when they were already there. That made
the register think that something else had it open.
Instead of reporting an error and declining to load the file (XML)
or failing to enter a value (SQL) when a bad date is found in the
database, use a 0 time stamp (1970-01-01 00:00:00 UTC). Adds a warning
in SQL backends; there was one already in XML.
1. Don't even check for price/exchange rate on template transactions,
there's no point.
2. Extract function get_transaction_currency:
a. Check all split commodities are valid, abort transaction creation if
not.
b. If the template transaction's currency isn't used by any of the
splits set the new transaction's currency to the first-found currency if
there is one, otherwise to the first-found commodity.
3. Fix a minor typo in a comment.
Limit the range of the random value to 1..1000 to prevent overflows,
particularly in number-of-periods or number-of-years variables.
While we're at it, g_random_int and g_random_int_range return ints so
piping the result through gnc_double_to_numeric() doesn't make much
sense. That's removed, we just construct a gnc_numeric.
Provide a backup recovery function that instead of dropping primaries
and restoring backups merges the primaries and backups. This should
handle a worst-case safe-save failure where the backup tables don't
have a complete set of rows for some reason.
We don't use floats in GnuCash, we use doubles (and those as little as
possible), but dbd-sqlite3 is broken in that it stores only floats.
Simply casting floats to doubles introduces bogus additional digits
that can cause round-trip tests to fail. Instead convert floats to
doubles by multiplying by 10E6, rounding, then dividing by 10E6.
Removes a public function, GncDbiSqlConnection::table_manage_backup that
should have been private in the first place.
Better encapsulates table renames and drops with private functions and
handles cases where there exist some primary tables and some backup tables.
SQLite backend.
Release Note: This bug caused data loss if you saved your SQLite3
database to a different file or database.
The problem is that in SQLite3 (though not in MySQL or PgSQL) the
subquery ((SELECT DISTINCT guid FROM transactions)) (note the double
parentheses) returns only the first guid in the subquery's results.
Some transactions are loaded by special queries and those queries are
also used to retrieve the transaction's slots so they weren't affected.
First change is to ensure gncEntry rounding is consistent. Internally
calculated values in the entry are never rounded. Consumers of
gncEntry's calculated values can request them either rounded or not.
Next use a pragmatical approach for calculating values on invoices based on
the entry values: do the rounding such that we never
create an unbalanced transaction while posting
That means
- round each entry's net value before summing them in net total
- accumulate all tax totals on invoice level per tax account before rounding
and round before before summing them in a global tax total
Hopefully this will catch a few more rounding issues in this area.
A complete solution can only offered if we allow users to manually correct
tax entries. This requires changes to user interface and data format
so that's not going to happen in gnucash 3.x.
strptime/strftime supports various modifiers to their parameters.
'E' and 'O': alternate locale-specific formats
(used in default format for Persian, Oriya, Azerbaijani)
'-': padding
(used in default format for Czech)
GnuCash passes dates as integer y/m/d without using locale-specific
formats, so we need to strip out 'E' and 'O' from the format when
scanning dates or determining separators in gnc-date.
None of '-', 'E', or 'O' are supported by boost (and '-' causes
errors), so strip them out from formatters in gnc-datetime as well.
See https://bugzilla.gnome.org/show_bug.cgi?id=795247.
This will probably need more refinement because the multiplications
are still missing rounding methods, but the changes in this commit
will already allow guile code to correctly create entries.
Clearer syntax helped find flawed test - while set-tm:mday directly
accepts 1-31, set-tm:mon accepts 0-11 to represent 1-12, therefore
must minus 1. set-tm:year accepts 92 to represent 1992, therefore must
minus 1900.
We want to sanitize render-options-changed, therefore it must return
an html-object. Unfortunately this is not accessible to
app-utils/options.scm. If we move this function to
report-system/html-utilities.scm, it can access html-objects.
Also rename it to gnc:html-render-options-changed
The swig 3.0 generated python wrappers trigger a warning converted into an error issued
by gcc 8.0 for using strncpy as follows:
strncpy(buff, "swig_ptr: ", 10);
The reason is this call will truncate the trailing null byte from the string.
This appears to have been fixed in swig master already but that's not released yet
so let disable the warning when compiling the swig wrappers until it is.
This commit contains another round of cleanups in the
timespec to time64 conversion. There were a number of
false assumptions that time64 = 0 would be a bad date
in the xml parser. This commit corrects enough of them to
eliminate the bug. Further cleanup is probably advised but
can be done at a later stage.
Also Bug 791825 - Accounting Period dates off by 1.
The DST start/end dates were reversed *and* the DST offset had the wrong
sign in Windows, resulting in the effective timezone always being one to
the west off (i.e. PDT was -9 and PST was -8).
Take 3: Catch encoding exceptions from trying to read a string into
Scheme using scm_from_utf8_string and try again using
scm_from_locale_string. If that throws too, give up and log a
warning.
First, remove the unnecessary locale push & pop on <CT_TIME64>load.
Second, the registry accesses were caused by using g_win32_get_locale
to convert the Microsoft locale strings to POSIX ones. We don't care
what kind of string we get as long as we can pass it back to setlocale,
so remove that.
Third, gnc_push/pop_locale were used only in backend/dbi in a
very limited way and did much more than was necessary, so
convert them to C++ inlines in gnc-backend-dbi.hpp that does
only what we need them to.
properly - corrupted business data
Turned out to be a pointer/value mismatch between <CT_NUMERIC>load()
and most of the setter functions, so the address was getting set
as the value.
This crash started to appear as of commit 80dbb9940b because the sequence
of split loading has changed as a result of the query optimizations.
Invoice transactions get loaded before the general transaction loading happens.
However because of this, when an invoice transaction was encountered again
during general transaction loading, it was (correctly) not created again
AND (incorrectly) not opened for subsequent editing. This caused
an assert to fail when the splits for this transaction are loaded
shortly afterwards. The solution is simply to ensure all transactions
are opened for editing during the general transaction loading call.
set/get_locale are apparently very expensive on Mingw64, and setting
the C locale for extracting a string is unnecessary.
Unfortunately the released version of libdbi still uses strtod so
setting the C locale *is* still necessary for retrieving floats
and doubles and for passing queries.
Thanks to Mkubat for the diagnosis.
The mingw-w64 toolchain bizarrely substitutes scm_to_locale_string()
for scm_to_utf8_string(). This results in latin1 (yeah, "locale" is
a lie) instead of utf8 which causes an assertion in
g_utf8_collate_key().
Perhaps equally bizarre, the compiler doesn't make the substitution
with scm_to_utf8_stringn(), so use that instead.
Bills and invoices that are posted and subsequently unposted again still store their
posted account internally as a convenience to the user (upon reposting the old
account will be offered by default) so it's not a reliable test for the posted state.
The posted transaction on the other hand is guaranteed to only exist when the invoice is
posted. This should fix a slew of small and perhaps larger side effects, such as
a posted bill still appearing as editable, critical warnings when creating new bills/invoices
and so on.
An odd corner case: BST apparently came off of DST at 23:00 26 Oct 2014,
so midnight that day was ambiguous about being DST or not; that causes
the local_date_time constructor to throw in spite of the tm.is_dst element
being 0 (meaning pick standard time).
Instead of just failing in that case, try constructing a local_date_time
three hours later then adjust it back three hours. If *that* doesn't work
then throw a std::invalid argument.
After much thrashing this turned out to be caused by a date string
with a 3-digit year and that caused an unexpected boost::bad_cast
exception from boost::posix_time::time_from_string().
To prevent that and anything like it, pre-parse the string with
regular expressions to classify them and split out the timezone
if there is one. If neither (perhaps eventually none) of the
regexes match throw std::invalid_argument. The C function will
catch this and return 0.
91f4b19 changed the test for gncInvoiceDateExists from date != 0
to date != INT64_MAX, which isn't backwards compatible, so test
for both. But the submitted file had a posted date of -1 so
gncInvoiceIsPosted returned true anyway. That's not consistent with
the logic in dialog-invoice.c, which checked gncInvoicePostedAcc != NULL,
a better test. The result was that the "Post" button lit up but
gncInvoicePost returned immediately, doing nothing, so change
gncInvoiceIsPosted to use gncInvoicePostedAcc instead.
Be smarter about what is path and key for each slot.
Instead of assuming a slash is always a path separator (first attempt
on unstable) or never a separator (second attempt),
track the parent path while loading kvp slots from the db
and deduce the slot's name by substracting this parent path.
There were several problems that broke the Imap Editor that have been
fixed due to kvp changes. The import-map-bayes entries were being added
to the tree view based on the number token entries squared. Retrieving
import-map entries resulted in an empty list and also deleting entries
from the tree view failed.
When accounts are deleted that have an open register window a component
not found error is triggered. After the account is destroyed, a call to
'gnc_resume_gui_refresh' calls 'gnc_gui_refresh_internal' and this then
calls 'find_component_ids_by_class' which is used in order resulting in
'register-single' being unregistered before 'GncPluginPageRegister' and
hence 'ld' being freed but the register not knowing this. Reversing the
list fixes this.
Replacing libgncmod-python, libgncmod-core-utils-python, and
libgncmod-app-utils-python with _sw_core_utils and _sw_app_utils.
The latter two are the modules that init.py wants to load and with
Python3 Swig appears to no longer make them available via libgncmod.
Note that there may still be some problems with actually using the
console, but it at least loads at startup without complaint.
Although the PR was titled "Some [fixes] for missing transient parent
warnings", it actually has a bugfix and an assorted collection of
GUI cleanups that happen to include a couple of transient-parent fixes.
Four date elements were affected: GncEntry::date, GncEntry::date_entered,
GncInvoice::opened, and GncInvoice::posted. The problem arose during the
cleansing of Timespec from the reports; the setter functions for those
elements were converted to time64 but no provision was made to the SQL
backend to pass them time64 instead of Timespec*.
This commit adds a new column type, CT_TIME64, and changes the column
types for those elements to CT_TIME64.
The import-map-bayes uses a three-part key that uses the same delimiter
as a path and the SQL backend was throwing away everything except the
account guid.
Added more slot types including import-map-bayes ones to the test xml
file to help debug this and to catch it in the future.
Also don't pass std::string.c_str() to a std::vector<std::string>
constructor, just pass the string.
This was producing a date corresponding to 01-01-1970. make-zdate was
being used in aging.scm as a hack to search all splits prior to
end-date. fix the date query logic to set begin-date match to #f.
Found with -Wimplicit-fallthrough, but older versions of GCC don't support it. So it will take some effort to add it to the build config.
I think it did not result in a reproducible bug.
In a previous commit I wrongly believed xgettext might not find
our strings if we used the fully namespaced boost::locale::translate or
aliased bl::translate instead of just translate. However that's
wrong, so I have changed the code to always use bl::translate.
At present these are
- log.conf
- config-user.scm (previously config.user-2.0)
Implement one-time migration of these files to the proper directory if they previously exist
Look for a log.conf file in gnc_userconfig_dir instead of gnc_userdata_dir
Note this commit also uses boost::locale::translate for the first time.
We may need to fine-tune our use, but in the current state strings marked
for translation using that function are already picked up for gnucash.pot.
Turns out the stream facet parser is really slow. Since we have a
well-constrained universe of input formats we don't really need that
overhead. This got a 33% improvement in loading a large SQLite database.
Two preferences were being used heavily in the drawing of the register
so it makes sense for them to be saved in the split_register structure
so they can be easily be referenced.
transaction number or split action (requires at least GnuCash 2.5.0)
Strip leading delimiters from KVP keys when reading them from the
database. Leading delimiters are incorrectly included in databases
created with GnuCash 2.6.x.
Move the CSS settings file to a location based on XDG_CONFIG_HOME which
is OS dependant but with slight changes for Windows and OS X.
- Windows: CSIDL_APPDATA/Gnucash
- OS X: $HOME/Application Support/Gnucash
- Linux: $XDG_CONFIG_HOME/gnucash (or the default $HOME/.config/gnucash)
This commit rewrites income-gst-statement.scm to be a derivation
of transaction.scm, modifying the options database and passing
custom arguments to the trep-renderer.
This should hopefully reduce risk of errors.
This commit adds 2 additional helper calls, primarily useful for
derived reports.
gnc:option-make-internal! will hide an existing option. e.g.
a derived report can set the value for a Display/* option and
hide it from the user.
gnc:unregister-option will unregister option. This is primarily
useful for derived options e.g. another report copies from
transaction.scm and removes some options and recreates them with
different parameters.
For example, unregister existing option from section "Accounts"
name "Accounts", and recreate with different parameters e.g.
limited account types.
This commit will change (infobox) to a general-purpose
renderer for "all options changed by user" in options.scm
and can be inserted into any report. It reduces the
number of strings required.
- Don't attempt to create a subdirectory of a non-existing home directory (use tmpdir as base directory in that case)
- Make sure all tests run in an environment with GNC_BUILDDIR and GNC_UNINSTALLED set. Otherwise
the one-shot old .gnucash to new GNC_DATA_HOME migration may already have run at build time,
preventing us from informing the user a run time.
- Re-enable the userdata-dir with invalid home test (linux only).
It will no longer attempt to use /home/janssege/.gnucash. That was
requiring lots of extra conditions.
It will also default to a base directory (gnc_data_home) in the
build dir if it detects the code is run during building or testing.
That again allows to simplify it as there's no need for temp dir
juggling in case the build environment doesn't have a writable home dir.
The swig wrappers don't really depend on git (but rather on swig) and there can be
situations the builder wants to generate the wrappers also from a tar ball.
This commit fixes business reports caused by 91f4b19039
whereby the posted-date for unposted invoice was changed
from 0 to MAXINT. Now we need to use (gncInvoiceIsPosted)
or (gncInvoiceDateExists) instead.
When building from git it will add targets to generate the swig files.
When building from tarball it will just point at the generated source
files from the tarball.
When building from git it will add targets to generate the swig files.
When building from tarball it will just point at the generated source
files from the tarball.
Several were only there because they used to be generated via autogen.sh
and hence had to be included because autogen.sh was not supposed to be run
in a tarball based build.
A few others aren't clear so I have left them in for now:
- gnucash.1 (man page)
- gnucash-design.info (because it's unclear what we want to do with that one)
These files should clearly be in the tarball:
- gnucash.pot -> for our translators
- ChangeLog -> can't be generated outside of a git repo
- gnc-vcs-info.h -> can't be generated outside of a git repo
- guile/python bindings -> to avoid a swig dependency when building from tarball
- the two dist_add_... macros now both take a list of file names
as argument so more files can be added at once to the dist tarball.
- dist_add_generated now creates the right target by itself. There's
no need to pass one any more
- make the swig generated *.py module files explicit output files
- change a couple of custom_targets into custom_commands. The only
reason they were defined as targets was to ensure they got built
before the dist tarball. This is now properly handled by the
dist_add_... macros.
- correctly handle dependency on swig-runtime.h (using OBJECT_DEPENDS
was not the way to do it according to that property's help page)
cmake with unix makefiles fails to resolve dist dependencies
added from COPY_FROM_BUILD if these dependencies aren't built yet.
This commit replaces the COPY_FROM_BUILD based logic with two new functions
'dist_add_configured' and 'dist_add_generated' to indicate which files should
be included in the dist tarball. The latter also adds a target level dependency
to the dist tarball custom command. Hence the former should
be used for files that get generated during a cmake run while the latter
should be used for files generated as the result of a 'make/ninja-build' run
(like files for which an add_custom_command rule exists).
Note: this commit also temporarily disables the dist target when building
from a tarball (and hence it won't be tested in distcheck either). This
will be handled in a future commit.
This includes removal of the now unused make-gnucash-potfiles.in,
checking for CMakeLists.txt rather than Makefile.am in gnc-vcs-info,
upating the HACKING file,
and generally updating references to autotools.
I have kept "Makefile.*" exclude patterns in our CMakeLists.txt files
because they may still be lingering in the source directory from
previous autogen.sh runs. At some point these should probably be
removed as well still, together with the gitignore references to them.
Conflicts:
- gnucash/gnome-utils/gnc-main-window.c
I have chosen to ignore the changes pulled in from maint. The same change will
be reimplemented in a followup commit.
- po/ru.po
Same here. The translation conflicted too much and there's a
translation update available for unstable I will pull in soon.
