Use built-in glib functions to retrieve the list of per-currency price
lists, concatenate them into a single list, instead of doing it all in
hand-rolled loops.
Sorting is preformed by the calling GncTreeViewPrice so this removes
sorting from gnc_pricedb_nth_price.
There's no concurrency concern because gnc_pricedb_nth_price is a
GUI callback and so must run in the GUI thread.
It seems that std::locales created by boost::locale::generator are
not entirely compatible: If used to create a new locale with a facet
for boost::date_time one ends up with the C locale and the facet.
For the time being avoid the problem by using boost::locale to format
dates and times. std::chrono gets calendar functions in C++20 so we
can switch date-time backends once we can adopt it.
We can't use std::locale::global because all streams imbue it by
default and if it's not 'C' (aka std::locale::classic) then we
must imbue all the streams that we don't want localized, and that's
most of them.
Provides error checking for setting the C++ locale from the environment.
This is necessary both because the environment might have an invalid
locale, which would cause an unhandled exception crash.
On windows std::locale("") can't handle some Microsoft-style locale
strings (e.g. Spanish_Spain) so we use boost::locale's gen("") function
to set the locale--though even that can't handle a Microsoft-style
locale string with an appended charset (e.g. Spanish_Spain.1252) and
that's what glibc's setlocale(LC_ALL, NULL) emits.
Testing notes: Based on the averages of 3 runs, the net
user CPU to save the XML file I use is:
10.2 seconds without this change
7.6 seconds with this change
In my environment the first call to the format routine
in question, the call that sets the cache value, is at
the end of the XML load.
The refactoring provides roughly 10% reduction in user CPU
use for XML file load by moving an expensive function
to within an if-clause where the result is used. The diff looks
like a full re-write but only the if statements, indenting,
and commentary changed.
This value is queried on each comparison of split or txn sort function,
which means it is called quite a lot. Avoiding the KVP lookup of this
property gains a lot in terms of CPU cycles.
when moving entry between accounts
When using the cut transaction option the 'associated file' value was
not being pasted to the new transaction. Added scheme code to get this
value and save it to new transaction when using 'cut/copy' and then
'paste' operations. When using the duplicate option, a dialogue allows
you to keep the copied association or not. It does not get copied for
autocomplete.
This involves renaming 3 functions:
gnc_uri_get_protocol -> gnc_uri_get_scheme
gnc_uri_is_known_protocol -> gnc_uri_is_known_scheme
gnc_uri_is_file_protocol -> gnc_uri_is_file_scheme
The *_protocol variants are marked as deprecated.
Additionally a number of local variables have been renamed from
protocol to scheme to support this change.
- gnc_uri_get_components will now return NULL as protocol if the input is a normal
file system path instead of a uri (it used to return 'file')
- gnc_uri_get_protocol will now return NULL if the input is a normal
file system path instead of a uri (it used to return 'file')
- gnc_uri_is_file_protocol now returns FALSE if protocol is NULL (it used to return TRUE)
- gnc_uri_is_file_uri now returns FALSE if input is a normal file
system path instead of a uri (it used to return TRUE)
- a new function gnc_uri_targets_local_fs will return TRUE only if its input
is either a file uri or a normal file system path. This function is now mostly
used instead of gnc_uri_is_file_uri in the current code base
- a new function gnc_uri_is_uri is added to check whether its input
is a valid uri (has protocol, path and hostname for non-file uris)
Previously the account color slot has been populated with "Not Set"
when any field for the account has been edited and saved. This routine
should run once and remove all such entries.
There are a very few left that need deeper study, but this gets
rid of most of the noise. For the most part it's just getting rid of
extra variables or removing an assignment that is always
replaced later but before any reads of the variable. A few are
discarded result variables.
It can't, because if b is 0 the function would have
returned already; since b.m_hi is 0 b.m_lo can't be. The assert
reassures clang that this is the case.
We need to compare the magnitudes of the remainder and the denominator
in order to round negative numbers correctly. Note that while gnc_numeric
is constrained to a positive denominator the C++ rounding functions cannot
assume that constraint in all cases.
Combined with the previous commit, this fixes
Bug 796949 - Incorrect conversion of 0,01 USD to EUR
So that the returned price tuple has the two commodities of interest
converted to a common currency. Before the first pair that that shared
any random currency would be returned, perhaps creating an absurd result.
This function complements gnc_time64_get_day_begin/end. There was
time64CanonicalDayTime but this returned noon of the given day, where we
want 10:59am in most cases. I haven't changed time64CanonicalDayTime
directly as that may break assumptions in other parts of the code.
Instead I have created a new function that can be gradually introduced.
date-posted to not be saved.
Check the stored GDate for being in the GncDateTime range as well
as the GDate range before returning it and check trans->date-posted
against INT64_MAX instead of 0 before changing it.
The first fix for this bug handled structs tm with ambiguous times.
This one fixes the GncDate constructor when the time is ambiguous
because it's in the DST-change hour, using the same add 3 hours,
construct the LDT, and subtract the 3 hours from the result.
The string constructor handles only simple-offset HH:MM timezones and so
is immune to the bug.
And don't ask to save a not-dirty or empty book, fixing
Bug 794870 - If no book is opened, gnucash still asks if the user wants
to save changes when opening a file
These are queried continuously by the owner tree view (on Customer/Vendor/Employee
Overview pages) and recalculating them is an expensive operation.
The cache will be invalidated each time a lot reated to the owner
changes (modify or delete). The net effect is a huge responsiveness
improvement of said overviews in case of a large book.
Any operation that can overflow will throw an underflow if it's a
negative number. The C interface needs to catch both to prevent
unhandled exception crashes,
The core issue was that the delete visitor was never called because its parameter
type (char *) didn't match the boost::variant type (const char *).
Fixing the visitor's parameter type also require a const_cast
back to char * because that's what g_free takes as argument.
The rest of this commit is merely fixing KvpValue instantiations that
tried to create a char* KvpValue from a stack based const string instead
of a heap allocated one. That would bomb out on calling the
delete visitor.
Splits were not marked for deletion if the transaction is read-only
and the account is not marked for deletion yet. The net result is
that split will not be freed later on.
However xaccSplitDestroy is also called from a Transaction's do_destroy.
At that point accounts are not necessarily marked for deletion yet (like
is the case when a datafile is closed). This turned out to be a problem
for invoice post transactions (which are also read only) and hence
would cause memory to leak.
Casting a char* to a struct containing a uint32_t is not universally safe
due to alignment constraints on reads on some platforms. Copy our possibly
unaligned source data into an aligned area of memory to avoid SIGBUS on
armhf.
Reported by vorlonofportland in PR#403. This commit the John's optimized
version of Vorlon's proposed fix.
- add -Wno-deprecated-declarations to CXX_FLAGS as well. This was
reported by vorlonofportland in PR#401 to become necessary for glib 2.58
as that has deprecated g_type_class_add_private which appears in our
c++ code.
- change -Wno-deprecated-register into -Wregister. The former appeared to
be a clang dialect and alias for the latter (see
https://github.com/Barro/compiler-warnings for an overview of clang
and gcc warnings). It was moved to global CXX_FLAGS as it can only be
added for g++.
The KVP value for the qof_book_get_num_days_autoreadonly was being
called many times so it makes sense to cache it in the book to avoid
the KVP lookup.
test_suite_gncInvoice sets up the test suite. It's not part of the test
runtime, so stack variables in it have gone out of scope by the time the
tests are actually run. Making invoiceData static makes it permanent so
it exists at runtime.
First, save isn't necessary if the book is dirty, so don't... but that
means that the book has to be marked dirty after a session swap. No more
laziness.
Second, regardless of the outcome of inner_main_add_price_quotes the
session must be destroyed to remove the lock.
A couple of cleanups in QofSessionImpl::save as well: Rewrote the
descriptive comment to reflect how it really works when the backend has
gotten disconnected and removed the superfluous qof_book_set_backend
with the backend that we'd *just gotten from the book*.
Change all instances of bugzilla.gnome.org to bugs.gnucash.org, reflecting
our migration to a self-hosted bug tracker.
Inform the Translation Project Coordinator at release that this affects
translatable strings and that all message catalogs have been updated.
I think this crash is triggered because the 'account' variable
defaults to the first available AR account. If there's no AR account
it becomes null, and querying null's default book leads to segfault.
I guess I can fix segfault too by fixing gnc_account_get_book.
Change some plain string literals to std::string constants, which helps
avoiding typos and also saves some string constructors/destructors
in the KVP lookup. Nevertheless the functions in Account.cpp do not
contribute that much to the overall UI speed, but whatever.
The function qof_book_use_split_action_for_num_field gets called quite a
lot in each register display refresh (due to sorting all splits from
Split.x's xaccSplitOrder function), but it always used to use a KVP
lookup, which is rather expensive compared to accessing a gboolean member
variable.
To get rid of this cost, I had to remove the KVP lookup in this
simple-looking function. The pattern is this: A gboolean cache variable is
introduced, along with an isvalid flag. The lookup makes the expensive
KVP lookup once, then caches the value. The GObject property mechanism
offers a callback for when the setter was called, which is used to mark
the cached value as invalid. A parallel setter method (here:
qof_book_set_option) also just marks the cache as invalid. This covers
all setters, and the getters will use the cached value except for their
first invocation.
The NUM_FIELD_SOURCE feature was introduced in 2012 by the very large
commit 7cdd7372 and apparently its costs never were a problem
until the KVP lookup became more costly due to the std::vector
construction and destruction.
Turns out that the on-the-fly conversion from const char* (the KVP_OPTION_PATH
constants) to std::string with their immediate deletion afterwards is
a quite costly operation. Avoiding this is surprisingly easy: Just keep
local std::string objects at hand, and they don't have to be created
and deleted anymore.
The more optimized solution might be to turn the std::vector<std::string>
into a std::vector<GQuark>, but this commit at least improves the picture for now.
First change is to ensure gncEntry rounding is consistent. Internally
calculated values in the entry are never rounded. Consumers of
gncEntry's calculated values can request them either rounded or not.
Next use a pragmatical approach for calculating values on invoices based on
the entry values: do the rounding such that we never
create an unbalanced transaction while posting
That means
- round each entry's net value before summing them in net total
- accumulate all tax totals on invoice level per tax account before rounding
and round before before summing them in a global tax total
Hopefully this will catch a few more rounding issues in this area.
A complete solution can only offered if we allow users to manually correct
tax entries. This requires changes to user interface and data format
so that's not going to happen in gnucash 3.x.
strptime/strftime supports various modifiers to their parameters.
'E' and 'O': alternate locale-specific formats
(used in default format for Persian, Oriya, Azerbaijani)
'-': padding
(used in default format for Czech)
GnuCash passes dates as integer y/m/d without using locale-specific
formats, so we need to strip out 'E' and 'O' from the format when
scanning dates or determining separators in gnc-date.
None of '-', 'E', or 'O' are supported by boost (and '-' causes
errors), so strip them out from formatters in gnc-datetime as well.
See https://bugzilla.gnome.org/show_bug.cgi?id=795247.
This will probably need more refinement because the multiplications
are still missing rounding methods, but the changes in this commit
will already allow guile code to correctly create entries.
This commit contains another round of cleanups in the
timespec to time64 conversion. There were a number of
false assumptions that time64 = 0 would be a bad date
in the xml parser. This commit corrects enough of them to
eliminate the bug. Further cleanup is probably advised but
can be done at a later stage.
Also Bug 791825 - Accounting Period dates off by 1.
The DST start/end dates were reversed *and* the DST offset had the wrong
sign in Windows, resulting in the effective timezone always being one to
the west off (i.e. PDT was -9 and PST was -8).
Bills and invoices that are posted and subsequently unposted again still store their
posted account internally as a convenience to the user (upon reposting the old
account will be offered by default) so it's not a reliable test for the posted state.
The posted transaction on the other hand is guaranteed to only exist when the invoice is
posted. This should fix a slew of small and perhaps larger side effects, such as
a posted bill still appearing as editable, critical warnings when creating new bills/invoices
and so on.
An odd corner case: BST apparently came off of DST at 23:00 26 Oct 2014,
so midnight that day was ambiguous about being DST or not; that causes
the local_date_time constructor to throw in spite of the tm.is_dst element
being 0 (meaning pick standard time).
Instead of just failing in that case, try constructing a local_date_time
three hours later then adjust it back three hours. If *that* doesn't work
then throw a std::invalid argument.
After much thrashing this turned out to be caused by a date string
with a 3-digit year and that caused an unexpected boost::bad_cast
exception from boost::posix_time::time_from_string().
To prevent that and anything like it, pre-parse the string with
regular expressions to classify them and split out the timezone
if there is one. If neither (perhaps eventually none) of the
regexes match throw std::invalid_argument. The C function will
catch this and return 0.
91f4b19 changed the test for gncInvoiceDateExists from date != 0
to date != INT64_MAX, which isn't backwards compatible, so test
for both. But the submitted file had a posted date of -1 so
gncInvoiceIsPosted returned true anyway. That's not consistent with
the logic in dialog-invoice.c, which checked gncInvoicePostedAcc != NULL,
a better test. The result was that the "Post" button lit up but
gncInvoicePost returned immediately, doing nothing, so change
gncInvoiceIsPosted to use gncInvoicePostedAcc instead.
There were several problems that broke the Imap Editor that have been
fixed due to kvp changes. The import-map-bayes entries were being added
to the tree view based on the number token entries squared. Retrieving
import-map entries resulted in an empty list and also deleting entries
from the tree view failed.
Found with -Wimplicit-fallthrough, but older versions of GCC don't support it. So it will take some effort to add it to the build config.
I think it did not result in a reproducible bug.
Turns out the stream facet parser is really slow. Since we have a
well-constrained universe of input formats we don't really need that
overhead. This got a 33% improvement in loading a large SQLite database.
This commit fixes business reports caused by 91f4b19039
whereby the posted-date for unposted invoice was changed
from 0 to MAXINT. Now we need to use (gncInvoiceIsPosted)
or (gncInvoiceDateExists) instead.
When building from git it will add targets to generate the swig files.
When building from tarball it will just point at the generated source
files from the tarball.
Several were only there because they used to be generated via autogen.sh
and hence had to be included because autogen.sh was not supposed to be run
in a tarball based build.
A few others aren't clear so I have left them in for now:
- gnucash.1 (man page)
- gnucash-design.info (because it's unclear what we want to do with that one)
These files should clearly be in the tarball:
- gnucash.pot -> for our translators
- ChangeLog -> can't be generated outside of a git repo
- gnc-vcs-info.h -> can't be generated outside of a git repo
- guile/python bindings -> to avoid a swig dependency when building from tarball
- the two dist_add_... macros now both take a list of file names
as argument so more files can be added at once to the dist tarball.
- dist_add_generated now creates the right target by itself. There's
no need to pass one any more
- make the swig generated *.py module files explicit output files
- change a couple of custom_targets into custom_commands. The only
reason they were defined as targets was to ensure they got built
before the dist tarball. This is now properly handled by the
dist_add_... macros.
- correctly handle dependency on swig-runtime.h (using OBJECT_DEPENDS
was not the way to do it according to that property's help page)
cmake with unix makefiles fails to resolve dist dependencies
added from COPY_FROM_BUILD if these dependencies aren't built yet.
This commit replaces the COPY_FROM_BUILD based logic with two new functions
'dist_add_configured' and 'dist_add_generated' to indicate which files should
be included in the dist tarball. The latter also adds a target level dependency
to the dist tarball custom command. Hence the former should
be used for files that get generated during a cmake run while the latter
should be used for files generated as the result of a 'make/ninja-build' run
(like files for which an add_custom_command rule exists).
Note: this commit also temporarily disables the dist target when building
from a tarball (and hence it won't be tested in distcheck either). This
will be handled in a future commit.
Conflicts:
- gnucash/gnome-utils/gnc-main-window.c
I have chosen to ignore the changes pulled in from maint. The same change will
be reimplemented in a followup commit.
- po/ru.po
Same here. The translation conflicted too much and there's a
translation update available for unstable I will pull in soon.
Resolved conflicts:
gnucash/gnome-search/dialog-search.c
gnucash/gnome-utils/gnc-splash.c
gnucash/import-export/aqb/gnc-plugin-aqbanking.c
gnucash/import-export/import-account-matcher.c
po/de.po
src/report/standard-reports/budget-barchart.scm
src/report/standard-reports/budget.scm
Note the de.po conflict was resolved by dropping all changes
and merging the file again with gnucash.pot (generated after
the merge but before committing).
GLib's scribbling of freed memory is enabled on Arch so attempting to
read the deleted members of inst and book crash instead of reaturning
invalid results. These weren't really useful tests anyway.
The first attempt to fix this, a17bc85a, doesn't work because the
boost::date_time constructor gets enough information in most cases to
generate a date, just not the one we expect. This change looks for '-' in
the fourth position and if it's there assumes iso-extended format, otherwise
it assumes delimiter-less ISO without the 'T', i.e. %Y%m%d%H%M%S.
Set a feature to prevent versions older that 2.6.20 from loading a
database from which they cannot read the dates. Ideally we would do this
only if the database is written to, but the current persistence design
includes committing back to the database during the load so the net
effect is that the flag would be set anyway.
This allows direct conversion between Scheme numbers and gnc_numeric
without the performance or accuracy penalties arising from using doubles
as an intermediary.
This commit introduces a new feature flag:
GNC_FEATURE_GUID_FLAT_BAYESIAN. It signifies that the bayes import map
data are stored flat and by guid. Any time bayes import map data are
accessed, they are converted if necessary.
We need to allocate enough space for the terminating null character.
Also, I double-checked the documents for std::basic_string::c_str () and
verified that it does guarantee the terminating null, so I put a comment
in the code that depends on that.
Before, it was necessary to remove '/' from tokens so that they won't be
divided up within kvp. Now that kvp doesn't parse tokens, it's okay to
pass '/', and it's better not to translate user-provided tokens if at all
possible.
The conversion assumed there were only three levels to bayes import
map kvp: IMAP token, user-supplied token, GUID/account name. In
actuality, since user-supplied tokens could have the delimiter in them,
there could be several. This fix takes that into account like so:
IMAP token, potentially several user-supplied tokens, GUID/account name.
The import map is undergoing two conversions at the same time: account names
to guids and an hierarchical representation to a flat representation in KVP.
The bayes data are stored in the KVP store. Before this commit, they are
stored under /import-map-bayes/<token>/<account guid>/count (where count
is the datum that "matters" in bayes matching).
The problem with this is that any token including the kvp delimiter
(currently '/') gets divided, and is not found correctly during bayes
kvp searching. The quickest solution to this is to replace all "/"
characters with some other character. That has been done, along with a
re-structuring of the bayes matching code to take advantage of c++
features to make the code more concise and readable.
Also modified some test functions to fix leaks and double-frees: the
same kvp value can't be in the kvp tree twice.
Also, when I added code to clean up after the tests, some things started
breaking due to double-delete. Apparently const_cast was hiding some
programming errors. Really? You don't say? When giving a GUID* to KvpValue,
the latter takes ownership of the former.
The nested representation was very noisy. Now, the string representation
shows one line per value with the full prefix which is also more
expressive than the old version.
The template avoids the need to cast to and from void*, and adds flexibility to
the targeted function's signature.
test-stuff.h defines a macro, "failure" which is used as an identifier
in the standard IO library, so I moved any inclusion of test-stuff.h to
the last include position so that "failure" wouldn't be defined before
the IO library was included.
Since Account.c is now Account.cpp, the function signatures look a bit
different internally. The tests rely on function signatures in error
messages. Instead of trying to figure out what the exact
function signature might be, I use a substring matching strategy to
ensure that the correct error was issued.
