2020-06-30 GnuCash IRC logs

00:09:38 *** Unhammer has quit IRC
00:25:14 *** Mechtilde has joined #gnucash
00:44:50 *** Unhammer has joined #gnucash
00:44:50 *** ChanServ sets mode: +v Unhammer
00:48:01 *** angel has quit IRC
00:55:30 *** sbluhm has joined #gnucash
01:09:53 *** storyjesse has quit IRC
01:32:33 *** sbluhm has quit IRC
01:49:54 *** sbluhm has joined #gnucash
01:49:54 *** ChanServ sets mode: +v sbluhm
02:00:15 *** suukim has joined #gnucash
02:01:06 *** Mechtilde has quit IRC
02:12:53 *** marusich has joined #gnucash
02:12:53 *** ChanServ sets mode: +v marusich
02:14:56 *** CDB-Work has quit IRC
02:16:09 *** sbluhm has quit IRC
02:18:53 *** jervin has quit IRC
02:28:18 *** sbluhm has joined #gnucash
02:28:18 *** ChanServ sets mode: +v sbluhm
02:32:27 *** sbluhm has quit IRC
02:46:00 *** jervin has joined #gnucash
02:46:06 *** jervin has quit IRC
02:55:03 *** sbluhm has joined #gnucash
02:55:03 *** ChanServ sets mode: +v sbluhm
02:58:50 *** Hamaryns has joined #gnucash
02:58:50 *** ChanServ sets mode: +v Hamaryns
03:03:00 *** Mechtilde has joined #gnucash
03:14:06 *** User_ has joined #gnucash
03:42:21 *** suukim has quit IRC
03:42:43 *** mauritslamers has quit IRC
03:46:39 *** fell has joined #gnucash
03:46:39 *** ChanServ sets mode: +o fell
03:54:11 *** suukim has joined #gnucash
05:05:30 *** Aussie_matt has quit IRC
05:08:08 *** chf has quit IRC
05:09:10 *** chf has joined #gnucash
05:27:51 <chris> the budget chart report is a major wart because it has start/end dates rather than budget period start/end
05:48:58 <chris> hmm looks like I completely forgot to upgrade budget-barchart to use chartJS
05:53:25 *** ramontjunior has quit IRC
06:01:21 *** ramontjunior has joined #gnucash
06:11:52 *** Hamaryns has quit IRC
06:26:24 *** Hamaryns has joined #gnucash
06:26:24 *** ChanServ sets mode: +v Hamaryns
06:33:04 *** Jimraehl1 has joined #gnucash
06:34:01 *** Jimraehl1 has quit IRC
06:55:41 *** Hamaryns has quit IRC
07:20:38 *** Hamaryns has joined #gnucash
07:20:38 *** ChanServ sets mode: +v Hamaryns
07:23:08 *** David has quit IRC
07:23:15 *** David has joined #gnucash
07:35:10 *** Agfarmer18 has joined #gnucash
07:37:50 *** Agfarmer18 has quit IRC
08:12:00 *** Daresbury has joined #gnucash
08:12:15 <Daresbury> Hello is anyone available please?
08:14:36 *** ChanServ sets mode: +v Daresbury
08:15:38 <Daresbury> Hello is anyone available please?
08:16:09 <chris> "Please don't ask to ask, just ask and wait! (Possibly a few hours!!)"
08:20:14 <Daresbury> Thanks Chris - new here so need to learn the etiquate! I need help with an odd ball transaction import issue - long story short - spent weeks building out ledgers and other supporting info and sought to import transactions. It fails, sits there sulking not responding. Created a minimal configureation and importing transactions works OK. Import
08:20:14 <Daresbury> accounts.csv and try to import transactions and it dies again. Looks as though there is some issue with my accounts bu I have no idea where to start diagnostics - on build 3.1 11th Apr 2020.
08:21:03 <chris> is your CSV valid?
08:21:29 <Daresbury> CSV is exported from a GnuCash set up.
08:21:50 <chris> how many lines?
08:22:39 <Daresbury> 162
08:23:02 <Daresbury> That is there are 161 ledgers (accounts)
08:23:34 <chris> so is the CSV describing the account structure? not the transactions?
08:25:02 <Daresbury> Two CSVs Transactions (22 lines) Accounts (161) - transaction import fails when configured with all accounts
08:25:44 <Daresbury> Clean simple build (checking account) the transactions import OK
08:26:15 <Daresbury> 'build out' with all of the accounts and the attempt to import the same transactions an it dies.
08:29:27 *** chris has quit IRC
08:32:42 *** chris has joined #gnucash
08:32:42 *** ChanServ sets mode: +v chris
08:33:07 <warlord> Daresbury, you cannot import an account structure.
08:33:16 <warlord> only transactions are (can be) imported
08:33:22 <warlord> Any "extra" accounts are ignored
08:34:45 <Daresbury> Well FILE - IMPORT has "Import Accounts from csv..."
08:34:55 *** dnbNCoffee has joined #gnucash
08:35:55 <warlord> Well, so it does.. I am unaware of how that particular feature works. gjanssens would be your guy.
08:35:55 <Daresbury> But that is not really the issue - I have a fully built instance of GnuCash which fails to import transactions - as a DIAGNOSTIC I set up a new minimal GnUCash instance
08:37:21 <Daresbury> A clean build imports the transactions - I abandon the trans action import - import the accounst CSV - then try to import the transactions and it fails ... just as it does with the origonal build
08:37:41 <Daresbury> I conclude there may be an issue with the accounts
08:38:28 <chris> warlord: is https://xkcd.com/1172/ familiar?
08:46:15 *** Hamaryns has quit IRC
08:50:47 <fell> Daresbury: can you find acluein the https://wiki.gnucash.org/wiki/Tracefile?
08:53:14 *** dnbNCoffee has quit IRC
08:54:38 *** Hirppa has joined #gnucash
08:56:26 <Daresbury> Thanks fell - looking for it now
09:00:30 *** chris has quit IRC
09:05:18 *** chris has joined #gnucash
09:05:18 *** ChanServ sets mode: +v chris
09:14:15 <warlord> chris, LOL
09:16:48 *** sbluhm has quit IRC
09:34:34 *** chris has quit IRC
09:34:46 <Daresbury> Well after some poking around, I found that the UK currency sign on Account Description and or notes might be the culprit here. In the accounts CSV it appeared as A(with ^)£30 - changing this to 30UKD in the CSV seemed to eliminate the issue - went back to my origonal file, removed all £ symbols from the accounts and it now seems to import transac
09:34:46 <Daresbury> tions albeit whole numbers of pounds are being processed as pence!
09:36:56 <warlord> Hmm, wonder if it's an ISO vs UTF8 encoding issue?
09:37:52 *** chris has joined #gnucash
09:37:52 *** ChanServ sets mode: +v chris
09:41:38 <Daresbury> Warlord - I would agree that is a relevant possiblity through the account export / import process, but totally internal to the product?? Also why would it only 'cough' when importing transactions?
09:45:28 <warlord> Daresbury, no idea. I honestly wasn't even aware of the Import Accounts feature until today!
09:45:59 <warlord> As for why £ would break import -- if the encoding is wrong it breaks the regex used to detect currency amounts.
09:48:13 <Daresbury> ...importing accounts is a red herring here ... it was part of a diagnotic process ... during which I noticed the abberation of £ symbols. It seems to be the presence of a £ symbol on an account when entered through the GUI causes the import of TRANSACTIONS to fail. I will see if I can reproduce the issue and raise a bug reprot.
09:49:46 *** sbluhm has joined #gnucash
09:49:46 *** ChanServ sets mode: +v sbluhm
09:55:06 *** David1 has joined #gnucash
09:55:58 *** David has quit IRC
09:55:58 *** David1 is now known as David
10:18:39 *** hussam has quit IRC
10:24:57 *** codesmyt_ has joined #gnucash
10:26:37 *** codesmythe has quit IRC
10:31:54 *** kael has joined #gnucash
10:31:54 *** ChanServ sets mode: +v kael
10:37:22 *** Mechtilde has quit IRC
10:46:56 <chris> https://www.reddit.com/r/gnucash they seem happy enough
11:00:23 *** hussam has joined #gnucash
11:00:23 *** ChanServ sets mode: +v hussam
11:07:59 *** chris is now known as chris_afk
11:16:43 *** omnireq has joined #gnucash
11:16:56 *** jralls_afk has joined #gnucash
11:16:56 *** ChanServ sets mode: +o jralls_afk
11:17:44 *** jralls has quit IRC
11:24:12 *** ArtGravity has joined #gnucash
11:24:12 *** ChanServ sets mode: +v ArtGravity
11:36:50 *** rdswift has left #gnucash
11:37:34 <Daresbury> Does a .db.ses belong to GnuCash? I'm still trying to bottom out transaction imports going unresponsive. I can see such a file being update every 15 minutes in the Temp filestore. Also empty mat-debug-99999.log files.
11:41:32 *** sbluhm has quit IRC
11:43:25 *** keiffer has joined #gnucash
11:43:39 *** Agfarmer18 has joined #gnucash
11:53:58 *** guak has joined #gnucash
12:00:14 <Daresbury> Well after an hour of pateince, with nothing appear to be working on the UI, the import has now worked. :D
12:01:29 <warlord> Oh really? It just took an hour to process?!?
12:01:54 <warlord> Or are you saying you waited an hour, retried, and it worked this time?
12:02:46 *** ramontjunior has quit IRC
12:03:42 <Daresbury> No, on checking it took circa 1hr15 to process 274 transactions into a build with 162 accounts (ledger lines). Well that's Q1 loaded ... 3 more to go!
12:05:45 <warlord> WOW.
12:06:08 <warlord> I think chris_afk , jralls_afk , and gjanssens (who isn't online) should hear about that.
12:07:15 *** jralls_afk is now known as jralls
12:07:42 <Daresbury> It is not as though this is a slow machine - dual core 2.2GHz - 8GB memory
12:08:29 <warlord> Right; it shouldn't take 1h15m to process 274 transactions and 162 accounts.
12:10:02 <jralls> A core duo is a pretty slow machine by today's standards, but I agree that 45 seconds per transaction isn't right. However we have lots of CSV import users and this is the first time that I've heard of poor performance.
12:11:09 *** gjanssens has joined #gnucash
12:11:09 *** ChanServ sets mode: +o gjanssens
12:11:21 <warlord> And there is gjanssens !
12:11:42 <warlord> Must've had burning in his ears. LOL
12:11:48 <Daresbury> I am sure you have - this is my first use of the product, so I have no bench mark. I though I had broken the build.
12:13:10 <Daresbury> I built a minimal virgin build, ran in 22 transactions fairly swiftly - then I imported my account structure and tried again ... it just died forever.
12:14:17 <Daresbury> I noted UK £ text in Account description / notes looked odd so I edited those out, re-ran the diagnostics and it seemed better (it did not die all together).
12:15:18 <Daresbury> Tried the Q1 import 274 transactions and it took 85 minutes. I also note some of the auto mappings do not work!
12:16:33 <Daresbury> Oh COCK! The dates are all screwed!
12:18:55 *** jralls has quit IRC
12:19:58 *** Agfarmer18 has quit IRC
12:24:02 *** jralls has joined #gnucash
12:24:02 *** ChanServ sets mode: +o jralls
12:26:49 <Daresbury> @jralls - how does the improt work? When I ran a diagnostic trace it rather looked as though each imported transaction was being checked "[qof_query_run_internal] matching objects=11c95f80" against EVERY account.
12:26:49 <gncbot> Daresbury: Error: "jralls" is not a valid command.
12:29:54 *** jralls_afk has joined #gnucash
12:29:54 *** ChanServ sets mode: +o jralls_afk
12:30:05 *** jralls has quit IRC
12:30:09 *** jralls_afk is now known as jralls
12:30:25 <jralls> Comcast is being really flakey today.
12:31:37 <Daresbury> If that is correct, in the case of my import of 274 transactions to a configuration of 161 accounts, that is £44k matches. About 10 per second.
12:35:12 <Daresbury> Forget that - I'm looking at the wrong trace file! Something is though taking an inordinate length of time!
12:35:59 *** kael has quit IRC
12:37:16 <gjanssens> Ok, that's slow... I'd have to profile to know more. But that won't be anytime soon unfortunately.
12:37:16 <gncbot> gjanssens: Sent 3 days, 17 hours, and 53 minutes ago: <jralls> I just back-checked the import map editor and the Online Accounts button has been there since 3.0, so I'm going to remove that image and demote the invalid mapping dialog item (it *is* new) to Importing Transactions.
12:37:17 <gncbot> gjanssens: Sent 3 days, 17 hours, and 51 minutes ago: <jralls> Also the link to the release notes doesn't work very well so I removed it again. There aren't any anchors in news.phtml and linking directly to a .news presents it with no styling.
12:38:14 <jralls> gjanssens about that last, I found the anchors and put the link back in.
12:39:46 <jralls> Daresbury, is your account tree and transaction import real data or something you'd be willing to share?
12:40:23 <Daresbury> It is real, but quite happy to share - at the moment it is an empty build.
12:46:59 <jralls> Please open a bug report then, and attach the account and transaction CSVs. Do check and obfuscate anything in the transactions you might think sensitive.
12:47:01 <Daresbury> Note Gnu is using circa 50% CPU - in other words nearly one core to itself
12:47:47 <jralls> As one would expect if it's running without competition and in an inefficient loop.
12:48:26 <gjanssens> jralls: ok
12:48:41 <jralls> It's very much single threaded.
12:49:13 <Daresbury> Thank you for all of th ehelp here today. I will let this import run, then assemble a bug report with as much information on reproducibility as I can muster.
12:52:42 <jralls> Super, thanks!
12:54:14 *** marusich has quit IRC
12:58:39 <jralls> warlord, we really need to do something about codesigning the windows builds. Step one is getting a certificate. Do you have a CA that you like?
13:00:42 *** Agfarmer18 has joined #gnucash
13:04:59 *** kael has joined #gnucash
13:04:59 *** ChanServ sets mode: +v kael
13:09:51 <jralls> gjanssens, I asked Benno (who's also the Dutch team leader) about Mark Haanen. He hasn't heard from him either so he's asked the team if anyone would volunteer to take over GnuCash.
13:10:37 <gjanssens> jralls: I did the same thing but hadn't had a chance to let you know
13:10:42 <gjanssens> GTG...
13:11:01 *** gjanssens has quit IRC
13:32:05 <jralls> chris_afk: Re window decorations I see 3.11 and 4.0 both using the native icons and 3.8 using the adwaita ones. Mingw-w64 did a mass upgrade last month, I imagine that the gtk-win32 folks did something.
13:33:46 <warlord> jralls, I use LetsEncrypt -- but that wouldn't work for this.
13:34:53 <jralls> Right, that's only for SSL.
13:46:53 *** kael has quit IRC
13:49:23 *** David has quit IRC
13:49:28 *** David has joined #gnucash
13:53:31 <warlord> jralls, I don't have any particular preference for software-signing CA
13:53:36 <jralls> Quite a range in price: $500/year at digicert to $85/year at comodo.
13:53:49 <warlord> Perfect use of Tip-Jar money :)
13:54:09 <jralls> Absolutely.
14:05:54 <jralls> It's not just sending money, though. https://support.ksoftware.net/support/solutions/articles/35844-what-are-the-requirements-to-obtain-a-code-signing-certificate-
14:19:33 <finster> hello all. i just saw you released gnucash 4.0. thanks a lot for your hard work!
14:20:10 <finster> fell: you there?
14:21:16 <warlord> What is OV vs EV?
14:22:03 <finster> warlord: ov is organization, e.g. your company gets sent a letter so the CA knows the address is correct
14:22:26 <finster> warlord: ev is an extended validation, like here https://www.globalsign.com/en/ssl-information-center/what-is-an-extended-validation-certificate
14:22:52 <finster> *organization validation
14:24:24 *** Mechtilde has joined #gnucash
14:27:59 <warlord> The issue is that there is no "GnuCash" organization.
14:28:39 <warlord> So either one of us devs goes as an individual (but we cannot get an EV cert as an individual), or someone (me?) gets a cert as a company (e.g. IHTFP Consulting, Inc)
14:36:07 *** sbluhm has joined #gnucash
14:36:07 *** ChanServ sets mode: +v sbluhm
14:37:39 *** frakturfreak has joined #gnucash
14:37:39 *** ChanServ sets mode: +v frakturfreak
14:37:42 *** suukim has quit IRC
14:38:51 <jralls> warlord, EV is for drivers, we need only OV.
14:41:11 <jralls> Do you have the necessary documentation for IHTFP to get verified? Otherwise I think one of us would have to do it as an individual as I've done with Apple.
14:41:21 *** David has quit IRC
14:41:24 *** finster has quit IRC
14:41:26 *** David has joined #gnucash
14:41:40 *** keiffer1 has joined #gnucash
14:41:52 <jralls> BIAB, have to mail my tax returns.
14:42:32 *** keiffer has quit IRC
14:42:32 *** keiffer1 is now known as keiffer
14:52:31 *** AdrienM has joined #gnucash
15:11:12 <warlord> jralls, do we want it to be signed by IHTFP? Or signed by one of the individual developers?
15:11:44 <warlord> jralls, I do have the documentation, somewhere, in one of the 30 boxes in my still-packed office.
15:12:26 *** Mechtilde has quit IRC
15:15:08 *** kael has joined #gnucash
15:15:08 *** ChanServ sets mode: +v kael
15:16:24 <warlord> jralls, for example the phone is in my name.
15:16:36 <warlord> So I would probably need to get a DUNS
15:27:17 <jralls> I don't think at matters who is on the certificate as long as it's signed.
15:31:22 <fell> finster:ja
15:32:07 <warlord> Ooh, I do have a DUNS number for IHTFP :)
15:33:48 <jralls> I saw at least one SO comment saying that that speeds up the process considerably.
15:37:13 *** Daresbury has quit IRC
15:40:15 *** Daresbury has joined #gnucash
15:42:58 <warlord> Or maybe not.. Hmm.. DNB has info about my company but the lookup isn't giving me a DUNS number
15:54:03 <jralls> You could try requesting one at https://www.dnb.com/duns-number.html, middle button at the bottom.
15:58:45 <warlord> I have one.
15:58:54 <warlord> Just took a while to actually find it..
15:59:14 <warlord> But this begs the question: Do we want GnuCash signed by IHTFP Consulting?
16:00:37 *** AdrienM has quit IRC
16:09:45 *** sbluhm has quit IRC
16:11:55 *** Agfarmer18 has quit IRC
16:13:24 *** David has quit IRC
16:13:29 *** David has joined #gnucash
16:29:34 *** User_ has quit IRC
16:29:45 <jralls> warlord, Like I said, I don't think it matters who does the signing, what matters is that it's signed. The Apple bundles use my Apple Developer Program certificate. Nobody has balked at that.
16:31:36 <warlord> Okay. Let me try to find the documentation.
16:32:02 <warlord> Ooh, that was easy -- found my articles of Incorporation. ;)
16:33:20 <warlord> I dont know what else I'll need besides the AoI and DUNS#
16:33:34 <warlord> (I suppose I can supply a bank statement if necessary).
16:34:32 *** David has quit IRC
16:34:38 *** David has joined #gnucash
16:49:23 <warlord> I do, however, need to wait for some DNB data to update.
16:53:59 <Daresbury> jralls - bug report raised as requested - cheers
16:57:07 *** User has joined #gnucash
17:12:00 *** frakturfreak has quit IRC
17:19:28 *** User has quit IRC
17:39:26 *** phebus has quit IRC
17:44:24 <chris_afk> I'm 99.99% sure there's O(N^x) in the xaccTransCommitEdit when autobalancing, where x >= 2
17:45:52 <jralls> Daresbury, It might amuse you to learn that it took 19 minutes to import your test data on my 3 GHz 8-core Xeon Mac Pro.
17:45:54 <warlord> Why would there be an O(N^2), let alone larger, operation in xaccTransCommitEdit?
17:46:58 <warlord> (then again, it also depends on what N there is; if it's only N in #splits in a transaction then if it's n^2 or n^3 then it shouldn't matter)
17:47:21 <chris_afk> warlord: this from experimenting a few months ago
17:47:45 *** phebus has joined #gnucash
17:47:45 *** ChanServ sets mode: +v phebus
17:47:46 <chris_afk> IIRC something something scanning all accounts at each commit
17:48:22 <chris_afk> it makes all imports very slow eventually
17:48:29 *** chris_afk is now known as chris
17:49:27 <Daresbury> jralls, Thank you for promptly reporducing the problem. As I think you said, it is single threaded so can only use one of the 8 cores ... 19 minutes is still a long time ... did you try listing the mappings ... that takes long enough to make a brew.
17:50:32 <jralls> Daresbury, No, one thing at a time. I had the profiler on it and I'm looking at the results.
17:51:17 <Daresbury> OK Thanks - 23:00 UK time so I'll be calling it a night soon.
17:52:37 <Daresbury> In case you have not seen the bug update - without Baysian matching the import is nearly instant, but there is no automatic transaction posting.
17:55:30 <chris> I tracked a QIF importer slow performance in some CommitEdit... can't remember whether Trans or Account
17:56:22 <chris> I think it was xaccAccountCommitEdit
17:57:07 <chris> e.g. recalc all split balances, also scan all these splits' peers' accounts, ensure their balances are clear etc. O(N^x).
17:57:28 *** kael has quit IRC
18:04:07 *** kael has joined #gnucash
18:04:07 *** ChanServ sets mode: +v kael
18:07:36 *** Daresbury is now known as Daresbury[A]
18:07:36 * Daresbury[A] is now away - Reason : Sleep hours!
18:35:29 *** jervin has joined #gnucash
18:36:14 <CDB-Man> chris: let me know when you're awake
18:37:10 *** David has quit IRC
18:37:35 *** David has joined #gnucash
18:45:54 <CDB-Man> hmm, on gnucash-4.0-2020-06-30-git-4.0-4-g2610b5f57+.setup.exe the alphavantage currency lookup is choking. anyone else see that?
18:46:27 *** kael has quit IRC
18:48:40 *** heredoc has quit IRC
18:49:04 <jralls> CDB-Man, It's unlikely to be in GnuCash. The only commits have been to the advanced portfolio report.
18:49:32 <CDB-Man> the last version I used prior to today's nightly is gnucash-3.905-2020-06-20-git-3.905-4-g3d5606a54+.setup.exe
18:49:48 <CDB-Man> does F::Q fail the currency lookup for you?
18:50:37 <jralls> gnc-fq-dump seems to be hanging on a currency lookup.
18:51:04 <CDB-Man> OK, so it's not just me (and therefore not my key expiring or something)
18:58:57 <jralls> No, looks like Alphavantage is having trouble, gnc-fq-dump alphavantage CSCO pukes too, gnc-fq-dump yahoo_json CSCO works fine.
18:59:40 *** omnireq has quit IRC
19:19:30 *** heredoc has joined #gnucash
19:37:03 <warlord> chris, If that's the issue then perhaps it should be wrapped in a single transaction, and/or disabling GUI events, to prevent all those re-computations until the end?
19:41:50 *** Aussie_matt has joined #gnucash
20:18:40 *** guak has quit IRC
20:40:55 *** jralls_afk has joined #gnucash
20:40:55 *** ChanServ sets mode: +o jralls_afk
20:41:53 *** jralls has quit IRC
21:42:41 *** chris has quit IRC
22:27:14 *** Daresbury[A] has quit IRC
22:30:43 *** Daresbury has joined #gnucash
23:19:17 *** chris has joined #gnucash
23:19:17 *** ChanServ sets mode: +v chris
23:28:54 *** ArtGravity has quit IRC
23:45:46 *** angel has joined #gnucash