Latest revision as of 20:20, 22 January 2019

Budgets in Gnucash: A History

This page is intended to document the history, current status and design decisions in the development of Budgeting functionality for the G2 port of Gnucash. First person pronouns refer to Chris Shoemaker.

For the combined list of requirements for Gnucash budgets that were bourne out of all these discussions, see Budget Requirements wiki page.

Preface

Budgeting functionality in Gnucash has a long and (to me, at least) somewhat frustrating history. I'm attempting to document some of that history here because it provides some of the rationale for some of the design decisions I've made in my recent design for budgeting. I make no claim that this recounting is purely unbiased (which, incidentally, no history is) because I'm also using it as a forum for me to weigh-in on debates that raged long before I ever used Gnucash. I'm "calling 'em like I see 'em", and "pulling no punches" to use some idioms, but I do try to remain objective and I think that those places where I'm expressing my opinion are, hopefully, obvious.

Ancient Times (Before November 2000)

As early as July 2000, there was some work on implementing budgets in Gnucash. Bryan Larsen wrote a scheme report titled "budget report", that required the user to enter their budget directly into the scm file. But when the reporting system in Gnucash was updated, this report was not converted. By November 2000, Bryan considered this report unmaintained, and hoped it would be replaced with something better. The lead developers recognized budgeting as one of the most significant "missing features" of Gnucash, and hoped to include it in the next release.

Serious Discussions (November 2000 – May 2001)

In November 2000, one of the earliest recorded collaborative efforts to design a budgeting system began with almost simultaneous mailing list posts by James LewisMoss <jimdres (at) mindspring.com> and Joshua Sled (who later became one of the lead developers). Jim took an approach of soliciting requirements from list members, while Josh collected a large body of notes and posted them for discussion. (Incidentally, the lack of budgeting functionality in Gnucash was one of the factors that prompted Josh Sled to join the development effort in the first place.)

Both of these threads generated a fair amount of discussion about budgets, but no clear design consensus was reached. There was, however, some rejection of what did _not_ constitute budget functionality. For example, despite what several people claimed to desire, Josh kept the discussion on-track by explaining that budgets cannot actually constrain real-world income and expenses.

One of the most remarkable features of this period, and one that can probably only be appreciated from the comfort of several years distance, was the rather low signal-to-noise ratio in the budget discussions. Surrounding a small handful of people who had at least some clear vision of the problem space were many well-meaning, vocal, people who only side-tracked the discussions because of a lack of familiarity with software development, double-entry accounting, or even basic accounting practices.

Much of the discussion was of no particular value for achieving budget functionality, covering instead the types of psychological or physical methods people used for self-imposed fiscal discipline; it often sounded like an overly enthusiastic self-help seminar extolling the virtues of financial planning. As an example of the difficult environment for collaborative design, take this quote:

   "The principle of budgeting says that we make contributions each
   time we are paid into various savings accounts to cover each kind
   of expense. –there are also [various savings accounts], which
   we make regular contributions to. From the point of view of our
   budget, they are an expense–" <name protected> 02 Dec 2000

(For non-accounting-minded readers: Money deposited to your savings account is _never_ an expense.) Nevertheless, there was also much useful discussion, too, often driven by Josh's motivation to contribute to development.

Even though it claimed not to be a design document, Josh's notes included helpful definitions and typical use-cases covering a broad range of budget types. They also listed some informal requirements for a budgeting system, and even some concrete menu design and GUI-layout. His notes also appear to have introduced the term "savings goal" (which is apparently a Quicken term for something using virtual accounts to "hide" money in accounts, without reconcilable transactions, unrelated to budgeting) to the Gnucash devel-list discussion.

In these early design discussions, two topics became especially important: the relationship between scheduled transactions and budgeting, and the concept of budgeting categories.

Josh's notes were instrumental in focusing the discussion onto what was generally considered to be the other significant "missing feature" — scheduled transactions (SX). There was, however, some disagreement about the exact design relationships between budgeting and SX.

Aaron Peromsik tended to view them as complementary, but distinct:

   "Quicken draws a distinction between "budgeting" functionality and
   "scheduled transactions." I find the scheduled transactions useful
   as reminders, but I don't use the auto-entry functionality because
   I don't want things entered as paid until I actually mail the
   check."  Aaron Peromsik <aperomsik (at) mail.com> , 23 Nov 2000

But the nature of the distinction remained unclear:

   "I agree there is a distinction, but I see scheduled transactions
   being very useful for budgeting, so for the moment I'm including
   them together in my head." James LewisMoss
   <jimdres (at) mindspring.com>, 23 Nov 2000

   "There are two major approaches to budgeting that I am aware of,
   and while I agree that thinking about them 'together' is necessary
   to getting a design, they tend to make the head hurt." Christopher
   Browne, <cbbrowne (at) hex.net> 28 Nov 2000.

Christopher Browne then went on to make the distinction quite clear in a post in which he names them the "accountability model" and the "pragmatic adding scheduled transactions".

After a couple of months of discussion, Aaron summed up the current state of the arguments over SX vs budgeting:

   "In all the (recently sparse) discussion of budgeting and
   scheduled transactions, people seem to like to lump scheduled
   transactions in with budgeting. I still think this is a
   mistake. Budgeting is a big scary complex subsystem which many
   people will not use at all, whereas scheduled transactions can be
   a pretty straightforward affair which is more broadly applicable."
   Aaron Peromsik <aperomsik (at) mail.com> 14 Jan 2001

Even though there remained some disagreement about the relationship of budgeting to SX, Josh did something very wise: He focused on what there seemed to be greater agreement about — SX:

   "This is true [that lumping the two together is a mistake].  I
   agree with the assertion that budgeting may not be used by all;
   perhaps there is some smaller subsystem for scheduled
   transactions, which the budgeting subsystem can utilize…" Joshua
   Sled <jsled (at) asynchronous.org> 14 Jan 2001

This marked Josh's turn toward SX, so that two weeks later Josh wrote:

   "I've been focusing recently on scheduled transactions, since
   that's what people more seem to want [more than budgeting], and
   the budgeting stuff should make use of a separable
   scheduled-transaction subsystem.  My hope is that soon/eventually,
   someone else will finish/polish/maintain the scheduled transaction
   stuff while I focus on budgeting…" Joshua Sled
   <jsled (at) asynchronous.org> 1 Feb 2001

Even though the exact relationship had not been hammered out, the developers generally viewed SX as somehow either complementary or essential to budgeting and very useful in their own right. Josh's willingness to work on the part which he viewed as better defined but less satisfying was very much responsible for Gnucash gaining SX functionality.

Nevertheless, serious design decisions about budgets remained unanswered, particularly regarding the concept of budgeting categories:

   "The budgeting categories can span multiple expense
   accounts/categories, so they should be external… I think."
   Joshua Sled <jsled (at) asynchronous.org> 1 Dec 2000

   "I think the budgeting categories [bcats] are external from the
   accounts to a large degree… while they may reference the
   accounts which the bcat refers to [expense or liability or
   whatever], the information about budgeting items crosses over
   expense/account category boundaries." Joshua Sled
   <jsled (at) asynchronous.org> 13 Feb 2001

To which Gregory Novak responded:

   "True, but I think that asking the user to define an entire
   additional additional set of categories for budgeting is asking
   for trouble.  I believe I've seen talk of associating a "category"
   with every transaction in addition to the account info.  If
   budgeting could be folded into this category specification, then
   we're golden, but if not, adding budgeting category information on
   top of this would be unwieldy.

   One thing may be to just ask the user how much will be transfered
   to/from each account, regardless of the account's categorization
   or whatever.  It seems to me that this would give you a lot of
   generality with very little effort in setting things up (for the
   user)." Gregory Novak <Gregory.Novak (at) cern.ch> 14 Feb 2001

Josh remained unconvinced:

   "Maybe, maybe not.  It's all about presentation and user interface.

   I do believe that the budgeting categories are not going to be the
   same as the expense categories… they will generally be a subset
   [and, in my case they might very well be a one-to-one map], but
   the user should be able to create a set of budgeting categories on
   the level which she wishes…

   For instance, I may have expense [sub-]categories for ea. type of
   donation I make to various organizations.  But I would probably
   budget this on a yearly scale as 'Donations'.  Entertainment, as
   well… I might have categories for books and movies and painting
   supplies and whatnot, but I'd probably just budget the category as
   a whole.

   Taken to the other extreme, I might have a lot of budget
   categories, but no expense categories. I may not care about
   tracking my expenses [no matter how easy GnuCash makes it :)], but
   I still want to make a budget over some general expense
   categories, and just leave the rest of my expenses [and my
   spending allowance] as the difference between known income and
   known expenses…  I don't think I'm quite understanding your
   budgeting world view…" Joshua Sled <jsled (at) asynchronous.org> 15
   Feb 2001

There was almost no discussion of the design of budgeting between Feb 15 and Jun 15, 2001. The single thread of discussion during this period evidences the continuing disagreement over budget categories. Aaron clearly understood Josh's definition of budget categories and thought that it had no place in his concept of budgeting. Specifically referring to budget categories, Aaron wrote in response to Josh:

   [Josh:] "The forthcoming budgeting stuff should be very cool,
   too… is budgeting something you'd think you'd use?  If so, I'd
   like to bounce ideas off you as we get further along in the
   thoughts/design of it all."

   [Aaron:] "Let's put it this way: In theory, yes… but in
   practice, if budgeting is nearly as complicated as it is in
   Quicken, or nearly as complicated as your previous descriptions
   (was it all the way back in December or February) then I want no
   part of it."  <aperomsik (at) mail.com> 20 May 2001

This difference of opinion went unresolved, and this issue would resurface at a later time.

Design Diffusion (June 2001 – October 2001)

September and October generated a huge flurry of budgeting discussion. The confusion in these discussions was captured by Derek Atkins:

   "I think part of the problem in this discussion is that people are
   coming from completely different angles and the nomenclature is
   extremely confusing.  Some people seem to be looking at budgeting
   as a way to describe Gnucash accounts; some people seem to think
   of budgeting as a side line that is separable from Gnucash
   accounts.  This dichotomy appears to be causing some conversant
   trouble.  Unfortunately I don't have a good approach to solve this
   problem." Derek Atkins <warlord (at) MIT.EDU> 01 Oct 2001

In my opinion, there was one important distinction that has been left unmade even since the early discussions: the difference between capital budgeting (sometimes called investment budgeting) and cash budgeting. (Capital budgeting is planning for some far-term, usually one-time and hypothetical, expense — often with a feasibility evaluation. Cash budgeting is the recording of predicted, often recurring, usually not-so-hypothetical, incomes and expenses — usually with on-going comparison of actuals and planned.) This confusion was only somewhat troublesome when people were still working out the distinction between SX and budgeting, but now that Josh had worked out the design of SX and the discussion turned toward budgeting, the confusion was more acute.

The confusion of these two types of budgets had two main negative effects. First, constant miscommunications on this point prevented any focus on the actual design of budgeting.

Josh used the term savings goals to mean something similar to capital budgeting and he was especially interested in the potential for interactively examining financial hypotheticals. Therefore he devoted much discussion to a design that would suit these purposes. On the other hand, he clearly had cash budgeting squarely in view, too:

   "The more important analysis is the historical one, comparing
   budgeting constraints to actual spending." Joshua Sled
   <jsled (at) asynchronous.org> 22 Jan 2001

Josh would usually lump capital and cash budgeting together in his discussions (Although, in a reposting of his design notes, Josh's "Basic" vs. "Advanced" budget functions vaguely resemble cash vs. capital budgeting.) and others would disagree with his design proposals because they were considering only cash budgeting. For example, in response to Josh's idea for a GUI that would allow capital budgeting-style analysis, Bryan wrote:

   "I disagree.  As far as I'm concerned, you create a budget, you
   balance it, then you report on it continuously for 1 year.  Then
   you create a new budget with the information you learned by
   completely blowing your budget the previous year.  :)", Bryan
   Larsen <blarsen (at) AnalogSynthesis.com> 26 Jul 2001

In response to Josh's allusions to capital budgeting features, the typical reaction was to conclude that there was still disagreement about what a budgeting system really was supposed to do. This resulted in several more discussions that started out like:

   "This is where we may have to discuss the role of the budgeting
   system."  30 Sep 2001 <nigel_gnucash-devel (at) unos.net>

And continued back-and-forth with Josh describing a mixture of cash and capital budgeting features and others saying that some of those features weren't properly part of a budgeting system.

The second negative impact of the confusion of budget types was that, in Josh's continuing design notes, the broader scope of a budgeting system that supported both capital and cash budgeting without any clear distinction between the two made the design unclear and unconvincing. Josh seemed fully conscious of this, and he occasionally expressed uncertainty with his own design notes.

   "I've been playing around with designing structures to capture
   budgeting in GnuCash, and figure out how budgeting integrates with
   the rest of the program.

   When you get closer to the end, it starts to unravel a bit… it's
   in a weird state in my head at the moment… not quite coherent,
   definitely not finished." Joshua Sled <jsled (at) asynchronous.org> 22
   Jan 2001

   "I've spent many mornings of weekend at the coffee shop up on the
   corner writing out pages of notes on budgeting.  Unfortunately,
   they always seem to start with 'Okay… blank slate:' and end with
   'hmmm.'" Joshua Sled <jsled (at) asynchronous.org> 25 Jul 2001

   "I tried very persistently for a few months to pin down a good
   budgeting design, and failed to come up with something that said
   'that's it' to me immediately." Joshua Sled
   <jsled (at) asynchronous.org> 18 Sep 2001

One of the contentious design points was the resurfacing budget categories concept. Josh's position seemed unchanged:

   "And would you agree that, while similar to the account hierarchy
   in concept and perhaps application, the budgeting hierarchy
   shouldn't be forced to be the same as the expense hierarchy?"
   Joshua Sled <jsled (at) asynchronous.org> 30 Sep 2001

There was some struggle to come up with a good example of a need for budget categories that didn't map one-to-one onto accounts. In the end, the Josh settled on an example where expenses were tracked at a finer grain (Entertainment:Movies, Entertainment:Videos) than they were budgeted for (just Entertainment). In my opinion, this example is not at all convincing, since the budgeted quantity _is_ for an account, just not the leaf accounts.

The responses to complex (meaning not one-to-one mapping to accounts) budget categories varied from strong disagreement to apathy to reluctant acceptance.

Interestingly, in the responses concerning budget categories, one small exchange appears to have gone almost completely unnoticed:

   [Philip:] "And I come back to a question that was asked some time
   ago, (and not answered to my knowledge) where do I place the shoes
   I bought for soccer.  Is that a sports expense or a clothes
   expense?  How do I let it be both?  Thus is this a sports budget
   item or a clothes item.", Phillip Shelton <shelton (at) usq.edu.au>

   [Matt:] "I think that would be a task for the "action" field
   listed on the devel page for gnucash.  The idea is to be able to
   assign transactions to action fields that would allow you to group
   them in addition to categories.", Matt Kowske
   <jmkowske (at) students.wisc.edu>

   [Philip:] "What if I want to assign more than one type of action
   per split?"  27 Sep 2001

No one responded to Philip's question, but it would come up again… XXX

My own summary of those discussions would be that Josh was still gathering input about user experiences and still propelling the discussion by his motivation. But he was also still failing to synthesize capital and cash budgeting into one design. Nevertheless, despite the lack of a clear design for a budgeting system, 2001 saw tremendous progress on SX.

The Dark Ages (November 2001 – March 2003)

The rest of 2001 and all of 2002 passed with no new discussions or progress toward budgeting functionality in Gnucash.

In January 2003, there were some brief discussions about adapting the cash-flow report into a budget report. Also at that time, Matthew Vanecek announced that he planned to work on budgeting sometime in the future.

Revival (April 2003 – September 2003)

By April 2003, Dave Fancella started to design a budgeting system from scratch. His design called for including the budget data in the KVPs of the Account objects, which Matthew strongly objected to. His design doesn't appear to have resulted in any implementation, but the mailing list discussion did make one design decision quite obvious: Budgets needed to be first-class entities in order to support multiple budgets. What also became evident was that Matthew was in firm agreement with Josh regarding BudgetCategories, but he didn't push the issue when Dave's psuedo-code clearly supported only a one-to-one mapping onto Accounts.

On September 1, 2003, in his first post to the devel mailing list, Darin Willits <darin (at) willits.ca> introduced himself, outlined a new design for the budgeting system, and said he was ready to begin implementation. Darin's design had one substantial advantage: he concentrated exclusively on cash budgeting, which made the project reasonable for the few weeks he expected to be able to spend coding. (By September 3, the GUI design was complete and by September 17, most of Darin's code was complete.)

The gnucash developers had not created a good design for a budgeting system for many years, so it was unfortunate, but not surprising, that Darin had not simply discovered the perfect design out of the blue in one day. Stewart Wright and others spoke up:

   "I don't want to put a dampener on this, but I think a clearer
   focus on what is to be done is a better approach than for someone
   to go out and start coding what they would like to see. Code is a
   'Good Thing(TM)' but there are still two different camps at
   least." Stewart V. Wright <svwright+lists (at) amtp.liv.ac.uk> Sep 1
   2003

Also, despite the fact that Darin's design was clearly addressing only the problem of cash budgeting, much of the design commentary was still focused on trying to clarify the distinction between cash and capital budgeting for the people who saw no distinction. Matthew Vanecek was confusing cash and capital budgeting in his discussions, and this may have been a reason why he was strongly advocating the complex BudgetCategories design. In fact, he separated the concepts of BudgetCategories and Accounts even more than Josh had ever done:

   "I *would* like to see the capability to create a budget category
   w/o an associated account." Matthew Vanecek <mevanecek (at) yahoo.com>
   Sep 1 2003

Even while Darin implemented his design, the mailing list rediscussed what exactly a budget was and what a budgeting system should do. Stephen Cuppet even proposed an alternate complete design with a one-to-one mapping. While most of the conversation was full of miscommunications about cash vs. capital budgeting, some of the discussion did try to respond to Darin's design, and some of that response was highly critical, especially on the point of budget categories:

   "As for the concept of 'categories'.  This does not exist.  The
   actual data has accounts.  The budget should have accounts."
   Stephen Cuppett <scuppett (at) nc.rr.co> Sep 2 2003

   "I guess doing government accounting for the last 5 years has me
   jaded, but I can't for the life of me see why people are wanting
   "virtual" budget accounts.  This makes no sense.  You shouldn't be
   able to budget to a category in which you can't post actuals.
   True often you budget at some roll up higher than where you spend,
   but why budget for somewhere you can't spend.  Maintaining a
   second set of categories seems silly." Derek Neighbors
   <derek (at) gnue.org> Sep 3 2003

The right solution seemed so obvious to the professional accountant that he viewed complex BudgetCategories as crazy:

   "Maybe I'm psycho but why wouldn't you just make your accounts:

   Expenses:Food
   Expenses:Food:Dining
   Expenses:Food:Groceries

   Then you choose to budget at the "Food" level and expend at the
   "sub" level.  This is how most budgets work.  You have a chart of
   accounts with each element having different "levels".  Often you
   budget at a higher level than you expend.  I see where you are
   trying to be flexible, but I think you are making it more
   complicated than it needs to be. In order to have some resemblance
   to commitment control you really need your budget accounts to
   mirror your expense accounts even if the budget is placed at a
   different level in the hierarchy tree than the expense."  Derek
   Neighbors <derek (at) gnue.org> Sep 3 2003

Then came perhaps the most frustrating point in the history of Gnucash's budgeting: in response to some of the clearest design decision material the list had seen for years, here's what passed for design rationale:

   "Some food is not food. =P Perhaps some people would rather see a
   flatter view, rather than drilling down.  Me, I like more of a
   tree view like above.  Different folks…

   In any case, you ought to be able to link multiple accounts to a
   single budget category, even if only to satisfy people that set up
   like 'Expenses:Grocery' and 'Expenses:Fine Dining' accounts."
   <name protected> Sep 3 2003

With an illogical non-sequitur, Derek's concerns were brushed away. Even when Phil <sublime78ska (at) comcast.net> expressed strong support for Derek Neighbors's evaluation, he was dismissed with the explanation that complex BudgetCategories were needed for capital budgeting (which Darin's design never covered).

There was probably no clearer indication that design decisions were being based on the desire for capital budgeting ("financial simulation") and then applied to an implementation of cash budgeting than Matthew's response to David Ayers:

   [David:] "Are we not confusing budgeting as commonly understood
   with financial simulation?  I submit that potential purchases do
   not affect a budget. […] Otherwise budgeting becomes a game
   without any real impact."

   [Matthew:] "YES!! YES!! YES!! YES!! YES!! YES!! My point exactly!!
   The above is precisely what we need out of a budgeting tool!!!"

   [Matthew:] "I do see a need for profiling possible scenarios,
   and how a future purchase will affect my budgetary needs. […] An
   operating budget, however, like you say below, should experience
   minimal changes" Sep 4 2003

But even if the distinction between "profiling possible scenarios" and an "operating budget" was becoming clear, it was too late to affect the design of Darin's implementation. Darin submitted his patch on Sep 17, 2003.

Missing Time (October 2003 – August 2004)

Strangely, there's no evidence that anyone even looked at Darin's patch until Derek Atkins reviewed it on Feb 1, 2004. Derek made no comments on the code's design. He had only minor concerns about file names, coding style, and infrastructure integration. Despite the minor nature of Derek's comments, Darin didn't get around to addressing them until I emailed him in Aug 2004, asking for his patch. Darin's budget code was committed to the gnome-2 branch of CVS on Aug 4, 2004.

Budgets in Gnucash: Another Look (August 2003 - GnuCash 2.6)

In August 2003, I began to investigate the possibility of budgeting from within Gnucash. I had been using Gnucash with an external budget for a while, and running budgetary reports had become quite tedious. Like more than 10 developers before me, I knew I wanted budgeting functionality in Gnucash, and I was willing to code it.

I got my feet wet by reviewing 4 years of budget design discussions, and by looking at Darin's code. One of the things I liked about Darin's design was the clear focus on cash budgeting. I knew that cash budgeting was exactly what I needed because it was exactly what I was already doing externally. I had some small interest in, but almost no need for, the capital budgeting envisioned by Josh and others.

One of the more obvious short-comings of Darin's code was the lack of budget persistence. Eager to climb the learning curve of Gnucash development, I quickly implemented budget persistence in the file back-end for all parts of the budget except for BudgetCategories. It wasn't until I saw Darin's implementation of BudgetCategories that I realized the design implications for the budgeting system.

Budget [Non-]Categories

Perhaps seeing the implementation in detail made things clearer for me; or perhaps I'm just naturally opinionated. In any case, I don't feel _any_ of the ambivalence that was sometimes expressed in the discussions on budget categories. I am in whole-hearted agreement with Stephen, Phil, Derek Neighbors and everyone else who thought that complex mappings of budget categories to accounts was crazy.

In my mind, cash budgets are very simple: they record predictions about real accounts. It's difficult for me to understand why complex budget categories were attractive to so many people. (Is an old DOS version of MYM responsible?) My only conclusion is that they were perceived to solve some problem that I don't consider to be part of the cash budgeting problem. It may not be convincing to budget category supporters, but I do have a few points to add to the debate, all really about what cash budgeting is _not_:

 You can always come up with useful, fine-grained reports
 after-the-fact.  You don't have to make budgets complicated just to
 be able to run complicated reports.

 You can't create a budget before you have an account hierarchy.  You
 have to make the account hierarchy first -- really.

 Not every account is a leaf account.  This is good.  Having a good
 tree structure is what allows you to "roll up" and look at income,
 expenses, and budgets at different levels of detail.

 It's not the place of a budget to try to compensate for a poor choice of
 account hierarchy.

 Slicing "across the tree" (sometime called "2D") _is_ sometimes
 useful.  But this is not unique or essential to budgets.  "Action
 fields" address that problem and budgets shouldn't.

Additionally, after thinking a bit about the design requirements of a capital budgeting system, I'm convinced that even a complex BudgetCategory-style mapping to accounts in insufficient to handle the capital budgeting scenarios. This is because the capital budgeting task is quite unstructured, and could involve "hypothetical" accounts which could have a much less concrete definition than real accounts do.

Incidentally, I think about 70% of the perceived need for capital budgeting functionality in Gnucash could be satisfied with one (relatively) rudimentary feature: the ability to forcast future account balances based on SX.

Current Design (up through 2.6.x)

Two big pieces of the budgeting system had yet to be written: persistence and reporting. In addition, changing such a fundamental design feature (as removing budget categories) required substantial modification to the two big pieces of already-implemented budget functionality: budget initialization and budget editing.

Based on the intrusiveness of the required changes, and the amount of additional functionality remaining to be implemented, I reluctantly decided that the incremental effort of redesigning even the existing portions of budgeting was small compared to the benefit of some improvements to the GUI design: Darin's budget initialization was a multi-page druid that was I felt was GUI-overkill and buggy anyway -- I replaced it with a single options dialog. Darin's budget editing was in a stand-alone "budget workbench" that I thought was too disconnected from the rest of the GUI -- I replaced it with a spreadsheet page in the main window.

As of April 2005, I am reasonably pleased with the current state of my implementation. I think it's usable and I believe that it has all the functionality I need.

In addition, I think my budgeting system more cleanly integrated with the existing Gnucash architecture. E.g. it uses the GncPlugin and GncPluginPage APIs; it uses Gnucash's component manager API for detecting modifications to budgets; it uses Gnucash's TreeViewAccount widget for budget editing; it uses KVPs for hierarchical data storage, etc. There are still a few minor things I'd like to tweak, but that will probably always be true.

Recurrences

Even though recurrences are a _very_ small part (~ 300 LOC) of budgeting, they may prove to be the most controversial, because Gnucash already has FreqSpec. I had originally used FreqSpec in my design of budgets, some subtle bugs prompted me to look at the implementation of FreqSpec. I observed several things that I didn't like about FreqSpec:

• unverifiable algorithms: I understood what the code was supposed to do but some parts (like the parts labeled "CHECK ME") were so complex that I couldn't verify by inspection that they were correct. This was especially annoying because I knew that the correct algorithm could be expressed in a simple way.

• unnecessary duplication: I would say that the chief design problem with FreqSpec is that it treats each frequency type as a new case, and with different algorithms and more functions. This is completely unwarranted and overly-complex. A unified treatment is found in Recurrence.c

• GUI stuff in the engine: The comments ("Ideally this is not here, but what can you do?") indicate that someone knew this was bad, but didn't fix it.

• heavyweight: It sub-classed QofEntity. My view of Recurrences was that they should be only slightly larger than simple dates, and making them QofEntities was crazy.

• unneeded data fields: I guess if you think every frequency type is unique, you'll find some use for unique data fields.

• a long TODO list and many FIXMEs: It seemed that the design is suffering from its own complexity. There are more things to fix in FreqSpec than ever _went_in_ to Recurrence.

• an ugly union: There are *way* too many fields to keep track of, and for what reason? This is simply symptomatic of the incorrect assumption that every frequency type is fundamentally different.

• a poor implementation of FreqSpec composition: For much a simpler way, see Recurrence.c.

• misspellings in the public API: e.g. ANUALLY

• warnings about unmaintained functions: e.g. xaccFreqSpecSetUIType

• incorrect names for public API elements: e.g. UIFREQ_TRI_ANUALLY

• Although not really part of FreqSpec, I wanted a GUI presentation that was MUCH simpler than what existed for FreqSpec, and it looked like writing one for FreqSpec was going to be a real pain. (I'm a bit of a GUI minimalist. see e.g. gnc-recurrence.)

• incompleteness: There's currently no way to specify the recurrence "every month on the 30th (or the last day of the month if there there is no 30th), beginning Feb 28, 2005" or any other recurrence of this class. (One could argue that this is a fault with the GUI, not with FreqSpec, but, in reality, this is really kludgy to solve in the GUI and very clean to handle in Recurrence. Strangely enough, considering the amount of code behind FreqSpec, it doesn't even capture the concept of a beginning of the recurrence -- that's left to be handled on your own. Recurrence handles that by itself.)

Even though I would have loved to just use FreqSpec, I knew that I could have something bug-free and verifiably correct in a few hours, with 1/10th the code, and capturing more needed functionality (start date) than FreqSpec did. So I wrote Recurrence which worked fine, and I never gave it another thought.

That is, until several months later when Derek Atkins gave me a little trouble about it on IRC. I agreed to reevaluate my decision after I completed the budgeting code (which I guess is about now) and I also promised to find a test case for the bug in FreqSpec before I started any non-budget related hacking.

Well, I've re-evaluated things now, and taken another closer look at FreqSpec. In hindsight, I think it's even clearer to me that Recurrence was a good idea. FreqSpec is just plain complicated, and a royal pain to debug (I found that failing test case, and put it into the unit test.) It does weird stuff like measure the relative offset between two dates by first measuring their relative offsets to some arbitrary third date and then comparing offsets. Plus, at the time I wrote Recurrence, I didn't even realize that, even if FreqSpec hadn't been quirky, it only offers half of what I wanted it to.

In summary, Recurrence is a much simpler design than FreqSpec and they offer more functionality than FreqSpec. I believe that the disadvantage of having two ways to represent a recurring event is out-weighed by the advantages of completeness, correctness and simplicity offered by Recurrence.

Episode 4 - A new hope (Future directions)

GnuCash version 2.6.x budgets allow users to set budget amounts for each account by period. Reports can be run to provide extra information. Despite this, many users (especially personal finance users) still want more out of budgeting. The breadth of opinions on the subject (coupled with a massive amount of history present) has resulted in common themes continually repeating themselves without resolution. For example, built in "zero sum budgeting" capability, and being able to view actual vs budgeted on the same screen. Looking at this situation, Matt G (An amateur Gnucash user wanting a better method of budgeting in Gnucash) decided to perform some analysis and construct the section below on Gnucash requirements. This is intended to bring together all of the design decisions in a clear and balanced discussion on what Gnucash should have in place for budgeting. From here, bug requests or enhancement requests can be filed and actioned with all available programmers being able to align their efforts. Whilst this history will (and should) be maintained to understand the context, the requirements will explain to any person what the intent is and how the gui should be constructed.

APPENDIX

A collection of devel-list posts from people offering to work on budgeting:

"For people like me, running gnucash as a personal (family) financial program, budgeting is pretty important. Folks bother to go through the pain of entering in every bloody transaction for years at a time, because they want to achieve greater financial security and prosperity. Planning for income and expenses, and making sure that something is left over to save or invest, is really one of the main points of the exercise. So I hope that these matters will continue to receive some development attention." Mike Sabin <m_sabin (at) mindspring.com>

"Also, I'm perhaps interested in contributing to Gnucash. Specifically, I want budgeting to work. Now I'm using a silly system involving a spreadsheet." Gregory Novak <greg (at) novak.cern.ch> 12 Feb 2001

"Hello all, I've joined this list because I saw in the archives that there was discussion of a "budget GUI" feature. I currently use q*cken (please excuse my language) for all my finances, but I would love to switch over to gnucash, thus eliminating one more barrier to being 100% linux.

) However, my primary usage of the aforementioned windoze program right

now is the budgeting feature, and I really can't do without it. How is progress on the budget gui? If help is needed, I would be glad to contribute. I have fairly extensive experience in C and C++, but no experience in programming x-windows unfortunately. But I am eager to get my hands dirty." Ansel Freniere <ansel (at) lambdares.com> 13 Feb 2001

"I am a reasonably experienced scheme programmer, and am strongly motivated to help get at least some basic budgeting functionality up and running soon. Is there anything I can do to help the project along?" Larry Hunter <hunter (at) 180glencoe.com> 30 Jun 2001

"Hi, I'm converting to linux and I would like to help add budgeting functionality to gnucash so I can use it. I have lots of time and some experience with C/C++, but i'm new to linux programming. I would like to work on the design document and coding as much as possible, and I have lots of ideas for budgeting based on the spreadsheet I currently use (if you want to see it, just tell me and i'll send you a copy). I'll read up on the current documents and discussions, but I'd like to know what to do to get to work as soon as I can." Richard Garand <krogoth2 (at) softhome.net>

"I'm willing to help with Gnucash development. My preferred project would be budgeting since that is the weakness in Gnucash that I need filled. If that is being worked on, contact me and I'll help with design, coding, testing and/or documentation. If it is not being worked on, I'll start." Phil Longstaff <plongstaff (at) newearth.org> Aug 17 2003

"As I have now moved completely away from Quicken to GnuCash, I would like to help out.

My main interests (at the moment) are adding budgeting (home) and mysql (or generic sql) backend.

The current version I am using is 1.8.9-1 from Debian/Sarge. I would like to get in and get started on something, but I don't want to reduplicate effort. So I guess I need which project needs the most help at the moment.

I have experience in C/C++/FORTRAN/perl/ etc. Know some lisp. I have done Xtoolkit and Motif coding. Currently I am a unix architect/admin for about every flavor of unix there is (my company has never met an OS it didn't like). I have also done DBA work and coding against mysql, informix, sybase, db/2, and ingres. I have also managed some small web servers and all email for an ISP. I see on the help wanted place on the gnucash site, webmasters are needed. Perhaps I can help out in that area." James Richardson <james (at) jamesrichardson.org> May 28 2004

Add-On: "The Little Budget Tool"

A working add-on tool has been programmed by Gabriel M. Beddingfield called "The Little Budget Tool". Download the source code from http://www.teuton.org/~gabriel/lb. It has an optional GUI, using Qt.

• Basic instructions
• Configure, make and make install lb.
• Go to the example subdirectory ~/lb-1.0.0/example
• Examine the budget.csv file.

From your gnucash list of accounts make a suitable budget.csv for yourself. (no, we can't tell you much you should spend each month). If your accounts are in the form Income:job1 Income:interest and Expense:groceries Expense:rent you won't have to tweak the settings in a .lbrc file to have it work quickly.

Extract your actual expenditure from gnucash

(from the man page)

From the accounts screen (the main window when you open GnuCash), select from the top menu Reports | Income & Expense | Cash Flow. By default it will give you a summary for all the account for the entire year. However, you probably want to get a report for last month. Or for a 4-week period corresponding to your paycheck. Here's how to tweak it for what you need:

1. Click the Options button.

2. Under the General tab, select the start and end dates for the period you are interested in. (E.g. From: Start of previous month, To: End of previous month.)

3. Under the Accounts tab, select the account that you want to study. By default, it has all the Assets accounts selected. However, for most personal budgeting, you may want to just select the "Assets:Current Assets:Checking Account." This will track everything going into and out of that account. To unselect the other accounts, just click on them.

4. Click OK.

5. Click the Export button.

6. In the "Save" dialog box, give the name and location for where you want to save the report. It's recommended to include the .html extension. (E.g. report.html)

You're done. You can run an LB report with this.

Running a LB report:

From the command line, you enter:

   $ lb -b budget.csv -r report.html

And you'll get a report that merges the data from each.

Running the gui

It accepts all the same command-line arguments as lb. You can use it like this:

   $ lbgui -b *.csv -r *.html

Then select Edit | Edit Report to tweak it.

Useful things

You can run a different budget for each month (power bills come every 3 months and water bills every four months for me) and run comparisons against the actual expenditure for each month as well as doing a year to date summary.

Zero-Sum Budgeting with GnuCash: A Fresh Approach

Zero-Sum Budgeting with a virtual envelope system (similar to the YNAB approach) works very well in GnuCash for personal finances. Examples and suggestions to avoid some irritations can be found on the Zero-Sum Budgeting page.

Alternatively, check out this page: http://allmybrain.com/2008/12/15/better-budgeting-with-gnucash/. Note that this method does not use the budget plugin of gnucash - it creates artifical accounts that are your budgeted amounts, with you doing extra bookkeeping entries to get it to work.