Revision as of 20:29, 24 March 2013

A good set of tests is a critical requirement for modern software development, both to ensure the quality of the product and to help developers to quickly modify code without introducing bugs or causing regressions. The literature on software testing is vast; excellent tutorials and references are available both online and in print.

All developers are encouraged to review tests and ensure that any work is covered by both acceptance and unit tests.

Current Test Architecture

Acceptance Tests

Acceptance tests in Gnucash are based on a home-grown (or anonymously sourced) set of macros and functions which can be found in src/test-core. The quality, scope, and coverage of these tests varies greatly; some parts of GnuCash are not tested lightly or not at all, others are tested fairly extensively. While I've labelled the "acceptance tests", in many cases they're written more as unit tests whose scope is a single function rather than a whole module.

Unit Tests

Unit testing, invented by Kent Beck in the early 1990s, seeks to test all execution paths in single functions with as much isolation from the rest of the system as possible.

GnuCash has adopted the GLib testing framework to facilitate unit testing. Muslim Choclov wrote unit tests for the most important modules in LibQOF as a GSoC2011 project. Work continues to get all of LibQOF and the engine fully tested to facilitate major architectural changes needed to make GnuCash a proper database application.

See also https://developer.gnome.org/glib/stable/gtester.html for the actual command line tool to run tests.

Running Unit Tests

The collection of all implemented tests in gnucash is run by running

make check

If any tests failed, you can run the respective executable in the subdirectory yourself, such as test-qof. The argument --verbose might be helpful.

Unit Test Policies

• Getter/Setter functions which only set or retrieve an instance member variable do not need to be tested.
• Convenience functions which only wrap another function to change the function's name or to provide a default argument do not require testing.
• Composed functions, or functions which simply string together a series of calls to other functions, need not be tested if the called functions are all tested, have no side effects, and where the composed function has only one flow of control.
• There is some disagreement among testing gurus about whether all execution paths in a function should be exercised in a single test function or separately in a test function per execution path. Use your judgement here. Remember that the dictum of Agile Development is to write a little bit at a time and to refactor as often as you need to. That applies as much to test code as it does to application code. It's OK to change your mind!
• Similarly there is tension among the gurus about how much to make a test program dependent upon, and how much to use mock objects to replace actual dependency code. Keeping in mind the goal of a short code-compile-test cycle, use your judgement.

Unit Test Facilities

To set up unit testing in a directory:

• Create a "test" directory if there isn't one already
• Create a Makefile.am in that test directory, again if there isn't one already.
• Add the line

 include $(top_srcdir)/test-templates/Makefile.decl

in the Makefile.am and in every parent directory's Makefile.am that doesn't already have that line. If a parent directory has more than one subdirectory and not all of them have unit tests, create or add to the variable TEST_SUBDIRS.

• You will be adding one or more test programs. These are declared by adding their names to the TEST_PROGS variable, then defining the usual automake variables for each. For example:

 TEST_PROGS += test-foo test-bar
 noinst_PROGRAMS = $(TEST_PROGS)
 
 test_foo_SOURCES = foo.c baz.c utest-foo.c
 test_foo_HEADERS = foo.h 
 test_foo_LDADD   = ../libgncmod-pepper.la
 (And so on, also for test_bar)

There is a test support module with some useful functions for controlling logging and signals in src/test-core. If you use it, add

 ${top_builddir}/src/test-core/libtest-core.la

to your test program's LDADD variable.

Test Coverage Status

Unit Tests

LibQOF

Done:

• qofbackend
• qofbook
• qofinstance
• qofobject
• qofsession
• gnc-date
• kvp_frame

Engine

Done:

• Account

In Progress:

• Split

Known Test Needs

• GtkAction callbacks referenced directly in GtkBuilder UI files need at a minimum "presence testing" so that make check will fail if the callbacks don't compile for some reason.