Data Validation

Data scrubbing, repairing and forward migration routines. More...

Files
file	Scrub.h
	convert single-entry accounts to clean double-entry
file	Scrub2.h
	Utilities to Convert Stock Accounts to use Lots.
file	Scrub3.h
	High-Level API for imposing Lot constraints.
file	ScrubBusiness.h
	Cleanup functions for business objects.

Double-Entry Scrubbing
Convert single-entry accounts to clean double-entry Provides a set of functions and utilities for checking and repairing (formerly called 'scrubbing clean') single-entry accounts so that they can be promoted into self-consistent, clean double-entry accounts. Basically and additionally, this file collects all functions that turn old (deprecated) data structures into the current new data model. The ScrubOrphans() methods search for transacations that contain splits that do not have a parent account. These "orphaned splits" are placed into an "orphan account" which the user will have to go into and clean up. Kind of like the unix "Lost+Found" directory for orphaned inodes.
void	gnc_set_abort_scrub (gboolean abort)
	The gnc_set_abort_scrub () method causes a currently running scrub operation to stop, if abort is TRUE; gnc_set_abort_scrub(FALSE) must be called before any scrubbing operation.
gboolean	gnc_get_abort_scrub (void)
gboolean	gnc_get_ongoing_scrub (void)
	The gnc_get_ongoing_scrub () method returns TRUE if a scrub operation is ongoing.
void	xaccTransScrubOrphans (Transaction *trans)
	The xaccTransScrubOrphans() method scrubs only the splits in the given transaction.
void	xaccAccountScrubOrphans (Account *acc, QofPercentageFunc percentagefunc)
	The xaccAccountScrubOrphans() method performs this scrub only for the indicated account, and not for any of its children.
void	xaccAccountTreeScrubOrphans (Account *acc, QofPercentageFunc percentagefunc)
	The xaccAccountTreeScrubOrphans() method performs this scrub for the indicated account and its children.
void	xaccSplitScrub (Split *split)
	The xaccSplitScrub method ensures that if this split has the same commodity and currency, then it will have the same amount and value. More...
void	xaccTransScrubSplits (Transaction *trans)
	The xacc*ScrubSplits() calls xaccSplitScrub() on each split in the respective structure: transaction, account, account & it's children, account-group.
void	xaccAccountScrubSplits (Account *account)
void	xaccAccountTreeScrubSplits (Account *account)
void	xaccTransScrubImbalance (Transaction *trans, Account *root, Account *parent)
	The xaccScrubImbalance() method searches for transactions that do not balance to zero. More...
void	xaccAccountScrubImbalance (Account *acc, QofPercentageFunc percentagefunc)
void	xaccAccountTreeScrubImbalance (Account *acc, QofPercentageFunc percentagefunc)
void	xaccTransScrubCurrency (Transaction *trans)
	The xaccTransScrubCurrency method fixes transactions without a common_currency by looking for the most commonly used currency among all the splits in the transaction. More...
void	xaccAccountScrubCommodity (Account *account)
	The xaccAccountScrubCommodity method fixed accounts without a commodity by using the old account currency and security. More...
void	xaccAccountTreeScrubCommodities (Account *acc)
	The xaccAccountTreeScrubCommodities will scrub the currency/commodity of all accounts & transactions in the specified account or any child account. More...
void	xaccAccountTreeScrubQuoteSources (Account *root, gnc_commodity_table *table)
	This routine will migrate the information about price quote sources from the account data structures to the commodity data structures. More...
void	xaccAccountScrubKvp (Account *account)
	Removes empty "notes", "placeholder", and "hbci" KVP slots from Accounts. More...
void	xaccAccountScrubColorNotSet (QofBook *book)
	Remove color slots that have a "Not Set" value, since 2.4.0, fixed in 3.4 This should only be run once on a book.
void	xaccTransScrubPostedDate (Transaction *trans)
	Changes Transaction date_posted timestamps from 00:00 local to 11:00 UTC. More...

Lot Management Routines
Provides the low-level API for checking and repairing ('scrubbing clean') the usage of Lots and lot balances in stock and commodity accounts. Broken lots are repaired using a first-in, first-out (FIFO) accounting schedule. This is a 'low-level' API in the sense that each routine accomplishes only one particular task needed to clean up a Lot. To clean up a Lot as a whole, you almost certainly want to use one of the high-level API routines from the Scrub3.h file.
void	xaccAccountAssignLots (Account *acc)
	The xaccAccountAssignLots() routine will walk over all of the splits in an account, and make sure that each belongs to a lot. More...
void	xaccLotFill (GNCLot *lot)
	The xaccLotFill() routine attempts to assign splits to the indicated lot until the lot balance goes to zero, or until there are no suitable (i.e. More...
void	xaccLotScrubDoubleBalance (GNCLot *lot)
	The xaccLotScrubDoubleBalance() routine examines the indicated lot. More...
gboolean	xaccScrubMergeSubSplits (Split *split, gboolean strict)
	The xaccScrubMergeSubSplits() routine will merge together all of the splits that were at one time split off from this split, but are no longer needed to be kept separate. More...
gboolean	xaccScrubMergeLotSubSplits (GNCLot *lot, gboolean strict)
	The xaccScrubMergeLotSubSplits() routine does the same as the xaccScrubMergSubSplits, except that it does it for all of the splits in the lot.

High-Level Lot Constraint
Provides the high-level API for checking and repairing ('scrubbing clean') the usage of Lots and Cap Gains transactions in stock and commodity accounts.
gboolean	xaccScrubLot (GNCLot *lot)
	The xaccScrubLot() routine makes sure that the indicated lot is self-consistent and properly balanced, and fixes it if its not. More...
void	xaccAccountScrubLots (Account *acc)
	The xaccAccountScrubLots() routine makes sure that every split in the account is assigned to a lot, and that then, every lot is self-consistent (by calling xaccScrubLot() on each lot). More...
void	xaccAccountTreeScrubLots (Account *acc)

Cleanup functions for business objects
Provides the high-level API for checking and repairing ('scrubbing clean') the various data objects used by the business functions.
gboolean	gncScrubBusinessLot (GNCLot *lot)
	The gncScrubBusinessLot() function makes sure that the indicated lot has all the correct properties required for a lot used in the business features. More...
gboolean	gncScrubBusinessSplit (Split *split)
	The gncScrubBusinessSplit() function will fix all issues found with the given split. More...
void	gncScrubBusinessAccountLots (Account *acc, QofPercentageFunc percentagefunc)
	The gncScrubBusinessAccountLots() function will call gncScrubBusinessLot() on each lot in the given account. More...
void	gncScrubBusinessAccountSplits (Account *acc, QofPercentageFunc percentagefunc)
	The gncScrubBusinessAccountSplits() function will call gncScrubBusinessSplit() on each split in the given account.
void	gncScrubBusinessAccount (Account *acc, QofPercentageFunc percentagefunc)
	The gncScrubBusinessAccount() function will call all scrub functions relevant for a given account on condition the account is a business related account (Accounts Receivable or Accounts Payable type). More...
void	gncScrubBusinessAccountTree (Account *acc, QofPercentageFunc percentagefunc)
	The gncScrubBusinessAccountTreeLots() function will call gncScrubBusinessAccount() on the given account and its sub accounts.

Detailed Description

Data scrubbing, repairing and forward migration routines.

These routines check and repair data, making sure that it is in a format that the current version of the GnuCash Engine likes. These routines serve both to provide backwards compatibility with older versions of GnuCash, and to fix or at least paper over possible current problems.

It is typically expected that the scrub routines are run over newly imported data, as well as during data file input.

In some cases, it is entirely appropriate to invoke these routines from the GUI, to validate that the user input through the GUI is in a format that the system likes. This includes things like balancing individual transactions, or assigning splits to lots, so that capital gains can be computed.

Function Documentation

gncScrubBusinessAccount()

void gncScrubBusinessAccount	(	Account *	acc,
		QofPercentageFunc	percentagefunc
	)

The gncScrubBusinessAccount() function will call all scrub functions relevant for a given account on condition the account is a business related account (Accounts Receivable or Accounts Payable type).

This routine is the primary routine for fixing all (known) issues in a business account.

Definition at line 736 of file ScrubBusiness.c.

{
    if (!acc) return;
    if (FALSE == xaccAccountIsAPARType (xaccAccountGetType (acc))) return;

    gncScrubBusinessAccountLots (acc, percentagefunc);
    gncScrubBusinessAccountSplits (acc, percentagefunc);
}

gncScrubBusinessAccountLots()

void gncScrubBusinessAccountLots	(	Account *	acc,
		QofPercentageFunc	percentagefunc
	)

The gncScrubBusinessAccountLots() function will call gncScrubBusinessLot() on each lot in the given account.

This routine is the primary routine for ensuring that the lot structure of every lot of a business account is in good order.

Definition at line 620 of file ScrubBusiness.c.

{
    LotList *lots, *node;
    gint lot_count = 0;
    gint curr_lot_no = 0;
    const gchar *str;
    const char *message = _( "Checking business lots in account %s: %u of %u");

    if (!acc) return;

    if (gnc_get_abort_scrub())
        (percentagefunc)(NULL, -1.0);

    if (FALSE == xaccAccountIsAPARType (xaccAccountGetType (acc))) return;

    str = xaccAccountGetName(acc);
    str = str ? str : "(null)";

    ENTER ("(acc=%s)", str);
    PINFO ("Cleaning up superfluous lot links in account %s\n", str);
    xaccAccountBeginEdit(acc);

    lots = xaccAccountGetLotList(acc);
    lot_count = g_list_length (lots);
    for (node = lots; node; node = node->next)
    {
        GNCLot *lot = node->data;

        PINFO("Start processing lot %d of %d",
              curr_lot_no + 1, lot_count);

        if (curr_lot_no % 100 == 0)
        {
            char *progress_msg = g_strdup_printf (message, str, curr_lot_no, lot_count);
            (percentagefunc)(progress_msg, (100 * curr_lot_no) / lot_count);
            g_free (progress_msg);
        }

        if (lot)
            gncScrubBusinessLot (lot);

        PINFO("Finished processing lot %d of %d",
              curr_lot_no + 1, lot_count);
        curr_lot_no++;
    }
    g_list_free(lots);
    xaccAccountCommitEdit(acc);
    (percentagefunc)(NULL, -1.0);
    LEAVE ("(acc=%s)", str);
}

gncScrubBusinessLot()

gboolean gncScrubBusinessLot

(

GNCLot *

lot

)

The gncScrubBusinessLot() function makes sure that the indicated lot has all the correct properties required for a lot used in the business features.

Currently this function only does one thing: eliminate lot link transactions between invoice lots and payment lots (which were generated by GnuCash versions 2.6.0-2.6.3). Lot links between invoices and credit notes will still remain.

Scrubbing the lot may cause subsplits to be merged together, i.e. for splits to be deleted. This routine returns true if any splits were modified or deleted.

Definition at line 467 of file ScrubBusiness.c.

{
    gboolean splits_deleted = FALSE;
    gboolean dangling_payments = FALSE;
    gboolean dangling_lot_link = FALSE;
    Account *acc;
    gchar *lotname=NULL;

    if (!lot) return FALSE;
    lotname = g_strdup (gnc_lot_get_title (lot));
    ENTER ("(lot=%p) %s", lot, lotname ? lotname : "(no lotname)");

    acc = gnc_lot_get_account (lot);
    if (acc)
        xaccAccountBeginEdit(acc);

    /* Check invoice link consistency
     * A lot should have both or neither of:
     * - one split from an invoice transaction
     * - an invoice-guid set
     */
    gncScrubInvoiceState (lot);

    // Scrub lot links.
    // They should only remain when two document lots are linked together
    xaccScrubMergeLotSubSplits (lot, FALSE);
    splits_deleted = gncScrubLotLinks (lot);

    // Look for dangling payments and repair if found
    dangling_lot_link = gncScrubLotIsSingleLotLinkSplit (lot);
    if (dangling_lot_link)
    {
        dangling_payments = gncScrubLotDanglingPayments (lot);
        if (dangling_payments)
            splits_deleted |= gncScrubLotLinks (lot);
        else
        {
            Split *split = gnc_lot_get_earliest_split (lot);
            Transaction *trans = xaccSplitGetParent (split);
            xaccTransDestroy (trans);
        }
    }

    // If lot is empty now, delete it
    if (0 == gnc_lot_count_splits (lot))
    {
        PINFO("All splits were removed from lot, deleting");
        gnc_lot_destroy (lot);
    }

    if (acc)
        xaccAccountCommitEdit(acc);

    LEAVE ("(lot=%s, deleted=%d, dangling lot link=%d, dangling_payments=%d)",
            lotname ? lotname : "(no lotname)", splits_deleted, dangling_lot_link,
            dangling_payments);
    g_free (lotname);

    return splits_deleted;
}

gncScrubBusinessSplit()

gboolean gncScrubBusinessSplit

(

Split *

split

)

The gncScrubBusinessSplit() function will fix all issues found with the given split.

Current checks are:

• check if the split is part of a transaction that was generated as the result of a doubly posted invoice/bill/credit note. Refer to https://bugs.gnucash.org/show_bug.cgi?id=754209 to learn how this could have happened in the past. If such a transaction is found, its read-only status is removed and a warning is written to the trace file. Considering the user may already have added a correcting transaction we leave it up to the user to decide whether to also delete the transaction or not.
• remove empty splits, on condition they aren't part of an invoice transaction. In this case the function returns true so the caller knows a split was removed.

Definition at line 529 of file ScrubBusiness.c.

{
    Transaction *txn;
    gboolean deleted_split = FALSE;

    if (!split) return FALSE;
    ENTER ("(split=%p)", split);

    txn = xaccSplitGetParent (split);
    if (txn)
    {
        gchar txntype = xaccTransGetTxnType (txn);
        const gchar *read_only = xaccTransGetReadOnly (txn);
        gboolean is_void = xaccTransGetVoidStatus (txn);
        GNCLot *lot = xaccSplitGetLot (split);
        GncInvoice *invoice = gncInvoiceGetInvoiceFromTxn (txn);
        Transaction *posted_txn = gncInvoiceGetPostedTxn (invoice);

        /* Look for transactions as a result of double posting an invoice or bill
         * Refer to https://bugs.gnucash.org/show_bug.cgi?id=754209
         * to learn how this could have happened in the past.
         * Characteristics of such transaction are:
         * - read only
         * - not voided (to ensure read only is set by the business functions)
         * - transaction type is none (should be type invoice for proper post transactions)
         * - assigned to a lot
         */
        if ((txntype == TXN_TYPE_NONE) && read_only && !is_void && lot)
        {
            const gchar *memo = _("Please delete this transaction. Explanation at https://wiki.gnucash.org/wiki/Business_Features_Issues#Double_posting");
            gchar *txn_date = qof_print_date (xaccTransGetDateEntered (txn));
            xaccTransClearReadOnly (txn);
            xaccSplitSetMemo (split, memo);
            gnc_lot_remove_split (lot, split);
            PWARN("Cleared double post status of transaction \"%s\", dated %s. "
                  "Please delete transaction and verify balance.",
                  xaccTransGetDescription (txn),
                  txn_date);
            g_free (txn_date);
        }
        /* Next check for transactions which claim to be the posted transaction of
         * an invoice but the invoice disagrees. In that case
         */
        else if (invoice && (txn != posted_txn))
        {
            const gchar *memo = _("Please delete this transaction. Explanation at https://wiki.gnucash.org/wiki/Business_Features_Issues#I_can.27t_delete_a_transaction_of_type_.22I.22_from_the_AR.2FAP_account");
            gchar *txn_date = qof_print_date (xaccTransGetDateEntered (txn));
            xaccTransClearReadOnly (txn);
            xaccTransSetTxnType (txn, TXN_TYPE_NONE);
            xaccSplitSetMemo (split, memo);
            if (lot)
            {
                gnc_lot_remove_split (lot, split);
                gncInvoiceDetachFromLot (lot);
                gncOwnerAttachToLot (gncInvoiceGetOwner(invoice), lot);
            }
            PWARN("Cleared double post status of transaction \"%s\", dated %s. "
            "Please delete transaction and verify balance.",
            xaccTransGetDescription (txn),
                  txn_date);
            g_free (txn_date);
        }
        /* Next delete any empty splits that aren't part of an invoice transaction
         * Such splits may be the result of scrubbing the business lots, which can
         * merge splits together while reducing superfluous lot links
         */
        else if (gnc_numeric_zero_p (xaccSplitGetAmount(split)) && !gncInvoiceGetInvoiceFromTxn (txn) && !is_void)
        {
            GNCLot *lot = xaccSplitGetLot (split);
            time64 pdate = xaccTransGetDate (txn);
            gchar *pdatestr = gnc_ctime (&pdate);
            PINFO ("Destroying empty split %p from transaction %s (%s)", split, pdatestr, xaccTransGetDescription(txn));
            xaccSplitDestroy (split);
            g_free (pdatestr);

            // Also delete the lot containing this split if it was the last split in that lot
            if (lot && (gnc_lot_count_splits (lot) == 0))
                gnc_lot_destroy (lot);

            deleted_split = TRUE;
        }

    }

    LEAVE ("(split=%p)", split);
    return deleted_split;
}

xaccAccountAssignLots()

void xaccAccountAssignLots

(

Account *

acc

)

The xaccAccountAssignLots() routine will walk over all of the splits in an account, and make sure that each belongs to a lot.

Currently, the default (and only implemented) assignment policy is a FIFO policy: Any splits that are not in a lot will be used to close the oldest open lot(s). If there are no open lots, a new lot will be started. By trying to close the oldest lots, this effectively implements a FIFO accounting policy.

The xaccAccountAssignLots() routine will walk over all of the splits in an account, and make sure that each belongs to a lot.

If a split does not belong to any lots, poke it into one.

Definition at line 59 of file Scrub2.cpp.

{
    if (!acc) return;

    ENTER ("acc=%s", xaccAccountGetName(acc));
    xaccAccountBeginEdit (acc);

restart_loop:
    for (auto split : xaccAccountGetSplits (acc))
    {
        /* If already in lot, then no-op */
        if (split->lot) continue;

        /* Skip voided transactions */
        if (gnc_numeric_zero_p (split->amount) &&
                xaccTransGetVoidStatus(split->parent)) continue;

        if (xaccSplitAssign (split)) goto restart_loop;
    }
    xaccAccountCommitEdit (acc);
    LEAVE ("acc=%s", xaccAccountGetName(acc));
}

xaccAccountScrubCommodity()

void xaccAccountScrubCommodity

(

Account *

account

)

The xaccAccountScrubCommodity method fixed accounts without a commodity by using the old account currency and security.

Definition at line 1228 of file Scrub.cpp.

{
    gnc_commodity *commodity;

    if (!account) return;
    if (xaccAccountGetType(account) == ACCT_TYPE_ROOT) return;

    commodity = xaccAccountGetCommodity (account);
    if (commodity) return;

    /* Use the 'obsolete' routines to try to figure out what the
     * account commodity should have been. */
    commodity = xaccAccountGetCommodity (account);
    if (commodity)
    {
        xaccAccountSetCommodity (account, commodity);
        return;
    }

    commodity = DxaccAccountGetCurrency (account);
    if (commodity)
    {
        xaccAccountSetCommodity (account, commodity);
        return;
    }

    PERR ("Account \"%s\" does not have a commodity!",
          xaccAccountGetName(account));
}

xaccAccountScrubKvp()

void xaccAccountScrubKvp

(

Account *

account

)

Removes empty "notes", "placeholder", and "hbci" KVP slots from Accounts.

Definition at line 1374 of file Scrub.cpp.

{
    GValue v = G_VALUE_INIT;
    gchar *str2;

    if (!account) return;
    scrub_depth++;

    qof_instance_get_kvp (QOF_INSTANCE (account), &v, 1, "notes");
    if (G_VALUE_HOLDS_STRING (&v))
    {
        str2 = g_strstrip(g_value_dup_string(&v));
        if (strlen(str2) == 0)
            qof_instance_slot_delete (QOF_INSTANCE (account), "notes");
        g_free(str2);
    }

    qof_instance_get_kvp (QOF_INSTANCE (account), &v, 1, "placeholder");
    if ((G_VALUE_HOLDS_STRING (&v) &&
        strcmp(g_value_get_string (&v), "false") == 0) ||
        (G_VALUE_HOLDS_BOOLEAN (&v) && ! g_value_get_boolean (&v)))
        qof_instance_slot_delete (QOF_INSTANCE (account), "placeholder");

    g_value_unset (&v);
    qof_instance_slot_delete_if_empty (QOF_INSTANCE (account), "hbci");
    scrub_depth--;
}

xaccAccountScrubLots()

void xaccAccountScrubLots

(

Account *

acc

)

The xaccAccountScrubLots() routine makes sure that every split in the account is assigned to a lot, and that then, every lot is self-consistent (by calling xaccScrubLot() on each lot).

This routine is the primary routine for ensuring that the lot structure, and the cap-gains for an account are in good order.

Most GUI routines will want to use one of these xacc[*]ScrubLots() routines, instead of the various component routines, since it will usually makes sense to work only with these high-level routines.

Definition at line 159 of file Scrub3.cpp.

{
    LotList *lots, *node;
    if (!acc) return;
    if (FALSE == xaccAccountHasTrades (acc)) return;

    ENTER ("(acc=%s)", xaccAccountGetName(acc));
    xaccAccountBeginEdit(acc);
    xaccAccountAssignLots (acc);

    lots = xaccAccountGetLotList(acc);
    for (node = lots; node; node = node->next)
    {
        GNCLot *lot = GNC_LOT(node->data);
        xaccScrubLot (lot);
    }
    g_list_free(lots);
    xaccAccountCommitEdit(acc);
    LEAVE ("(acc=%s)", xaccAccountGetName(acc));
}

xaccAccountTreeScrubCommodities()

void xaccAccountTreeScrubCommodities

(

Account *

acc

)

The xaccAccountTreeScrubCommodities will scrub the currency/commodity of all accounts & transactions in the specified account or any child account.

Definition at line 1293 of file Scrub.cpp.

{
    if (!acc) return;
    scrub_depth++;
    xaccAccountTreeForEachTransaction (acc, scrub_trans_currency_helper, nullptr);

    scrub_account_commodity_helper (acc, nullptr);
    gnc_account_foreach_descendant (acc, scrub_account_commodity_helper, nullptr);
    scrub_depth--;
}

xaccAccountTreeScrubQuoteSources()

void xaccAccountTreeScrubQuoteSources	(	Account *	root,
		gnc_commodity_table *	table
	)

This routine will migrate the information about price quote sources from the account data structures to the commodity data structures.

It first checks to see if this is necessary since, for the time being, the quote information will still be written out as part of the account. Just in case anyone needs to fall back from CVS to a production version of code.

Parameters

root	A pointer to the root account containing all accounts in the current book.
table	A pointer to the commodity table for the current book.

Definition at line 1351 of file Scrub.cpp.

{
    gboolean new_style = FALSE;
    ENTER(" ");

    if (!root || !table)
    {
        LEAVE("Oops");
        return;
    }
    scrub_depth++;
    gnc_commodity_table_foreach_commodity (table, check_quote_source, &new_style);

    move_quote_source(root, GINT_TO_POINTER(new_style));
    gnc_account_foreach_descendant (root, move_quote_source,
                                    GINT_TO_POINTER(new_style));
    LEAVE("Migration done");
    scrub_depth--;
}

xaccLotFill()

void xaccLotFill

(

GNCLot *

lot

)

The xaccLotFill() routine attempts to assign splits to the indicated lot until the lot balance goes to zero, or until there are no suitable (i.e.

unassigned) splits left in the account. It uses the default accounting policy to choose the splits to fill out the lot.

Definition at line 92 of file Scrub2.cpp.

{
    Account *acc;
    Split *split;
    GNCPolicy *pcy;

    if (!lot) return;
    acc = gnc_lot_get_account(lot);
    pcy = gnc_account_get_policy(acc);

    ENTER ("(lot=%s, acc=%s)", gnc_lot_get_title(lot), xaccAccountGetName(acc));

    /* If balance already zero, we have nothing to do. */
    if (gnc_lot_is_closed (lot))
    {
    LEAVE ("Lot Closed (lot=%s, acc=%s)", gnc_lot_get_title(lot),
           xaccAccountGetName(acc));
    return;
    }
    split = pcy->PolicyGetSplit (pcy, lot);
    if (!split)
    {
    LEAVE ("No Split (lot=%s, acc=%s)", gnc_lot_get_title(lot),
           xaccAccountGetName(acc));
    return;   /* Handle the common case */
    }

    /* Reject voided transactions */
    if (gnc_numeric_zero_p(split->amount) &&
            xaccTransGetVoidStatus(split->parent))
    {
    LEAVE ("Voided transaction (lot=%s, acc=%s)",
           gnc_lot_get_title(lot), xaccAccountGetName(acc));
    return;
    }

    xaccAccountBeginEdit (acc);

    /* Loop until we've filled up the lot, (i.e. till the
     * balance goes to zero) or there are no splits left.  */
    while (1)
    {
        Split *subsplit;

        subsplit = xaccSplitAssignToLot (split, lot);
        if (subsplit == split)
        {
            PERR ("Accounting Policy gave us a split that "
                  "doesn't fit into this lot\n"
                  "lot baln=%s, isclosed=%d, aplit amt=%s",
                  gnc_num_dbg_to_string (gnc_lot_get_balance(lot)),
                  gnc_lot_is_closed (lot),
                  gnc_num_dbg_to_string (split->amount));
            break;
        }

        if (gnc_lot_is_closed (lot)) break;

        split = pcy->PolicyGetSplit (pcy, lot);
        if (!split) break;
    }
    xaccAccountCommitEdit (acc);
    LEAVE ("(lot=%s, acc=%s)", gnc_lot_get_title(lot), xaccAccountGetName(acc));
}

xaccLotScrubDoubleBalance()

void xaccLotScrubDoubleBalance

(

GNCLot *

lot

)

The xaccLotScrubDoubleBalance() routine examines the indicated lot.

If it is open, it does nothing. If it is closed, it then verifies that the lot is 'double balanced'. By 'double balance', we mean that both the sum of the split amounts is zero, and that the sum of the split values is zero. If the lot is closed and the sum of the values is not zero, the lot is considered to have a 'realized gain or loss' that hadn't been correctly handled. This routine then creates a balancing transaction to so as to record the realized gain/loss, adds it to the lot, and adds it to a gain/loss account. If there is no default gain/loss account, it creates one.

Definition at line 160 of file Scrub2.cpp.

{
    gnc_commodity *currency = nullptr;
    SplitList *snode;
    GList *node;
    gnc_numeric zero = gnc_numeric_zero();
    gnc_numeric value = zero;

    if (!lot) return;

    ENTER ("lot=%s", gnc_lot_get_title(lot));

    for (snode = gnc_lot_get_split_list(lot); snode; snode = snode->next)
    {
        Split *s = GNC_SPLIT(snode->data);
        xaccSplitComputeCapGains (s, nullptr);
    }

    /* We double-check only closed lots */
    if (FALSE == gnc_lot_is_closed (lot))
    {
    LEAVE ("lot=%s is closed", gnc_lot_get_title(lot));
    return;
    }

    for (snode = gnc_lot_get_split_list(lot); snode; snode = snode->next)
    {
        Split *s = GNC_SPLIT(snode->data);
        Transaction *trans = s->parent;

        /* Check to make sure all splits in the lot have a common currency */
        if (nullptr == currency)
        {
            currency = trans->common_currency;
        }
        if (FALSE == gnc_commodity_equiv (currency, trans->common_currency))
        {
            /* This lot has mixed currencies. Can't double-balance.
             * Silently punt */
            PWARN ("Lot with multiple currencies:\n"
                   "\ttrans=%s curr=%s", xaccTransGetDescription(trans),
                   gnc_commodity_get_fullname(trans->common_currency));
            break;
        }

        /* Now, total up the values */
        value = gnc_numeric_add (value, xaccSplitGetValue (s),
                                 GNC_DENOM_AUTO, GNC_HOW_DENOM_EXACT);
        PINFO ("Split=%p value=%s Accum Lot value=%s", s,
               gnc_num_dbg_to_string (s->value),
               gnc_num_dbg_to_string (value));

    }

    if (FALSE == gnc_numeric_equal (value, zero))
    {
        /* Unhandled error condition. Not sure what to do here,
         * Since the ComputeCapGains should have gotten it right.
         * I suppose there might be small rounding errors, a penny or two,
         * the ideal thing would to figure out why there's a rounding
         * error, and fix that.
         */
        PERR ("Closed lot fails to double-balance !! lot value=%s",
              gnc_num_dbg_to_string (value));
        for (node = gnc_lot_get_split_list(lot); node; node = node->next)
        {
            Split *s = GNC_SPLIT(node->data);
            PERR ("s=%p amt=%s val=%s", s,
                  gnc_num_dbg_to_string(s->amount),
                  gnc_num_dbg_to_string(s->value));
        }
    }

    LEAVE ("lot=%s", gnc_lot_get_title(lot));
}

xaccScrubLot()

gboolean xaccScrubLot

(

GNCLot *

lot

)

The xaccScrubLot() routine makes sure that the indicated lot is self-consistent and properly balanced, and fixes it if its not.

This is an important routine to call if the amount of any split in the lot is changed. That's because (obviously) changing split values is guaranteed to throw off lot balances. This routine may end up closing the lot, or at least trying to. It will also cause cap gains to be recomputed.

Scrubbing the lot may cause subsplits to be merged together, i.e. for splits to be deleted. This routine returns true if any splits were deleted.

Definition at line 85 of file Scrub3.cpp.

{
    gboolean splits_deleted = FALSE;
    gnc_numeric lot_baln;
    gboolean opening_baln_is_pos, lot_baln_is_pos;
    Account *acc;
    GNCPolicy *pcy;

    if (!lot) return FALSE;
    ENTER ("(lot=%p) %s", lot, gnc_lot_get_title(lot));

    acc = gnc_lot_get_account (lot);
    pcy = gnc_account_get_policy(acc);
    xaccAccountBeginEdit(acc);
    xaccScrubMergeLotSubSplits (lot, TRUE);

    /* If the lot balance is zero, we don't need to rebalance */
    lot_baln = gnc_lot_get_balance (lot);
    PINFO ("lot baln=%s for %s", gnc_num_dbg_to_string (lot_baln),
           gnc_lot_get_title(lot));
    if (! gnc_numeric_zero_p (lot_baln))
    {
        SplitList *node;
        gnc_numeric opening_baln;

        /* Get the opening balance for this lot */
        pcy->PolicyGetLotOpening (pcy, lot, &opening_baln, nullptr, nullptr);
        PINFO ("lot opener baln=%s", gnc_num_dbg_to_string (opening_baln));

        /* If the lot is fat, give the boot to all the non-opening
         * splits, and refill it */
        opening_baln_is_pos = gnc_numeric_positive_p(opening_baln);
        lot_baln_is_pos = gnc_numeric_positive_p(lot_baln);
        if ((opening_baln_is_pos || lot_baln_is_pos) &&
                ((!opening_baln_is_pos) || (!lot_baln_is_pos)))
        {
rethin:
            for (node = gnc_lot_get_split_list(lot); node; node = node->next)
            {
                Split *s = GNC_SPLIT(node->data);
                if (pcy->PolicyIsOpeningSplit (pcy, lot, s)) continue;
                gnc_lot_remove_split (lot, s);
                goto rethin;
            }
        }

        /* At this point the lot is thin, so try to fill it */
        xaccLotFill (lot);

        /* Make sure there are no subsplits. */
        splits_deleted = xaccScrubMergeLotSubSplits (lot, TRUE);
    }

    /* Now re-compute cap gains, and then double-check that.
     * But we only compute cap-gains if gains are possible;
     * that is if the lot commodity is not the same as the
     * currency. That is, one can't possibly have gains
     * selling dollars for dollars.  The business modules
     * use lots with lot commodity == lot currency.
     */
    if (gains_possible (lot))
    {
        xaccLotComputeCapGains (lot, nullptr);
        xaccLotScrubDoubleBalance (lot);
    }
    xaccAccountCommitEdit(acc);

    LEAVE ("(lot=%s, deleted=%d)", gnc_lot_get_title(lot), splits_deleted);
    return splits_deleted;
}

xaccScrubMergeSubSplits()

gboolean xaccScrubMergeSubSplits	(	Split *	split,
		gboolean	strict
	)

The xaccScrubMergeSubSplits() routine will merge together all of the splits that were at one time split off from this split, but are no longer needed to be kept separate.

Splits might be split up if they need to be divided over multiple lots; they can be merged back together if the lots change. In particular, two sub-splits may be merged if they are in the same lot, or in no lot. Note that, by definition, all subsplits belong to the same transaction.

There are two ways to find matching subsplits. The first way will consider splits to be subsplits only if they are explicitly marked as such while splitting the original split. Set strict to TRUE for this matching algorithm.

The second way is more relaxed. It will consider any two splits that happen to be part of the same lot and the same transaction to be subsplits. Set strict to FALSE for this matching algorithm.

The routine returns TRUE if a merger was performed, else it returns FALSE.

Definition at line 318 of file Scrub2.cpp.

{
    gboolean rc = FALSE;
    Transaction *txn;
    SplitList *node;
    GNCLot *lot;

    if (strict && (FALSE == is_subsplit (split))) return FALSE;

    txn = split->parent;

    // Don't mess with splits from an invoice transaction
    // Those are the responsibility of the business code
    if (gncInvoiceGetInvoiceFromTxn (txn)) return FALSE;

    lot = xaccSplitGetLot (split);

    ENTER ("(Lot=%s)", gnc_lot_get_title(lot));
restart:
    for (node = txn->splits; node; node = node->next)
    {
        Split *s = GNC_SPLIT(node->data);
        if (xaccSplitGetLot (s) != lot) continue;
        if (s == split) continue;
        if (qof_instance_get_destroying(s)) continue;

        // Don't mess with splits from an invoice transaction
        // Those are the responsibility of the business code
        if (gncInvoiceGetInvoiceFromTxn (s->parent)) return FALSE;

        if (strict)
        {
            /* OK, this split is in the same lot (and thus same account)
             * as the indicated split.  Make sure it is really a subsplit
             * of the split we started with.  It's possible to have two
             * splits in the same lot and transaction that are not subsplits
             * of each other, the test-period test suite does this, for
             * example.  Only worry about adjacent sub-splits.  By
             * repeatedly merging adjacent subsplits, we'll get the non-
             * adjacent ones too. */
            if (!xaccSplitIsPeerSplit (split, s))
                continue;
        }

        merge_splits (split, s);
        rc = TRUE;
        goto restart;
    }
    if (rc && gnc_numeric_zero_p (split->amount))
    {
        time64 pdate = xaccTransGetDate (txn);
        gchar *pdatestr = gnc_ctime (&pdate);
        PWARN ("Result of merge has zero amt!");
        PWARN ("Transaction details - posted date %s - description %s", pdatestr, xaccTransGetDescription(txn));
        g_free (pdatestr);
    }
    LEAVE (" splits merged=%d", rc);
    return rc;
}

xaccSplitScrub()

void xaccSplitScrub

(

Split *

split

)

The xaccSplitScrub method ensures that if this split has the same commodity and currency, then it will have the same amount and value.

If the commodity is the currency, the split->amount is set to the split value. In addition, if this split is an orphan, that is fixed first. If the split account doesn't have a commodity declared, an attempt is made to fix that first.

Definition at line 430 of file Scrub.cpp.

{
    split_scrub_or_dry_run (split, false);
}

xaccTransScrubCurrency()

void xaccTransScrubCurrency

(

Transaction *

trans

)

The xaccTransScrubCurrency method fixes transactions without a common_currency by looking for the most commonly used currency among all the splits in the transaction.

If this fails it falls back to using the old account currency and security fields of the parent accounts of the transaction's splits.

Definition at line 1111 of file Scrub.cpp.

{
    SplitList *node;
    gnc_commodity *currency;

    if (!trans) return;

    /* If there are any orphaned splits in a transaction, then the
     * this routine will fail.  Therefore, we want to make sure that
     * there are no orphans (splits without parent account).
     */
    xaccTransScrubOrphans (trans);

    currency = xaccTransGetCurrency (trans);
    if (currency && gnc_commodity_is_currency(currency)) return;

    currency = xaccTransFindCommonCurrency (trans, qof_instance_get_book(trans));
    if (currency)
    {
        xaccTransBeginEdit (trans);
        xaccTransSetCurrency (trans, currency);
        xaccTransCommitEdit (trans);
    }
    else
    {
        if (nullptr == trans->splits)
        {
            PWARN ("Transaction \"%s\" has no splits in it!", trans->description);
        }
        else
        {
            SplitList *node;
            char guid_str[GUID_ENCODING_LENGTH + 1];
            guid_to_string_buff(xaccTransGetGUID(trans), guid_str);
            PWARN ("no common transaction currency found for trans=\"%s\" (%s);",
                   trans->description, guid_str);

            for (node = trans->splits; node; node = node->next)
            {
                Split *split = GNC_SPLIT(node->data);
                if (nullptr == split->acc)
                {
                    PWARN (" split=\"%s\" is not in any account!", split->memo);
                }
                else
                {
                    gnc_commodity *currency = xaccAccountGetCommodity(split->acc);
                    PWARN ("setting to split=\"%s\" account=\"%s\" commodity=\"%s\"",
                           split->memo, xaccAccountGetName(split->acc),
                           gnc_commodity_get_mnemonic(currency));

                    xaccTransBeginEdit (trans);
                    xaccTransSetCurrency (trans, currency);
                    xaccTransCommitEdit (trans);
                    return;
                }
            }
        }
        return;
    }

    for (node = trans->splits; node; node = node->next)
    {
        Split *sp = GNC_SPLIT(node->data);

        if (!gnc_numeric_equal(xaccSplitGetAmount (sp),
                               xaccSplitGetValue (sp)))
        {
            gnc_commodity *acc_currency;

            acc_currency = sp->acc ? xaccAccountGetCommodity(sp->acc) : nullptr;
            if (acc_currency == currency)
            {
                /* This Split needs fixing: The transaction-currency equals
                 * the account-currency/commodity, but the amount/values are
                 * inequal i.e. they still correspond to the security
                 * (amount) and the currency (value). In the new model, the
                 * value is the amount in the account-commodity -- so it
                 * needs to be set to equal the amount (since the
                 * account-currency doesn't exist anymore).
                 *
                 * Note: Nevertheless we lose some information here. Namely,
                 * the information that the 'amount' in 'account-old-security'
                 * was worth 'value' in 'account-old-currency'. Maybe it would
                 * be better to store that information in the price database?
                 * But then, for old currency transactions there is still the
                 * 'other' transaction, which is going to keep that
                 * information. So I don't bother with that here. -- cstim,
                 * 2002/11/20. */

                PWARN ("Adjusted split with mismatched values, desc=\"%s\" memo=\"%s\""
                       " old amount %s %s, new amount %s",
                       trans->description, sp->memo,
                       gnc_num_dbg_to_string (xaccSplitGetAmount(sp)),
                       gnc_commodity_get_mnemonic (currency),
                       gnc_num_dbg_to_string (xaccSplitGetValue(sp)));
                xaccTransBeginEdit (trans);
                xaccSplitSetAmount (sp, xaccSplitGetValue(sp));
                xaccTransCommitEdit (trans);
            }
            /*else
              {
              PINFO ("Ok: Split '%s' Amount %s %s, value %s %s",
              xaccSplitGetMemo (sp),
              gnc_num_dbg_to_string (amount),
              gnc_commodity_get_mnemonic (currency),
              gnc_num_dbg_to_string (value),
              gnc_commodity_get_mnemonic (acc_currency));
              }*/
        }
    }

}

xaccTransScrubImbalance()

void xaccTransScrubImbalance	(	Transaction *	trans,
		Account *	root,
		Account *	account
	)

The xaccScrubImbalance() method searches for transactions that do not balance to zero.

If any such transactions are found, a split is created to offset this amount and is added to an "imbalance" account.

The xaccScrubImbalance() method searches for transactions that do not balance to zero.

Parameters

trans	The Transaction
root	The (hidden) root account, for the book default currency.
account	The account whose currency in which to balance.

Definition at line 835 of file Scrub.cpp.

{
    gnc_numeric imbalance;

    if (!trans) return;

    ENTER ("()");

    /* Must look for orphan splits even if there is no imbalance. */
    xaccTransScrubSplits (trans);

    /* Return immediately if things are balanced. */
    if (xaccTransIsBalanced (trans))
    {
        LEAVE ("transaction is balanced");
        return;
    }

    if (! xaccTransUseTradingAccounts (trans))
    {
        gnc_transaction_balance_no_trading (trans, root, account);
        LEAVE ("transaction balanced, no managed trading accounts");
        return;
    }

    xaccTransClearTradingSplits (trans);
    imbalance = xaccTransGetImbalanceValue (trans);
    if (! gnc_numeric_zero_p (imbalance))
    {
        PINFO ("Value unbalanced transaction");

        add_balance_split (trans, imbalance, root, account);
    }

    gnc_transaction_balance_trading (trans, root);
    if (gnc_numeric_zero_p(xaccTransGetImbalanceValue(trans)))
    {
        LEAVE ("()");
        return;
    }
    /* If the transaction is still not balanced, it's probably because there
       are splits with zero amount and non-zero value.  These are usually
       realized gain/loss splits.  Add a reversing split for each of them to
       balance the value. */

    gnc_transaction_balance_trading_more_splits (trans, root);
    if (!gnc_numeric_zero_p(xaccTransGetImbalanceValue(trans)))
        PERR("Balancing currencies unbalanced value");

}

xaccTransScrubPostedDate()

void xaccTransScrubPostedDate

(

Transaction *

trans

)

Changes Transaction date_posted timestamps from 00:00 local to 11:00 UTC.

11:00 UTC is the same day local time in almost all timezones, the exceptions being the -12, +13, and +14 timezones along the International Date Line. If Local time is set to one of these timezones then the new date_posted time will be adjusted as needed to ensure that the date doesn't change there. This change was made for v2.6.14 to partially resolve bug 137017.

Definition at line 1533 of file Scrub.cpp.

{
    time64 orig = xaccTransGetDate(trans);
    if(orig == INT64_MAX)
    {
    GDate date = xaccTransGetDatePostedGDate(trans);
    time64 time = gdate_to_time64(date);
    if(time != INT64_MAX)
    {
        // xaccTransSetDatePostedSecs handles committing the change.
        xaccTransSetDatePostedSecs(trans, time);
    }
    }
}