GncTxImport Class Reference

The actual TxImport class It's intended to use in the following sequence of actions: More...

#include <gnc-import-tx.hpp>

Public Member Functions
	GncTxImport (GncImpFileFormat format=GncImpFileFormat::UNKNOWN)
	Constructor for GncTxImport.
	~GncTxImport ()
	Destructor for GncTxImport.
void	file_format (GncImpFileFormat format)
	Sets the file format for the file to import, which may cause the file to be reloaded as well if the previously set file format was different and a filename was already set. More...
GncImpFileFormat	file_format ()
void	multi_split (bool multi_split)
	Toggles the multi-split state of the importer and will subsequently sanitize the column_types list. More...
bool	multi_split ()
void	base_account (Account *base_account)
	Sets a base account. More...
Account *	base_account ()
void	currency_format (int currency_format)
int	currency_format ()
void	date_format (int date_format)
int	date_format ()
void	encoding (const std::string &encoding)
	Converts raw file data using a new encoding. More...
std::string	encoding ()
void	update_skipped_lines (std::optional< uint32_t > start, std::optional< uint32_t > end, std::optional< bool > alt, std::optional< bool > errors)
uint32_t	skip_start_lines ()
uint32_t	skip_end_lines ()
bool	skip_alt_lines ()
bool	skip_err_lines ()
void	separators (std::string separators)
std::string	separators ()
void	settings (const CsvTransImpSettings &settings)
bool	save_settings ()
void	settings_name (std::string name)
std::string	settings_name ()
void	load_file (const std::string &filename)
	Loads a file into a GncTxImport. More...
void	tokenize (bool guessColTypes)
	Splits a file into cells. More...
std::string	verify (bool with_acct_errors)
void	create_transactions ()
	This function will attempt to convert all tokenized lines into transactions using the column types the user has set. More...
bool	check_for_column_type (GncTransPropType type)
void	set_column_type (uint32_t position, GncTransPropType type, bool force=false)
std::vector< GncTransPropType >	column_types ()
std::set< std::string >	accounts ()

Data Fields
std::unique_ptr< GncTokenizer >	m_tokenizer
	Will handle file loading/encoding conversion/splitting into fields.
std::vector< parse_line_t >	m_parsed_lines
	source file parsed into a two-dimensional array of strings. More...
std::multimap< time64, std::shared_ptr< DraftTransaction > >	m_transactions
	map of transaction objects created from parsed_lines and column_types, ordered by date

Detailed Description

The actual TxImport class It's intended to use in the following sequence of actions:

• set a file format
• load a file
• optionally convert it's encoding
• parse the file into lines, which in turn are split up in columns the result of this step can be queried from tokenizer
• the user should now map the columns to types, which is stored in column_types
• last step is convert the mapped columns into a list of transactions
• this list will then be passed on the generic importer for further processing

Definition at line 104 of file gnc-import-tx.hpp.

Member Function Documentation

base_account()

void GncTxImport::base_account

(

Account *

base_account

)

Sets a base account.

This is the account all import data relates to. As such at least one split of each transaction that will be generated will be in this account. When a base account is set, there can't be an account column selected in the import data. In multi-split mode the user has to select an account column so in that mode the base_account can't be set.

Parameters

base_account

Pointer to an account or NULL.

Definition at line 177 of file gnc-import-tx.cpp.

{
    if (m_settings.m_multi_split)
    {
        m_settings.m_base_account = nullptr;
        return;
    }

    auto base_account_is_new = m_settings.m_base_account == nullptr;
    m_settings.m_base_account = base_account;

    if (m_settings.m_base_account)
    {
        auto col_type_it = std::find (m_settings.m_column_types.begin(),
                m_settings.m_column_types.end(), GncTransPropType::ACCOUNT);
        if (col_type_it != m_settings.m_column_types.end())
            set_column_type(col_type_it - m_settings.m_column_types.begin(),
                            GncTransPropType::NONE);

        if (base_account_is_new)
        {
            /* Set default account for each line's split properties */
            for (auto line : m_parsed_lines)
                std::get<PL_PRESPLIT>(line)->set_account (m_settings.m_base_account);
        }
        else
        {
            /* Reparse all of the lines with the new base account's commodity. */
            tokenize(false);
        }

    }
}

create_transactions()

void GncTxImport::create_transactions

(

)

This function will attempt to convert all tokenized lines into transactions using the column types the user has set.

Creates a list of transactions from parsed data.

The parsed data will first be validated. If any errors are found in lines that are marked for processing (ie not marked to skip) this function will throw an error.

Parameters

skip_errors

true skip over lines with errors

Exceptions

throws

std::invalid_argument if data validation or processing fails.

Definition at line 724 of file gnc-import-tx.cpp.

{
    /* Start with verifying the current data. */
    auto verify_result = verify (true);
    if (!verify_result.empty())
        throw std::invalid_argument (verify_result);

    /* Drop all existing draft transactions */
    m_transactions.clear();

    m_parent = nullptr;

    /* Iterate over all parsed lines */
    for (auto parsed_lines_it = m_parsed_lines.begin();
            parsed_lines_it != m_parsed_lines.end();
            ++parsed_lines_it)
    {
        /* Skip current line if the user specified so */
        if ((std::get<PL_SKIP>(*parsed_lines_it)))
            continue;

        /* Should not throw anymore, otherwise verify needs revision */
        create_transaction (parsed_lines_it);
    }
}

encoding()

void GncTxImport::encoding

(

const std::string &

encoding

)

Converts raw file data using a new encoding.

This function must be called after load_file only if load_file guessed the wrong encoding.

Parameters

encoding

Encoding that data should be translated using

Definition at line 256 of file gnc-import-tx.cpp.

{

    // TODO investigate if we can catch conversion errors and report them
    if (m_tokenizer)
    {
        m_tokenizer->encoding(encoding); // May throw
        try
        {
            tokenize(false);
        }
        catch (...)
        { };
    }

    m_settings.m_encoding = encoding;
}

file_format()

void GncTxImport::file_format

(

GncImpFileFormat

format

)

Sets the file format for the file to import, which may cause the file to be reloaded as well if the previously set file format was different and a filename was already set.

Parameters

format

the new format to set

Exceptions

std::ifstream::failure

if file reloading fails

Definition at line 87 of file gnc-import-tx.cpp.

{
    if (m_tokenizer && m_settings.m_file_format == format)
        return;

    auto new_encoding = std::string("UTF-8");
    auto new_imp_file = std::string();

    // Recover common settings from old tokenizer
    if (m_tokenizer)
    {
        new_encoding = m_tokenizer->encoding();
        new_imp_file = m_tokenizer->current_file();
        if (file_format() == GncImpFileFormat::FIXED_WIDTH)
        {
            auto fwtok = dynamic_cast<GncFwTokenizer*>(m_tokenizer.get());
            if (!fwtok->get_columns().empty())
                m_settings.m_column_widths = fwtok->get_columns();
        }
    }

    m_settings.m_file_format = format;
    m_tokenizer = gnc_tokenizer_factory(m_settings.m_file_format);

    // Set up new tokenizer with common settings
    // recovered from old tokenizer
    m_tokenizer->encoding(new_encoding);
    load_file(new_imp_file);

    // Restore potentially previously set separators or column_widths
    if ((file_format() == GncImpFileFormat::CSV)
        && !m_settings.m_separators.empty())
        separators (m_settings.m_separators);
    else if ((file_format() == GncImpFileFormat::FIXED_WIDTH)
        && !m_settings.m_column_widths.empty())
    {
        auto fwtok = dynamic_cast<GncFwTokenizer*>(m_tokenizer.get());
        fwtok->columns (m_settings.m_column_widths);
    }

}

load_file()

void GncTxImport::load_file

(

const std::string &

filename

)

Loads a file into a GncTxImport.

This is the first function that must be called after creating a new GncTxImport. As long as this function didn't run successfully, the importer can't proceed.

Parameters

filename

Name of the file that should be opened

Exceptions

may	throw std::ifstream::failure on any io error

Definition at line 377 of file gnc-import-tx.cpp.

{

    /* Get the raw data first and handle an error if one occurs. */
    try
    {
        m_tokenizer->load_file (filename);
        return;
    }
    catch (std::ifstream::failure& ios_err)
    {
        // Just log the error and pass it on the call stack for proper handling
        PWARN ("Error: %s", ios_err.what());
        throw;
    }
}

multi_split()

void GncTxImport::multi_split

(

bool

multi_split

)

Toggles the multi-split state of the importer and will subsequently sanitize the column_types list.

All types that don't make sense in the new state are reset to type GncTransPropType::NONE. Additionally the interpretation of the columns with transaction properties changes when changing multi-split mode. So this function will force a reparsing of the transaction properties (if there are any) by resetting the first column with a transaction property it encounters.

Parameters

multi_split

Boolean value with desired state (multi-split vs two-split).

Definition at line 145 of file gnc-import-tx.cpp.

{
    auto trans_prop_seen = false;
    m_settings.m_multi_split = multi_split;
    for (uint32_t i = 0; i < m_settings.m_column_types.size(); i++)
    {
        auto old_prop = m_settings.m_column_types[i];
        auto is_trans_prop = ((old_prop > GncTransPropType::NONE)
                && (old_prop <= GncTransPropType::TRANS_PROPS));
        auto san_prop = sanitize_trans_prop (old_prop, m_settings.m_multi_split);
        if (san_prop != old_prop)
            set_column_type (i, san_prop);
        else if (is_trans_prop && !trans_prop_seen)
            set_column_type (i, old_prop, true);
        trans_prop_seen |= is_trans_prop;

    }
    if (m_settings.m_multi_split)
        m_settings.m_base_account = nullptr;
}

tokenize()

void GncTxImport::tokenize

(

bool

guessColTypes

)

Splits a file into cells.

This requires having an encoding that works (see GncTxImport::convert_encoding). Tokenizing related options should be set to the user's selections before calling this function. Notes: - this function must be called with guessColTypes set to true once before calling it with guessColTypes set to false.

• if guessColTypes is true, all the column types will be set GncTransPropType::NONE right now as real guessing isn't implemented yet

  Parameters

  guessColTypes

  true to guess what the types of columns are based on the cell contents

  Exceptions

  std::range_error

  if tokenizing failed

Definition at line 405 of file gnc-import-tx.cpp.

{
    if (!m_tokenizer)
        return;

    uint32_t max_cols = 0;
    m_tokenizer->tokenize();
    m_parsed_lines.clear();
    for (auto tokenized_line : m_tokenizer->get_tokens())
    {
        auto length = tokenized_line.size();
        if (length > 0)
        {
            auto pretrans = std::make_shared<GncPreTrans>(date_format(), m_settings.m_multi_split);
            auto presplit = std::make_shared<GncPreSplit>(date_format(), currency_format());
            presplit->set_pre_trans (std::move (pretrans));
            m_parsed_lines.push_back (std::make_tuple (tokenized_line, ErrMap(),
                                      presplit->get_pre_trans(), std::move (presplit), false));
        }
        if (length > max_cols)
            max_cols = length;
    }

    /* If it failed, generate an error. */
    if (m_parsed_lines.size() == 0)
    {
        throw (std::range_error (N_("There was an error parsing the file.")));
        return;
    }

    m_settings.m_column_types.resize(max_cols, GncTransPropType::NONE);

    /* Force reinterpretation of already set columns and/or base_account */
    for (uint32_t i = 0; i < m_settings.m_column_types.size(); i++)
        set_column_type (i, m_settings.m_column_types[i], true);
    if (m_settings.m_base_account)
    {
        for (auto line : m_parsed_lines)
            std::get<PL_PRESPLIT>(line)->set_account (m_settings.m_base_account);
    }

    if (guessColTypes)
    {
        /* Guess column_types based
         * on the contents of each column. */
        /* TODO Make it actually guess. */
    }
}

Field Documentation

m_parsed_lines

std::vector<parse_line_t> GncTxImport::m_parsed_lines

source file parsed into a two-dimensional array of strings.

Per line also holds possible error messages and objects with extracted transaction and split properties.

Definition at line 163 of file gnc-import-tx.hpp.

---

The documentation for this class was generated from the following files:

• gnc-import-tx.hpp
• gnc-import-tx.cpp