Transaction Filters

Transaction stream can be filtered to select which transactions are part of reports and calculations.

Transaction stream can be filtered based on several attributes of single transaction. If transaction is not included into stream and it is filtered out, then all data regarding that transaction will not be used with any calculation. It is also possible to filter transactions based on sharding scheme.

Transaction filters can be combined with logical AND, OR and NOT operations and it is possible to create filter trees by combining logical filters and filters for single transactions.

Usage of transaction filters:

  • See Usage Guide for using transaction filters with tackler-cli.jar CLI application.

  • See Server API document for using transaction filters with embedded tackler-core.

  • See Client API document for data model of transaction filters for JS/JVM tackler-api.

  • For actual definition of transaction filters, keep reading this document.

Logical transaction filters

There are following logical transaction filters:

  • AND: all filters must select txn, so that it will be selected

  • OR: if any of filters select txn, then it’s selected

  • NOT: reverses result (selected will be unselected, unselected will be selected)

Transaction property filters

Transaction filters select transactions based on single transaction’s properties and attributes.

  • Transaction header

    • Time (Begin, End)

    • Code

    • Description

    • UUID

    • Geo Location

    • Txn Comments

  • Postings

    • Account

    • Amount (account and amount: less, equal, greater)

    • Commodity

    • Posting Comment

Transaction filter definitions and JSON Serialization

Transaction filters can be defined in and decoded from JSON.

Regualar expression syntax used by Transaction filters uses regular expression syntax specified by Java.

Below is filter format definition syntax in JSON pseudo-format.

For actual working JSON filter examples, see test suite of Transaction Filters: core → tests → filter → TxnFilterJsonTest

Top Level component

Each transaction filter definition contains a root element which is defined as:

{ "txnFilter" : <TxnFilter> }

where TxnFilter is definition of top-level transaction filter. That top-level transaction filter can be logical filter or transaction property filter.

Logical Filters

Logical filters combine one or multiple transaction filters by logical operation.

AND

Logical AND filter contains two or more filters, and selects transactions which are selected by all contained filters.

 { "TxnFilterAND" : { "txnFilters" : [ <TxnFilter>, <TxnFilter> ... ] } }
OR

Logical OR filter contains two or more filters, and selects transactions which are selected by any of contained filters.

{ "TxnFilterOR" : { "txnFilters" : [ <TxnFilter>, <TxnFilter> ... ] } }
NOT

Logical NOT filter contains a single filter and it reverses the result of contained filter.

E.g. it selects all transactions which are not selected by contained filter. Similarly it unselects all transactions which are selected by contained filter.

{ "TxnFilterNOT" : { "txnFilter" : <TxnFilter> } }

Transaction property filters

Transaction property filters select transactions based on properties of single transaction.

Txn Timestamp: Begin

Timestamp Begin filter selects transactions which timestamp is on specified time or after it.

Time is expressed in ISO 8601 format with zone, e.g. 2018-01-01T10:11:22.345+02:00 or 2018-01-01T08:11:22.345Z.

{
  "TxnFilterTxnTSBegin" : {
    "begin" : "ISO 8601 timestamp"
  }
}
Txn Timestamp: End

Timestamp End filter selects transactions which timestamp is before specified time.

Time is expressed in ISO 8601 format with zone, e.g. 2018-01-01T10:11:22.345+02:00 or 2018-01-01T08:11:22.345Z.

{
  "TxnFilterTxnTSEnd" : {
    "end" : "ISO 8601 timestamp"
  }
}
Txn Code

Txn Code filter selects transactions which code matches specified regular expression.

{
  "TxnFilterTxnCode" : {
    "regex" : "<regex>"
  }
}
Txn Description

Txn Description filter selects transactions which description matches specified regular expression.

{
  "TxnFilterTxnDescription" : {
    "regex" : "<regex>"
  }
}
Txn UUID

Txn UUID filter selects transactions which UUID is same as specified.

{
  "TxnFilterTxnUUID" : {
    "uuid" : "<UUID>"
  }
},
Geo Location

Transaction Geo Filters selects transactions which geographic location is inside Bounding Box defined by the filter. See Transaction Geo Filters documentation for how these filters selects transactions.

2D Bounding Box (Latitude, Longitude)
# BBoxLatLon will ignore altitude,
# e.g. it will select 3D transaction if it fits 2D BBox.
{
  "TxnFilterBBoxLatLon" : {
    "south" : <number: min latitude>,
    "west" :  <number: min longitude>,
    "north" : <number: max latitude>,
    "east" :  <number: max longitude>
  }
}
3D Bounding Box (Latitude, Longitude, Altitude)
# BBoxLatLonAlt will select only 3D transactions with altitude,
# e.g. it will not select any 2D txn.
{
  "TxnFilterBBoxLatLonAlt" : {
    "south" :  <number: min latitude>,
    "west" :   <number: min longitude>,
    "depth" :  <number: min altitude>,
    "north" :  <number: max latitude>,
    "east" :   <number: max longitude>,
    "height" : <number: max altitude>
  }
}
Txn Comments

Txn Description filter selects transactions which have a comment which matches specified regular expression.

{
  "TxnFilterTxnComments" : {
    "regex" : "<regex>"
  }
}

Transaction Posting filters

Posting Account

Posting Account filter selects transactions which have an account which matches specified regular expression.

{
  "TxnFilterPostingAccount" : {
    "regex" : "<regex>"
  }
}
Posting Amount (equal)

Posting Amount (egual) selects transactions which have a posting for specified account (regex) with exactly same amount as specified amount.

 Q: Why there is also account regex as parameter?
 A: For consistency with less and greater, where it's mandatory.

{
  "TxnFilterPostingAmountEqual" : {
    "regex" : "<regex>",
    "amount" : <BigDecimal>
  }
}
Posting Amount (less)

Posting Amount (less) selects transactions which have a posting for specified account (regex) with amount that is less than specified amount.

 Q: Why there is also account regex as parameter?
 A: Sum of all postings inside transaction must be zero.
    If you select "less than some positive amount",
    then all transactions will match, because there must
    be postings with negative amounts in every transaction
    to zero out whole transaction.

{
  "TxnFilterPostingAmountLess" : {
    "regex" : "<regex>",
    "amount" : <BigDecimal>
  }
}
Posting Amount (greater)

Posting Amount (greater) selects transactions which have a posting for specified account (regex) with amount that is greater than specified amount.

 Q: Why there is also account regex as parameter?
 A: Sum of all postings inside transaction must be zero.
    If you select "more than some negative amount",
    then all transactions will match, because there must
    be postings with positive amounts in every transaction
    to zero out whole transaction.

{
  "TxnFilterPostingAmountGreater" : {
    "regex" : "<regex>",
    "amount" : <BigDecimal>
  }
}
Posting Commodity

Posting Commodity selects transactions which have a posting with commodity which matches specified regular expression.

{
  "TxnFilterPostingCommodity" : {
    "regex" : "<regex>"
  }
},
Posting Comment

Posting Commodity selects transactions which have a posting with comment which matches specified regular expression.

{
  "TxnFilterPostingComment" : {
    "regex" : "<regex>"
  }
}

There are also several examples of complex Transaction filters in test suite: core → tests → filter → TxnFilterJsonTest