Event bars

Time bars are a choice, not a law. Because the Databento adapter constructs bars from raw trades, it can close a bar on any accumulation rule. Algolang implements fourteen kinds. Ten are de Prado’s,¹ from the Advances in Financial Machine Learning family (time, tick, volume, dollar, and the three imbalance and three runs kinds); the other four — range, renko, streak, and vwap_anchor — are our own additions in the same spirit:

Trade direction (the +1/-1 each trade contributes) is pinned: the data’s aggressor side when present, else the tick rule (sign of the price change, carrying the previous direction on a zero change).

All non-time kinds take --threshold in the kind’s own unit. The sample file holds 35,670 shares (about $9.3M notional) across 1,000 trades, so:

$ bin/algo run ... --strategy bin/printbars --inputs Print=false \
    --symbol MSFT --bar-kind volume --threshold 5000
algo: stdio-dbn-v1 cannot carry volume bars; preferring stdio-pb-v1
algo: simulation resolution falls back to bar: volume bars have no time-addressable trade windows
  request:    MSFT BARS kind=volume threshold=5000
  bars:       8 main after replicate (+ 0 warmup), 8 loop, 0 orders

$ bin/algo run ... --bar-kind dollar --threshold 1000000
  request:    MSFT BARS kind=dollar threshold=1e+06
  bars:       10 main after replicate (+ 0 warmup), 10 loop, 0 orders

Eight 5,000-share bars and ten million-dollar bars from the same 23 minutes of trading – activity bars sample the market by what it did, not by the clock.

Note the two stderr lines, which you will see on every activity-bar run: the raw-DBN data wire only has record types for time bars at native cadences, so the engine quietly prefers the protobuf wire (stdio-dbn-v1 cannot carry volume bars); and tick fill-simulation needs time-addressable bar windows, so the simulator drops to bar resolution. The same DBN demotion note appears for time bars at intervals DBN has no native record for.

Construction statistics

A bar built from the trade stream knows more than its OHLCV: how many trades made it, their volume-weighted price, and which side was the aggressor. Every event bar marketfeed serves, and every bar the Databento adapter builds from trades (time bars included), carries those figures on bar.Stats (GLE-386):

Availability is explicit. bar.Stats is nil on a bar that was not built from trades: a stored OHLCV time bar from marketfeed, a bar read from a CSV file, and every bar on the CSV strategy wire, which has no slot for the figures. Inside the block a zero is a real zero: SellVolume == 0 means no seller-initiated volume traded, not that the figure is unknown. The rule is the trade count: a wire bar whose tick_count is 0 has no block, whatever its other statistic fields hold. Check for nil before reading:

if st := bar.Stats; st != nil && st.AggressorVolume() > 0 {
    imbalance := (st.BuyVolume - st.SellVolume) / st.AggressorVolume()
    ...
}

Price frame. VWAP is a price, so on an adjusted bar (a split-adjusted equity, a back-adjusted continuous contract) it is adjusted with the bar’s OHLC; Notional stays in raw venue prices, so there VWAP is not Notional / Volume. marketfeed builds event bars for dated contracts only, so a continuous run carries no statistics today.

Kit helpers. strategies/kit turns the block into signals, each returning (value, ok) with ok false when the bar has no statistics or the denominator is not positive:

A NaN in a StatColumn column propagates through any rolling indicator fed from it; to sum a statistic over bars that may lack one, skip those bars the way FlowDelta does.

Example. strategies/demo/go/flowimbalance sums the order flow of the last Window bars, enters when the net aggressor share reaches Entry and the bar closed on the same side of its VWAP, and goes flat when the flow turns against its position. make run-flowimbalance runs it on the bundled sample:

$ make run-flowimbalance
[flowimbalance] INFO: flowimbalance: 2022-06-10T12:32:06Z MSFT ratio=-0.3540 disp=-0.0185 avgtrade=36.11 pos=0 target=-100
[flowimbalance] INFO: flowimbalance: fill flowimbalance_1654864326154532805 sell 100 MSFT @ 262.1000
[flowimbalance] INFO: flowimbalance: 2022-06-10T12:32:06Z MSFT ratio=-0.5325 disp=0.0262 avgtrade=148.30 pos=-100 target=-100
...
[flowimbalance] INFO: flowimbalance: 2022-06-10T12:34:05Z MSFT ratio=0.2190 disp=0.2162 avgtrade=33.93 pos=-100 target=100
[flowimbalance] INFO: flowimbalance: fill flowimbalance_1654864445805958673 buy 200 MSFT @ 261.5000

The first decision comes at the fifth bar (Window=5): sellers initiated 35% more of the known-aggressor volume than buyers and the bar closed below its VWAP, so the strategy goes short 100 shares; when the flow turns it reverses with one 200-share order. Two bars can share a close second on a busy tape (the bar’s close is its last trade’s timestamp).

An end-to-end test (engine/gle386_eventstats_e2e_test.go) checks every logged figure against the bars the adapter serves on the data wire.

Expectation mode

For the imbalance and runs families a static threshold is naive: the “right” imbalance depends on recent activity. --threshold-mode expectation replaces the fixed threshold with the AFML EWMA scheme – the expected bar size and the expected signed rate are estimated from the bars already closed, and each new bar closes when its accumulation beats the current expectation:

$ bin/algo run ... --strategy bin/printbars --inputs Print=false \
    --symbol MSFT --bar-kind imbalance_tick --threshold-mode expectation \
    --ewma-span-bars 20 --ewma-span-signal 20 --init-expected-size 50
  request:    MSFT BARS kind=imbalance_tick
  bars:       25 main after replicate (+ 0 warmup), 25 loop, 0 orders

The four parameters: --ewma-span-bars (span of the expected-bar-size EWMA), --ewma-span-signal (span of the signed-rate EWMA), --init-expected-size (the seed: expected trades per bar before any bar has closed), and --init-signed-rate (the signed-rate seed, default 0.5). The first two and the size seed are required in expectation mode; static mode is the default and requires --threshold instead.

Constructor lead-in

An event bar’s constructor carries state from one bar to the next. A volume bar fills from wherever the feed began, and in expectation mode the expected bar size and signed rate are EWMAs of the bars already closed. A run that starts at S therefore builds its first bars from a cold constructor, and they are not the bars a run started earlier serves after S. --lead-in D (config key lead-in, RunConfig.LeadIn; GLE-397) asks the adapter to feed the trades from S - D through the same constructor and to discard the bars that close before S. The bars served are then exactly the bars a request from S - D serves that close at or after S. The discarded bars never reach the strategy and are not its warm-up: a strategy’s Lookback warm-up is a separate matter.

The adapter reports what it did on the request’s final envelope (a data.LeadInReport): where the warm-up began, whether it was clamped, and how many warm-up bars it discarded. A clamped lead-in means the adapter held no data as far back as asked and began at its first trade instead. The engine notes the report on stderr, and records a clamp as a run degradation of kind lead_in_clamped (GLE-407). The note is printed with the run’s data degradations and written to degradation.json on the runs that write one. An event series’ pre-roll asks for the lead-in too, and a clamp of the pre-roll fetch whose bars it keeps is recorded under the series’ name followed by pre-roll. A lead-in is refused on time bars (they have no constructor state), when negative, and when the adapter does not advertise it for the bar kind. It needs the pb data wire, because the CSV get line has no field for it or its report: auto negotiation prefers pb for a lead-in run, and a forced CSV wire is refused. The bar cache keys on the lead-in, so a lead-in request never reuses the bars of a cold request or of another lead-in. A cache hit keeps the warmed bars but not the adapter’s report, and the note says so: --cache refresh fetches afresh and reports again.

Example. make run-leadin runs the construction statistics example, flowimbalance, on 1,000-share MSFT volume bars from 12:40 with a five-minute lead-in. The volume bars therefore continue a construction that began at 12:35. The strategy’s five-bar window is filled before the first decision by the event-bar pre-roll, so it decides from the first bar. The pre-roll reaches back to the sample’s first trade (12:30:01), where no lead-in can warm the constructor, and the run records that. LEADIN=15m reaches before the sample’s first trade for the run’s own bars too, so that request is clamped as well. (A repeat run is served from the bar cache, which keeps no report, so it notes the cache instead; --cache refresh reports again.)

$ make run-leadin
algo: series MSFT@1m: 5m0s lead-in: the constructor warmed from 2022-06-10T12:35:00Z (8 warm-up bars discarded)
[flowimbalance] INFO: flowimbalance: 2022-06-10T12:40:01Z MSFT ratio=-0.8298 disp=0.0610 avgtrade=22.75 pos=0 target=0
[flowimbalance] INFO: flowimbalance: 2022-06-10T12:41:00Z MSFT ratio=-0.7213 disp=0.3842 avgtrade=22.40 pos=0 target=0
[flowimbalance] INFO: flowimbalance: 2022-06-10T12:43:42Z MSFT ratio=-0.3580 disp=-0.4098 avgtrade=25.15 pos=0 target=-100
...
  bars:       11 main after replicate (+ 5 warmup), 11 loop, 3 orders
algo: data degradation: MSFT@1m pre-roll: the 5m0s lead-in was clamped: the adapter's data begins at 2022-06-10T12:30:01Z, at or after the start, so the constructor was not warmed

$ make run-leadin LEADIN=15m
algo: series MSFT@1m: the 15m0s lead-in was clamped: the adapter's data begins at 2022-06-10T12:30:01Z, so the constructor warmed for 9m59s, not 15m0s (24 warm-up bars discarded)
...
  bars:       10 main after replicate (+ 5 warmup), 10 loop, 2 orders
algo: data degradation: MSFT@1m: the 15m0s lead-in was clamped: the adapter's data begins at 2022-06-10T12:30:01Z, so the constructor warmed for 9m59s, not 15m0s
algo: data degradation: MSFT@1m pre-roll: the 15m0s lead-in was clamped: the adapter's data begins at 2022-06-10T12:30:01Z, at or after the start, so the constructor was not warmed

Without a lead-in the first volume bar starts filling at 12:40 and the bars cut at different trades. The clamped request warms the constructor from 12:30:01 and serves ten bars rather than eleven, because its bars cut at different trades again. An end-to-end test (engine/gle397_leadin_e2e_test.go) checks the defining property on the bundled adapter for volume bars and expectation-mode imbalance bars. For each kind, the bars served from 12:40 with a lead-in equal the bars an earlier request serves from 12:40, and the report gives the warm-up’s start and its discarded count, both with and without a clamp. engine/gle407_e2e_test.go checks that a clamped run, and only a clamped one, records the lead_in_clamped degradation for its own bars, and that the pre-roll’s clamp is recorded once.

In an adapter. An adapter advertises a lead-in per bar kind with SupportedBarKind.ServesLeadIn. The data SDK refuses any lead-in the adapter did not advertise before OnGetBars runs, and hands the admitted duration over as BarRequest.LeadIn. The adapter feeds its constructor from req.Start - req.LeadIn, or from its first record when that is later. It drops the bars that close before req.Start, then calls ctx.SetLeadInReport(data.LeadInReport{Start, Clamped, WarmupBarsDiscarded}) before returning, and the SDK puts the report on that request’s final envelope. The bundled Databento adapter serves a lead-in for every event-bar kind, and so does marketfeed (GLE-402), which clamps the warm-up to the covered tape segment containing the start. The current archive holds no trade tapes, so marketfeed’s lead-in is exercised on fixture tapes.