Intraday grids

Implemented (GLE-207). By default an intraday bar sits on one fixed grid: multiples of the bar size counted from 1 January 1970, UTC. A 7m bar drifts against the clock from day to day, and a 4h bar can mix two trading days. An intraday grid restarts the bars at every trading window instead, and cuts the last bar of each window short at its close.

A series asks for a grid by pairing a boundary session with an intraday time-bar interval (whole seconds, 1s up to but excluding 24h):

bin/algo run --adapter "$MF_ADAPTER" --adapter-inputs Dir=/usr/local/marketfeed \
  --strategy bin/printbars --symbols fut:XCME:ES:M24 \
  --bar-interval 7m --session rth --data-protocol pb \
  --date-from 2024-05-22T00:00:00Z --date-to 2024-05-23T00:00:00Z
BoundaryWindows
utc-dayeach UTC civil day
etheach extended-hours leg of each trade date (ES: 17:00 CT the prior evening to 16:00 CT)
rtheach regular session (ES: 08:30 to 16:00 CT)
sessionper trade date: ETH where the date’s session era declares one, else RTH

The run above serves 65 bars for 22 May 2024: 13:30–13:37Z, …, 20:51–20:58Z, then 20:58–21:00:01Z. An ETH or RTH window is closed at its close, so the closing second belongs to the last bar, which ends one second after the close. Bars carry nominal instants: ts_open is the grid position, and ts_close is the bar’s nominal end, not the last trade. The series label reads fut:XCME:ES:M24@7m-rth (SYMBOL@INTERVAL-BOUNDARY). eth, rth and session at 1d or 1w remain the trading-day containers. utc-day at 1d or 1w asks for UTC days rather than the exchange’s local days (GLE-1342), from an adapter that lists utc-day among its sessions (adapter does not serve session "utc-day" otherwise; marketfeed does not list it yet, GLE-1347); any other width of a day or more is refused (session "utc-day" serves intraday intervals and whole days, not 36h0m0s), as is --include-path on it.

The capability probe and the revision. A grid’s windows come from the adapter’s calendars, so a run cannot be planned against one calendar and served from another. Before fetching any bar, and before the series cache is consulted, the engine runs the adapter once per adapter command line, symbol and boundary (--verify runs the same probe):

marketfeed-adapter --intraday-grid-capability fut:XCME:ES:M24 rth Dir=/usr/local/marketfeed
{"version":1,"boundary":"rth","revision":"c5071ba4…2035"}

The engine sends the adapter inputs as one JSON object on the probe’s stdin, as DataInit carries them, never on the command line where other local users could read them. NAME=VALUE arguments, as above, are for manual use and override the same names from stdin. The answer must be exactly one JSON line with version 1, the requested boundary, and a 64-character lowercase hex revision. The probe has a 5-second timeout; the child is always reaped, and its stdout is bounded at 64 KiB. Any other outcome refuses the run: - a non-zero exit (an adapter without grids exits 2; one built on an older SDK ignores the flag and exits 1); - malformed output; - a timeout.

The refusal reads engine: series fut:XCME:ES:M24@7m-rth: adapter "…" does not advertise intraday-grid-v1 for boundary "rth"; upgrade the adapter, quoting the adapter’s stderr. Every grid request then carries the revision in GetDataRequest.grid_revision. The SDK refuses a grid request whose revision is missing (intraday grid revision is missing; query the adapter capability first) or no longer current (intraday grid revision changed (requested …, current …); restart the request). The revision joins the series cache key.

Adapter side (Go SDK). An adapter serves grids by implementing data.IntradayGridProvider:

IntradayGridCapability(sym, boundary string) (data.IntradayGridCapability, error)

It returns Version: 1, the boundary and its revision. data.Run answers the probe from it: inputs are applied and OnInit runs, with no handshake. The SDK admits a grid request only when the adapter implements the interface, and it checks the request against the adapter’s current revision. A grid request does not consult SessionProvider: utc-day need not be listed. Grid bars travel on stdio-pb-v1, or on stdio-csv-v1 without a path. DBN refuses them (intraday grids cannot travel on stdio-dbn-v1; …), because its OHLCV record rebuilds ts_close as ts_open + interval. Automatic negotiation demotes DBN for any session request.

Paths. --include-path works on the grids of a dated contract, an equity and a continuous symbol, which walks the path of the contract it holds (GLE-1332, Path resolution). With --path-interval d, every piece must sit on the bar’s own d grid from its open: - a piece starts at ts_open + j·d and ends at min(start + d, ts_close), so the last piece is cut with the bar; - pieces ascend, and empty pieces are omitted; - the pieces must reconstruct the bar’s OHLCV exactly.

The engine checks this in place of the epoch rule that every piece spans exactly d.

Warm-up. Grid bars are not evenly spaced in time: they restart each session, and a closed market has none. So a grid warm-up counts bars instead of assuming a span: 1. It fetches the max(lookback·T, 14 days) before the first bar. 2. While it has fewer distinct bars than the lookback, it widens the span by max(2, ceil(1.5·lookback / have)), down to the series’ coverage start, for at most six fetches. 3. It keeps the newest lookback bars, and warns on a shortfall as the epoch warm-up does.

Restrictions. Grid series refuse: - --replicate (intraday grid replication is not supported); - non-time bar kinds; - the TRADES schema.

Continuous symbols on a grid. A bare root or a :cont composite in the run grammar reads the feed’s grid composite on a grid (GLE-443): --symbol fut:XNYM:CL --bar-interval 4h --session eth. - The probe and the revision. The engine probes the grid on the composite symbol, and every composite read carries the revision, as a plain grid series’ reads do. The revision is part of the composite cache key. - The warm-up counts bars. It follows the plain grid law above, through reads pinned to the run’s snapshot (as a continuous series’ warm-up is, GLE-393) and bounded by the served schedule’s first span. Each read must reproduce the run’s bars. - The path. The grid composite is read with the run’s path request (GLE-1332): --include-path and --path-interval put each held contract’s pieces under its bars, at its own prices. A utc-day grid’s merged parents are always read with their path for the engine’s own use (below); on a path run the remainder it executes keeps its pieces from the cut. - eth, rth and session grids never straddle a roll. The feed’s v2 cut is a trade-date boundary: it opens an eth or session window and lies outside every rth one. The incoming contract owns the cut instant (marketfeed GLE-442). - A utc-day grid does straddle. The cut (17:00 Chicago, 22:00 or 23:00 UTC) falls inside the UTC day, and the feed serves the parent across it merged: a chart bar folded from both contracts, under the composite’s own symbol, with no raw prices. The engine executes it in two parts (GLE-444): - The strategy sees the chart bar as served, in the run’s adjusted frame. Its raw prices (RawPrice) are the incoming contract’s from the cut. - The simulator rolls at the cut at the served seam pair, as on any roll (The instant law), then executes the bar’s remainder in the incoming contract from the cut onward. The remainder is folded from the bar’s path: the engine reads each merged parent once more, on its own, with its native path and pinned to the run’s snapshot, and folds the incoming contract’s constituents from the cut. The outgoing contract’s part before the cut is not executed. An order resting into the bar fills at the incoming contract’s first print from the cut. - The engine notes each such bar, and degradation.json records it: the 4h0m0s utc-day grid bar … straddles the roll at …: the strategy sees the merged chart bar, the simulator executes it from the cut in H26 (GLE-444). - A warm-up or reach-back read treats a merged parent the same way, so a utc-day warm-up runs through the rolls before the start, and a splice met only there is noted too (once). - A parent the feed serves tagged (the outgoing contract did not trade before the cut) is the incoming contract’s alone and executes as served.

make run-cl-grid runs smacross on the continuous crude-oil series’ 4h eth grid across the G26 to H26 roll of 14 January 2026. The warm-up holds the strategy’s nine bars, and the position held at the cut rolls with the series (excerpt):

[smacross] INFO: smacross: fast SMA(3) crossed above slow SMA(8); buying 1 fut:XNYM:CL
[smacross] INFO: smacross: fill smacross_1767625200000000000: bought 1 fut:XNYM:CL @ 57.85
...
Algolang run run-fut-xnym-cl-4h
  instance:   1 (fut:XNYM:CL@4h-eth), 17 orders
  request:    fut:XNYM:CL@4h-eth BARS
  fills:      bar path (none)
  bars:       120 main after replicate (+ 9 warmup), 120 loop, 17 orders
              18 fills, commission 0.00, realized PnL 2350.00
              open CLH26 -1 @ 64.0400 (last 65.7400, unrealized -1700.00)
              equity 100650.00 (capital 100000.00, return +0.6500%)

make run-cl-grid-utc runs the same strategy over the same window on the utc-day grid, where the roll falls inside the 20:00Z parent (excerpt):

Algolang run run-fut-xnym-cl-4h
  instance:   1 (fut:XNYM:CL@4h-utc-day), 15 orders
  fills:      bar path (none)
  bars:       123 main after replicate (+ 9 warmup), 123 loop, 15 orders
              17 fills, commission 0.00, realized PnL -1460.00
              open CLH26 +1 @ 65.5700 (last 65.7400, unrealized 170.00)
              equity 98710.00 (capital 100000.00, return -1.2900%)
...
Data notes  1 note
  roll straddles  fut:XNYM:CL@4h-utc-day  1 grid bar straddles a roll, 2026-01; the simulator executes each from the cut

Its degradation.json holds the note in full: “the 4h0m0s utc-day grid bar 2026-01-14T20:00:00Z..2026-01-15T00:00:00Z straddles the roll at 2026-01-14T23:00:00Z: the strategy sees the merged chart bar, the simulator executes it from the cut in H26 (GLE-444)”.

The grammar’s own refusals come first and read the same with or without a grid. The front named explicitly reads:

engine: series fut:XCME:ES:cont:ofs=F@7m-rth: fut:XCME:ES:cont:ofs=F names ofs=F, which a run symbol cannot carry: the plain composite fut:XCME:ES:cont is the root's own product, which need not hold the front; name it, or a deferred rank ofs=B1, ofs=B2 or ofs=B3

Another ofs= value reads … names the held rank "B4"; want ofs=B1, ofs=B2 or ofs=B3, and a second ofs= reads … repeats the ofs= qualifier. Any other qualifier (straddle=split) reads … carries the qualifier "<q>", which a continuous run symbol cannot carry; name the root fut:XCME:ES or the composite fut:XCME:ES:cont:adj=<panama|ratio|none> (the straddle policy and the session are the run's, not the symbol's). A declared bare root, :cont composite or deferred composite (ofs=B1..B3, GLE-410) joins the run as a continuous series (GLE-392). --verify starts no strategy, so it cannot see a declared series; a configured symbol outside the grammar stops --verify before it prints the plan.

The marketfeed adapter serves composite bars and answers their coverage. Since ER3 (GLE-325) the continuous orchestration consumes them for a run’s own series, and since GLE-392 for declared ones; the plain bar stream cannot, because each composite bar carries its dated contract’s symbol, not the series symbol. Report benchmark sleeves fetch :cont directly, outside the strategy’s bar stream and without asking for coverage; they need the pb wire (see Report benchmark findings).

Implementation anchors: engine/grid.go, engine/gridwire.go, sdk/data/grid.go, their *_test.go files, and internal/seriescache/gridrevision_test.go.