Release-stamped series

Release-stamped series

The dates on these series need care. A COT report describes positions on a Tuesday, but the CFTC publishes it that Friday at the earliest, and during the 2025 shutdown seven weeks later. A FRED payrolls observation dated 1 August describes a month whose figure appears in September. A backtest that used each value from the date it describes would trade on numbers nobody had yet. Marketfeed and the engine prevent that between them: marketfeed stamps every value at the instant it was first public, and the engine hands it to the strategy at that instant and not before.

How a release reaches a strategy

A bar read serves each release as one synthetic bar; the revisions are a separate read (Revisions: the OBSERVATIONS schema). Open, high, low and close all equal the value, and volume is 0. The bar closes at the instant the value was first public and opens at the previous release. The requested interval is ignored and the series’ own cadence is served: macro:FRED:VIXCLS at 1w or 1d is the same daily series, one bar per release. A cot symbol needs its group qualifier (cot:CFTC:13874A:MM). Besides open interest (OI) and each trader category’s net (long minus short), a group can name one leg of a category since GLE-413: -L its long positions, -S its short, -SP its spreading (not for non-reportables NR or producer/merchants PM), and -LCH, -SCH, -SPCH the CFTC’s published week-on-week change of each, with OI-CH for open interest. So cot:CFTC:13874A:LF-L is leveraged funds’ long positions in the E-mini S&P 500; kit.COTLegSymbols builds the pair for a product. A week the CFTC published no change for has no bar in the change series.

  • COT rows close at 16:00 New York on the day the CFTC actually published the report (it publishes at 15:30), usually the Friday after the Tuesday as-of date. The date comes from marketfeed’s release calendar, which records holiday weeks, the 2013, 2018-19 and 2025 shutdowns and the 2023 ION incident. The E-mini S&P 500 report as of 2025-09-30 closes on 2025-11-19. Reports published together, as in a shutdown catch-up, close a second apart in as-of order.
  • RBA FX rates close at 17:00 Sydney on the rate date, the RBA’s published release time.
  • FRED has two kinds of series. 52 serve each observation’s first-release value, closing on ALFRED’s first vintage date for the observation at the original publisher’s release time (BLS and BEA 08:30 New York, H.15 16:15, H.4.1 16:30). September 2026 payrolls close at 08:30 New York on 2026-10-02. The other five (VIXCLS, VXVCLS, DFEDTARU, RRPONTSYD, MORTGAGE30US) are known on their observation date and never revised, and close at 17:00 New York that day. Observations released together close a second apart in observation order.

Each stored row also records the period its value describes: the COT as-of date, the FRED observation date or the RBA rate date. The wire carries it as Bar.ref_date (proto field 27, YYYY-MM-DD; the optional eleventh CSV field on both bar lines, after SESSION, which is left empty when the bar has none), and a strategy reads it as Bar.Ref, that date’s UTC midnight (GLE-261). Key periods by Ref, never by OpenTime: on a release-stamped bar OpenTime is the previous release. Ref is zero on every price bar, and a period that was never published has no bar. The marketfeed adapter serves it from GLE-252; until that adapter is installed Ref is zero on these series too. A bar file for the file-csv adapter may add a ninth ref column (header symbol,ts_open,ts_close,open,high,low,close,volume,ref), and that adapter, like marketfeed’s, serves a cot:, macro: or fx:RBA: symbol’s rows verbatim at their own cadence whatever interval is requested. A strategy binary built from an SDK older than GLE-261 rejects a ref-dated CSV bar that has no session (its parser refuses the empty SESSION placeholder): rebuild strategies with the engine. A run’s series cache stores bars as protobuf, so it keeps ref_date; entries cached before the adapter served it lack it until refreshed (--cache refresh).

Any time interval will do, since it doesn’t change what is served; the strategies declare these series at 1w. A bar’s close is its release, not its open plus the interval, and only the pb and csv wires carry that close verbatim. So negotiation routes a run with a configured release series to one of them at any interval (GLE-1340), and a forced dbn request is refused rather than shifting the release. Negotiation happens before the strategy declares its series, so a run that declares one needs --data-protocol pb. Without it, the engine refuses the declared series before any fetch and names the flag (GLE-311). A run’s --session doesn’t reach these series: they have no trading day (GLE-536).

The run report names a release series by what is served, not by the interval it was asked for: macro:FRED:VIXCLS@release on the instance: and request: lines, macro:FRED:VIXCLS release on the summary plate and macro:FRED:VIXCLS @ release in report.html. The series’ identity keeps the interval: the run id (run-macro:FRED:VIXCLS-1w), result.json’s series Label and a declaration’s errors still say @1w. The bars: line counts every release delivered, about 250 a year for a daily FRED series.

The engine merges every series of a run into one stream ordered by close time (Backtest event ordering), so a release joins ctx.Series(id) when the run’s clock reaches its stamp. At any decision, ctx.Series(id).Close(0) is the latest value already published. kit.AsOfClose(series, cutoff) returns the latest value released by an explicit cutoff, normally the primary bar’s close; use it when the cutoff can be earlier than the current decision. At a co-close the release goes after the primary (the co-close order, Backtest event ordering): a release stamped at exactly a primary bar’s close arrives just after that bar’s decision and is first seen at the next one, so a decision cannot act at a close price that predates the release. For whole windows, kit has the alignment helpers (GLE-263; see Lining up series): AsOfCloses(aux, prim, n) (the value known at each of the last n primary closes, carried forward), JoinCloses(a, b, n) (an inner join on exact close time, for same-interval legs with missing bars), ReleaseClock (the aux value and the last primary close before each release, for correlations against a slow series), AgeAt/StaleLegs/StaleSeries (each leg’s age at the decision) and Cadence/CompareLegs (equal interval and session end, or the implicit lag between legs). Each helper’s doc comment states its pairing class; never correlate carried-forward as-of values.

Lookback and Bar(k) count releases. Lookback: 104 on a COT series holds the last 104 reports known. That is 104 as-of weeks, but not always the last 104 calendar weeks: since 2010, 64 of 874 weeks had no COT release and 62 had two or three, in holiday weeks and shutdown catch-ups. FRED first-release series are stamped in observation order, but a period that was never published has no row: CPIAUCSL has no October 2025, so from November 2025 on Bar(12) of that monthly series is 13 months back, not 12. Key such lookups by reference period instead (GLE-264): kit.PeriodsAgo(cpi, kit.PeriodMonth, 12) is year on year by Ref and reports a missing month (ok false, with that month’s Ref) rather than returning its neighbour; kit.RefValue looks up one period, kit.ShiftRef does the calendar arithmetic, and kit.PairByRef pairs two release series by period, each pair known at the later release. For a second view of the same series, sampled at the bars of a series the strategy chooses and holding each value as known at that bar, declare an aligned series (GLE-353): a daily view of a COT series repeats each report on every daily bar until the next one, with the report’s own release instant in SourceCloseTime. Keep the release view for statistics over the reports themselves.

Warmup counts releases. Before the run, the engine warms each irregular series up to its depth — the declaration’s own Lookback when it names one, else the global Lookback() — in release ROWS (GLE-262): it fetches back from at least a year, doubling the window until it holds that many releases or reaches the series’ earliest coverage. A monthly series declared with Lookback: 24 starts the run with its last 24 releases (fewer only when the archive starts later, which the run records as a warmup_shortfall degradation). Guard reads with Len() all the same.

Don’t add a publication lag of your own: the stamps already carry it.

Seeing the lag. printbars prints each bar’s open and close and, on a bar that has a Ref, the reference period and the calendar days from it to the release (lag, counted to the close’s date in the publisher’s zone: New York for CFTC and FRED, Sydney for RBA). With the adapters installed (Installing), a release-stamped series needs only the strategy, the symbol and the interval; with no --date-from or --date-to the run covers the series’ whole history, here the managed-money net of CFTC commodity code 007 from June 2006:

$ bin/algo run --strategy bin/printbars --symbols cot:CFTC:007:MM --bar-interval 1w
...
[printbars] [bar 1] cot:CFTC:007:MM open=2006-06-16T16:00:00 EDT (-04:00) close=2006-06-16T16:00:00 EDT (-04:00) O=41552 H=41552 L=41552 C=41552 V=0 ref=2006-06-13 lag=3d
[printbars] [bar 2] cot:CFTC:007:MM open=2006-06-16T16:00:00 EDT (-04:00) close=2006-06-23T16:00:00 EDT (-04:00) O=19831 H=19831 L=19831 C=19831 V=0 ref=2006-06-20 lag=3d
...
[printbars] [printbars] received 1061 bars total
  adapter:    marketfeed (stdio-pb-v1), inferred from /usr/local/algolang/adapters

A cot symbol still needs its group qualifier (:MM here): marketfeed refuses a bare cot:CFTC:007.

Over a window, the same output shows the 2025 shutdown in leveraged funds’ E-mini positions: the report as of 30 September published 50 days late, and the catch-up’s lag shrinking back to the usual 3 days by late December:

$ bin/algo run --strategy bin/printbars --symbols cot:CFTC:13874A:LF --bar-interval 1w \
    --date-from 2025-09-01T00:00:00Z --date-to 2026-01-01T00:00:00Z --report off
...
[printbars] [bar 4] cot:CFTC:13874A:LF open=2025-09-19T16:00:00 EDT (-04:00) close=2025-09-26T16:00:00 EDT (-04:00) O=-442953 H=-442953 L=-442953 C=-442953 V=0 ref=2025-09-23 lag=3d
[printbars] [bar 5] cot:CFTC:13874A:LF open=2025-09-26T16:00:00 EDT (-04:00) close=2025-11-19T16:00:00 EST (-05:00) O=-437245 H=-437245 L=-437245 C=-437245 V=0 ref=2025-09-30 lag=50d
[printbars] [bar 6] cot:CFTC:13874A:LF open=2025-11-19T16:00:00 EST (-05:00) close=2025-11-21T16:00:00 EST (-05:00) O=-456477 H=-456477 L=-456477 C=-456477 V=0 ref=2025-10-07 lag=45d
...
[printbars] [bar 16] cot:CFTC:13874A:LF open=2025-12-19T16:00:00 EST (-05:00) close=2025-12-23T16:00:00 EST (-05:00) O=-391100 H=-391100 L=-391100 C=-391100 V=0 ref=2025-12-16 lag=7d

The same run on macro:FRED:CPIAUCSL shows September 2025 CPI released 53 days after its period instead of the usual 41 to 44, and no October 2025 bar at all: ref steps from 2025-09-01 to 2025-11-01.

For positioning in one product, read that product’s own contract market (kit.CFTCCodeForSymbol: 13874A for the E-mini S&P 500, 088691 for COMEX gold), not a three-digit CFTC commodity sum or a -CMB consolidation. The sums mix contract sizes and unrelated markets. marketfeed serves the consolidations in E-mini equivalents throughout (GLE-364), but they take in the micro contracts from the 2 May 2023 report, a step the E-mini contract never had. marketfeed’s docs/marketfeed.md has the full stamping rules and data warnings.

Estimated release dates in early FRED history

ALFRED, the St. Louis Fed’s archive of past vintages, starts at a different date for each series. An observation older than a series’ ALFRED record has no usable first-release date, because ALFRED often adds such history in a vintage years later: DGS10’s 1962 values first appear in the 2005-12-20 vintage. Marketfeed therefore estimates the release as the observation date plus the series’ 90th-percentile release lag, measured over the ALFRED era in marketfeed’s 2026-09-28 audit, at the release’s time of day. Precisely, an observation is estimated when its date plus that lag falls before the series’ first ALFRED vintage.

The estimate can be early. If early releases were as slow as modern ones, about one estimated value in ten is stamped before it was really public, usually by days, so a backtest over those years can see it slightly early. Values just inside the record carry the record’s first vintage date, which can be later than their real release: late, never early. We accept this residual risk. Where a result depends on one of these series, start the backtest after its “Estimated through” date below, or check that the result survives doing so.

Most backtests start in 2010 or later; Databento’s futures history begins in June 2010. These series have estimated stamps in 2010 or later: STLFSI4 (to October 2022), SAHMREALTIME (July 2019), DTWEXBGS (January 2019), PCOPPUSDM (November 2014), BAA10Y, T10Y2Y, T10Y3M, T10YIE and T5YIE (January 2014), BOGMBASE (May 2013), RECPROUSM156N (June 2012), WALCL (June 2011), ANFCI and NFCI (May 2011), DCOILWTICO (March 2011), TEMPHELPS and USSLIND (November 2010), JTSQUL (May 2010), and DRTSCILM, M2V and USREC (January 2010). The spreads T10Y2Y and T10Y3M can be computed from DGS10, DGS2 and DGS3MO, whose records start in June 2005.

The table covers the 52 first-release series, measured from the lake on 2026-10-03. “Estimated through” is the last observation date with an estimated stamp; “Lag” is the estimate’s lag in calendar days. The five known-on-the-day series have no estimated stamps.

SeriesHistory fromALFRED record fromEstimated throughLag (days)
ANFCI1971-01-082011-05-252011-05-136
AWHMAN1932-01-011961-11-031961-09-0137
BAA10Y1986-01-022014-01-272014-01-224
BAMLH0A0HYM22023-09-292023-09-29none4
BOGMBASE1959-01-012013-07-112013-05-0157
CCSA1967-01-072009-09-102009-08-2212
CPIAUCSL1947-01-011972-07-211972-05-0153
CPILFESL1957-01-011996-12-121996-10-0149
DCOILWTICO1986-01-022011-04-062011-03-288
DFF1954-07-012005-06-282005-06-234
DFII102003-01-022005-10-122005-10-074
DGS101962-01-022005-06-282005-06-234
DGS21976-06-012005-06-282005-06-234
DGS3MO1981-09-012005-06-282005-06-234
DRTSCILM1990-04-012010-04-202010-01-0138
DTWEXBGS2006-01-022019-02-042019-01-257
GDP1946-01-011991-12-041991-07-01122
GDPC11947-01-011991-12-041991-07-01122
HOUST1959-01-011960-07-211960-06-0149
ICSA1967-01-072009-05-282009-05-165
INDPRO1919-01-011927-01-261926-11-0157
ISRATIO1948-01-011997-03-141996-12-0177
JTSQUL2000-12-012010-08-112010-05-0173
M1SL1959-01-011980-02-081979-12-0152
M2SL1959-01-011980-02-081979-12-0153
M2V1959-01-012010-07-302010-01-01121
MICH1978-01-011999-02-261999-01-0130
NEWORDER1968-02-011997-03-061997-01-0158
NFCI1971-01-082011-05-252011-05-136
PAYEMS1939-01-011955-05-061955-03-0139
PCEPI1959-01-012000-08-012000-05-0162
PCEPILFE1959-01-012000-08-012000-05-0162
PCOPPUSDM1980-01-012015-11-062014-11-01352
PERMIT1960-01-011999-08-171999-06-0150
RECPROUSM156N1967-02-012012-09-042012-06-0193
RSAFS1992-01-012001-06-132001-04-0146
SAHMREALTIME1959-12-012019-09-062019-07-0137
STLFSI41993-12-312022-11-102022-10-286
T10Y2Y1976-06-012014-01-272014-01-224
T10Y3M1982-01-042014-01-272014-01-224
T10YIE2003-01-022014-01-272014-01-224
T5YIE2003-01-022014-01-272014-01-224
TCU1967-01-011996-11-151996-09-0147
TEMPHELPS1990-01-012011-01-072010-11-0137
TOTRESNS1959-01-011996-12-121996-10-0155
UMCSENT1952-11-011998-07-311998-06-0130
UNRATE1948-01-011960-03-151960-02-0137
USREC1854-12-012011-05-022010-01-01463
USSLIND1982-01-012011-02-012010-11-0184
WALCL2002-12-182011-07-072011-06-291
WM2NS1980-11-032002-10-312002-09-2336
WTREGEN1986-01-082008-12-182008-12-101

First-release values, not revisions

Served FRED bars are first releases, and later revisions never change them. Marketfeed keeps every vintage in its revision store (vintages/FRED/<ID>.csv: observation date, vintage date, value). The bars stay first releases by choice; the trade-off is recorded here. The revisions travel separately, as the OBSERVATIONS schema (GLE-398, next section): every vintage of every period, or the series as known at an instant. Marketfeed’s adapter serves it for its 52 alfred-kind FRED series (GLE-403), and a strategy declares it as market events (Observations as events, GLE-408).

For first-release values:

  • No look-ahead through revisions. Each value is one a trader could have acted on at its close.
  • One value per observation, fixed for good. Results are reproducible, a later revision cannot change a past backtest, cached series stay valid, and live and backtest runs see the same history.
  • It is the number the market reacted to. Prices move on the first print, so a release-day signal sees what was traded on.
  • Simple delivery. One bar per observation, in release order, keeps the wire simple and Lookback meaningful.

Against:

  • It is not everything known at the time. On any date a trader also knew the revised values of earlier observations, while the strategy sees only first prints. Payrolls are revised in each of the two following months and benchmarked every year, so a 24-month payroll trend built from first prints follows a path nobody saw at any single moment.
  • Rebasings can leave steps. When an agency rebases a level series, consecutive first prints can come from different bases, and a change computed between them includes the rebasing. BLS applies its annual payroll benchmark with the January figure, released in February.
  • Revisions are news in themselves. The revisions to earlier months that come with each release are invisible.
  • Published models may not carry over. Studies, and models calibrated on today’s FRED, use latest-vintage data, so their thresholds may not suit first-release values.

Two other views were considered:

  • A real-time view: at each decision, the series’ whole history as of the latest vintage then. This is correct, but carried on the bars each release would hold a whole history. The OBSERVATIONS schema carries it instead: a stream of revisions, each stamped when it became public, and a snapshot read pinned with as_of. The bars and Lookback are unchanged.
  • A latest-vintage research series, clearly labelled. It helps comparisons with published work, but it has look-ahead by construction and must never feed a strategy. It is not served.

Both can be rebuilt from the revision store without refetching.

Revisions: the OBSERVATIONS schema

The OBSERVATIONS schema (data schema 16, GLE-398) carries a release-stamped series’ revision history. One record is one published value of one reference period:

  • ref_date: the period the value describes, as on the bars (2025-04-01 for the second quarter of 2025).
  • available_at: the instant the value became public, the release stamp. The record’s meta.ts_recv equals it (and meta.ts_event too, when set), and the engine orders on it.
  • value, or withdrawn when that vintage withdrew the period’s value (FRED’s .). A record carries exactly one of the two.
  • revision: 0 for the period’s first release, then 1, 2, … for each later vintage of the period.
  • vintage_date: the vintage the value belongs to (FRED’s realtime_start).

A request names one series and a window (start, end] on availability time: a record owns its instant, as a bar owns its close (The date range, GLE-1342). It reads in one of two ways:

  • Every revision. Without as_of, every record published in the window, in the order a replay needs: by available_at, then ref_date, then revision. An annual update republishes many periods at one instant, so they arrive together, in period order.
  • As known at an instant. With as_of, the series as it stood then: of the records published at or before as_of, each period’s newest, still windowed on available_at. This is the snapshot a decision at as_of could have used, revisions applied.

The bundled file-csv adapter serves the schema from a vintages CSV (symbol,ref_date,vintage_date,available_at,value; the value is a finite decimal, or . for a withdrawal). testdata/gle398/fred-gdp-vintages.csv holds GDP’s vintages for every quarter since 2023, copied from marketfeed’s revision store and stamped by marketfeed’s release rule for GDP (08:30 New York on the vintage date). Revision 0 of each quarter equals the first-release bar of testdata/gle357/FRED_GDP.csv, value and stamp. Every revision published from July to September 2026:

$ bin/algodata obs -adapter bin/file-csv-adapter \
    -adapter-inputs Path=testdata/gle357/FRED_GDP.csv,Observations=testdata/gle398/fred-gdp-vintages.csv \
    -symbol macro:FRED:GDP -start 2026-07-01 -end 2026-10-01
symbol,ref_date,revision,vintage_date,available_at,value
macro:FRED:GDP,2026-04-01,0,2026-07-30,2026-07-30T12:30:00Z,32475.21
macro:FRED:GDP,2026-04-01,1,2026-08-26,2026-08-26T12:30:00Z,32486.066
macro:FRED:GDP,2023-01-01,6,2026-09-30,2026-09-30T12:30:00Z,27241.282
macro:FRED:GDP,2023-04-01,5,2026-09-30,2026-09-30T12:30:00Z,27533.925
...
macro:FRED:GDP,2026-01-01,3,2026-09-30,2026-09-30T12:30:00Z,31906.274
macro:FRED:GDP,2026-04-01,2,2026-09-30,2026-09-30T12:30:00Z,32563.03

The second quarter’s advance and second estimates came out in July and August. The annual update on 30 September revised all 14 quarters at 08:30, so it is 14 records at one instant. The same series as known on 1 September 2026, one record per quarter:

$ bin/algodata obs -adapter bin/file-csv-adapter \
    -adapter-inputs Path=testdata/gle357/FRED_GDP.csv,Observations=testdata/gle398/fred-gdp-vintages.csv \
    -symbol macro:FRED:GDP -start 2025-01-01 -end 2026-10-01 -as-of 2026-09-01
symbol,ref_date,revision,vintage_date,available_at,value
macro:FRED:GDP,2023-01-01,5,2025-09-25,2025-09-25T12:30:00Z,27216.445
...
macro:FRED:GDP,2025-07-01,1,2026-01-22,2026-01-22T13:30:00Z,31098.027
macro:FRED:GDP,2025-10-01,2,2026-04-09,2026-04-09T12:30:00Z,31422.526
macro:FRED:GDP,2026-01-01,2,2026-06-25,2026-06-25T12:30:00Z,31865.721
macro:FRED:GDP,2026-04-01,1,2026-08-26,2026-08-26T12:30:00Z,32486.066

The 2023 to 2025 quarters stand at the 2025 annual update. The third quarter of 2025 shows the shutdown: its second vintage came out on 22 January 2026.

From marketfeed. Marketfeed’s adapter serves OBSERVATIONS for every alfred-kind FRED series in its archive (GLE-403), straight from the revision store, with a snapshot token. A close-kind series (VIXCLS and the other series known at their close) has no vintages and is refused. A period’s first release carries the same instant as its bar, and every later vintage is stamped at the series’ release time on its vintage date. The archive holds GDP’s whole history, so the 2026 annual update revises 22 quarters:

$ make marketfeed-adapter
$ bin/algodata obs -adapter bin/marketfeed-adapter \
    -symbol macro:FRED:GDP -start 2026-07-01 -end 2026-10-01
symbol,ref_date,revision,vintage_date,available_at,value
macro:FRED:GDP,2026-04-01,0,2026-07-30,2026-07-30T12:30:00Z,32475.21
macro:FRED:GDP,2026-04-01,1,2026-08-26,2026-08-26T12:30:00Z,32486.066
macro:FRED:GDP,2021-01-01,8,2026-09-30,2026-09-30T12:30:00Z,22704.65
...

Marketfeed’s stamping law, its token and its cache are described in its own documentation (docs/marketfeed.md, “OBSERVATIONS”).

Serving it from an adapter. An adapter implements data.ObservationSource and advertises OBSERVATIONS in Capabilities(). An adapter that holds a series’ history can hand it to data.SelectObservations, which numbers the revisions, applies as_of and the window, and returns the records in wire order. It reads Ref and Vintage as UTC calendar days and AvailableAt as a wall-clock instant, as the wire carries them:

func (a *MyAdapter) OnGetObservations(ctx *data.DataContext, req data.ObservationRequest, emit chan<- []data.Observation) error {
    history, err := a.vintages(req.Symbol) // every vintage of the series, any order
    if err != nil {
        return err
    }
    emit <- data.SelectObservations(history, req)
    return nil
}

The schema travels on the protobuf data wire only. The SDK refuses it on DBN and CSV with bad_request, and an adapter that does not advertise it answers unsupported_schema.

Reading it in the engine. DataClient.ServesObservations() reports the capability, and FetchObservations(symbol, start, end, engine.ObservationPins{AsOf, SnapshotToken}, batchSize) returns the records published in the engine’s half-open [start, end), sent to the adapter as (start-1ns, end-1ns]. It refuses a response that breaks the wire contract: - a record for another symbol; - a malformed ref_date or vintage_date; - a ts_recv, or a ts_event that is set, other than the record’s available_at; - a value present on a withdrawn record, or missing on any other; - a non-finite value; - a record outside the window, or after as_of; - records out of order; - a skipped revision, or under as_of a second record for one period.

The engine checks the response after reading all of it, so a refused response leaves the connection usable.

COT reports beyond the release calendar

Marketfeed’s COT release calendar (fetch/cot/releases.csv in marketfeed) lists every report week from 2006-06-13. In the marketfeed commit pinned here it ends with the report as of 2026-09-22. A report after the calendar’s last week is stamped at the usual Friday, 16:00 New York, if the nightly run sees it within 12 hours of that time, and otherwise at the time it was first seen. Such a stamp can be late but never early. It records our run, not the CFTC’s release, so a late or missed nightly run stamps a report later than the CFTC published it, and a later run keeps the stamp an earlier one chose.

Append each year’s dates to the calendar when the CFTC publishes its schedule. Extending the calendar re-stamps weeks already stored only when they are fetched again; marketfeed’s docs give the procedure.

Live trading

Today. This build’s live mode is an emulator-driven harness (The venue seam and the current live path). A COT or FRED series a strategy declares is reference data there: its bars come from the data the run fetched at start, and it steps nothing at the venue (GLE-1385 8a). A continuous futures series (a bare root or a :cont composite) runs live on the emulated venue under lifetime v1, from the composite the run fetched at start (Continuous futures in live mode, GLE-1385 8b). A live run also fetches all its data when it starts and nothing afterwards. No strategy that reads these series can run live yet; this subsection records the rules that apply once one can. GLE-295 tracks the work.

The rule. Live market data comes only from Databento, through marketfeed’s gateways (The live source). COT, FRED and RBA series will not be streamed at all: a live run reads them from the marketfeed archive and treats them exactly as a backtest does: each release joins the strategy’s series when the clock reaches its stamp, Lookback counts releases, and the values are first releases. Warmup at startup reads the same rows a backtest would. The stamps are stable, because first-release values never change and marketfeed keeps the stamp an earlier run chose, so a restarted strategy rebuilds the history it had.

Where live and backtest differ: arrival. A backtest assumes each value is in hand at its stamp. A live run has it only once the archive has fetched it and the engine has read it. With only marketfeed’s nightly run (11:30 Melbourne, which is 01:30 UTC under AEST and 00:30 UTC under AEDT), each source reaches the archive this long after its stamp at the earliest:

SourceStampReaches the archive
COTFriday 16:00 New York3.5 to 5.5 hours later, Friday evening New York
FRED, BLS and BEA releases08:30 New York11 to 13 hours later
FRED daily rates (H.15)16:15 New York, the next business day3 to 5.5 hours later
FRED known-on-the-day series (VIXCLS)17:00 New York2.5 to 4.5 hours later
RBA17:00 Sydneyabout 18.5 hours later

A decision between a stamp and its arrival sees the value in a backtest but not live. Take a strategy deciding at the CME equity close, 16:00 Chicago (17:00 New York). A backtest uses Friday’s COT report at Friday’s close; a live run first has it on Monday. A payrolls figure released at 08:30 is in hand by that day’s close in a backtest, and live only at the next session’s close. The same applies to the H.15 rates stamped at 16:15 New York.

Making them agree. There are two ways; live runs should use the first.

  • Fetch promptly. Schedule archive fetches shortly after each release as well as the nightly run: COT on Fridays after 15:30 New York, FRED on release mornings after 08:30 New York and daily after 16:15 New York for H.15, and RBA after 17:00 Sydney. The engine must then re-read the archive during the run, which is not built (GLE-295): a live run reads it only at startup. FRED posts a release some minutes after the agency publishes it, so even a prompt fetch lands the value after its stamp. That matters only to a decision inside that window, such as an intraday bar closing just after 08:30.
  • Delay the backtest. A backtest could instead move each release to when the live setup would have it, for example the next scheduled fetch. That makes backtests pessimistic and matches a nightly-only live setup exactly. It is not built either.

Rows stored before their stamp. A row can reach the archive before its stamp. On 2026-10-03 the archive already held T10Y2Y, T10Y3M, T10YIE and T5YIE for 2026-10-02, stamped 2026-10-05 16:15 New York because of the one-business-day floor. A live run must hold such a row until its clock reaches the stamp, as a backtest does, and never deliver it early.

Releases that land after their stamp. When a live run finds a release whose stamp has already passed, it should add the release to the series at once, before the next decision, and never re-run a decision already made. The protocol has a late flag for this: the SDK appends a late bar to the series without calling OnBar. The engine does not set the flag yet. The run should also record each such release’s stamp and arrival time, so live-versus-backtest reconciliation can explain the decisions that differ.

The earlier gaps, live.

  • Estimated early FRED dates don’t affect live decisions: by the time a live run starts, those values are long public.
  • First-release values apply unchanged, and they are what keeps live history equal to backtest history. A revision published while the strategy runs never alters what it has seen.
  • COT reports beyond the calendar are the normal live case unless the calendar is kept current. A report the nightly run sees within 12 hours of the usual Friday is stamped at that Friday. A later one is stamped when first seen, which is when the live run had it. If the calendar is later extended over such a week and the week fetched again, its stamp moves back to the CFTC’s release, and later backtests see the report earlier than the live run did.