Declaring data

A one-series strategy gets its bar series from the run config and declares nothing. When you want more – a second series to filter against, or a symbol that isn’t in the run’s universe at all – you declare it, and the engine fetches and presents it before your first bar. Two optional interfaces and one handler cover the whole feature.

Extra series: DataRequirer. Return one DataRequirement per series. Each has a stable SeriesID you address it by, a Symbol, a Schema (the bar kind, interval and construction), and a per-series Lookback:

func (s *TwoSpeed) DataRequirements() []algolang.DataRequirement {
    return []algolang.DataRequirement{{
        SeriesID: "trend",
        Symbol:   s.Symbol,
        Lookback: 50,
        Schema:   algolang.DataSchemaSpec{Kind: algolang.Bars, BarKind: algolang.BarsTime, Interval: "1d"},
    }}
}

Each declared series warms up to its own Lookback, and a release-stamped series counts releases (GLE-262). The data wire is negotiated before the strategy declares anything, so a strategy that declares a release-stamped series (cot:, macro: or fx:RBA:, declared at 1w) needs --data-protocol pb: DBN has no 1w record type. Without the flag, a run whose configured series negotiated DBN stops before fetching any declared series, with an error that names the series and the flag (GLE-311). A continuous run reads declarations as a plain run does (GLE-392; see Declaring extra data on a continuous run). See also strategy-declared data follow-ons.

Besides bar series, a declaration can name an instrument’s trade tape (Schema: algolang.DataSchemaSpec{Kind: algolang.Trades}): the engine then sends you every trade between bars rather than a bar series. See Market events.

Each series has its own construction (GLE-390). A declared bar series says how its bars are built. With BarKind left unspecified and no construction field set, it takes the run’s construction, as before. With a BarKind, the schema is that series’ own complete construction: a field it leaves zero takes the kind’s default, never the run’s. So one run can mix 1,000-share volume bars, one-minute time bars and tick-imbalance bars:

A field the kind does not use is refused, naming the field, rather than dropped. The SDK checks it before the declaration leaves the strategy, and the engine checks it again. Two series with the same symbol, interval, session and effective construction are the same series. So a declaration that matches a configured series refines it, whether it spells the construction out or not, and one that differs only in its construction is a new series. A series with an event-bar construction of its own is named by it in the run summary, its lead-in note and a duplicate refusal (MSFT@volume:t=1000). Two series with one identity are refused before any fetch: on a volume run, a series declaring the run’s own volume construction beside the run’s series serves the same bars twice. A series’ own construction travels on the pb strategy wire only: the CSV declaring record has no slots for it and refuses it, naming the series. A declared session or construction an event series cannot take, such as a session or a path on volume bars, is refused before any fetch. So is a series the negotiated data wire cannot carry: an event series on DBN needs --data-protocol pb.

Example. strategies/demo/go/volumetrend (make run-volumetrend) trades MSFT on the run’s 1,000-share volume bars from the bundled trade sample. It declares two more series, each with a construction of its own:

• trend is one-minute time bars, a trend filter: the last close above the mean of the last five.
• flow is tick-imbalance bars with a static threshold of 15 and a five-minute lead-in, an order-flow confirmation: the last imbalance bar closed up.

On each volume bar it wants a long of 100 shares when the trend is up and the flow confirms, and flat once the trend turns down:

func (v *VolumeTrend) DataRequirements() []algolang.DataRequirement {
    return []algolang.DataRequirement{
        {SeriesID: "trend", Symbol: v.symbol, Lookback: v.TrendN + 1,
            Schema: algolang.DataSchemaSpec{Kind: algolang.Bars, BarKind: algolang.BarsTime, Interval: v.Trend}},
        {SeriesID: "flow", Symbol: v.symbol, Lookback: 2,
            Schema: algolang.DataSchemaSpec{Kind: algolang.Bars, BarKind: algolang.BarsImbalanceTick,
                Threshold: v.Imbalance, LeadIn: 5 * time.Minute}},
    }
}

$ make run-volumetrend
algo: series MSFT@imbalance_tick:t=15:lead=5m0s: 5m0s lead-in: the constructor warmed from 2022-06-10T12:35:00Z (6 warm-up bars discarded)
[volumetrend] INFO: volumetrend: 2022-06-10T12:41:00Z close=261.88 trend=261.70/261.45 up=true flow_up=true pos=0 target=100
[volumetrend] INFO: volumetrend: 2022-06-10T12:43:42Z close=261.55 trend=262.30/261.68 up=true flow_up=true pos=100 target=100
[volumetrend] INFO: volumetrend: 2022-06-10T12:44:10Z close=261.48 trend=261.51/261.75 up=false flow_up=true pos=100 target=0
...
[volumetrend] INFO: volumetrend: 2022-06-10T12:46:53Z close=262.00 trend=261.82/261.82 up=true flow_up=true pos=0 target=100
[volumetrend] INFO: volumetrend: 2022-06-10T12:48:02Z close=262.02 trend=262.00/261.76 up=true flow_up=false pos=100 target=100
[volumetrend] INFO: volumetrend: 2022-06-10T12:50:50Z close=261.32 trend=261.87/261.97 up=false flow_up=false pos=100 target=0
...
  instance:   1 (MSFT@volume:t=1000 + MSFT@imbalance_tick:t=15:lead=5m0s + MSFT@1m), 4 orders
  bars:       33 main after replicate (+ 10 warmup), 33 loop, 4 orders
algo: data degradation: MSFT@imbalance_tick:t=15:lead=5m0s pre-roll: the 5m0s lead-in was clamped: the adapter's data begins at 2022-06-10T12:30:01Z, at or after the start, so the constructor was not warmed
              4 fills, commission 0.00, realized PnL -83.00

The volume bars, the one-minute bars and the imbalance bars arrive merged by close, so each decision sees the latest bar of every series. Every series is pre-rolled before the first decision, the two event series included (GLE-407), so the first volume bar already sees an imbalance bar that closed up. The imbalance series’ lead-in is its own: the run itself has none. Its pre-roll reaches back to the sample’s first trade, where the lead-in cannot warm the constructor, and the run records that. An end-to-end test (engine/gle390_volumetrend_e2e_test.go) runs the example on the bundled adapter. It checks one decision per volume bar, that both declared series feed the decisions, the imbalance series’ own lead-in, and a fill for every change of target. engine/gle390_construction_fetch_test.go checks that every construction field and the session of a declared series reach the fetch. A declared series can carry a session of its own as well: sessionref declares an E-mini contract’s custom-window day session as a daily reference beside its hourly bars (see custom session windows).

The run config’s series[] entries take the same construction keys; see Run matrices.

You read a declared series exactly like any other, by its SeriesID:

trend := ctx.Series("trend")
if trend.Len() >= 2 && trend.Close(0) >= trend.Close(1) {
    // the daily trend is up
}

The trend series above shares the strategy’s configured symbol, so it is a refinement – a second interval of a symbol already in the run, allowed under any policy. (A declared eq: series is served as the feed serves it, raw or adjusted by its adj= qualifier, and takes no corporate-action accounting; a symbol whose books must follow its splits and dividends belongs in the run config, GLE-328.) Declaring a series for a symbol that is not in the run config is a universe expansion, and that is where the two universes matter.

Two universes: data vs. venue. “See a symbol’s data” and “trade a symbol” are kept separate. The data universe is every series you declare: the engine fetches it and you read it through ctx.Series. The venue universe is only the symbols you may trade – your configured symbols plus any you explicitly declare tradeable. A symbol you pull in only for data – a volatility index, a sector gauge, a basket of names you screen but never trade – is in the data universe only, and an order on it is denied. To trade an expanded symbol, name it tradeable too:

func (s *Pairs) DataRequirements() []algolang.DataRequirement {
    return []algolang.DataRequirement{{
        SeriesID: "hedge", Symbol: "NQ", Lookback: 20,
        Schema: algolang.DataSchemaSpec{Kind: algolang.Bars, BarKind: algolang.BarsTime, Interval: "1m"},
    }}
}
func (s *Pairs) TradeableSymbols() []string { return []string{"NQ"} }  // admit NQ to the venue

Every tradeable symbol must also be a declared bar series (its positions mark from its bars). Expanding into a new tradeable symbol is governed by the run’s universe-expansion policy – by default a new symbol may be pulled in for data but not traded (see “Universe expansion” under config files). Pulling in data-only reference symbols is allowed by default.

Reacting to the resolved universe: OnUniverseResolved and ctx.Instrument. Once the engine has resolved the (possibly expanded) universe – after InitAck, before your first bar – it pushes the instrument metadata and calls your optional OnUniverseResolved(ctx). From then on ctx.Instrument(sym) answers metadata questions (exchange, asset class, tick size, point value) locally, the way ctx.Position answers position questions:

func (s *Pairs) OnUniverseResolved(ctx *algolang.Context) error {
    ii := ctx.Instrument("NQ")
    ctx.Logf("NQ resolved: exchange=%s tick=%v", ii.ExchangeMIC, ii.TickSize)
    return nil  // returning an error refuses the run
}

ctx.Instrument is empty during OnInit (the push hasn’t arrived yet); if you need configured-instrument metadata that early, read cfg.Instruments in OnInit instead.

Lining declared series up: SeriesAligner. A declared weekly or quarterly series keeps its own clock: ctx.Series("cot") holds one bar per report. To read it beside the primary row for row, declare an aligned series on it as well, {SeriesID: "cot_d", Source: "cot"}, and read ctx.Series("cot_d"): one sample per primary bar, each the report known at that bar’s close. See Aligned series.

A strategy that declares no extra data and no tradeables – one that implements only OnBar – sees exactly the single-series behaviour of the rest of this chapter: its data and venue universes are just its configured symbols, and none of this applies. Declaring strategies must speak the protobuf wire (stdio-pb-v1); the engine refuses to start a declaring strategy on the CSV protocol.

Declaring extra data on a continuous run

A strategy on a continuous series declares extra data as it would on a dated contract (GLE-392). strategies/demo/go/cotcarry trades the continuous E-mini S&P 500 daily series and declares two release-stamped references beside it:

• the weekly CFTC leveraged-funds report on E-mini S&P 500 futures (cot:CFTC:13874A:LF);
• the monthly unemployment rate (macro:FRED:UNRATE).

func (s *COTCarry) DataRequirements() []algolang.DataRequirement {
    weekly := algolang.DataSchemaSpec{Kind: algolang.Bars, BarKind: algolang.BarsTime, Interval: "1w"}
    return []algolang.DataRequirement{
        {SeriesID: "cot", Symbol: s.COT, Lookback: s.COTWindow, Schema: weekly},
        {SeriesID: "jobless", Symbol: s.Jobless, Lookback: 4, Schema: weekly},
    }
}

It holds one ES lot while the adjusted close is above its 50-day SMA, the newest COT report sits in the lower half of its 26-report range, and unemployment is no higher than three releases earlier; otherwise it is flat. Each release arrives as a bar of its own series at its release time, so a daily decision sees only what was known at its close. make run-cotcarry runs it over the archive, 2018 through 2024 (excerpt):

Algolang run run-fut-xcme-es-1d
  adapter:    marketfeed-adapter (stdio-pb-v1)
  strategy:   cotcarry ("COT carry") via stdio-pb-v1
  instance:   1 (fut:XCME:ES@1d + cot:CFTC:13874A:LF@1w + macro:FRED:UNRATE@1w), 74 orders
  request:    fut:XCME:ES@1d,cot:CFTC:13874A:LF@1w,macro:FRED:UNRATE@1w BARS
  coverage:   cot:CFTC:13874A:LF: finest supportable granularity: 1w (limited by marketfeed-adapter, 2018-01-01T00:00:00Z..2025-01-01T00:00:00Z)
  coverage:   macro:FRED:UNRATE: finest supportable granularity: 1d (limited by marketfeed-adapter, 2018-01-01T00:00:00Z..2025-01-01T00:00:00Z)
  data:       4,191 records, 640,926 bytes in 464.9 ms (9,014 rec/s, 1.3 MiB/s)
  bars:       2,258 main after replicate (+ 81 warmup), 2,258 loop, 74 orders
  simulator:  resolution=bar bracket_ambiguity=conservative limit_fill=conservative slippage_ticks=0
              90 fills, commission 0.00, realized PnL 16237.50
              equity 116237.50 (capital 100000.00, return +16.2375%)

The 81 warm-up bars are 51 ES bars (Lookback(), the 50-day SMA plus the current bar; GLE-393 warms a continuous series up) and the two references’ 26 and 4 releases, so the first session of 2018 already decides on a full SMA.

A declared continuous series works the same way. Declaring fut:XCME:NQ beside a configured fut:XCME:ES delivers NQ’s adjusted bars, under the declared SeriesID (GLE-394), and rolls NQ by its own served schedule. Orders on NQ are denied unless the strategy also names it in TradeableSymbols. The same declaration on a plain run (a dated contract, say) joins NQ through the continuous module too (TestGLE424DeclaredContinuousSeries, TestGLE392DeclarationsBesideContinuousAndOnAPlainRun).