Architecture

Package map

Process boundaries

The engine launches one adapter and one strategy process for a run. The simulator and venue adapters live inside the engine. Each run owns its strategy state and account; a multi-symbol run remains one strategy instance.

run config -> engine -> data client <-> adapter SDK -> archive/files
                 |
                 +-> strategy host <-> strategy SDK -> OnBar
                 |
                 +-> venue (simulator or IBKR) -> fills/account state
                 |
                 +-> recorder -> result.json, CSVs, report.html

Data protocol preference is DBN, protobuf, then CSV. Strategy preference is protobuf, then CSV; a run under --lifetime v2 offers CSV v2 ahead of CSV v1. Session/path requirements can demote DBN during automatic negotiation. JSONL and Unix-domain-socket protocol names are reserved but have no implementation. A message’s presence in the schema does not imply that the engine drives it.

The handshake

Every child process speaks the same two-line ASCII handshake, implemented in internal/wire/wire.go:

1. Child writes ALGO/1 protocols=stdio-pb-v1,stdio-csv-v2,stdio-csv-v1 sdk=go-0.1.0 (wire.FormatHello) and flushes.
2. Engine reads it (wire.ParseHello), intersects the child’s offer with its own preference list (wire.Negotiate – the engine’s order wins), and replies ALGO/1 use=stdio-pb-v1 (wire.FormatUse).
3. Both sides switch to the negotiated protocol on the same pipes.

The spawn-and-handshake mechanics live in engine/negotiator.go::spawn. Two details there are load-bearing:

• One bufio.Reader per child, for its whole lifetime. The handshake line and every subsequent frame or CSV line come off the same buffered reader (child.r). Constructing a second reader after the handshake would silently discard whatever the first one had already buffered. The SDKs obey the same rule on their side (sdk/strategy/run.go::runWith, sdk/data/run.go::run).
• stderr is an explicit os.Pipe, not cmd.StderrPipe(), so cmd.Wait does not race the stderr-scanner goroutine that re-prints child diagnostics prefixed [name]. A reaped child’s scanner gets a bounded grace period (stderrDrainTimeout, 1s) because a grandchild that inherited the write end can hold the pipe open forever.

Framing

internal/wire/frame.go implements the binary framing shared by stdio-pb-v1 and the stdio-dbn-v1 control plane: a 4-byte big-endian length prefix followed by a serialized Envelope protobuf, capped at wire.MaxFrame (16 MiB). wire.EnvelopeWriter/wire.EnvelopeReader reuse their buffers across calls; neither is safe for concurrent use.

stdio-csv-v1 is newline-delimited comma-separated text with no quoting: fields can never contain commas. Free prose (names, log messages) is sanitized into lookalike characters (internal/wire/csv_strategy.go::SanitizeCSVText); semantic values that cannot ride the grammar are rejected, never rewritten (validateCSVValue), because silently rewriting an identifier would corrupt its meaning. stdio-csv-v2 percent-escapes every field of its command and event lines instead, so they carry any text losslessly (Escaping and fixed arity).

Flush-per-frame is an invariant, not an optimization choice. Both wires are strict request/response at every step, so a buffered-but- unflushed message deadlocks both processes. Every write path in the codebase flushes immediately: engine/negotiator.go::conn.writeEnvelope and conn.writeLine, the strategy SDK’s send/writeLine closures in sdk/strategy/run_pb.go, run_csv.go and run_csv2.go, and the data SDK’s sdk/data/run.go::wireConn (which additionally holds a mutex so source goroutines can emit Log lines while a record batch is streaming). The one deliberate exception is the engine’s stdio-csv-v2 warm-up block, buffered and flushed once as a whole, which a strategy answers only after its last bar (Warm-up blocks). The engine writes events without reading the strategy’s output, so every stdio strategy wire can deadlock when a strategy’s event handlers log heavily (a known limit, GLE-446; see The engine host).