Bid-Ask Spread Estimators

CorwinSchultzSpread

func (s Series) CorwinSchultzSpread(h, l, c Series, n int) Series

CorwinSchultzSpread estimates the bid-ask spread from the high, low and close of each bar by the estimator of Corwin and Schultz 2012, Journal of Finance 67(2), which the R package bidask calls CS2: each pair of consecutive bars gives an estimate that is clipped at 0, the n clipped estimates are averaged, and the paper’s overnight-gap adjustment is applied. The paper is “A Simple Way to Estimate Bid-Ask Spreads from Daily High and Low Prices”. The receiver is not used.

Pair t, for t >= 1, is the pair of bars t-1 and t with the prior close c[t-1]. With hl = ln(h) - ln(l) the log range of a bar, beta = hl[t]^2 + hl[t-1]^2 is the sum of the two bars’ squared log ranges (the per-bar Parkinson terms of ParkinsonVolatility), and gamma = (max(ln h[t] + g, ln h[t-1]) - min(ln l[t] + g, ln l[t-1]))^2 is the squared two-bar log range after the gap adjustment g. The paper’s alpha is (sqrt(2 beta) - sqrt(beta)) / (3 - 2 sqrt(2)) - sqrt(gamma / (3 - 2 sqrt(2))), evaluated here as the equal (1 + sqrt(2)) (sqrt(beta) - sqrt(gamma)), with 1 + sqrt(2) the float64 2.4142135623730949; a negative alpha is set to 0, the paper’s rule for negative two-day spreads; and the paper’s S = 2 (exp(alpha) - 1) / (1 + exp(alpha)) is evaluated as the equal 2 tanh(alpha/2), which keeps its accuracy for a small alpha.

The gap adjustment (the paper’s section II.A) is g = max(0, ln c[t-1] - ln h[t]) + min(0, ln c[t-1] - ln l[t]). It acts only when c[t-1] lies outside [l[t], h[t]], a low above the prior close giving g < 0 and a high below it giving g > 0; it shifts bar t’s logarithms by g, so that the bar touches the prior close, and leaves bar t’s own log range unchanged. It corrects overnight gaps and so rarely acts on intraday bars.

The window n counts two-bar pairs: the value at bar t is the mean of the estimates of pairs t-n+1 .. t, which read bars t-n .. t and closes t-n .. t-1, and n = 1 gives the per-pair estimate. For comparison, the width of the R package bidask is n + 1, as it counts bars; the sl of Lopez de Prado’s Snippet 19.1 averages beta only and equals n only at 1; and a month of daily bars is about n = 20.

The value is a proportion of price in [0, 2) in exact arithmetic (float64 reaches 2 only for a bar range of a factor of about 3.6e16), where 0.01 is a spread of 1%; it is not a per-bar rate and is not annualised. Two identical bars, with the prior close inside them, give beta = 2 hl^2 and gamma = hl^2, hence alpha = hl and a value of 2 (h - l) / (h + l), all of the range read as spread; a high of 101 and a low of 99 give 0.02 to within 1e-14. The estimate is an input for trading costs and liquidity regimes; the sources give it no evidence as a source of returns.

Inputs h, l and c of unequal lengths give an empty Series, whatever n is. The three inputs are one bar set, so they are not right-aligned as the base and comparison series of the relative methods are. Any n < 1 or n >= len(h) gives len(h) zeros, as a value needs n + 1 bars.

Bars 0 .. n-1 are the warm-up and are +0, not NaN, and the first valid index is n. A value of 0 is also a legitimate reading, given by clipped pairs and flat bars, so mask the warm-up with SetN(n, math.NaN()) before reading 0 as no spread.

A high or low that is zero, negative, NaN or +Inf at bar p makes pairs p and p+1 NaN, and with them the values at bars max(p, n) .. p+n, as far as the series runs; such a close at bar p spoils pair p+1 only, and with it the values at bars max(p+1, n) .. p+n. Later values recover, every other value keeps its bits, the last close is not read, and a positive subnormal price is valid. Bars are not otherwise validated: a bar with h < l is computed as it is and gives a finite value that is not negative, and a close outside its own bar is accepted, as it is read only as the prior close of the next pair. A flat bar (h = l) gives exactly +0 for its pair when the other bar has h >= l, so a window of flat bars is +0; the paper’s substitutions for infrequent trading (its section II) are not applied.

Other variants are not provided. Lopez de Prado’s Snippet 19.1 (Advances in Financial Machine Learning, 2018), and mlfinlab after it, average beta over the window before forming alpha, clip after and make no gap adjustment, a different estimator for n > 1 that equals this one at n = 1 when every close lies within the next bar’s range. The CS of bidask averages unclipped estimates and by default returns the absolute value of the mean. The unadjusted form, without the gap adjustment, is not provided either, nor is the Corwin-Schultz volatility of Snippet 19.2 and the paper’s spread-adjusted variance estimator, where the book’s sqrt(8/pi) and the paper’s 1/(4 ln 2) give values that differ by a factor of about 1.0435.

Each value costs O(n) and is computed afresh from its own window, with no running sum, so the method is window-local and start-invariant: dropping leading bars leaves every later value’s bits unchanged. The arithmetic after the logarithms rounds every product on its own, but math.Log and math.Tanh can differ in the last bit between amd64 and arm64, and so can the values: by a few units in the last place in absolute terms, which the cancellation in alpha can make a large relative difference in a small estimate.

RollSpread

func (s Series) RollSpread(n int) Series

RollSpread returns Roll’s estimate of the bid-ask spread from the serial covariance of the receiver’s one-bar changes over the trailing n pairs of changes. It is the measure of Roll 1984 (“A Simple Implicit Measure of the Effective Bid-Ask Spread in an Efficient Market”, Journal of Finance 39(4)): bid-ask bounce makes successive changes negatively correlated, with cov(d[t], d[t-1]) = -S^2/4 for a spread S, so that S = 2 sqrt(-cov).

With d[t] = s[t] - s[t-1], the value at bar t is 2 sqrt(max(0, -cov)), where cov is the population covariance, divisor n, of the n pairs (d[i], d[i-1]) for i = t-n+1 .. t. It is computed through the composition d := s.Delta(1); cv := d.Covariance(d.Shift(1), n), and from bar n+1 it equals s.Delta(1).Covariance(s.Delta(1).Shift(1), n).Mul(-1).Max(0).Sqrt().Mul(2) bit for bit.

The value is the full spread, as in Roll, in Corwin and Schultz, in bidask and in mlfinlab; the c of Lopez de Prado’s Advances in Financial Machine Learning (2018), section 19.3.2, is the half-spread, half of this value. A window whose covariance is positive gives 0, following the rule that positive correlations imply zero spreads, rather than the 2 sqrt(|cov|) of mlfinlab and of bidask’s default or a signed root.

The bidask and mlfinlab packages use the sample covariance, divisor n-1: where cov < 0 their value is sqrt(n/(n-1)) times this one, and where cov > 0 they give 2 sqrt(|cov|) against 0 here. No sample sibling is provided. Roll’s efficient-price variance is not exported either; with cv as above it is d.StdDev(n).Squared().Add(cv.Mul(2)), valid from bar n+1.

The receiver is used as given and the value is in the units of its changes: raw prices give a spread in price units, and s.Log() gives a proportional spread, bidask’s form and the percentage form of Roll’s own estimates, in which 0.01 is 1%. The estimate is an input for trading costs and liquidity regimes; the sources give it no evidence as a source of returns.

A bounce s[t] = m + c(-1)^t, with c > 0 and a full spread of 2c, gives, in exact arithmetic, 4c for even n, twice that spread, because an alternation is not Roll’s independent trade direction, and 4c sqrt(1 - 1/n^2) for odd n; with c = 0.5 the value is 2 for even n and 1.8856180831641267 for n = 3.

Any n < 2 or n > len(s)-2 gives len(s) zeros, as n pairs of changes need n + 1 changes and so n + 2 bars. Bars 0 .. n are the warm-up and are +0, not NaN, and the first valid index is n+1. The method writes those zeros itself, because the composition reads Delta’s 0 at bar 0 and Shift’s 0 pad there. A value of 0 is also a frequent legitimate reading, a quarter to a half of the values of a random walk, wherever the window’s covariance is positive, so mask the warm-up with SetN(n+1, math.NaN()) before reading 0 as no spread.

A NaN or an infinity at bar p makes the changes d[p] and d[p+1] non-finite, and with them the values at bars max(p, n+1) .. p+n+1 NaN; later values recover, and every other value keeps its bits. A change that overflows to an infinity does the same for every window that holds it. Any finite value is valid: zero and negative raw prices are computed as they are, while s.Log() turns them into an infinity or NaN, which gives that footprint. A window whose sum of products exceeds the float64 range gives a covariance of +Inf or -Inf, as Covariance documents, and so a value of 0 or +Inf.

Each value costs O(n) and depends only on its own window, so the method is window-local and start-invariant. Covariance rounds every product on its own and math.Sqrt is correctly rounded, so the bits are the same on every platform for the same input; s.Log() calls math.Log, which can differ in the last bit between amd64 and arm64.

Compositions

The price-impact measures below mask their values with NaN before their first valid index, where bar 0 has no return and the windows are partial, and a zero-volume or unchanged window that would divide by zero is NaN, not the 0 that Div gives. Prices are positive and finite, volumes and dollar volumes non-negative and finite, and lengths are in bars; the measures are in the units of the dollar volume passed in.

Amihud’s illiquidity and the Amivest liquidity ratio (Amihud; Fosback)

Yakov Amihud (“Illiquidity and stock returns: cross-section and time-series effects”, Journal of Financial Markets 5, 2002, pp. 31-56, eq. (1)) defines illiquidity as “the average ratio of the daily absolute return to the (dollar) trading volume on that day”, “the daily price response associated with one dollar of trading volume”, and relates it to the Amivest measure, “the ratio of the sum of the daily volume to the sum of the absolute return”. Norman Fosback’s liquidity beta (Stock Market Logic, 1976, ch. 62, pp. 244-245), “the number of dollars of volume that has historically been required to move a stock up or down 1%”, is that ratio with the change in percent. t6Illiquidity returns the mean over n bars of |r|/DV, r the log return from LogRatio, and the ratio of the n-bar sums of DV and of the absolute percent changes, in dollars per 1%.

r := c.LogRatio(c.Shift(1))
illiq := r.Abs().Div(dv).MaskNonFinite(dv.Log()).Average(n).SetN(n, math.NaN())
mv := c.RateOfChange(1).Abs().Sum(n)
liq := dv.Sum(n).Div(mv).MaskNonFinite(mv.Log()).SetN(n, math.NaN())

Inputs: closes; the bar’s dollar volume dv (c.Mul(v) is the proxy; a vendor’s dollar volume where one exists); n bars (Amihud averages over a year; mlfinlab’s code uses 20 and Jansen’s 21).
First valid index: n for both, the value of the masks; LogRatio is NaN at bar 0, where Log().Delta(1) gives a fake 0 return. MaskNonFinite(dv.Log()) makes the ratio NaN on a zero-volume bar (its log is -Inf), and on an infinite dollar volume or a NaN close too, so the n windows holding it are NaN in illiq, where Div alone would count a 0, and MaskNonFinite(mv.Log()) makes a window of unchanged closes NaN in liq, where Div would read no liquidity.
Anchors: on openquant’s hand vector (closes 100, 110 and 99, dollar volume 1e6, n = 2) illiq is (ln 1.1 + |ln 0.9|)/2e6 to rounding of the logarithm and liq is 99,999.99999999997; LogRatio equals Log().Delta(1) bit for bit from bar 1. With the simple return R, the percent change over 100, 1/(100 liq) is the dollar-volume-weighted mean of the daily ratios |R|/DV, rr.Div(dv).VWMA(dv, n) for rr = |R|, to rounding on windows with volume on every bar (a zero-volume bar drops out of the VWMA form and stays in the ratio of sums), where illiq is their equal-weighted mean, so the two measures have no exact identity.
Traps: Amihud’s return is CRSP’s simple daily return; c.RateOfChange(1).Div(100) in place of r changes the measure by the second-order term R^2/2 and reads 0, not NaN, at bar 0.
Variant: Amihud’s scale is illiq.Mul(1e6), after the average (his “multiplied by 10^6”; 1e6 is not a power of two, so scaling before the average differs in the last bit).
Variant: over the last n bars with volume, Amihud’s “days for which data are available” read as a count of trading days rather than a calendar window: r.Abs().Div(dv).AverageWhere(dv.GT(0), n), held on a zero-volume bar.
Evidence: Amihud’s cross-section (NYSE stocks, 1963 to 1996) supports the measure’s premium, not a trading rule, and Fosback states his view without data (“Experience shows that, in fact, they do just that”). Evidence, not a reference.
Pinned by TestRecipeT6Illiquidity.

Kyle’s and Hasbrouck’s lambdas on bars (Lopez de Prado)

Marcos Lopez de Prado (Advances in Financial Machine Learning, Wiley, 2018, ch. 19, pp. 286-290) estimates Kyle’s lambda by regressing the price change on the signed volume b_t V_t, and Hasbrouck’s by regressing the log return on the signed square root of dollar volume, both without an intercept, and finds “that the t-values associated with these microstructural estimates are more informative than the (mean) estimates themselves”. t6PriceImpact fits both by BetaOrigin over n bars and composes Kyle’s t-value with n - 1 degrees of freedom from window sums, the book’s coefficient over the square root of its variance (Snippet 17.4, p. 259). The trade sign b is an input: the book’s tick rule on closes is b := c.Delta(1).Hysteresis(0, 0) (0 until the first change; its “b_0 is arbitrarily set to 1” is c.Delta(1).SetN(1, 1).Hysteresis(0, 0)), and a vendor’s net volume sv gives sv.Div(sv.Abs()).

r := c.LogRatio(c.Shift(1))
dp := c.Delta(1)
x := b.Mul(v)
kyle := dp.BetaOrigin(x, n).SetN(n, math.NaN())
sxy := dp.Mul(x).Sum(n)
t := sxy.Mul(math.Sqrt(float64(n-1))).Div(x.Squared().Sum(n).Mul(dp.Squared().Sum(n)).Sub(sxy.Squared()).Sqrt()).
    SetN(n, math.NaN())
hasb := r.BetaOrigin(b.Mul(dv.Sqrt()), n).SetN(n, math.NaN())

Inputs: closes; the trade sign b (+1, -1 or 0); volumes and dollar volumes; n >= 2 bars (mlfinlab’s 20). A zero-volume bar is a zero regressor row, which drops out of a fit through the origin, so the lambdas need no mask there.
First valid index: n for all three, the value of the masks, so that each window holds n real changes; hasb is NaN while a window holds bar 0’s NaN return.
Anchors: kyle equals the ratio of window sums dp.Mul(x).Sum(n).Div(x.Squared().Sum(n)), here bit for bit, and on openquant’s hand vector of price changes 0.2, -0.1 and 0.4 on signed volumes 10, -5 and 20 the slope is 0.02; the t-value is within 1e-14 relative of exact rational sums away from a perfect fit.
Traps: with b from the bar’s own close change both lambdas are sign-free: kyle is dp.Abs().BetaOrigin(v, n) and hasb is r.Abs().BetaOrigin(dv.Sqrt(), n) bit for bit once b is set, so they measure the size of a move per unit of volume; on bars the book’s b is “the volume-weighted average of the trade signs” (exercise 19.5). The t-value loses accuracy as the fit nears perfection: an exact fit on integers gives 0 (Div by an exact 0) where the t-value is infinite, and a near-perfect one in binary64 a huge finite value, such as 88,080,384 on the hand vector, or NaN; realistic windows sit far from that (an uncentred r-squared of at most 0.83 on mlfinlab’s E-mini dollar bars). Hasbrouck’s regressor b sqrt(DV) stands in for the book’s within-bar sum of b sqrt(pV).
Variant: mlfinlab’s bar forms are means of ratios, dp.Abs().Div(v).Average(n) for Kyle and r.Abs().Div(dv.Sqrt()).Average(n) for Hasbrouck, and its “t-value” is the coefficient over its variance, not over its standard error.
Variant: the book’s Amihud regression, r.Abs().BetaOrigin(dv, n).SetN(n, math.NaN()).
Evidence: the book’s histograms of the three lambdas on E-mini S&P 500 futures (Figures 19.1 to 19.3) state no window or bar, and its preference for t-values is the author’s observation (“Although the literature does not mention this observation”). Evidence, not a reference.
Pinned by TestRecipeT6PriceImpact; runnable example ExampleSeries_BetaOrigin_kyleLambda.

Volatility Estimators SuperTrend