Relative: Correlation

BandCorrelation

func (s Series) BandCorrelation(s2 Series, n, period int, bw float64) Series

BandCorrelation returns the rolling n-bar correlation of s and s2 after each is passed through a band-pass filter centred on period bars with fractional bandwidth bw.

Each input’s overlap is passed through Ehlers’s band-pass filter, BandPass(period, bw), as printed in his Stocks & Commodities articles (for example “Truncated Indicators”, July 2020). The value at a bar is the Pearson correlation of the trailing n filtered values. It uses the shared co-moment kernel, with the filtered s2 as the regressor and the filtered s as the response. Take the overlap views a of s and b of s2, and let fy = a.BandPass(period, bw) and fx = b.BandPass(period, bw). From overlap index n+1 on, the value equals fy.Correlation(fx, n) bit for bit.

s is the base and s2 the comparison; the inputs are right-aligned at their most recent bar, only the trailing min(len(s), len(s2)) bars are paired, the result has len(s), and bars before the overlap are zero.

The first valid index, counted from the start of the overlap, is n+1, and the minimum overlap is n+2. The first valid index is the first window that excludes indices 0 and 1, where BandPass is structurally zero. Any of the following gives a series of len(s) zeros, in O(len(s)) time whatever the size of n: an empty overlap, n < 2, period < 1, a NaN or infinite bw, or an overlap shorter than n+2. Every period of at least 1 is valid, however large, and costs O(len(s)) time.

The filter is recursive, so it is not start-invariant. Each value depends on every bar of the overlap before it, and moving the start of the overlap changes it.

The output lies in [-1, 1], and a window in which either filtered series is flat gives exactly 0. The start-up transient of the filter is not masked; it decays by about exp(-pibw) per period, so discard about 2period/bw bars (about 7 periods at bw = 0.3, 20 at bw = 0.1).

BandPass reads every input from index 0 on. A NaN or an infinity at overlap index p therefore makes every output from max(p, n+1) on NaN, because the filter carries it forever.

s.BandCorrelation(s, n, period, bw) is exactly 1 wherever the filtered window is not flat. The bandwidth bw is a fraction of the centre frequency, typically 0.1 to 0.5 (Ehlers’s Stocks & Commodities listings use 0.1 to 0.25). Where cos(2pibw/period) < 0, that is for period/4 < bw < 3*period/4 give or take whole periods, the coefficient is below -1 and the filter is unstable; the outputs there are NaN or meaningless.

Correlating two markets within one cycle band isolates their co-movement at that period. Use it to compare them at a chosen cycle length, free of the trends and noise outside the band.

Verification: two independent references agree with it; it equals Correlation of the BandPass series from the first valid index on, bit for bit; and it is exactly 1 for s2 = s.

Correlation

func (s Series) Correlation(s2 Series, n int) Series

Correlation returns the rolling Pearson correlation of s with s2 over the trailing n right-aligned bars.

It is the correlation coefficient of Markos Katsanos (Intermarket Trading Strategies, 2008, equations 2.1 to 2.3) and MetaStock’s Correl: on each bar, the Pearson correlation Sxy/sqrt(Sxx*Syy) of the n most recent paired values of s and s2, with the sums centred within the window.

The first valid index is len(s) - min(len(s), len(s2)) + n - 1 and the minimum overlap is n. An empty overlap, n < 2 or n > min(len(s), len(s2)) gives a series of len(s) zeros, in O(len(s)) time whatever the size of n.

Pair j of the overlap is s[len(s) - min(len(s), len(s2)) + j] with s2[len(s2) - min(len(s), len(s2)) + j], and the window ending at pair j holds pairs j-n+1 to j. Any finite level of either sign is valid. The kernel uses centred, power-of-two-scaled, window-local sums: each variable is anchored at the first value of its window and scaled by a power of two before the deviations are formed, so levels near 1e9 or 1e300 are safe, the powers of two cancel in the ratio, and the result is bitwise start-invariant (a bar’s value does not depend on where the series begins). The correlation is symmetric, so s.Correlation(s2, n) equals s2.Correlation(s, n) bit for bit on equal lengths, and it is invariant under multiplying either input by a power of two, provided no product overflows or rounds in the subnormal range.

The result lies in [-1, 1]. A flat window, in which either variable is constant over the n bars, gives exactly 0. At n = 2 the value is the product of the signs of the two one-bar changes, exactly 1, -1 or 0, where the general formula rounds to ±0.9999999999999999 on about one random window in nine. A NaN or an infinity at pair p gives NaN at exactly the windows containing it, pairs p to p+n-1 from the first valid pair on (fewer than n near either end of the overlap), and later outputs recover. Before the first valid index the output is 0, so a fixed-threshold gate such as r < 0.5 or r < 0.8 is true on every warm-up bar and must be masked, for example with Window(-f) where f is the first valid index, or by starting the test there. Forward-filled bars (the Epps effect) bias the value towards 0; align the two inputs by date before calling.

Correlations of trending levels can be spurious (George Udny Yule; Clive Granger and Paul Newbold): Katsanos’s Table 6.1 shows the DAX against the lagged S&P 500 at 0.913 on prices but 0.134 on daily returns and 0.661 on weekly returns, so ReturnCorrelation is the recommended default and Correlation is for readers who want the level relationship itself. His Chapter 16 filter Correl(C,SEC2,130,0)<.5 is Correlation(s2, 130), which needs a minimum overlap of 130, and the Correlation(C,IND,126) of his RSMK system needs 126. Published level windows and gates in Stocks & Commodities: 30 days (Thom Hartle, and the editor’s sidebar to Michael Turner’s 1999 article), 40 bars with a gate below -0.40 (Katsanos, gold against European bank futures), 52 weeks with ±0.5 as “strong” (Katsanos), 63 days and two years (Jim Bianco), and 50 months of monthly averages (Eric Sharp); Cassandra Wang plots a 13-week correlation of XLE with SPY on adjusted weekly closes (January 2018), s.Correlation(spy, 13) on true weekly bars, not a strided daily series. Hartle’s point is that a turn in the correlation is the signal, not its level. The sign of a correlation may choose the orientation of a spread (long/long for inverse pairs, John Murphy and David Hirschfeld) or the inverted scale of an overlay (Roger Pilloton), but the library does not switch automatically.

Deviations from the published code: the published raw-sum formula (SumXY - nXbarYbar)/sqrt(…) cancels at price levels (Bianco’s ten bond and CRB closes lose about four of their digits to it) and the centred kernel does not; the n = 2 sign rule replaces the formula on two-bar windows; and the output before the first valid index is zero rather than empty.

Compositions that start from it are listed under “Compositions” in the “Relative: Correlation” category.

Verification: two independent references, a Python reference in exact rational arithmetic and a Julia reference, and the exact identities with existing methods: s.Correlation(s, n) is 1 and s.Correlation(s.Mul(-1), n) is -1 on windows that are not flat, the Pearson correlation of each window’s fractional ranks equals SpearmanRankCorrelation(n, s2) on equal-length finite inputs for n up to about 300,000, where Spearman’s plain sums of ranks are exact, and sign(s.Correlation(ramp, n)) equals sign(s.Slope(n)) for the ascending unit-step ramp c, c+1, c+2 and so on, wherever Slope’s result does not underflow (it is normal, or the exact slope is zero).

CorrelationBreak

func (s Series) CorrelationBreak(s2 Series, n, n2 int) Series

CorrelationBreak returns a Fisher z test statistic comparing the correlation of s and s2 over the latest n bars with that over the n2 bars before them.

At each bar the Pearson correlation ra of the latest n paired bars and the correlation rb of the n2 paired bars before them are computed exactly as Correlation computes them, with one kernel of capacity max(n, n2); each is transformed to Fisher’s z with the Fisher method’s expression, the correlation clamped to [-0.999, 0.999] and then ln((1+r)/(1-r))*0.5 taken; and the output is the difference of the two z values divided by the standard error se = sqrt(1/(n-3) + 1/(n2-3)) of Fisher (1921), evaluated as (float64(fisherZ(ra)) - float64(fisherZ(rb)))/se with se computed once per call. Under a constant correlation of independent bivariate normal pairs the statistic is approximately standard normal, so a magnitude beyond about 2 flags a change in the relationship at roughly the five per cent level; heavy-tailed returns inflate its variance, about threefold for Student’s t with three degrees of freedom.

The first valid index, counted from the start of the overlap, is n+n2-1, and the minimum overlap is n+n2. An empty overlap, n < 4, n2 < 4 or n+n2 > min(len(s), len(s2)) gives a series of len(s) zeros, in O(len(s)) time whatever the sizes of n and n2; the last test never forms the sum n+n2, so no argument, math.MaxInt included, can overflow.

Each output depends only on its own paired window of n+n2 bars, the latest n and the n2 before them, so the series is start-invariant: dropping leading bars of both inputs leaves the later outputs unchanged bit for bit. Any finite data is valid, and both correlations are centred within their windows, so levels, returns and back-adjusted prices all serve.

A window holding a NaN or an infinity in either input gives a NaN correlation and a NaN output, so a non-finite value at overlap index p makes NaN exactly the outputs from p to p+n+n2-1 that lie at or after the first valid index, and later outputs recover. A flat window gives a correlation of 0 and a z of 0. CorrelationBreak(s, s, n, n2) is exactly 0 wherever neither window is flat, both correlations being 1 and clamped to 0.999 by the transform, and wherever both are; with one flat window it is 0.5ln(1999)/se, about 3.8002/se, or its negation. The transform’s clamp to [-0.999, 0.999] bounds each z by 0.5ln(1999), so no output overflows.

With overlapping k-bar returns the bars are not independent and the effective sample size is smaller than n, about n_eff = 3kn/(2k*k+1) for uncorrelated returns, so multiply the output by se/sqrt(1/(n_eff-3) + 1/(n2_eff-3)) before reading it as a z score. Murphy, Intermarket Analysis (Wiley 2004), chapter 15, pages 238 to 239, warns that intermarket correlations lapse and sometimes change for good, as bonds and stocks decoupled after 1997; the test itself is original to this library. The fast-in-slow nested form is the composition s.Correlation(s2, nf).Fisher().Sub(s.Correlation(s2, ns).Fisher()).Div(se): its windows share bars, so the two z values are correlated and se overstates the spread of their difference, and it reads as a scale rather than a test. Published practice judges a relationship by the stability of block correlations rather than by a break test: Saitta’s mean and mean absolute deviation of five four-year block correlations (Stocks & Commodities 18:7), Katsanos’s calendar-year tables and nested trailing windows (Stocks & Commodities volumes 30:1 and 38:9), and Lederman’s whole-period return correlation of 0.823 hiding calendar-year values from 0.225 to 0.941 (Stocks & Commodities 14:8). The rolling form of that practice is a composition: sample s.Correlation(s2, n) with Value(0), Value(n), Value(2n) and so on and take the mean and the mean absolute deviation in the consumer; calendar-year blocks need timestamps and belong upstream of nseries.

Verification: two independent references, a Python reference and a Julia reference, each in exact rational arithmetic with a high-precision square root and logarithm, agree with it; and it equals s.Correlation(s2, n).Fisher().Sub(s.Correlation(s2, n2).Fisher().Shift(n)).Div(se) bit for bit after warm-up.

CorrelationTrend

func (s Series) CorrelationTrend(n int) Series

CorrelationTrend returns Ehlers’s Correlation Trend Indicator, the correlation of the trailing n values with a straight line rising over the same bars.

It is the Pearson correlation of each n-bar window of s with the ramp 0, 1, …, n-1. It uses the shared co-moment kernel of Correlation, with the ramp as the regressor and the window of s as the response. The source is Ehlers, “Correlation As A Trend Indicator”, Stocks & Commodities, May 2020. His code correlates the closes with a line. Its Y = -count over Close[count] pairs the same values in reverse order, with the line shifted by a constant, which leaves the correlation unchanged. Let t be the unit ramp t[i] = i, of the same length as s. The value equals s.Correlation(t, n) bit for bit, because the kernel anchors each window of t at its first value and so sees exactly 0 to n-1.

The first valid index is n-1, and the minimum length is n. An empty s, n < 2 or n > len(s) gives a series of len(s) zeros. That takes O(len(s)) time whatever the size of n.

It is window-local and start-invariant: each value depends only on the n bars of its own window.

The output lies in [-1, 1]. A flat window gives exactly 0, which is the rule of the correlation family; Ehlers’s code holds the previous value instead where its raw sums show no variance; in floating point they often do show a little, and it then prints a value near 0. A rising line gives exactly 1, and a falling one exactly -1, wherever the kernel reads the window as a line, for example on integer ramps. At n = 2 the value is the sign of the latest change. A NaN or an infinity at index p makes the outputs on [p, p+n-1] NaN, clipped to the valid bars from n-1 to len(s)-1, and later outputs recover.

Values near +1 mean a steady rise, values near -1 a steady fall and values near 0 no linear trend. Unlike Slope, it is scale-free. Its sign is that of Slope(n) wherever Slope’s result does not underflow, and its square is R2(n) up to rounding. For a pair of series, s.LogRatio(s2).CorrelationTrend(n) gives the trend of the relationship line.

Verification: two independent references agree with it, and a translation of Ehlers’s published code agrees to within 1e-9 off flat windows; it equals s.Correlation(t, n) against a unit ramp t bit for bit; and its sign is that of Slope(n) and its square is R2(n) up to rounding.

Covariance

func (s Series) Covariance(s2 Series, n int) Series

Covariance returns the rolling population covariance of s and s2 over the trailing n right-aligned bars.

It is the mean product of the centred deviations of the n most recent paired values, the sum over the window of (s - mean(s))*(s2 - mean(s2)) divided by n, computed by the shared co-moment kernel behind Correlation: each variable is anchored at the window’s first value and scaled by a power of two before its deviations are formed, and the output is the kernel’s scaled cross sum restored to real units with math.Ldexp and then divided by float64(n), in that order. The anchoring spares levels near 1e9 or 1e300 the cancellation of a raw-sum formula, and the result is within a few ulps of the window’s absolute co-moment scale. Katsanos, Intermarket Trading Strategies (Wiley 2008), equation 2.2 on page 17, divides by n-1 instead, the sample covariance; multiply by n/(n-1) for it.

The first valid index, counted from the start of the overlap, is n-1, and the minimum overlap is n. An empty overlap, n < 2 or n > min(len(s), len(s2)) gives a series of len(s) zeros, in O(len(s)) time whatever the size of n.

Each output depends only on its own paired window of n bars, so the series is start-invariant: dropping leading bars of both inputs leaves the later outputs unchanged bit for bit. Any finite data is valid, including negative levels and back-adjusted prices, because the deviations are centred within the window. The method is symmetric: s.Covariance(s2, n) equals s2.Covariance(s, n) bit for bit on inputs of equal length.

A window holding a NaN or an infinity in either input gives NaN, so a non-finite value at overlap index p makes NaN exactly the outputs from p to p+n-1 that lie at or after the first valid index, and later outputs recover. A window in which either variable is flat gives exactly 0. The co-moment sum is restored to real units before the division by n, so a window whose sum of products exceeds the binary64 range, n times the covariance, gives an infinity of its sign even where the covariance itself would fit: deviations of ±1e154 at n = 2 give +Inf.

The covariance is the unnormalised correlation, in the product of the two inputs’ units, and the numerator of the slope of s regressed on s2 and of the beta of s against s2. Katsanos’s Table 2.2, fifteen days of percentage changes of gold and the dollar index, gives a population covariance of -0.1931488, printed there as the sample value -0.207.

Verification: two independent references, a Python reference and a Julia reference, both in exact rational arithmetic, agree with it; it equals the shared kernel’s covariance, the kernel behind Correlation, whose correlation and slope are tied to the existing R2 and Slope; the ramp t = 0, 1, …, n-1 gives exactly float64(nn-1)/12 against itself and float64(nn-1)/4 against 3t+7; and Katsanos’s Table 2.2 gives -0.1931488.

DeltaCorrelation

func (s Series) DeltaCorrelation(s2 Series, n, k int) Series

DeltaCorrelation returns the rolling Pearson correlation of the k-bar price changes of s and s2 over the trailing n changes, which is valid for spreads, rates and back-adjusted futures.

It is the difference form of the return correlation of Markos Katsanos (Intermarket Trading Strategies, 2008): each input is first reduced to its k-bar change v[j] - v[j-k], the expression of Momentum and Delta, and the two change series are then correlated as Correlation does, over n right-aligned changes, with Sxy/sqrt(Sxx*Syy) on sums centred within the window. After the warm-up it is the Momentum(k) composition, s.Momentum(k).Correlation(s2.Momentum(k), n), which untrimmed contaminates the first windows with Momentum’s leading zeros.

The first valid index is len(s) - min(len(s), len(s2)) + n + k - 1 and the minimum overlap is n + k, so the earliest change in the first window is a real one. An empty overlap, n < 2, k < 1, k >= min(len(s), len(s2)) or n > min(len(s), len(s2)) - k gives a series of len(s) zeros, in O(len(s)) time whatever the size of n or k.

Pair j of the overlap is s[len(s) - min(len(s), len(s2)) + j] with s2[len(s2) - min(len(s), len(s2)) + j]; the change at pair j is the difference from pair j-k, and the window ending at pair j holds the changes of pairs j-n+1 to j. Any finite values are valid, of either sign and through zero: the difference of an arbitrary additive level is well defined, so this is the form for Panama back-adjusted futures, spreads, rates and other zero-crossing series, where a percentage means nothing. A difference that overflows gives an infinity, as Delta does, and hence NaN. The kernel uses centred, power-of-two-scaled, window-local sums, so changes from levels near 1e9 or 1e300 are safe and the result is bitwise start-invariant (a bar’s value does not depend on where the series begins).

The result lies in [-1, 1]. Flat prices give all-zero changes, and a window in which either change series is flat gives exactly 0. At n = 2 the value is the product of the signs of the two one-bar changes in the changes, exactly 1, -1 or 0. A NaN or an infinity at pair p makes the changes at pairs p and p+k NaN, so from the first valid index on the output is NaN at exactly the windows containing them, pairs p to p+n-1 and p+k to p+k+n-1 (fewer near either end of the overlap; one run when k <= n, two when k > n), and later outputs recover; a zero price makes nothing NaN. Before the first valid index the output is 0, so a fixed-threshold gate such as r < 0.5 or r < 0.8 is true on every warm-up bar and must be masked, for example with Window(-f) where f is the first valid index, or by starting the test there. Forward-filled bars (the Epps effect) bias the value towards 0; align the two inputs by date before calling.

No source publishes windows or thresholds for difference correlation; the return-correlation windows, 52 weekly changes of one bar or 50 changes of three bars, are reasonable starting points, but thresholds such as Katsanos’s 0.8, which he fitted to percentage returns, should be fitted again. The changes are in the units of each input, but the correlation does not depend on those units, so a spread in points and a rate in basis points compare directly, and a correlation of trending levels can still be spurious, for the same reasons as for Correlation.

Deviations from the published code: the difference in place of the percentage return, so a zero or negative base is ordinary data rather than a division hazard; NaN only from a non-finite price or an overflowing difference; and zeros before the first valid index.

Verification: two independent references, a Python reference in exact rational arithmetic and a Julia reference, and the exact identity with existing methods: s.Momentum(k).Correlation(s2.Momentum(k), n) from the first valid index on equal-length finite inputs, with DeltaCorrelation(s, s.Mul(-1), n, k) at -1 on windows whose changes are not flat.

DetrendedCorrelation

func (s Series) DetrendedCorrelation(s2 Series, n int) Series

DetrendedCorrelation returns the rolling correlation of s and s2 over n bars after removing each series’ linear trend within the window.

Within each paired window of n bars it is the partial correlation of s and s2 given time: with rab the Pearson correlation of s2 and s, rat that of the ramp t = 0, 1, …, n-1 and s, and rbt that of the ramp and s2, each computed exactly as Correlation computes it, the output is (rab - ratrbt)/sqrt((1 - ratrat)(1 - rbtrbt)) clamped to [-1, 1]. Go evaluates it as num := rab - float64(ratrbt), then rad := (1 - float64(ratrat))(1 - float64(rbtrbt)), and then math.Max(-1, math.Min(1, num/math.Sqrt(rad))), with 0 when rad <= 0. In exact arithmetic this equals the Pearson correlation of the two series’ within-window least-squares residuals on time (Yule 1907; for detrending, Frisch and Waugh 1933 and Lovell 1963).

The first valid index, counted from the start of the overlap, is n-1, and the minimum overlap is n. An empty overlap, n < 3 or n > min(len(s), len(s2)) gives a series of len(s) zeros, in O(len(s)) time whatever the size of n.

Each output depends only on its own paired window of n bars, so the series is start-invariant: dropping leading bars of both inputs leaves the later outputs unchanged bit for bit. Any finite data is valid, including negative levels and back-adjusted prices, because every correlation is centred within the window. The method is symmetric, s.DetrendedCorrelation(s2, n) equals s2.DetrendedCorrelation(s, n) bit for bit on inputs of equal length, and every output that is not NaN lies in [-1, 1].

The float64 conversions are rounding barriers: they make DetrendedCorrelation(s, s, n) exactly 1 on every window where the kernel’s correlation of s with time is not exactly 1 or -1, because there num equals the value q = 1 - float64(ratrat), rad is float64(qq) and math.Sqrt returns exactly q for it, on every architecture and with or without fused multiply-add. A window where the kernel’s correlation of either series with time is exactly 1 or -1 gives rad = 0 and the output 0, and a flat window gives 0. That set is not quite the set of exact lines: a window within rounding of a line can read as one (0 where the exact value is 1), and a long exact line whose slope has many significant bits can miss it (1 against itself where the exact value is 0). A window holding a NaN or an infinity in either input gives NaN, so a non-finite value at overlap index p makes NaN exactly the outputs from p to p+n-1 that lie at or after the first valid index, and later outputs recover. No output overflows: the correlations are clamped and the quotient is clamped again.

It removes each series’ linear trend, which inflates the plain correlation of two trending series, the drift that makes any two rising markets look related, and so measures the co-movement of their departures from trend. Vigderhous’s SPRM controls for the market instead of time, a partial correlation given a third series, which is not provided. No existing method equals it.

Verification: two independent references agree with it, a Python reference that correlates explicit least-squares residuals in exact rational arithmetic with a high-precision square root, and a Julia reference from the three-correlation formula in exact rational arithmetic with a BigFloat square root; no identity reaches an existing method, so the alternative oracle is those exact references together with the exact self, symmetry and exact-line identities: DetrendedCorrelation(s, s, n) is exactly 1 where the kernel’s correlation of s with time is not 1 or -1, and exactly 0 through rad <= 0 where it is.

ReturnCorrelation

func (s Series) ReturnCorrelation(s2 Series, n, k int) Series

ReturnCorrelation returns the rolling Pearson correlation of the k-bar percentage returns of s and s2 over the trailing n returns.

It is the return correlation of Markos Katsanos (Intermarket Trading Strategies, 2008): each input is first reduced to its k-bar signed percent change, (v[j]/v[j-k] - 1)*100 on a positive base, which is RateOfChange’s exact expression, and (v[j]/(-v[j-k]) + 1)*100 on a negative one, the change over the magnitude of the base; the two return series are then correlated as Correlation does, over n right-aligned returns, with Sxy/sqrt(Sxx*Syy) on sums centred within the window.

The first valid index is len(s) - min(len(s), len(s2)) + n + k - 1 and the minimum overlap is n + k, so the earliest return in the first window is a real one. An empty overlap, n < 2, k < 1, k >= min(len(s), len(s2)) or n > min(len(s), len(s2)) - k gives a series of len(s) zeros, in O(len(s)) time whatever the size of n or k.

Pair j of the overlap is s[len(s) - min(len(s), len(s2)) + j] with s2[len(s2) - min(len(s), len(s2)) + j]; the return at pair j is the change from pair j-k, and the window ending at pair j holds the returns of pairs j-n+1 to j. The valid data are prices that do not cross zero, normally positive ones: the signed percent change gives numbers on negative prices, a zero or non-finite base gives NaN, never a fake zero return, and near a zero crossing the return is unbounded. A percentage of an arbitrary additive level means nothing, so Panama back-adjusted futures, spreads, rates and other zero-crossing series should use DeltaCorrelation. The kernel uses centred, power-of-two-scaled, window-local sums, so returns from levels near 1e9 or 1e300 are safe and the result is bitwise start-invariant (a bar’s value does not depend on where the series begins).

The result lies in [-1, 1]. Flat non-zero prices give all-zero returns (flat zero prices give NaN, every base being zero), and a window in which either return series is flat gives exactly 0. At n = 2 the value is the product of the signs of the two one-bar changes in the returns, exactly 1, -1 or 0. A NaN or an infinity at pair p makes the returns at pairs p and p+k NaN, so from the first valid index on the output is NaN at exactly the windows containing them, pairs p to p+n-1 and p+k to p+k+n-1 (fewer near either end of the overlap; one run when k <= n, two when k > n), and later outputs recover; a zero price at pair p makes only the return at pair p+k NaN, where it is the base. Before the first valid index the output is 0, so a fixed-threshold gate such as r < 0.5 or r < 0.8 is true on every warm-up bar and must be masked, for example with Window(-f) where f is the first valid index, or by starting the test there. Forward-filled bars (the Epps effect) bias the value towards 0; align the two inputs by date before calling.

Returns rather than levels are the recommended basis, because trending prices “can produce deceptively high correlations” (Katsanos, Stocks & Commodities, February 2026), where his weekly ReturnCorrelation(spy, 52, 1) is the measure. Other published parameterisations: his decoupling gate ReturnCorrelation(spy, 50, 3).Value() < 0.8 (July 2017); his book’s fifteen-day returns over 300 days in the regression divergence indicator; Perry Kaufman’s recommended band of roughly 0.30 to 0.70 for pairs (March 2014) and his 20- or 40-day return correlation above 0.50 (October 2021); on true weekly bars, ReturnCorrelation(spy, 13, 1) as the return form of Cassandra Wang’s 13-week level correlation of XLE with SPY; and Jerome Lederman’s level correlations of 0.83 to 0.955 against return correlations of 0.099 to 0.347 for the same indices. The composition a.RateOfChange(k).Window(-k).Correlation(b.RateOfChange(k).Window(-k), n) over Overlap differs in its domain policy (0 at a zero base, the signed base on a negative one) and, untrimmed, contaminates the first windows.

Deviations from the published code: the signed percent change in place of MetaStock’s and AmiBroker’s subtract-first ROC, the same number up to rounding on positive bases and the opposite sign on negative ones; NaN at a zero base; and zeros before the first valid index.

Compositions that start from it are listed under “Compositions” in the “Relative: Correlation” category.

Verification: two independent references, a Python reference in exact rational arithmetic and a Julia reference that also translates Katsanos’s published return-correlation code, and the exact identity with existing methods: s.RateOfChange(k).Correlation(s2.RateOfChange(k), n) from the first valid index on equal-length inputs whose bases are all finite and positive (at an infinite base RateOfChange gives -100 where this method gives NaN), with ReturnCorrelation(s, s.Mul(2), n, k) at 1 where s.Mul(2) is finite and self-correlation at 1, and ReturnCorrelation(s, s.Mul(-1), n, k) at -1, on windows whose returns are not flat.

RoofingCorrelation

func (s Series) RoofingCorrelation(s2 Series, n, hpLen, ssLen int) Series

RoofingCorrelation returns the rolling n-bar correlation of s and s2 after each is passed through a hpLen-bar high-pass filter and an ssLen-bar SuperSmoother.

Each input’s overlap is passed through a roofing filter in Ehlers’s sense (Cycle Analytics for Traders, Wiley, 2013): a two-pole high-pass filter, here HighPass(hpLen) in the Butterworth form of his later articles, then SuperSmoother(ssLen). Together they pass cycles between about ssLen and hpLen bars; his printed defaults are hpLen = 48 and ssLen = 10. The value at a bar is the Pearson correlation of the trailing n filtered values. It uses the shared co-moment kernel, with the filtered s2 as the regressor and the filtered s as the response.

Take the overlap views a of s and b of s2, and let fy = a.HighPass(hpLen).SuperSmoother(ssLen) and fx = b.HighPass(hpLen).SuperSmoother(ssLen). From overlap index n+2 on, the value equals fy.Correlation(fx, n) bit for bit. Correlating trending levels gives spurious correlation (Granger and Newbold, “Spurious regressions in econometrics”, Journal of Econometrics 2(2):111-120, 1974), and the high-pass stage removes those trends.

The first valid index, counted from the start of the overlap, is n+2, and the minimum overlap is n+3. The first valid index is the first window that excludes indices 0 to 2, where the filtered series are structurally zero. Any of the following gives a series of len(s) zeros, in O(len(s)) time whatever the size of n: an empty overlap, n < 2, hpLen < 1, ssLen < 1, or an overlap shorter than n+3. Every hpLen and ssLen of at least 1 is valid, however large, and costs O(len(s)) time.

The filters are recursive, so it is not start-invariant. Each value depends on every bar of the overlap before it, and moving the start of the overlap changes it.

The output lies in [-1, 1] for finite inputs whose second differences do not overflow, and a window in which either filtered series is flat gives exactly 0. The start-up transient of the filters is not masked: discard about 2hpLen bars when ssLen is well below hpLen, as with the defaults, and about 2(hpLen+ssLen) bars otherwise.

HighPass never reads the first value of its input. A NaN or an infinity at overlap index 0 of either input therefore changes nothing. One at overlap index p >= 1 makes every output from max(p, n+2) on NaN, because the filters carry it forever.

s.RoofingCorrelation(s, n, hpLen, ssLen) is exactly 1 wherever the filtered window is not flat. Take an integer-valued s and s2 = s + u*i + v, with integers u and v of moderate magnitude. The high-pass filter removes the line exactly, and the value is exactly 1 wherever the filtered window is not flat. An integer-valued ramp filters to exactly 0 and so gives 0.

Correlations of series that the filter has removed are meaningless, because only rounding residue is left. For example, a non-integer ramp such as 0.1*i+0.3 leaves a residue of about 1e-15 after HighPass. Two such residues can correlate at nearly -1 or 1, and at n = 2 exactly, by the sign rule.

Use it to measure how two markets move together within a band of cycles, free of the trends that dominate the correlation of their levels.

Verification: two independent references, which implement the library’s HighPass and SuperSmoother recursions, agree with it; it equals Correlation of the HighPass then SuperSmoother series from the first valid index on, bit for bit; and it is exactly 1 for s2 = s and for an integer series against itself plus an integer line.

SignCorrelation

func (s Series) SignCorrelation(s2 Series, n, k int) Series

SignCorrelation returns the rolling average over n bars of the product of the signs of the k-bar changes of s and s2, a concordance measure between -1 and 1.

At each pair j of the overlap from k onwards the k-bar changes are the differences between pair j and pair j-k of each input, the changes that Momentum(k) returns, and each is reduced to its sign, 1, -1 or 0. The output is the integer sum of the products of the two signs over the n most recent changes divided by n, float64(sum)/float64(n) in that order, so it lies in [-1, 1] in steps of 1/n: 1 when the two series moved the same way on every one of the n bars and -1 when they always moved opposite ways. A zero k-bar change in either series contributes 0, which shrinks the value on illiquid or forward-filled data.

The first valid index, counted from the start of the overlap, is n+k-1, and the minimum overlap is n+k. An empty overlap, n < 2, k < 1 or n+k > min(len(s), len(s2)) gives a series of len(s) zeros, in O(len(s)) time whatever the sizes of n and k; the last test never forms the sum n+k, so no argument, math.MaxInt included, can overflow.

Each output depends only on its own paired window of n+k bars, so the series is start-invariant: dropping leading bars of both inputs leaves the later outputs unchanged bit for bit. Any finite data is valid: the changes are differences, whose sign on positive levels matches that of simple and logarithmic changes, so the method is safe on back-adjusted futures and on negative levels, and scaling either input by a power of two leaves it unchanged unless a scaled value overflows or underflows. The n products are kept in an integer ring with their running sum and a count of the unusable changes in the window, so each bar costs O(1) and the sum is exact and does not drift.

A change is unusable when either of its two operands is not finite, and an output is NaN when any of its n changes in either series is unusable, so a non-finite value at overlap index p makes NaN exactly the outputs from p to p+n-1 and from p+k to p+k+n-1 that lie at or after the first valid index, and later outputs recover. An overflowing difference of finite values is an infinity with a sign and counts normally. A window in which either series never changes gives exactly 0.

For jointly normal changes with zero mean the Pearson correlation of the changes is approximately sin(pi*v/2), with v this value (Sheppard 1899 and Blomqvist 1950); a drift in either series biases the mapping upwards. With no zero changes it equals 2I - 1, where I is the concordance index with which Harding and Pagan (2002) compare business cycles, with the signs of k-bar changes in place of their dated cycle phases. The percent-agreement variant, the number of positive products divided by the number of non-zero products, is a composition: Sum(n) of the positive products divided by Sum(n) of the non-zero ones, which gives 0 when there are none. The phi coefficient of up and down changes, which corrects for unbalanced margins when one series rises far more often than the other, is not provided.

Verification: two independent references, a Python reference in exact integers and a Julia reference, agree with it exactly; on finite inputs it equals, bit for bit after warm-up, the composition that takes the signs of s.Momentum(k) and s2.Momentum(k) through GT(0) and LT(0), multiplies them, sums the products with Sum(n) and divides by float64(n); s2 = s gives the share of non-zero changes in the window and s2 = s.Mul(-1) its negation.

XCorrelation

func (s Series) XCorrelation(s2 Series, n int) Series

XCorrelation returns an exponentially weighted correlation of s with s2, seeded from the first n-bar window and then updated with smoothing factor 2/(n+1).

It is an exponentially weighted Pearson correlation: it applies the exponential weighting of RiskMetrics (RiskMetrics Technical Document, 4th edition, J.P. Morgan/Reuters, 1996, section 5.2, on exponentially weighted covariances and correlations) to deviations from exponentially weighted means, rather than to returns taken as zero-mean. The means and variances are updated as in West, “Updating mean and variance estimates: an improved method”, Communications of the ACM 22(9):532-535, 1979. Let alpha = 2/(n+1). Let x be the overlap of s2, which is the regressor, and y the overlap of s, which is the response.

At overlap index n-1, the seed bar, the shared co-moment kernel fits the first n-bar window, f := k.fit(x[:n], y[:n]). The means mx and my of the state are the fit’s real-unit means, f.xbar and f.ybar. The values vx, vy and c are its centred sums of squares and of cross-products, scaled back to real units, divided by n, that is, its population variances and covariance: vx = math.Ldexp(f.sxx/float64(n), 2f.ex), vy = math.Ldexp(f.syy/float64(n), 2f.ey) and c = math.Ldexp(f.sxy/float64(n), f.ex+f.ey). Dividing by n before scaling back gives the same bits as dividing afterwards wherever the result is a normal number, and keeps the moments finite for deviations up to the top of the float64 range, where the rebuilt centred sums would overflow; where the result is subnormal the last bits can differ, and accuracy is not claimed there. The seed bar’s value is the kernel’s correlation of that window, which is bit for bit the value of Correlation there. The seed bar emits that value directly because c/sqrt(vx*vy), formed from the seeded state, would not reproduce it bit for bit.

Each later bar updates the state in the Go operation order below. Each product dxdx, dydy and dxdy is formed on its own and only then scaled by alpha, so exchanging x and y, or negating one of them, gives every operation the same operands up to order and sign; this keeps the update symmetric in x and y where multiply-adds are fused, as on arm64, where each alpha(…) is fused with the addition before it. The float64 conversions only mark those roundings; no addition takes the products they wrap:

dx := x - mx
dy := y - my
mx += alpha * dx
my += alpha * dy
vx = (1 - alpha) * (vx + alpha*float64(dx*dx))
vy = (1 - alpha) * (vy + alpha*float64(dy*dy))
c = (1 - alpha) * (c + alpha*float64(dx*dy))

The value is 0 if vx <= 0 or vy <= 0. Otherwise it is c/sqrt(vxvy), clamped to [-1, 1]. When the product vxvy falls below the smallest normal number, 0x1p-1022 (a subnormal product has too few bits), or overflows to +Inf, the value is computed with one square root per variance instead, larger first, (c/sqrt(max(vx, vy)))/sqrt(min(vx, vy)), which does not depend on the order of the inputs, or as c/vx when the two variances are equal. At overlap index t, each bar after the seed bar carries the weight alpha*(1-alpha)^k, where k is its distance back from t. The seed window as a whole carries the remaining weight, (1-alpha)^(t-n+1).

The first valid index, counted from the start of the overlap, is n-1, and the minimum overlap is n. An empty overlap, n < 2 or n > min(len(s), len(s2)) gives a series of len(s) zeros. That takes O(len(s)) time whatever the size of n.

It is recursive, with infinite memory, so it is not start-invariant. Each value depends on every bar of the overlap before it, and moving the start of the overlap changes it. After the seed it costs O(1) time per bar.

The output lies in [-1, 1] while the deviations from the means stay between about 1e-162 and 1e154 in magnitude; below that the variances underflow and the output is 0, and above it a variance or the covariance overflows and later outputs are 0 or NaN. The clamp keeps [-1, 1] over that whole range, but accuracy to a few units in the last place holds only while both variances are normal numbers (deviations above about 1e-154). A NaN or an infinity at overlap index p makes every output from max(p, n-1) on NaN. Inside the seed window it spoils the seed; after the seed window it spoils the state, which never recovers.

The output is +0 at every bar, from the seed bar on, where the trailing n values of either input in the overlap are all equal. This is Correlation’s rule for a flat window, applied to the same n bars, so XCorrelation and s.Correlation(s2, n) are 0 by flatness on the same bars. As there, +0 and -0 count as equal, and a NaN takes precedence. The rule changes the output only: the state goes on updating through a flat stretch, and the value follows it again from the first bar where neither input’s trailing n values are all equal. When the rule first applies to a run that starts after the seed window, the flat bars carry 1-(1-alpha)^n of the weight, from about 89 per cent at n = 2 down to 1-e^-2, about 86.5 per cent, for large n. The value steps to 0 there from the recursion’s value, as Correlation’s does at a flat window: at n = 2 a single unchanged value gives 0, and a single move inside a long halt gives n-1 bars computed from it, then 0 again.

Without the rule, a flat stretch would show rounding noise in float64. The flat input’s deviation from its mean, its variance and the covariance shrink by the factor 1-alpha per bar at first, but the mean stops moving once alpha times the deviation is below half a unit in the last place of the mean, after about 15*(n+1) bars at ordinary levels, and the deviation freezes at up to about (n+1)/4 units in the last place. The rule applies once a value has repeated n-1 times, long before that. An input that is not exactly flat but moves only in its last bits, such as a derived series, or that decays towards 0 into the subnormal range, is outside the rule; there the value can become rounding noise anywhere in [-1, 1], so do not read meaning into it. After a long halt of both inputs the state holds little but the halt (after about 15*(n+1) bars, only rounding residuals), so the first bars after both resume are dominated by those bars alone: the first reads near -1 or 1, as Correlation’s first window after a halt of n-1 bars reads -1 or 1. Read the value from about n bars after a long halt.

s.XCorrelation(s, n) is exactly 1, and s.XCorrelation(s.Mul(-1), n) exactly -1, at every valid bar where the variances are positive and finite and the trailing n values of s are not all equal (both are 0 there), because the two states are the same expressions and equal variances give c/vx. s.XCorrelation(s2, n) equals s2.XCorrelation(s, n) bit for bit on inputs of equal length.

The X marks the exponential form, at the front of the name as in XAverage and XMA (BollingerX and IntermarketDisparityX put it at the end). Take the overlap views a of s and b of s2. Up to rounding, the means mx and my track b.XAverage(n) and a.XAverage(n), since all of them start from the first window’s mean. The centre of mass of the weights is (1-alpha)/alpha = (n-1)/2 bars back, so the effective memory is about n bars. Use it for a smooth correlation over long histories: after the seed each bar costs O(1) within a call, but each call recomputes the whole overlap. Compare Correlation for a hard n-bar window.

Verification: two independent references, a Python reference in exact rational arithmetic with high-precision square roots and a Julia reference, agree with it; it equals Correlation at the seed bar, and at every bar where the trailing n values of either input are all equal (both are 0); at ordinary levels self-correlation is 1, negation gives -1 and it is symmetric, bit for bit; and it tracks the closed-form exponentially weighted Pearson correlation.

Compositions

Katsanos’s correlation gates

Markos Katsanos uses a rolling correlation as a gate rather than a signal. His Chapter 16 asset-allocation entry (Intermarket Trading Strategies, 2008) requires the 130-bar correlation of the asset with the S&P 500 to be below 0.5 (Correl(C,SEC2,130,0) < .5), so that the rotation buys what moves independently of the benchmark. In “Trading The Nikkei” (Stocks & Commodities 35:8, 2017) his decoupling gate requires the 50-bar correlation of the 3-bar returns of DXJ and SPY to be below 0.8. Each is a comparison of the trimmed correlation, 1 where the gate is open and 0 where it is shut.

a, b := s.Overlap(s2)
ch16 := a.Correlation(b, 130).Window(-129).LT(0.5)
decouple := a.ReturnCorrelation(b, 50, 3).Window(-52).LT(0.8)

First valid index: 129 for the Chapter 16 gate and 52 for the decoupling gate, counted from the start of the overlap (in general n - 1 and n + k - 1). Minimum overlap: 130 and 53 bars.
Trim first: the untrimmed comparison reads the warm-up zeros as a correlation of 0 and opens the gate on every one of those bars.
Domain: the correlation’s. A window with a non-finite value gives a NaN correlation, as does, for the decoupling gate, a zero price, which is a zero base for its return; LT returns 0 for a comparison with NaN, so the gate is shut there. A correlation exactly equal to the threshold shuts the gate too: LT is strict, as Katsanos’s < .5 is. A flat window has a correlation of exactly 0, which opens it, except that flat zero prices give NaN in the decoupling gate and shut it.
Anchors: each gate equals the untrimmed method’s LT from the first valid index on (E); with s2 == s, the Chapter 16 gate is shut on every window that is not flat (the correlation is 1).
Pinned by TestRecipeCorrelationGates; runnable example ExampleSeries_Correlation_gates.

Percent agreement of signs

The share of bars on which the two legs moved the same way, among the bars on which both moved: the percent-agreement form of SignCorrelation. Saitta’s quarterly “same or opposite” direction share (Stocks & Commodities 16:10) is this reading on non-overlapping quarters.

a, b := s.Overlap(s2)
p := a.Momentum(k).Mul(b.Momentum(k)).Window(-k)
agree := p.GT(0)
agreement := agree.Sum(n).Div(agree.Add(p.LT(0)).Sum(n)).Window(-(n - 1))

First valid index: n + k - 1, counted from the start of the overlap; element e is overlap bar n + k - 1 + e. Minimum overlap: n + k bars.
Parameters: n >= 2 and k >= 1; n = 1 or k = 0 makes a trim Window(0), which returns an empty series.
Counting: a bar with a zero change in either leg counts in neither total, and a NaN change compares false both ways and also counts in neither, unlike SignCorrelation, which keeps zero changes in its n and is NaN while a non-finite change is in its window. A window with no non-zero product gives 0, because Div by zero gives 0.
Anchors: s2 == s gives exactly 1 on windows with a change of at least about 1.6e-162 in size (E); a product of changes smaller than about 2.5e-324 underflows to 0 and counts in neither total; s2 = -s gives 0 (E); on windows without zero changes or underflowing products it is (1 + SignCorrelation(s2, n, k))/2 within 1e-14; it lies in [0, 1].
Pinned by TestRecipeT3SignAgreement; runnable example ExampleSeries_Momentum_signAgreement.

Relative: Channels Relative: Regression and Spread