Endpoint FFT

EndpointFFTSlope

func (s Series) EndpointFFTSlope(n int, threshold float64) Series

EndpointFFTSlope returns the one-bar change at the newest point of a noise-filtered Fourier fit to the last n log prices. It is Dennis Meyers’s deltaEP from “The Endpoint Fast Fourier Transform System”, Stocks & Commodities V17:5 (May 1999), pp. 199-208, where EPFFT is his name for the method: the value of the fit at the newest bar of the window less its value at the bar before, in flattened log units, with the chord slope b not added back.

For the window ending at bar t, with x_j = ln s[t-n+1+j] for j = 0 to n-1, the method takes the log prices and end-flattens them as Meyers’s Equation 1 does: a = x_0, b = (x_{n-1} - x_0)/(n-1) and y_j = x_j - (a + bj), so that y_0 and y_{n-1} are 0 in exact arithmetic. It pads y with zeros to N = 2n points (Meyers pads 256 to 512) and takes the unscaled forward transform F_k, the sum over j of y_j e^{-2 pi i jk/N}, for k = 0 to n; the input is real, so F_{N-k} is the conjugate of F_k. Fmax is the largest magnitude of F_k over the harmonics k = 2 to n, the Nyquist harmonic included, as in his Equation 2. His noise filter (Equations 3A to 3C, after Donoho) keeps harmonics 0 and 1 and thresholds every other index on its own. This method decides on the half spectrum and mirrors each decision: it keeps the mirror N-1 of harmonic 1 with it, and for each k from 2 to n drops harmonic k and its mirror N-k together when the magnitude of F_k is at most thresholdFmax, and keeps both otherwise. So a tie drops (his Equation 3A keeps on a strict >, and the filter of his earlier article, Stocks & Commodities V17:4, zeroes on <=), and the masked spectrum stays conjugate-symmetric, so its inverse is real. The earlier article is “The Discrete Fourier Transform Illusion”, Stocks & Commodities V17:4 (April 1999), pp. 151-158. The masked inverse, with the factor 1/N, is taken at the two newest points only, yep at n-1 and yepm1 at n-2, and the result is yep - yepm1. The transform is a direct DFT over N = 2n points, so N need not be a power of two, and its result equals an FFT’s to rounding.

The published parameters are n = 256 and a threshold of 0.15. A threshold of 0 keeps every harmonic with a non-zero magnitude, so the inverse returns the flattened window and, from bar n-1 on, the result equals, to rounding, the one-bar log change less the chord slope, for a price series c:

c.Log().Delta(1).Sub(c.Log().Delta(n-1).Div(float64(n-1)))

A finite threshold of 1 or more keeps only harmonics 0 and 1 (with the mirror of 1), since no harmonic from 2 to n has a magnitude above Fmax, and every such threshold gives the same result as a threshold of 1. A threshold of -0 acts as 0.

The warm-up reads 0, not NaN: bars 0 to n-2 are 0 and the first valid index is n-1. Since 0 is also a value, trim the warm-up with Window(-(n-1)) or mask it with SetN(n-1, math.NaN()). The parameters n < 2 or n > len(s), or a threshold that is NaN, negative or infinite, give len(s) zeros. A price that is not finite and positive (NaN, an infinity, zero or a negative number) at bar p gives NaN at bars max(p, n-1) to min(p+n-1, len(s)-1), exactly the windows that hold it, while warm-up bars stay 0; later windows recover. Positive subnormal prices are valid: the logarithms come from the package’s subnormal-safe ln, as those of Log do.

Each valid bar depends only on its own window, so the slope is causal, does not repaint and is start-invariant: a series that starts later gives the same values, bit for bit, at every bar whose window it holds whole. A constant window flattens to exact zeros, so every F_k is 0 and the result is exactly 0; zero results are +0. For a fixed mask the result is a fixed linear weighting of the log prices in the window, a linear FIR filter; only the magnitude test makes it adaptive.

The cost is O(n^2) per bar: the forward transform takes 2n(n+1) real multiply-adds and the two endpoints at most 8n (fewer as harmonics drop), so at most 2n(n+1) + 8n in all, about 1.3e5 at n = 256 (131,584 and 2,048), with O(n) tables and scratch and a slice of the len(s) logarithms, all allocated once per call.

Every product is rounded on its own and every sum has a fixed order, so the result is fixed bit for bit on a given platform. Across platforms (arm64 against amd64) the cosine, sine and logarithm of the math package can differ in the last bit, so results agree to rounding rather than bit for bit, and a harmonic whose magnitude lies within rounding of threshold*Fmax can be kept on one platform and dropped on another.

Meyers’s EPFFT curve is the running sum dsumEP of these increments, a recipe rather than a method, with slope the result of this method:

dsum := slope.CumSum()

Its level depends on where the history starts; only its changes are start-invariant. Meyers trades a stop-and-reverse that reads only dsumEP: buy at the next open when dsum has risen more than ptup above its lowest value since the short entry, and sell, reversing to short, at the next open when it has fallen more than ptdn below its highest value since the long entry. His optimised ptup and ptdn are on an unstated scale (he prints the endpoint sum without the 1/N his narrative needs and cites MathCad 8, and the axis of his chart is unlabelled), so they do not transfer to the log units of this method. His evidence is the S&P 500 futures, walk-forward with three-year optimisation windows: out of sample from 1 January 1996 to 22 January 1999 the system made 15 trades with 8 winners (53%) and a net 105,900 dollars before costs, a profit factor of 3.74, compounding at 14.43% a year against 19.31% for buy-and-hold, so it lagged buy-and-hold. On log random walks with 1% a bar, n = 256 and a threshold of 0.15, the slope correlates about 0.73 to 0.76 with the one-bar change of c.Log().SuperSmoother(16) to c.Log().SuperSmoother(20), and about 0.79 to 0.80 with that change a bar later.

Meyers’s later “Trading The E-Mini” (Stocks & Commodities V21:1, January 2003) uses one-minute bars, n = 512 without padding and an optimised threshold of 0.13, and exponentiates the endpoints with the chord added back, so its increment is in price units and includes b. That form is not built; it would be a sibling method, EndpointFFTPriceSlope. Nor is there a level method: Meyers’s level yepab = exp(yep + a + b(n-1)) jitters as the window slides, and he discards it.

Follow-Through Index Regime Detection