The following is an excerpt from "PREDICTION", by John
Niels Bohr said “Prediction is very difficult, especially if it’s about the future.”. Actually, prediction is pretty easy in the context of . All you have to do is to assume the market will behave in the immediate future just as it has behaved in the immediate past. In this article we will explore several different techniques that put the philosophy into practice.
Linear extrapolation takes the philosophical approach quite literally. Linear extrapolation simply takes the difference of the last two bars and adds that difference to the value of the last bar to form the prediction for the next bar. The prediction is extended further into the future by taking the last predicted value as real data and repeating the process of adding the most recent difference to it. The process can be repeated over and over to extend the prediction even further.
Linear extrapolation is an FIR filter, meaning it depends only on the data input rather than on a previously computed value. Since the output of an FIR filter depends only on delayed input data, the resulting lag is somewhat like the delay of water coming out the end of a hose after it supplied at the input. Linear extrapolation has a negative group delay at the longer cycle periods of the spectrum, which means water comes out the end of the hose before it is applied at the input. Of course the analogy breaks down, but it is fun to think of it that way. As shown in Figure 1, the actual group delay varies across the spectrum. For frequency components less than .167 (i.e. a period of 6 bars) the group delay is negative, meaning the filter is predictive. However, the filter has a positive group delay for cycle components whose periods are shorter than 6 bars.
Here’s the practical ramification of the group delay: Suppose we are projecting the prediction 5 bars into the future. This is fine as long as the market is continued to trend up in the same direction. But, when we get a reversal, the prediction continues upward for 5 bars after the reversal. That is, the prediction fails just when you need it the most. An interesting phenomenon is that, regardless of how far the extrapolation extends into the future, the prediction will always cross the signal at the same spot along the time axis. The result is that the prediction will have an overshoot. The amplitude of the overshoot is a function of how far the extrapolation has been carried into the future.
But the overshoot gives us an opportunity to make a useful prediction at the cyclic turning point of band limited signals (i.e. oscillators having a zero mean). If we reduce the overshoot by reducing the gain of the prediction, we then also move the crossing of the prediction and the original signal into the future. Since the group delay varies across the spectrum, the effect will be less effective for the shorter cycles in the data. Nonetheless, the technique is effective for both discretionary trading and automated trading in the majority of cases.
EXPLORING THE CODE
Before we predict, we need to create a band limited indicator from which to make the prediction. I have selected a “roofing filter” consisting of a High Pass Filter followed by a Low Pass Filter. The tunable parameter of the High Pass Filter is HPPeriod. Think of it as a “stone wall filter” where cycle period components longer than HPPeriod are completely rejected and cycle period components shorter than HPPeriod are passed without attenuation. If HPPeriod is set to be a large number (e.g. 250) the indicator will tend to look more like a trending indicator. If HPPeriod is set to be a smaller number (e.g. 20) the indicator will look more like a cycling indicator. The Low Pass Filter is a Hann Windowed FIR filter whose tunable parameter is LPPeriod. Think of it as a “stone wall filter” where cycle period components shorter than LPPeriod are completely rejected and cycle period components longer than LPPeriod are passed without attenuation. The purpose of the Low Pass filter is to smooth the signal. Thus, the combination of these two filters forms a “roofing filter”, named Filt, that passes spectrum components between LPPeriod and HPPeriod.
Since working into the future is not allowed in EasyLanguage variables, we need to convert the Filt variable to the data array XX . The data array is first filled with real data out to “Length”. I selected Length = 10 simply to have a convenient starting point for the prediction. The next block of code is the prediction into the future. It is easiest to understand if we consider the case where count = 0. Then, in English, the next value of the data array is equal to the current value of the data array plus the difference between the current value and the previous value. That makes the prediction one bar into the future. The process is repeated for each value of count until predictions up to 10 bars in the future are contained in the data array. Next, the selected prediction is converted from the data array to the variable “Prediction”. Filt is plotted in Red and Prediction is plotted in yellow.
The Predict Extrapolation indicator is shown below for the Emini S&P contract using the default input parameters. Filt is plotted in red and Predict is plotted in yellow. The crossings of the Predict and Filt lines provide reliable buy and sell timing signals. There is some overshoot for the shorter cycle periods, for example in February and March 2021, but the only effect is a late timing signal. Further reducing the gain and/or reducing the BarsFwd inputs would provide better timing signals during this period.
Figure 2. Predict Extrapolation Provides Reliable Timing Signals
I have experimented with other FIR filters for predictions, but found none that had a significant advantage over linear extrapolation.
MESA is an acronym for Maximum Entropy Spectral Analysis. Conceptually, it removes spectral components until the residual is left with maximum entropy. It does this by forming an all-pole filter whose order is determined by the selected number of coefficients. It maximally addresses the data within the selected window and ignores all other data. Its resolution is determined only by the number of filter coefficients selected. Since the resulting filter is an IIR filter, a prediction can be formed simply by convolving the filter coefficients with the data. MESA is one of the few, if not the only way to practically determine the coefficients of a higher order IIR filter. Discussion of MESA is beyond the scope of this article.
TWO POLE IIR FILTER
While the coefficients of a higher order IIR filter are difficult to compute without MESA, it is a relatively simple matter to compute the coefficients of a two pole IIR filter.
(Skip this paragraph if you don’t care about DSP) We can locate the conjugate pole positions parametrically in the Z plane in polar coordinates. Let the radius be QQ and the principal angle be 360 / P2Period. The first order component is 2*QQ*Cosine(360 / P2Period) and the second order component is just QQ2. Therefore, the transfer response becomes:
H(z) = 1 / (1 - 2*QQ*Cosine(360 / P2Period)*Z-1 + QQ2*Z-2)
By mixing notation we can easily convert the transfer response to code.
Output / Input = 1 / (1 - 2*QQ*Cosine(360 / P2Period)* + QQ2*)
Output - 2*QQ*Cosine(360 / P2Period)*Output + QQ2*Output = Input
Output = Input + 2*QQ*Cosine(360 / P2Period)*Output - QQ2*Output
The Two Pole Predictor starts by computing the same “roofing filter” design as described for the Linear Extrapolation Predictor. The HPPeriod and LPPeriod inputs adjust the roofing filter to obtain the desired appearance of an indicator. Since EasyLanguage variables cannot be extended into the future, the prediction process starts by loading the XX data array with indicator data up to the value of Length. I selected Length = 10 simply to have a convenient place from which to start the prediction. The coefficients are computed parametrically from the conjugate pole positions and are normalized to their sum so the IIR filter will have unity gain at zero frequency.
The prediction is formed by convolving the IIR filter coefficients with the historical data. It is easiest to see for the case where count = 0. This is the initial prediction. In this case the new value of the XX array is formed by successively summing the product of each filter coefficient with its respective historical data sample. This process is significantly different from linear extrapolation because second order curvature is introduced into the prediction rather than being strictly linear. Further, the prediction is adaptive to market conditions because the degree of curvature depends on recent historical data. The prediction in the data array is converted to a variable by selecting the BarsFwd value. The prediction is then plotted in yellow, and is compared to the indicator plotted in red.
The Predict 2 Pole indicator is shown above being applied to the Emini S&P contract for most of 2021. The default parameters for the roofing filter and predictor were used. By comparison to the Linear Extrapolation prediction of Figure 2, the Predict 2 Pole indicator has a more consistent prediction. For example, there is little or no overshoot in February or March while still giving good predictions in April and May.
Input parameters can be varied to adjust the appearance of the prediction. You will find that the indicator is relatively insensitive to the BarsFwd input. The P2Period parameter primarily controls the gain of the prediction and the QQ parameter primarily controls the amount of prediction lead during trending sections of the indicator.
1. A more or less universal band limited “roofing filter” indicator was used to demonstrate the predictors. The HPPeriod input parameter is used to control whether the indicator looks more like a trend indicator or more like a cycle indicator. The LPPeriod input parameter is used to control the smoothness of the indicator.
2. A linear extrapolation predictor is formed by adding the difference of the two most recent data bars to the value of the last data bar. The result is considered to be a real data point and the process is repeated to extend the prediction into the future. This is an FIR filter having a one bar negative group delay at zero frequency, but the group delay is not constant across the spectrum. This variable group delay causes the linear extrapolation prediction to be inconsistent across a range of market conditions.
3. The degree of prediction by linear extrapolation can be controlled by varying the gain of the prediction to reduce the overshoot to be about the same amplitude as the peak swing of the indicator.
4. I was unable to experimentally derive a higher order FIR filter predictor that had advantages over the simple linear extrapolation predictor.
5. A Two Pole IIR predictor can be created by parametrically locating the conjugate pole positions.
6. The Two Pole predictor is a second order filter, which allows curvature into the prediction, thus mitigating overshoot. Further, the curvature is adaptive because the prediction depends on previously computed prediction values.
7. The Two Pole predictor is more consistent over a range of market conditions.
- Loxx's Expanded source types:
Library for expanded source types:
Explanation for expanded source types:
- Three different signal types: 1) Prediction/Filter crosses; 2) Prediction middle crosses; and, 3) Filter middle crosses.
- Bar coloring to color trend.
- Signals, both Long and Short.
- Alerts, both Long and Short.