-
74
C h a p t e r 4
PREDICTOR TESTING METHODS
The Predictors
Parallax has been producing neural-network based financial
predictors since 1990
and so it is an integral part of our business to validate these
predictors using
reliable statistical methods. With each predictor, we need to
answer the following
set of questions:
1. What price behavior is being predicted?
2. What is the effective duration of the predictor?
3. How statistically significant is the predictor at each time
step forward?
4. Is the predictor effective on all time scales?
5. Is the predictor effective on all financial series?
6. What combinations of predictors are the most effective?
The following section is an overview of a simple and reliable
statistical method
and an 8-year analysis of our present stable of predictors. In
order to carry out
this analysis, it is necessary to measure the price action
during the time period
immediately following each prediction event. We call this the
post-predictor or
outrun period.
-
75
Post-Predictor Z Scores
We are interested in characterizing the post-predictor time
period using a scale-
independent method that allows comparisons between financial
series. The Z
Score is the most appropriate measure for this job. A Z-Score is
the measure of
how many standard deviations price has moved away from its price
at the
prediction event, assuming that the probability of either an up
or down move is
random at 50%.
By measuring local volatility at the prediction event, a normal
probability
distribution can be drawn going forward in time that acts as a
roadmap for
subsequent price moves. The map is centered at the closing price
of the
prediction event. Each day the map widens according to normal
diffusion
velocities, which are proportional to 1/Time, representing the
region where the
future price is most likely to be found. An example of such a
probability map is
shown below for the stock Home Depot on Feb 3, 2005:
-
76
Viewed from the side at two time steps, diffusion acts to spread
out the region
where we might find the stock price as time elapses:
At each time step, there is a larger standard deviation and the
same mean. If we
represent the actual price achieved at each step in terms of
that standard
deviation, we produce a series of Z-Scores. For example, if the
price at a
prediction event is $5, and then moves to $7 on day ten with a
standard deviation
of $1.60, then the Z-Score on day ten would be (7-5)/1.6 = 1.25.
This means
that price moved 1.25 standard deviations above the price at the
prediction event.
Since the standard deviation continues to increase, in order to
maintain the same
Z, price would have to increase by the same relative amount.
Diffusion causes expected prices to spread out over time
-
77
If all post-predictor prices for all financial series are
converted to Z-Scores, and if
the predictors do not work, and if markets are random, then at
every time step, a
histogram of all the Z-Scores would be expected to result in a
standardized
normal distribution, with mean of zero and a standard deviation
of one N(0,1) as
shown below:
We of course hope that our predictors actually predict
non-random price
behavior, so the degree of deviation from the normal curve is
critical.
Are Financial Series Random?
We have assumed that price movement is best characterized by a
random walk
model, but this might not be the case. There is gathering
evidence for other
characteristic distributions such as a Biased Random Walk,
Truncated Levy
Flights, or Cauchy distribution. Our solution is to produce a
quantitative
background distribution based on randomly selected dates across
our entire 3000
day test period, and for all of the 2500 stocks being
considered. The figure below
-
78
shows this background distribution of Z-scores corresponding to
random
prediction dates, and a normal distribution based on a random
walk assumption
plotted together. It is clear from this figure that a strict
random walk assumption
is inappropriate during our 8 year test period. Instead there
appears to be a
positive return bias.
To illustrate this another way, we could ask what percentage of
buy predictions
are winners (Z>0) if the timing is random, and plot this
percentage each day
following the randomly selected purchase dates. Normally this
would be 50%,
but since the background distributions is positively biased, the
figure below
shows the percent winners for randomly selected buys climbs
steadily over time.
The reverse is of course true for randomly selected sells
(mirror image).
-
79
On a weekly scale the effect is even more pronounced, as shown
in the next
figure. I guess this could be called the dartboard effect, in
that even a random
selection of stocks during this period showed a 59% win rate at
30 weeks after
purchase, and shorting was decidedly unprofitable.
-
80
We will use these curves as benchmarks for our predictors over
this test period
and stock set. In order to have a non-random buy predictor then,
we need to see
a win rate in excess of 59% at 30 weeks for example, and sells
would need a win
rate better than 41%.
So far we have examined the background distribution of Z-Scores
and the
percentage of winners. What we still need to know is how much
gain is possible
from randomly selected buys and sells, so that each predictors
excess gain profile
can be evaluated. The pictures below show the background
performance for
randomly selected buys and sells:
-
81
-
82
Parallax Predictors
Using the randomly selected background distribution we found in
the last section,
we will now statistically evaluate our main stable of predictors
using a chi-squared
test. The Chi-squared test is designed for comparing
distributions and a p
-
83
ExtremeHurst Extension Buy & Sell Predictors
Our ExtremeHurst extension predictor is Parallaxs most
predictive tool. It
measures the degree of positive feedback present in price and
predicts when the
end-of-trend critical point has been reached using the principle
of discrete scale
invariance. Price behavior in the post-predictor period is
expected to be flat or
opposite of the preceding trend. On a daily scale, we have
accumulated Z-Scores
for all 30 days following the signal. The distribution of these
Z-Scores on day
eleven, shown in the figure below, shows the strong positive
bias, with 60% of
the returns being positive by day eleven, with a p-value =
0.00000005. This
means that these are not random signals, and that market
behavior leading up to
the prediction event conditioned what happened in the
post-prediction period.
ExtremeHurst is scale invariant by definition, so this same
effect should be
present on weekly, monthly, or intra-day periods as well.
-
84
The ExtremeHurst sell predictor is extremely fast, showing 56%
winners at only
4 days from the signal, with a Chi squared test p-value =
6.7x10-124. This means
that these are not random signals, and that market behavior
leading up to the
prediction event significantly conditioned what happened in the
post-
prediction period. ExtremeHurst is scale invariant by
definition, so this same
effect should be present on weekly, monthly, or intra-day
periods as
well.
-
85
Looking at the percentage of winning predictions for both
ExtremeHurst
predictors compared to the background rate yields the following
picture:
The way to interpret this picture is that we gain a significant
advantage over
random for both buy and sell predictors for at least 30 days.
Extension buy
signals yield the best percentage of winning predictions at 8%
more than the
random background after 22 days. This tells us that if we had a
large sample of
trades that were triggered by this predictor, we could expect
that at 22 days, a
total of 55%+8% = 63% of the trades would be ahead. Remember
that our
random trade sample had 55% winners at 22 days, so we would beat
it by 8%.
This chart also tells us that we can hold trades for a
relatively long time. Despite
the win rate staying constant, average portfolio returns would
climb steadily.
-
86
The sell signal is different and faster. We immediately achieve
a 7% edge above
random at 3 days and that edge diminishes steadily over 30 days.
This might
make us design a faster sell trading strategy than what we would
use for the long
side.
So far we have examined the overall distribution of Z-Scores for
ExtremeHurst,
and the percentage of times that a buy signal goes up or a sell
goes down. The
win rate is only part of the story however. What we still need
to know is how
much gain is possible from these buys and sells in excess of the
background
market performance. The pictures below show a gain analysis for
ExtremeHurst
buys and sells. Each plot shows the median percentage gain with
error bars at the
40th and 60th percentiles and the average of all returns. Note
that the average is
not as good a measure of performance because of the overemphasis
on outlier
signals. The count of signals is shown in parenthesis.
-
87
-
88
On a weekly scale, the power of our ExtremeHurst extensions
becomes even
more evident. Remember that 0% on the graph is equivalent to a
random win
rate, so any excursion above zero stacks the deck in our favor.
Weekly scale
extension buy predictions reach their maximum effect after 8
weeks, with an
overall win rate that is 15% over the random background. Again
we see that the
extension sell signals are faster and give only a 2.5% edge over
random.
-
89
-
90
Price Wizard Fundamental Value Predictor
Price is often out of sync with value, being bid up or down
based on future
expectations and irrational trend persistence. Our model
incorporates all major
stores of corporate value, normalized simultaneously for sector,
industry, and
economic factors. The figure below shows the Z-Scores 30 weeks
following a
stock moving to undervalued from overvalued. Note the
significant (p=0)
positive bias.
..
-
91
-
92
-
93
-
94
Precision Turn Cycle Predictor
..
-
95
-
96
/ColorImageDict > /JPEG2000ColorACSImageDict >
/JPEG2000ColorImageDict > /AntiAliasGrayImages false
/DownsampleGrayImages true /GrayImageDownsampleType /Bicubic
/GrayImageResolution 300 /GrayImageDepth -1
/GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true
/GrayImageFilter /DCTEncode /AutoFilterGrayImages true
/GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict >
/GrayImageDict > /JPEG2000GrayACSImageDict >
/JPEG2000GrayImageDict > /AntiAliasMonoImages false
/DownsampleMonoImages true /MonoImageDownsampleType /Bicubic
/MonoImageResolution 1200 /MonoImageDepth -1
/MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true
/MonoImageFilter /CCITTFaxEncode /MonoImageDict >
/AllowPSXObjects false /PDFX1aCheck false /PDFX3Check false
/PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true
/PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ]
/PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [
0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile ()
/PDFXOutputCondition () /PDFXRegistryName (http://www.color.org)
/PDFXTrapped /Unknown
/Description >>> setdistillerparams>
setpagedevice