RSM338: Applications of Machine Learning in Finance
Lecture 3: Predictability tests.
Goutham Gopalakrishna
Spring 2025
”In investing, what is comfortable is rarely profitable.” - Robert Arnott
Goutham Gopalakrishna Rotman-UToronto Spring 2025 1 / 30
Predictability
1 Predictability
2 Autocorrelation
3 Predictability Tests
4 Out of sample prediction
5 Certainty Equivalence
Goutham Gopalakrishna Rotman-UToronto Spring 2025 2 / 30
Predictability
Are Returns Predictable?
Why should we care about the predictability of asset returns?
Economic vs. Statistical Significance
How do we distinguish patterns that matter in the real world from mere statistical
flukes?
Return Horizon
Are we predicting tomorrow’s return, next month’s, or next year’s?
Concerns about Predictability:
1 Measurement errors of prices (Are we seeing “noise” or “signal”?)
2 Nonsynchronous trading (Not all markets trade at the same time)
3 Survivorship bias (Only success stories survive in datasets!)
4 Time-varying expected return (Conditions change over time)
5 Investor irrationality (Behavioral biases can create predictable patterns)
Goutham Gopalakrishna Rotman-UToronto Spring 2025 3 / 30
Predictability
Return Predictability Academic Literature is Huge
Nobody:
Absolutely nobody:
Me: ”How many academic articles exist about return predictability?”
Goutham Gopalakrishna Rotman-UToronto Spring 2025 4 / 30
Predictability
Return Predictability Academic Literature is Huge
Nobody:
Absolutely nobody:
Me: ”How many academic articles exist about return predictability?”
GPT: Given that the topic stretches back to at least the 1970s (and arguably earlier),
and each year brings new contributions, one can conservatively say the body of
literature on return predictability numbers in the thousands of articles.
Goutham Gopalakrishna Rotman-UToronto Spring 2025 4 / 30
Predictability
Return Predictability Academic Literature is Huge
Nobody:
Absolutely nobody:
Me: ”How many academic articles exist about return predictability?”
GPT: Given that the topic stretches back to at least the 1970s (and arguably earlier),
and each year brings new contributions, one can conservatively say the body of
literature on return predictability numbers in the thousands of articles.
Me: Does any provide a precise answer about how to predict returns?
Goutham Gopalakrishna Rotman-UToronto Spring 2025 4 / 30
Predictability
Return Predictability Academic Literature is Huge
Nobody:
Absolutely nobody:
Me: ”How many academic articles exist about return predictability?”
GPT: Given that the topic stretches back to at least the 1970s (and arguably earlier),
and each year brings new contributions, one can conservatively say the body of
literature on return predictability numbers in the thousands of articles.
Me: Does any provide a precise answer about how to predict returns?
GPT: No!
Goutham Gopalakrishna Rotman-UToronto Spring 2025 4 / 30
Predictability
Predictive Regression
Decompose realized returns:
rt = µt−1 + t ,
where µt−1 = E[rt | It−1 ] (expected return) and It−1 is the information set at t − 1.
A simple predictive model:
rt = α + β 0 xt−1 + t .
Interpretation
α: Baseline (intercept)
β: Sensitivity to predictive variables xt−1
t : Surprise component (unpredictable part)
Question: How might you choose xt−1 ? Could it be a macroeconomic indicator like
inflation or something as simple as last month’s return?
Goutham Gopalakrishna Rotman-UToronto Spring 2025 5 / 30
Predictability
Potential Predictive Variables
Common predictive variables:
Past returns (momentum or mean reversion?)
Variance (risk perceptions can drive future returns)
Dividend-to-price ratio, earnings-to-price ratio (market valuation measures)
Book-to-market ratio (value vs. growth patterns)
Interest rates: T-bill rate, inflation, term spread, default spread
Corporate issuing activity: Net equity expansion
Anything in It−1 that could shape future returns
Real-World Tie-In:
Hedge funds often use dozens of these signals (and more) to inform high-frequency
trades.
Are they always successful? Not necessarily—but the search for “the next
predictor” keeps going!
Goutham Gopalakrishna Rotman-UToronto Spring 2025 6 / 30
Autocorrelation
1 Predictability
2 Autocorrelation
3 Predictability Tests
4 Out of sample prediction
5 Certainty Equivalence
Goutham Gopalakrishna Rotman-UToronto Spring 2025 7 / 30
Autocorrelation
Correlation
Definition:
Cov(X , Y )
ρ(X , Y ) = p p ,
Var(X ) Var(Y )
where Cov(X , Y ) is the covariance between X and Y .
Intuitive Meaning
Correlation measures the linear relationship between two different variables.
ρ > 0: When one variable goes up, the other tends to go up.
ρ < 0: When one goes up, the other tends to go down.
ρ ≈ 0: No strong linear connection.
Financial Example
Stock Returns of Apple vs. Microsoft: A positive correlation suggests they often
move in the same direction—perhaps reflecting broader tech-sector trends or shared
market sentiment.
Goutham Gopalakrishna Rotman-UToronto Spring 2025 8 / 30
Autocorrelation
Interesting Financial Correlation Examples
Oil Prices and Airline Stocks:
Often exhibit a negative correlation: as oil prices rise, airline profits can be
squeezed, driving airline stocks lower.
Gold and Stock Market Indices:
Traditionally seen as safe havens, gold prices often have a negative correlation with
equity indices (e.g., S&P 500) when markets are in turmoil.
Consumer Sentiment and Retail Stocks:
Positive correlation: confident consumers might spend more, boosting retail stocks.
Cryptocurrencies and Tech Stocks:
Over certain periods, Bitcoin and tech stocks (e.g., Tesla, Nvidia) may show
surprisingly high correlation, reflecting speculative risk-on/risk-off attitudes. Could
be spurious correlation!
Exchange Rates and Commodities:
Some currencies (like AUD) correlate positively with commodity prices (iron ore,
etc.). Activity: Find out why.
Goutham Gopalakrishna Rotman-UToronto Spring 2025 9 / 30
Autocorrelation
Autocorrelation
Definition:
ρk = Corr(Xt , Xt−k ) for lag k,
measures how a time series (Xt ) correlates with its own past values.
Key Idea
If ρ1 is positive and large, it means Xt is often similar to Xt−1 .
If ρ1 is negative, then Xt tends to move in the opposite direction of Xt−1 .
Higher-order lags (ρk ) check relationships further back in time.
Goutham Gopalakrishna Rotman-UToronto Spring 2025 10 / 30
Autocorrelation
Autocorrelation
Definition:
ρk = Corr(Xt , Xt−k ) for lag k,
measures how a time series (Xt ) correlates with its own past values.
Key Idea
If ρ1 is positive and large, it means Xt is often similar to Xt−1 .
If ρ1 is negative, then Xt tends to move in the opposite direction of Xt−1 .
Higher-order lags (ρk ) check relationships further back in time.
Financial Intuition
Stock Returns:
A positive 1-day autocorrelation might indicate momentum. If the stock was up
today, there’s a slight chance it might be up tomorrow.
A negative autocorrelation might indicate mean reversion. If it rises today, it
might fall tomorrow.
Goutham Gopalakrishna Rotman-UToronto Spring 2025 10 / 30
Autocorrelation
Interesting Examples of Autocorrelation in Finance
Momentum Trading:
Some hedge funds look for short-term positive autocorrelation in returns—if a stock
is rising, they expect it to keep rising briefly.
Mean Reversion in Currencies:
Central banks often intervene if exchange rates drift too far; short-term negative
autocorrelation can appear as the exchange rate snaps back.
Volatility Clustering:
Not exactly return autocorrelation, but volatility often exhibits strong
autocorrelation: big moves today signal likely big moves tomorrow.
Bond Yields Over Time:
Long-term interest rates can show persistent trends (positive autocorrelation) or
cyclical patterns (oscillating autocorrelation).
High-Frequency Trading Spikes:
Autocorrelation at minute-level data can reveal microstructure patterns (like order
flow).
Goutham Gopalakrishna Rotman-UToronto Spring 2025 11 / 30
Autocorrelation
Why Autocorrelation Matters for Predictability
If returns (or some function of returns) are autocorrelated, there’s potential to
predict future values based on past values.
Many asset pricing models (e.g., CAPM) often assume no autocorrelation in
returns for efficient markets.
In practice, short-term autocorrelation might exist but often diminishes quickly
(due to trading or arbitrage).
Statistical Tools:
Autocorrelation function (ACF) plot.
Ljung-Box test to detect non-zero autocorrelations.
Durbin-Watson test for first-order autocorrelation in residuals.
Goutham Gopalakrishna Rotman-UToronto Spring 2025 12 / 30
Predictability Tests
1 Predictability
2 Autocorrelation
3 Predictability Tests
4 Out of sample prediction
5 Certainty Equivalence
Goutham Gopalakrishna Rotman-UToronto Spring 2025 13 / 30
Predictability Tests
Autocorrelation-Based Tests of Predictability
Sample autocorrelation:
PT −k T
t=1 (rt − r¯)(rt+k − r¯) 1 X
ρ̂(k) = PT , r¯ = rt .
t=1 (rt − r¯)
2 T t=1
Under i.i.d. returns:
√
T [ρ̂(1), ρ̂(2), . . . , ρ̂(m)]0 ∼ N (0m , Im ).
Python code:
from statsmodels.tsa.stattools import acf
rho = acf(r,nlags=24)
Goutham Gopalakrishna Rotman-UToronto Spring 2025 14 / 30
Predictability Tests
Autocorrelation-Based Tests of Predictability
Sample autocorrelation:
PT −k T
t=1 (rt − r¯)(rt+k − r¯) 1 X
ρ̂(k) = PT , r¯ = rt .
t=1 (rt − r¯)
2 T t=1
Under i.i.d. returns:
√
T [ρ̂(1), ρ̂(2), . . . , ρ̂(m)]0 ∼ N (0m , Im ).
Python code:
from statsmodels.tsa.stattools import acf
rho = acf(r,nlags=24)
Quick Poll: Who thinks yesterday’s stock return affects today’s?
Goutham Gopalakrishna Rotman-UToronto Spring 2025 14 / 30
Predictability Tests
Autocorrelation-Based Tests of Predictability
Sample autocorrelation:
PT −k T
t=1 (rt − r¯)(rt+k − r¯) 1 X
ρ̂(k) = PT , r¯ = rt .
t=1 (rt − r¯)
2 T t=1
Under i.i.d. returns:
√
T [ρ̂(1), ρ̂(2), . . . , ρ̂(m)]0 ∼ N (0m , Im ).
Python code:
from statsmodels.tsa.stattools import acf
rho = acf(r,nlags=24)
Quick Poll: Who thinks yesterday’s stock return affects today’s?
Let’s see if the data supports your intuition!
Goutham Gopalakrishna Rotman-UToronto Spring 2025 14 / 30
Predictability Tests
Autocorrelation-Based Tests of Predictability
Financial returns often assumed to be uncorrelated over time (random walk or
white noise).
Goal: Detect whether there is any autocorrelation (e.g., momentum or mean
reversion).
Two major approaches:
Portmanteau tests (Box-Pierce, Ljung-Box).
Variance Ratio (VR) test.
Implication: Rejection of the “no autocorrelation” hypothesis means potential
predictability in returns.
Goutham Gopalakrishna Rotman-UToronto Spring 2025 15 / 30
Predictability Tests
1. Portmanteau Tests (Ljung-Box, Box-Pierce)
Key Idea: Check if any sample autocorrelations at lags 1, 2, . . . , m are significantly
different from zero.
Box-Pierce QBP
m
X
QBP (m) = T ρ̂2k ∼ χ2m (if no autocorrelation).
k=1
Ljung-Box QLB
m
X ρ̂2k
QLB (m) = T (T + 2) ∼ χ2m .
T −k
k=1
Decision Rule: If Q > χ2m,α , reject no-autocorrelation for at least one lag.
Practical Use: Often used in residual diagnostics to confirm “white noise” after fitting
time series models.
Goutham Gopalakrishna Rotman-UToronto Spring 2025 16 / 30
Predictability Tests
1. Portmanteau Tests: Example
Data: 120 monthly returns (10 years).
Null Hypothesis: No autocorrelation at lags 1, 2, . . . , m.
Steps:
1 Estimate ρ̂1 , . . . , ρ̂m .
2 Compute
m
X ρ̂2k
QLB (m) = T (T + 2) .
T −k
k=1
3 Compare with χ2m distribution.
Interpretation:
If QLB (m) is large, we reject H0 .
Means there’s significant autocorrelation at some lag(s) among the first m.
Goutham Gopalakrishna Rotman-UToronto Spring 2025 17 / 30
Predictability Tests
2. Variance Ratio Test: Concept
Motivation: For a random walk, the variance of k-period returns is k times the
variance of 1-period returns.
The Variance Ratio:
2
σ
bk-period
VR(k)
c = 2 .
kσ
b1-period
Under a random walk, we expect VR(k)
c ≈ 1.
VR(k)
c > 1: suggests positive autocorrelation (momentum).
VR(k)
c < 1: suggests negative autocorrelation (mean reversion).
Why a Test?
Because we need to see if VR(k)
c significantly differs from 1. We form a Z -statistic
that (approximately) follows N (0, 1) under H0 (random walk).
Goutham Gopalakrishna Rotman-UToronto Spring 2025 18 / 30
Predictability Tests
2. Variance Ratio Test: Test Statistic
Test Statistic: Null hypothesis: H0 : VR(k)
c =1
VR(k)
c −1
Z (k) = q ∼ N (0, 1) (for large T ).
Var VR(k)
c
If |Z (k)| > zα/2 (e.g., 1.96 at 5%), we reject the null of a random walk for lag k.
Interpretation
- Positive Z (k) and large: VR(k)
c > 1 significantly (momentum).
- Negative Z (k) and large (in absolute value): VR(k)
c < 1 (mean reversion).
Goutham Gopalakrishna Rotman-UToronto Spring 2025 19 / 30
Predictability Tests
2. Variance Ratio Test: Example
Data: 5 years of daily returns (T ≈ 1250).
Compute:
2
σ
b5-day VR(5)
c −1
VR(5)
c = 2 , Z (5) = q .
5b
σ1-day
Var VR(5)
c
Note that H0 : VR(5)
c = 1 (random walk - no autocorrelation). Suppose
VR(5) = 1.6 and Z (5) = 2.4.
c
Compare |2.4| > 1.96 at 5% level =⇒ reject random walk at lag 5.
Conclusion: 5-day variance is significantly larger than 5 × variance(1-day), indicating
short-term momentum.
Multiple Horizons:
Check k = 2, 4, 8, . . . and potentially apply Chow-Denning multiple comparison
correction (outside the scope of this course).
Goutham Gopalakrishna Rotman-UToronto Spring 2025 20 / 30
Predictability Tests
Summary and Practical Insights
Ljung-Box / Box-Pierce:
Portmanteau tests checking multiple autocorrelation lags at once.
Q-statistic ∼ χ2m under no autocorrelation.
Variance Ratio Test:
Focuses on how multi-period variance scales with k.
Z -statistic ∼ N (0, 1) for large samples.
Good for detecting momentum (VR c > 1) or mean reversion (VR
c < 1).
When to Use:
Portmanteau Tests: Quick diagnostic to see if any of first m lags are non-zero.
VR Test: More direct check of random walk vs. momentum/reversal.
Caveat:
Short sample sizes or heavy tails can distort inference.
Adjust for heteroskedasticity (Lo and MacKinlay’s robust approach - outside the
scope of this course).
Goutham Gopalakrishna Rotman-UToronto Spring 2025 21 / 30
Predictability Tests
Econometric Issues of Predictive Regression
Challenges:
Stochastic regressors (the predictors themselves can change unpredictably)
OLS estimator consistency vs. bias (small sample issues)
Adjusting standard errors (time-series data can be tricky)
Risk of misspecification (we might leave out important factors)
Data mining concerns (beware of “cherry-picking” the best models)
Careful validation is essential!
Activity: Experiment in Excel or Python: Fit a predictive regression on a short dataset
vs. a long dataset. Compare your β estimates and see if they’re stable!
Goutham Gopalakrishna Rotman-UToronto Spring 2025 22 / 30
Out of sample prediction
1 Predictability
2 Autocorrelation
3 Predictability Tests
4 Out of sample prediction
5 Certainty Equivalence
Goutham Gopalakrishna Rotman-UToronto Spring 2025 23 / 30
Out of sample prediction
In-Sample vs. Out-of-Sample R 2
In-Sample R 2 :
Measures how well the model fits the data used to estimate it.
Computed on the same observations used for model training.
PN
2 (yt − ŷt )2
Rin-sample = 1 − Pt=1
N
.
2
t=1 (yt − ȳ )
Pitfall: Can be overly optimistic if the model is complex or if there is overfitting.
Out-of-Sample R 2 :
Measures predictive performance on new or held-out data.
Use a test/validation set not used in model estimation.
(yt − ŷt )2
P
Roos = 1 − P t∈test
2
2
.
t∈test (yt − ȳtrain )
Goal: Lower chance of overfitting. A better gauge of “real-world” performance.
Takeaway
A high in-sample R 2 doesn’t guarantee predictive power. Always check out-of-sample
R 2 to confirm your model generalizes well!
Goutham Gopalakrishna Rotman-UToronto Spring 2025 24 / 30
Out of sample prediction
Example: Predicting Next-Period Return
We collect historical data on market returns and some predictor xt .
We run a predictive regression on a rolling basis:
rt+1 = α + β xt + εt+1 .
α
bt , βbt are re-estimated each month with an expanding window of size M.
Goutham Gopalakrishna Rotman-UToronto Spring 2025 25 / 30
Out of sample prediction
Example: Predicting Next-Period Return
We collect historical data on market returns and some predictor xt .
We run a predictive regression on a rolling basis:
rt+1 = α + β xt + εt+1 .
α
bt , βbt are re-estimated each month with an expanding window of size M.
Forecasting:
µ
bt = α
bt + βbt xt .
The forecast error is ebt+1 = rt+1 − µ bt . After the initial window of M observations, we
eM+1 , ebM+2 , . . . , ebT }. Out-of-Sample R 2 is given by
get forecast errors {b
PT
ebt2
ROS2
= 1 − PTt=M+1 , (can be negative!)
2
t=M+1 e b0,t
Measures how much better (or worse) the predictive model is compared to the “no
regression” benchmark given below.
Goutham Gopalakrishna Rotman-UToronto Spring 2025 25 / 30
Out of sample prediction
Example: Predicting Next-Period Return
We collect historical data on market returns and some predictor xt .
We run a predictive regression on a rolling basis:
rt+1 = α + β xt + εt+1 .
α
bt , βbt are re-estimated each month with an expanding window of size M.
Forecasting:
µ
bt = α
bt + βbt xt .
The forecast error is ebt+1 = rt+1 − µ bt . After the initial window of M observations, we
eM+1 , ebM+2 , . . . , ebT }. Out-of-Sample R 2 is given by
get forecast errors {b
PT
ebt2
ROS2
= 1 − PTt=M+1 , (can be negative!)
2
t=M+1 e b0,t
Measures how much better (or worse) the predictive model is compared to the “no
regression” benchmark given below.
Benchmark:
Use the historical average return as the forecast.
Denote these forecast errors as {b
e0,M+1 , eb0,M+2 , . . . , eb0,T }, and use it in the
2
denominator of ROS .
Goutham Gopalakrishna Rotman-UToronto Spring 2025 25 / 30
Certainty Equivalence
1 Predictability
2 Autocorrelation
3 Predictability Tests
4 Out of sample prediction
5 Certainty Equivalence
Goutham Gopalakrishna Rotman-UToronto Spring 2025 26 / 30
Certainty Equivalence
Certainty Equivalence (CEV)
Mean-variance utility:
γ
U(Rp ) = µp − σp2 ,
2
|{z} |{z}
return
risk
where µp is the risk premium and σp2 is the variance of the portfolio.
Interpretation:
Certainty equivalence is how much the guaranteed return have to be for you to
feel equally good about holding a risky asset.
γ = risk aversion parameter (“How much do losses really hurt?”)
U(Rp ) is a simple way to balance return vs. risk.
Discussion:
Debate why an investor might accept a lower expected return if it comes with
lower variance.
Goutham Gopalakrishna Rotman-UToronto Spring 2025 27 / 30
Certainty Equivalence
Certainty Equivalence (CEV)
Suppose the investor can only invest in:
A risk-free T-bill (earning the risk-free rate µr ),
A market portfolio with mean return µt and variance σt2 .
Optimal weight in the market portfolio:
1
wt∗ = µt .
γ σt2
Reality: µt and σt2 are unknown, so the investor must forecast them:
1
ŵt = µ̂t .
γ σ̂t2
We forcast using first M observations. Then, compute out-of-sample excess return
of this portfolio for remaining T − M observations.
Out-of-sample excess return of this portfolio at time t + 1 is w
bt rˆt+1 .
Goutham Gopalakrishna Rotman-UToronto Spring 2025 28 / 30
Certainty Equivalence
Estimate of Certainty Equivalence for the Investor
Certainty Equivalence:
γ 2
bp − σ
Ub = µ b ,
2 p
where
µ
bp is the estimated portfolio risk premium,
bp2 is the estimated portfolio variance,
σ
both computed out-of-sample.
Out-of-Sample Estimates:
T −1 T −1
1 X 1 X 2
µ
bp = w
bt rt+1 , bp2 =
σ bt rt+1 − µ
w bp .
T −M T −M −1
t=M t=M
T : total number of observations,
M: number of periods used for in-sample estimation,
w
bt : predicted or estimated weight in the market for period t,
rt+1 : realized return at time t + 1.
Implication: A predictive regression is valuable if it delivers a higher Ub compared to a
competing model.
Goutham Gopalakrishna Rotman-UToronto Spring 2025 29 / 30
Certainty Equivalence
Methods for Improving Out-of-Sample Performance
Strategies for better predictions:
Impose nonnegativity on market risk premium (if it makes sense economically)
Model averaging techniques:
Median of various forecasts
Mean of various forecasts
Trimmed mean of various forecasts
Weighted averages based on past performance
Takeaway:
No single model has it all. Combining models can hedge against “one-trick pony”
forecasts.
The quest for the best model is ongoing, and is ever more important in the age of
Big Data. You are in the right place to learn how to do it!
Final Question: If you were running a hedge fund, how would you decide which
models to weight more heavily?
Goutham Gopalakrishna Rotman-UToronto Spring 2025 30 / 30
