Claremont Graduate University

MATH 387: Discrete Mathematical

Modeling

Implied Volatility Modeling

with Markov Switching
Autoregression

Authors:
Anthony Hollerich
Augusto Gonzalez
Bonorino

December 12, 2023

1 Introduction
Efforts to understand and predict future movements of uncertainty permeate
a diverse set of disciplines. Each with its own perspective of the meaning of
uncertainty. Economists attempt to quantify volatility as measures of poten-
tial risk of falling into a state of stagnant or decreasing growth. Financial
analysts and investors model volatility by capturing the movement of a finan-
cial asset as simple standard deviations from expected returns, attempting
to maximize profits while minimizing risk. The quest for predicting future
states of an underlying system has fueled quantitative and computational
innovations, which have helped formalized the field of risk management and
develop an appreciation of the complexity of such phenomena.
Two particular measures of asset risk have taken shape over the years.
Implied Volatility (IV) captures the uncertainty of the movement of a finan-
cial or economic asset (often a stock or index) by reflecting the expected
volatility of the options market; thus providing a forward-looking approach
by encoding the expected volatility from now until the option’s expiration.
In contrast, Historical Volatility (HV) offers a backward-looking alternative
by leveraging data of known prices over a fixed period of time to reflect
the actual volatility realized in the underlying market. There are numerous
techniques to measure and model each, we focus on the former.
Successful forecasting of IV from a trader’s point of view primarily in-
volves forecasting the direction of IV correctly; a correct magnitude for the
change is not as relevant. This is because option positions such as the strad-
dle will generate a profit if the IV moves in the correct direction, ceteris
paribus (the size of the profit is affected by the magnitude of change, how-
ever). The forecasting accuracy of the various models is first evaluated based
on sign: how many times does the sign of the change in the measure of IV
correspond to the direction forecasted by the model. However, the point
forecasts are also evaluated based on mean squared errors (MSE). Accurate
point forecasts can be valuable, for example, in risk management and asset
pricing applications [1].
The most widely used measure today is a volatility index constructed
by the Chicago Board of Option Exchanges’s (CBOE) from S&P 500 in-
dex option prices called the VIX, sometimes referred to as the ”fear index”.
Importantly, the VIX is believed to follow a complex autoregressive, multi-
variate, and heteroskedastic time-series that captures perceptions of risk or
market sentiment. We propose a Markov Switching Autoregression (MSAR)

1
framework as an alternative to traditional econometric models that often fall
short in this task due to the nuances of the VIX’s statistical properties. The
paper is organized into 5 sections. After a brief literature review on previous
attempts to model implied volatility, particularly the VIX, section 3 describes
the multiswitch problem that motivated efforts to merge markov chains with
regression models, and introduces the MSAR model. Section 5 presents the
dataset used to fit a baseline markov model and our MSAR implementa-
tion; we discuss performance and forecasting power of both models. Finally,
section 6 offers concluding thoughts and suggests potential improvements.

2 Literature Review
The VIX index, calculated by the Chicago Board Options Exchange (CBOE)
from S&P 500 index option prices, represents a critical measure of market
volatility. It has garnered significant interest in the literature due to its
role as a forward-looking volatility predictor, offering insights into market
sentiment and investor expectations. Scholars have emphasized the VIX’s
utility in financial risk management and market analysis [2].
However, traditional linear models like multivariate regressions or ARIMA
face challenges in accurately predicting the VIX [3] due to their inherent
linear nature. These models, while effective in many time-series analyses,
struggle with the VIX’s distinct regimes characterized by shifts in mean and
variance. This limitation has prompted the exploration of more advanced
methodologies capable of capturing such non-linear and multivariate dynam-
ics of time-series with inherent ”turning points”. Results from simulations
showcasing these limitations are provided in the Appendix.
Markov Switching Regressions (MSRs) [4] emerge as a promising alter-
native, leveraging the properties of Markov chains to accommodate regime
shifts in an autoregressive time-series [5]. MSRs offer a framework where
the parameters of interest, such as the mean and variance, can vary across
different states. This approach aligns with the observable characteristics of
the VIX, which exhibits periods of relative stability interspersed with high
volatility. By fitting a system of linear equations for each state and esti-
mating a transition matrix, MSRs enhance the predictive accuracy for such
complex time-series data.
In integrating macroeconomic variables, we follow the precedent set by
Prasad et al. [6]. Their application of machine learning techniques, such

2
as Light GBM and XG Boost, identified key economic indicators that sig-
nificantly predict the VIX. These findings guide our selection of exogenous
covariates, enhancing the model’s robustness and relevance to current eco-
nomic conditions.
Despite the advantages of MSRs, it’s crucial to acknowledge their limi-
tations. The MSR framework, like traditional regressions, is susceptible to
endogeneity and relies heavily on assumptions like the Markov property, sta-
tionarity, and time-invariant transition probabilities [7]. Moreover, the com-
plexity and stability of the model can be challenged as the number of regimes
increases, posing potential issues in model specification and interpretation.
Overall, this review underscores the evolving landscape of volatility pre-
diction. By transitioning from traditional linear models to more sophisticated
approaches like MSRs, we address the dynamic nature of financial markets.
However, this advancement does not come without new challenges and as-
sumptions, which must be carefully considered in empirical applications.

3 Multiswitch Problem
The concept of multiswitch problems emerges from the need to model sys-
tems that exhibit multiple regimes or states, each governed by distinct equa-
tions. Consequently, the statistical moments (mean and variance) of the
dependent variable of interest are conditionally dependent on the state of
the world. Traditional linear models often struggle in these contexts, as they
inherently assume a singular, consistent pattern throughout the observed
data. Mathematically, this can be conceptualized using a piecewise function,
where different linear models apply depending on the current state or regime.
A simplified two-state model could be represented as:
(
α1 + β1 Xt + ϵ1t , if State 1
Yt = (1)
α2 + β2 Xt + ϵ2t , if State 2
where Yt s the observed dependent variable at time t, Xt is a vector of
independent variables, α and β are the parameters unique to each state, and
ϵ denotes the state-dependent error terms of each equation.
Early proposals for solving the multiswitch problem relied on determinis-
tic solutions, such as indicator functions. Among these, Quandt’s D-method
and lambda-method are notable. The D-method involves splitting the sam-

3
ple into different regimes based on an exogenously determined variable, D.
Mathematically, it’s expressed as [4]:

Yt = (I − D) · Xβ1 + D · Xβ2 + W (2)

Where W = (I − D) · ϵ1 + D · ϵ2 denotes the latent and heteroskedastic
error terms, X is the matrix of observed variables, and D is a diagonal matrix
that serves as an indicator function to denote the current state.
The lambda-method, on the other hand, uses an endogenously deter-
mined switching variable, λ, to identify the regime. Thus, we estimate the
conditional density function of the dependent variable:

h(yi |xi ) = λ · f1 (yi |xi ) + (1 − λ) · f2 (yi |xi ) (3)

These methods assume that the transition between states is clearly de-
lineated and can be captured by specific, observable indicators. However, in
complex systems such as financial markets, regime switches are often driven
by a combination of observable and unobservable factors, making these ap-
proaches less effective. The stochastic nature of market dynamics, including
unobservable shifts in investor sentiment and market conditions, cannot be
adequately captured by these deterministic methods. This limitation paves
the way for the introduction of Markov chains as a solution, offering a prob-
abilistic approach to modeling regime switches in time-series, which will be
explored in the subsequent sections.

3.1 Markov Switching Models

These models stand out for their ability to incorporate the probabilistic na-
ture of regime changes, drawing on the principles of Markov chains. The key
innovation in Markov switching models is the integration of the Markov chain
property into a regression framework, allowing for the dynamic modeling of
time series data under varying regimes. It extends the λ−method presented
in the previous section by allowing the probabilities of state to be estimated
stochastically as transition probabilities via markov chains.
The foundation of a Markov switching model lies in its ability to represent
a time series as a mixture of several distinct models, each corresponding to
a different regime. The regime at any given time point is determined by a
state variable, assumed to follow a non-observable Markov chain. This state

4
variable, typically denoted as st , is crucial as it dictates the current regime
and, consequently, the parameters of the model.
Consider the structure of a basic Markov Switching Autoregressive (MSAR)
model: (
α0 + βYt−1 + ϵt , if st = 0,
Yt = (4)
α1 + βYt−1 + ϵt , if st = 1.
In this formulation, the model parameters vary depending on the state
variable st , which is governed by a Markov process. This state-dependency
is the essence of the Markov switching model, enabling the model to adapt
its behavior based on the prevailing regime and historical patterns.
The integration of the Markov chain into the regression model is achieved
through the state variable st . The Markov chain property posits that the
probability of being in a certain state at time t is dependent solely on the
state at time t − 1, encapsulating the memoryless characteristic of Markov
processes. This memoryless assumption is supported by the financial litera-
ture in Implied Volatility (IV) forecasting that finds the VIX to be accurately
represented by a first-order model since ”no second lags turned out to be sta-
tistically significant” [1].
The transition between states is quantified by a transition probability
matrix T , which is central to the estimation of the model:

 
P (st = 0|st−1 = 0) P (st = 1|st−1 = 0)
P =
P (st = 0|st−1 = 1) P (st = 1|st−1 = 1)
 
P00 P01
=
P10 P11

The elements of this matrix, Prs , indicate the probability of transitioning

from state r to state s. These probabilities are estimated from the data
and are fundamental in determining the dynamics of the regime switching.
It followsQithat the estimated transition probabilities λi are recovered from
′ ′
λi = λ0 j=1 Pj [4].
The superiority of the Markov switching approach in handling financial
data stems from its ability to model complex and often non-linear relation-
ships inherent in financial markets. This is crucial for accurately predicting
variables like the VIX index, which exhibit distinct regimes with varying
means and variances.

5
3.1.1 Data Requirements and Model Estimation
As in any markov chain framework, stationarity is a crucial prerequisite for
the effective application of Markov switching models. In cases where the
time series data exhibit non-stationarity, such as a unit root, differencing the
series is a necessary step before applying the model. This ensures that the
underlying assumptions of the model are satisfied, leading to meaningful and
interpretable results.
Markov switching models are estimated from the data by employing the
following steps:

1. Specification of Regimes and Model Structure: Initially, the

model’s structure, including the number of regimes and the form of the
autoregressive process within each regime, must be specified. This is
based on theoretical considerations and the nature of the data.

2. Estimation of Transition Probabilities: Central to Markov switch-

ing models is the estimation of transition probabilities that govern the
likelihood of moving from one regime to another. This is typically done
using maximum likelihood estimation methods.

3. Application of Expectation-Maximization (EM) Algorithm:

The EM algorithm is often employed in the estimation of Markov
switching models. The EM algorithm iterates between the following
two steps until convergence is reached, providing estimates of both the
model parameters and the latent state variables.:

(a) E-Step (Expectation): In this step, the latent state variable

(the regime) is estimated using filtering and smoothing algorithms,
such as the Kalman smoother.
(b) M-Step (Maximization): Here, the model parameters, includ-
ing the transition probabilities, are estimated given the current
regime estimates. This is typically achieved through maximum
likelihood estimation.

In summary, Markov Regime Switching models represent a sophisticated

approach to analyzing time series data, especially in contexts where the data
exhibit distinct regimes. By integrating the Markov chain property into a
regression framework, these models allow for a dynamic and probabilistic

6
understanding of regime changes, thereby enhancing the modeling and pre-
diction of complex time series behaviors.

4 Modeling the VIX

The CBOE VIX Index is a short-term measure of real-time risk in the stock
market and is viewed as a fear index. The day-to-day movements in the VIX
Index indicate how the market’s perceptions fluctuate over time, and it is an
important tool for risk management in the capital market. The movements
of the VIX Index from day to day are of interest, not only as a good check
on the shifting market perceptions of risk, but also for volatility trading,
using options strategies, or VIX futures. Some researchers [8, 9] believe the
VIX acts as a fear index or a market perception of risk, while others [10, 11]
propose risk handling and portfolio diversification.
We start by estimating a baseline Markov model that, at each time step,
looks back over a period of time, generates values for a number a variables,
matches the current values of the variables to times where the variables have
been similar, and takes a weighted average of the next week performance of
the VIX. Due to computing limitations and the large time frame that was
tested the model was ran once a week, for a total of 520 total datapoints.
The variables in question are as follows: Current VIX Price, VIX Previous
Week Change, QQQ Previous Week Change, QQQ Onbalance Volume, and
the difference between the QQQ 200 Day and 50 Day Moving Averages.
For each of the variables above a transition probability matrix is created by
counting the amount of times the particular situation occurred in the back-
testing data. This is more clear in a simplified example. Take QQQ Previous
Week Change, and create a 2x2 matrix: P . In the presented analysis, the
transition matrix P is created to denote the relationship between variations
in the QQQ Previous Week Change and the levels of the VIX. The rows of
the matrix correspond to distinct categories of QQQ Previous Week Change,
while the columns signify different VIX values. Specifically, Row 1 denotes
instances where QQQ experienced a positive percent change in the preceding
week, while Row 2 corresponds to scenarios where QQQ exhibited a negative
percent change.
Furthermore, Column 1 denotes instances where the VIX is above 25,
while Column 2 signifies occurrences with the VIX below 25. The analyti-
cal procedure involves conducting tests for each week over the past decade.

7
Should the QQQ demonstrate a positive percent change in the preceding
week and the VIX register a value above 25, 1 is added to the corresponding
cell in the matrix, in this case the first row and the first column of P .
Following the accumulation of data points over 520 weeks, the transition
matrix (P) is computed. This matrix is derived by dividing the frequency
of occurrences of a particular event by the total number of data points. An
illustrative example of P is provided below:
 
0.3 0.7
P =
0.4 0.6

Then, after P is fully created, the program calculates the current state of
the QQQ Previous Week Change variable. If the QQQ declined the previous
week, than the VIX would have a 40 percent change of below 25 and a 60
percent change of above 25 the following week. This process is repeated for
each of the five variables. Note, the example above was a simplified matrix
to demonstrate how the transition matrices are calculated, the true matrices
in the program are 8x8 matrices with more specific cutoffs to generate a
more accurate prediction. After the transition matrices have been created
for each of the five variables, a weighted average is taken to create the final
VIX prediction.
The R2 value of this method is 0.315. The graph can be seen below. The
model follows the direction of the changes, and is able to follow the spikes in
the testing data, however, it fails to account for sharp spikes, most notably,
the COVID spike. This is due to the fact that the model did not have reliable
past data to compare the values of the indicators in beginning of the COVID
pandemic because nothing like that had happened in the previous 10 years.
Figure 1 depicts the results of the baseline model, the blue line is the actual
VIX data, and the orange line is the predicted VIX.
We followed by fitting a multivariate MSAR model. The dataset used
consists of data for the VIX (V̂IX), and the following macroeconomic vari-
ables that the literature found to be relevant predictors: gold prices (GC=F),
Crude Oil Prices (CL=F), USD Index (DX-Y.NYB), and the financial stress
index (STLFSI4). The weekly sample size spans 13 years of data, from
01/17/2010 to 01/01/2023. Data for the financial stress index was fetched
from FRED, and the remaining variables were obtained from yahoo finance.
Finally, to correct for stationarity and/or large outliers, we applied a loga-
rithmic transformation to the first three covariates and a first-differences to

8
 Figure 1: Baseline model predictions

the VIX data; leaving FSI data unchanged. Figure 2 shows the resulting
series.
Interestingly, transforming the dependent variable eliminates a lot of rel-
evant information relevant for estimating transition probabilities. In short,
by normalizing the time-series we obfuscate the difference between regimes
and consequently the estimated filtered probabilities are almost equivalent.
Given that our primary objective is to estimate transition probabilities to
predict future direction of the VIX, we proceed by fitting the model to the
raw weekly VIX data instead.

4.1 Prediction
The model is implemented in Python via the statsmodels [12] package, which
provides an API to access a variety of econometric and time-series models.
We estimate an MSAR model with the weekly VIX as the endogenous vari-
able, the matrix of transformed variables as our exogenous variables, two
regimes, a constant trend, assumed an autoregressive order of one, and non-
switching autoregressive parameters. Table 1 describes the summary results
from the fitted model
The Markov Switching Model results reveal two distinct regimes with sig-
nificantly different baseline levels, as evidenced by the intercepts for Regime
0 (14.8930) and Regime 1 (19.8009). The non-switching parameters, particu-

9
 Figure 2: Transformed time-series

larly x3, show a substantial impact across both regimes, indicating that USD
index has a significant influence on the dependent variable. The autoregres-
sive component, ar.L1, with a coefficient of 0.83 suggests that the close value
of the VIX is highly influenced by its immediate past value, a common char-
acteristic in financial time series data. We believe that our large sample size
(677) and careful selection of covariates help explain the strong statistical
significance of the exogenous macroeconomic variables.
Figure 3 shows the in-sample predictions of our model. It is clear from
the plot that the MSAR model is able to captured the dynamics of the VIX’s
time series, but in-sample predictions only provide a limited understanding
of the model’s capability to generalize.
One limitation we found with statsmodels implementation of MSAR is

10
 Dep. Variable: Close No. Observations: 676
Model: MarkovAutoregression Log Likelihood -1740.139
Date: Mon, 11 Dec 2023 AIC 3500.277
Time: 12:20:15 BIC 3545.439
Sample: 01-17-2010 HQIC 3517.763
- 01-01-2023
Covariance Type: approx
coef std err z P> |z| [0.025 0.975]
const 14.8930 1.257 11.852 0.000 12.430 17.356
coef std err z P> |z| [0.025 0.975]
const 19.8009 0.675 29.330 0.000 18.478 21.124
coef std err z P> |z| [0.025 0.975]
x1 16.9873 5.496 3.091 0.002 6.215 27.760
x2 -5.8513 2.375 -2.463 0.014 -10.507 -1.196
x3 160.9426 26.129 6.159 0.000 109.730 212.155
x4 7.6144 0.483 15.773 0.000 6.668 8.561
sigma2 7.9047 0.778 10.156 0.000 6.379 9.430
ar.L1 0.8310 0.026 32.436 0.000 0.781 0.881

Table 1: Markov Switching Model Results

that they do not support out-of-sample predictions, thus requiring a man-

ual implementation. To do so, we follow the expected maximization (EM)
approach to estimating future movements of the dependent variable via max-
imum likelihood. Alternative approaches include Markov Chain Monte Carlo
(MCMC), which uses simulation integration, but due to time constraints and
unclear relative benefits we opted for the more straightforward implementa-
tion of EM by estimating

E[yt ] = λ1 (αˆ1 + βˆ1 · E(Xˆt−1 ) + λ2 (αˆ2 + βˆ2 · E(Xˆt−1 ) (5)

where λi denotes the filtered probabilities of each regime.
Filtered probabilities are used to estimate the likelihood of the system be-
ing in a particular state at a given time, based on the observed data up to that
point. They are essential for predicting the next state in the Markov Switch-
ing framework. The estimated coefficients are extracted from the summary

11
 Figure 3: MSAR in-sample predictions

statistics in Table 1, and the filtered transition probabilities are estimated by

taking the dot product of the transition matrix with the filtered probabilities
of the last observation (i.e., 2023-01-01); both parameters readily available
from the MSAR model. The predicted state probabilities for the next period
are:
     
0 0.0721 0.005339 0.07176427
· =
1 0.9279 0.994661 0.92823573
The high probability in the second regime (0.994661) for the last obser-
vation suggests that the model is quite confident about the regime the series
was in at the end of the sample. The predicted state probabilities for the next
period (0.07176427 for regime 0 and 0.92823573 for regime 1) indicate the
model’s expectation that the series is more likely to stay in the same regime
(regime 1). We can now leverage these estimated probabilities to forecast the
value of the VIX at future time steps. A custom python function is developed

12
for this purposes, available to the reader in the Appendix and supplementary
materials. The obtained forecasts are as follows:
Time Step Forecasted VIX
1 18.0565
2 17.7305
3 17.7540
4 17.7523
5 17.7524

Table 2: Forecasted VIX Values for Future Time Steps

The MSAR model forecasts a drop in the value of the VIX in upcoming
weeks, this means that we expect a downward movement of implied volatil-
ity in the markets but still remain in the same high-volatility regime. The
model’s prediction of a continued presence in Regime 1 aligns with current
market expectation of an unsettled market, potentially driven by macroe-
conomic factors, geopolitical tensions, or other systemic risks. Hence, these
results can be interpreted as a potential future decrease in Implied Volatility,
making its way to the low-volatility regime.
In light of these predictions, investors might adopt more conservative
strategies, prioritizing assets that are traditionally seen as less volatile or
hedging their portfolios to mitigate potential risks. For options traders, this
could mean increased demand for hedging instruments, leading to higher
premiums on options contracts. On a broader scale, these forecasts could in-
fluence financial institutions and policy makers in their risk assessment and
management strategies. However, it’s important to acknowledge the model’s
limitations and the inherently unpredictable nature of financial markets. Ex-
ternal shocks and unforeseen events can rapidly alter market dynamics, un-
derscoring the need for continual monitoring and adaptation of predictive
models to new data and changing conditions.

5 Conclusion & Further Thoughts

This paper embarked on a detailed exploration of the financial markets
through the lens of the Markov Switching Autoregressive (MSAR) model,
with a specific focus on forecasting the direction of the Volatility Index (VIX).
The MSAR model’s capacity to identify distinct regimes offered a nuanced

13
understanding of market fluctuations, a key element in our analytical frame-
work. The integration of macroeconomic variables and the examination of
their influence on the VIX further enriched our analysis, providing a multi-
faceted view of market behavior.
The findings of our study are both significant and timely. The identifica-
tion of two regimes, one characterized by higher volatility, presents a critical
insight into the market’s underlying mechanisms. Our model forecasts an
upsurge in market volatility, suggesting that we are likely entering a period
marked by increased economic uncertainty. This projection is particularly
relevant given the current global economic landscape, which is fraught with
challenges ranging from geopolitical tensions to shifting monetary policies.
For investors and market analysts, this implies a need for heightened vig-
ilance and a potential reevaluation of risk management strategies. It also
underscores the value of predictive modeling in navigating the complex and
often turbulent financial markets.
While our study contributes valuable perspectives to the field of financial
forecasting, it is not without its limitations. The MSAR model, despite its
robustness, is inherently sensitive to the parameters and assumptions under-
pinning it. Our reliance on historical data, while extensive, cannot fully ac-
count for the capricious nature of financial markets, where unforeseen events
can dramatically alter trajectories. Additionally, the lack of out-of-sample
predictions in our current framework points to an area ripe for further de-
velopment. Future research could focus on enhancing the model’s predictive
power, perhaps by incorporating real-time data analysis or exploring alterna-
tive econometric techniques. The continuous evolution of financial markets
necessitates an adaptable and forward-looking approach to modeling and
analysis.

References
[1] Katja Ahoniemi. Modeling and forecasting the vix index. International
Finance, 2008.

[2] Robert E. Whaley. The investor fear gauge. 2000.

[3] Joseph Muscat. Comparing forecast accuracy of arima modelling on

vix returns before and after the introduction of vix derivatives, 2022.
https://scholarship.claremont.edu/cmc_theses/2963.

14
 [4] Stephen M. Goldfeld and Richard E. Quandt. A markov model for
switching regressions. Journal of Econometrics, 1(1):3–15, 1973.

[5] James D. Hamilton. A new approach to the economic analysis of nonsta-

tionary time series and the business cycle. Econometrica, 57(2):357–384,
1989.

[6] Akhilesh Prasad, Priti Bakhshi, and Arumugam Seetharaman. The im-
pact of the u.s. macroeconomic variables on the cboe vix index. Journal
of Risk and Financial Management, 15(3), 2022.

[7] Hans-Martin Krolzig. Markov-switching vector autoregressions: Mod-

elling, statistical inference, and application to business cycle analysis.
1997.

[8] Peter Carr. Why is vix a fear gauge? Risk and Decision Analysis,
6(2):179–185, 2017.

[9] Ghulam Sarwar. Is vix an investor fear gauge in bric equity markets?
Journal of Multinational Financial Management, 22(3):55–65, 2012.

[10] Ashok Bantwa. A study on india volatility index (vix) and its perfor-
mance as risk management tool in indian stock market. PARIPEX -
INDIAN JOURNAL OF RESEARCH, 6(1), January 2017.

[11] Gaurav Jadhao and Abhijeet Chandra. Application of vix and entropy
indicators for portfolio rotation strategies. Research in International
Business and Finance, 42:1367–1371, 2017.

[12] Skipper Seabold and Josef Perktold. statsmodels: Econometric and

statistical modeling with python. In 9th Python in Science Conference,
2010.

15
Simulations
The simulated dataset was crafted based on an Autoregressive (AR) process
of order 4 (AR(4)), with a white noise error component characterized by a
standard deviation of 0.5. This dataset incorporates two covariates, x1 and
x2, both of which follow a normal distribution. Central to our simulation is
the inclusion of three distinct regimes, each signifying a change in the mean
value of the dependent variable. The ’y’ variable in our dataset is thus a
product of an autoregressive process that is influenced by its four preceding
values (or four lags). This process is further modulated by the covariates
x1 and x2, and is subject to shifts in mean value, contingent upon the time
index, while maintaining a fixed variance. This simulation framework is
designed to test candidate models, providing an insightful understanding of
their capability to model time-series with properties that characterize Implied
Volatility.

Multivariate regressions present challenges similar to those from machine

learning models, they fit the data better but are hard to interpret. Still,
despite the better fit, it is still not capable of capturing the regime transitions
which is our main interest. On the other hand, ARIMA models fall short in
capturing the non-linear properties common in time-series describing implied
volatility. Thus demanding transformations of the data that, as shown in
section 4.1, obfuscate the regime transitions characterizing the time-series of
interest.

16
 Simple Markov Switching Regression (MSR) provides an improvement
over traditional linear models by allowing the researcher to estimate a set of
linear equations conditional on the regime. This type of conditional mean
model allows us to accurately capture the change in expected value of our
regressions, which serves in estimating transition probabilities, but fails to
capture the variance within the regimes. We observe this limitation in the
in-sample predictions of the model.

17
 Finally, Markov Switching Autoregression (MSAR) is able to capture
the complex behavior of the time-series describing the variable of interest
within each regime. This upgrade is observed by the superior in-sample fit
of the MSAR model compared to all other models we experimented with.
Because of this superior performance on the simulated scenario, we decided
to implement it in our main analysis to model future direction of the VIX.

Forecasting
def MSAR forecast ( model , data , s t e p s =1):
”””
F o r e c a s t u s i n g a Markov S w i t c h i n g A u t o r e g r e s s i o n model .

: param model : A f i t t e d MSAR model .

: param d a t a : DataFrame c o n t a i n i n g t h e e x o g e n o u s v a r i a b l e s
for the l a s t observation .
: param s t e p s : Number o f s t e p s t o f o r e c a s t .
: r e t u r n : A l i s t c o n t a i n i n g t h e f o r e c a s t f o r each s t e p .
”””
# I n i t i a l i z e predictions

18
predictions = [ ]

# E x t r a c t p a r a m e t e r s from t h e model
mu1 = model . params [ ’ c o n s t [ 0 ] ’ ]
mu2 = model . params [ ’ c o n s t [ 1 ] ’ ]

# estimated c o e f f i c i e n t s of exogenous v a r i a b l e s
b e t a s = model . params . drop ( [ ’ c o n s t [ 0 ] ’ , ’ c o n s t [ 1 ] ’ ,
’ sigma2 ’ , ’ a r . L1 ’ ,
’ p[0 − >0] ’ , ’ p[1 − >0] ’ ] ) . v a l u e s
transMat = model . r e g i m e t r a n s i t i o n

# L a t e s t o b s e r v a t i o n f o r e x o g e n o u s v a r i a b l e s used
l a s t o b s = data . i l o c [ −1 , [ 1 , 3 , 4 , 6 ] ] . v a l u e s

# I n i t i a l regime p r o b a b i l i t i e s
s t a t e p r o b s = model . f i l t e r e d m a r g i n a l p r o b a b i l i t i e s . i l o c [ −1]

r e s h a p e d t r a n s i t i o n m a t r i x = transMat . r e s h a p e ( 2 , 2 )

# Forecasting loop
for in range ( s t e p s ) :
# C a l c u l a t e regime p r e d i c t i o n s
r e g i m e 1 p r e d = mu1 + np . dot ( bet as , l a s t o b s )
r e g i m e 2 p r e d = mu2 + np . dot ( bet as , l a s t o b s )

# Weighted sum o f regime p r e d i c t i o n s

weighted pred = regime1 pred ∗ state probs [ 0 ] + \
regime2 pred ∗ state probs [ 1 ]
p r e d i c t i o n s . append ( w e i g h t e d p r e d )

# Update s t a t e p r o b a b i l i t i e s
s t a t e p r o b s = np . dot ( r e s h a p e d t r a n s i t i o n m a t r i x , s t a t e p r o b s )

return p r e d i c t i o n s

19