Chapter 4: Models for Stationary Time Series Autocorrelation function:

ρk = { 1, k=0
−θ𝑘+θ1θ𝑘+1+θ2θ𝑘+2 +...+θ𝑞−𝑘θ𝑞
Moving Average Processes { 1+θ1²+θ2²+...+θ𝑞²
), k = 1,2,...,q-1
−θ𝑞
Terminology: Suppose {et} is a zero mean white noise process with var(et) = σe² { ), k=q
1+θ1²+θ2²+...+θ𝑞²
The process Yt = et- θ1et-1- θ2et-2 - …- θqet-q is called a moving average process of order q, MA(q). { 0, k>q
MA(1) process The salient feature is that the (population) ACF:
With q = 1, the MA process defined above becomes Yt = et- θ1et-1 ; this is called an MA(1) process. - for lags k = 1, 2, ...q. ρk is nonzero
For this process: - for all lags k > q ρk=0
- the mean is E( Yt ) = E (et- θ1et-1) = 0. Auto Regressive Processes
- the variance is γ₀ = var(et- θ1et-1) =σ²e(1 + θ²)
Terminology: Suppose {et} is a zero-mean white noise process with var(et) = σₑ².
Autocovariance function:
The autocovariance function for an MA(1) process is: The process Yt= ϕ₁Yt-1 + ϕ₂Yt-2 + ...+ ϕpYt-p +et is called an autoregressive process of order p, AR(p).
γk = { σ²e (1 + θ²), k=0
● In this model, the value of the process at time t, Yt , is a weighted linear combination of the values
{ -θσ²e , k=1
of the process from the previous p time points plus a "shock" or "innovation" term et at time t.
{ 0, k>1
Autocorrelation function: ● We assume that et, the innovation at time t, is independent of all previous valuesYt-1, Yt-2,...,.
The autocorrelation function for an MA(1) process is: ● We continue to assume that E(Yt) = 0. A nonzero mean could be added to the model by replacing
ρk = γk /γ₀ = { 1, k=0 Ytwith Yt-µ (for all t). This would not affect the stationarity properties.
−θ
{ 1+θ²
, k=1
AR(1) process
{ 0, k>1
MA(2) process With p = 1, the RA process defined above becomes Yt = φYt-1 + et ; this is called an AR(1) process.

With q = 2, the MA process defined above becomes Yt = et- θ1et-1- θet-2 ; this is called an MA(2) process. Note that:

For this process: - If φ = 1, this process reduces to a random walk.

- the mean is E( Yt ) = E (et- θ1et-1- θet-2) = 0. - If φ = 0, this process reduces to white noise.

- the variance is γ₀ = var(et- θ1et-1- θet-2) =σ²e(1 + θ1²+θ2²) Autocovariance function:

Autocovariance function: γk = φᵏ (σ²e / 1 - φ²) for k = 0,1,2,...

The autocovariance function for an MA(2) process is: Autocorrelation function:

γk = {(1 + θ1² + θ2² ) σ²e , k=0 ρk = γk / γ₀ = φᵏ for k = 0,1,2,...

{(-θ1+ θ1θ2 ) σ²e , k=1 Important: For an AR(1) process, because -1 < φ < 1, the (population) ACF ρk = φᵏ decays exponentially
{ -θ2 σ²e k=2 as k increases.
{ 0, k>2 - If φ is close to ±1, then the decay will be slower.
Autocorrelation function:
- If φ is not close to ±1, then the decay will take place rapidly.
The autocorrelation function for an MA(2) process is:
- If φ > 0, then all of the autocorrelations will be positive.
ρk = γ฀ /γ₀ = { 1, k=0
−θ1+θ1θ2 - If φ < 0, then the autocorrelations will alternate from negative (k=1), to positive (k=2), to negative
{ 1+θ1²+θ2²
), k=1 (k=3), to positive (k=4), and so on.
−θ2
{ 1+θ1²+θ2²
), k=2 Remember these theoretical patterns so that when we see sample ACFs (from real data), we can make
{ 0, k>2 sensible decisions about potential model selection.
MA(q) process
With q , the MA process Yt = et- θ1et-1- θ2et-2 - …- θqet-q is called an MA(q) process. Stationarity condition:

For this process: The AR(1) process is stationary if and only if |φ| < 1, that is, if -1 < φ < 1.

- the mean is E( Yt ) = 0. The AR(1) process is not stationary if |φ| ≥ 1.

- the variance is γ₀ = var(Yt) =(1 + θ1² + θ2²+...+θq² )σ²e