Introductory Econometrics
ECON2206/ECON3209
Slides02
Lecturer: Minxian Yang
ie_Slides02
my, School of Economics, UNSW
2. Simple Regression Model (Ch2)
2. Simple Regression Model
Lecture plan
Motivation and definitions
ZCM assumption
Estimation method: OLS
Units of measurement
Nonlinear relationships
Underlying assumptions of simple regression model
Expected values and variances of OLS estimators
Regression with STATA
ie_Slides02
my, School of Economics, UNSW
2. Simple Regression Model (Ch2)
Motivation
Example 1. Ceteris paribus effect of fertiliser on soybean yield
yield = 0 + 1ferti + u .
Example 2. Ceteris paribus effect of education on wage
wage = 0 + 1educ + u .
In general,
y = 0 + 1x + u,
where u represents factors other than x that affect y.
We are interested in
explaining y in terms of x,
how y responds to changes in x,
holding other factors fixed.
ie_Slides02
my, School of Economics, UNSW
2. Simple Regression Model (Ch2)
Simple regression model
Definition
y = 0 + 1x + u ,
y : dependent variable (observable)
x : independent variable (observable)
1 : slope parameter, partial effect, (to be estimated)
0 : intercept parameter (to be estimated)
u : error term or disturbance (unobservable)
The disturbance u represents all factors other than x.
With the intercept 0, the population average of u can
always be set to zero (without losing anything)
E(u) = 0 .
y = 0 + E(u) + 1x + u E(u)
ie_Slides02
my, School of Economics, UNSW
2. Simple Regression Model (Ch2)
Zero conditional mean assumption
y = 0 + 1x + u
y + y = 0 + 1(x + x)
+ u + u
If other factors in u are held fixed (u = 0), the ceteris
paribus effect of x on y is 1 :
= change
y = 1 x .
But under what condition u can be held fixed while x
changes?
As x and u are treated as random variables,
u is fixed while x varying is described as
the mean of u for any given x is the same (zero).
The required condition is
X=
X
X
X
...
E(u |X) = 0
0
0
0
E(u | x) = E(u) = 0 ,
known as zero-conditional-mean (ZCM) assumption.
1
ie_Slides02
my, School of Economics, UNSW
2. Simple Regression Model (Ch2)
Zero conditional mean assumption
Example 2. wage = 0 + 1educ + u
Suppose u represents ability.
Then ZCM assumption amounts to
E(ability | educ) = 0 ,
ie, the average ability is the same irrespective of the
years of education.
This is not true
if people choose the education level to suit their ability;
or if more ability is associated with less (or more)
education.
In practice, we do not know if ZCM holds and have to
deal with this issue.
ie_Slides02
my, School of Economics, UNSW
2. Simple Regression Model (Ch2)
Zero conditional mean assumption
Taking the conditional expectations of
y = 0 + 1x + u
for given x, ZCM implies
E(y | x) = 0 + 1x ,
known as the population regression function
(PRF), which is a linear function of x.
The distribution of y is centred about E(y | x).
Systematic part of y : E(y | x).
Unsystematic part of y : u.
ie_Slides02
my, School of Economics, UNSW
2. Simple Regression Model (Ch2)
Simple regression model
yi = 0 + 1xi + ui
y
distribution of u
conditional mean of y given x
(population regression line)
E(y| x = x3)
= 0 + 1x3
E(y| x = x2)
distribution of y
for given x = x3
E(y| x = x1)
x1
ie_Slides02
x2
x3
my, School of Economics, UNSW
x
8
2. Simple Regression Model (Ch2)
Observations on (x, y)
A random sample is a set of independent
observations on (x, y), ie, {(xi , yi), i = 1,2,...,n}.
At observation level, the model may be written as
yi = 0 + 1xi + ui , i = 1, 2, ..., n
where i is the observation index.
Collectively,
y1
x1 u1
y1 1
1
y
x u
y 1
1
2
2
2
, or 2
0
y
x
u
1
n
n n
yn 1
Matrix notation:
ie_Slides02
x1
u1
x2 0 u2
.
1
xn
u n
Y X B U.
my, School of Economics, UNSW
2. Simple Regression Model (Ch2)
Estimate simple regression
The model:
yi = 0 + 1xi + ui ,
i = 1, 2, ..., n
Let ( 0 , 1 ) be the estimates of (0 , 1).
Corresponding residual is
ui y i 0 1xi , i 1,2,..., n.
The sum of squared residuals (SSR)
n
SSR u ( y i 0 1 xi )2
i 1
2
i
i 1
indicates the goodness of the estimates.
Good estimates should make SSR small.
ie_Slides02
my, School of Economics, UNSW
10
2. Simple Regression Model (Ch2)
Ordinary least squares (OLS)
The OLS estimates ( 0 , 1 ) minimise the SSR:
( 0 , 1 ) minimiser of SSR.
Choose ( 0 , 1 ) to minimise SSR.
The first order conditions lead to
n
( y i 0 1xi ) 0,
mean residual = 0
i 1
n
(y
i 1
0 1 xi )xi 0.
covariance of
residual and x
=0
ie_Slides02
my, School of Economics, UNSW
11
2. Simple Regression Model (Ch2)
Ordinary least squares (OLS)
Solving the two equations with two unknowns gives
n
(x
i 1
x )( y i y )
2
(
x
x
)
i
0 y 1 x ,
i 1
where
1 n
y yi ,
n i 1
1 n
x xi .
n i 1
OLS requires the condition
n
2
(
x
x
)
0.
i
i 1
ie_Slides02
my, School of Economics, UNSW
12
2. Simple Regression Model (Ch2)
OLS regression line or SRF
For any set of data {(xi , yi), i = 1,2,...,n} with n > 2,
OLS can always be carried out as long as
n
2
(
x
x
)
0.
i
i 1
Once OLS estimates are obtained, y i 0 1xi
is known as the fitted value of y when x = xi.
By OLS regression line or sample regression
function (SRF), we refer to
y 0 1x,
which is an estimate of PRF E(y | x) = 0 + 1 x.
ie_Slides02
my, School of Economics, UNSW
13
2. Simple Regression Model (Ch2)
Interpretation of OLS estimate
In the SRF y 0 1x,
the slope estimate 1 is the change in y when x
increases by one unit: 1 y / x,
which is of primary interest in practice.
The dependent variable y may be decomposed either
as the sum of the SRF and the residual
y y u
or as the sum of the PRF and the disturbance.
y E( y | x ) u.
ie_Slides02
my, School of Economics, UNSW
14
2. Simple Regression Model (Ch2)
PRF versus SRF
Hope: SRF = PRF on average or when n goes to infinity.
y i 0 1 xi ui
(xi, yi)
y
sample
regression
line 0 1x
residual
ui
population
regression
line 0+ 1x
ie_Slides02
my, School of Economics, UNSW
15
2. Simple Regression Model (Ch2)
OLS example
10
15
20
wa ge 0.90 0.54 educ ,
Population : workforce in 1976
y = wage : hourly earnings (in $)
x = educ : years of education
OLS SRF : n = 526
wage
25
Example 2. (regress wage educ)
10
15
educ
Interpretation
Slope 0.54 : each additional year of schooling increases
the wage by $0.54.
Intercept -0.90 : fitted wage of a person with educ = 0?
SRF does poorly at low levels of education.
Predicted wage for a person with educ = 10?
ie_Slides02
my, School of Economics, UNSW
16
2. Simple Regression Model (Ch2)
Properties of OLS
The first order conditions:
n
(y
i 1
(y
i 1
0 1 xi ) 0,
0 1xi )xi 0
imply that
the sum of residuals is zero.
the sample covariance of x and the residual is zero.
the mean point ( x, y ) is always on the SRF (or OLS
regression line).
ie_Slides02
my, School of Economics, UNSW
17
2. Simple Regression Model (Ch2)
Sums of squares
Each yi may be decomposed into y i y i ui .
Measure variations from y :
Total sum of squares (total variation in yi ):
SST i 1 ( y i y )2 ,
n
Explained sum of squares (variation in y i ):
SSE i 1 ( y i y )2 ,
n
sum of squared Residuals (variation in u i ):
SSR i 1 u i2 .
n
It can be shown that
ie_Slides02
SST = SSE + SSR .
my, School of Economics, UNSW
18
2. Simple Regression Model (Ch2)
R-squared: a goodness-of-fit measure
How well does x explain y?
or how well does the OLS regression line fit data?
We may use the fraction of variation in y that is
explained by x (or by the SRF) to measure.
R-squared (coefficient of determination):
SSE
SSR
R2
1
.
Not advisable to put
SST
SST
too much weight on
larger
better fit;
0 R2 1.
R2,
R2 when evaluating
regression models.
eg. R2 = 0.165 for wa ge 0.90 0.54 educ ,
16.5% of variation in wage is explained by educ.
ie_Slides02
my, School of Economics, UNSW
19
2. Simple Regression Model (Ch2)
Effects of changing units of measurement
If y is multiplied by a constant c, then the OLS
intercept and slope estimates are also multiplied by c.
If x is multiplied by a constant c, then the OLS
intercept estimate is unchanged but the slope
estimate is multiplied by 1/c.
The R2 does not change when varying the units of
measurement.
eg. When wage is in dollars, wa ge 0.90 0.54 educ .
If wage is in cents, wa ge 90 54 educ .
ie_Slides02
my, School of Economics, UNSW
20
2. Simple Regression Model (Ch2)
Nonlinear relationships between x and y
The OLS only requires the regression model
y = 0 + 1x + u
to be linear in parameters.
Nonlinear relationships between y and x can be easily
accommodated.
1
0
lwage
eg. Suppose a better description
is that each year of education
increases wage by a fixed
percentage. This leads to
log(wage) = 0 + 1 educ + u ,
with %wage = (1001)educ
when u= 0.
OLS: lwa ge 0.584 0.083 educ ,
ie_Slides02
my, School of Economics, UNSW
10
15
educ
R 2 0.186
21
2. Simple Regression Model (Ch2)
Nonlinear relationships between x and y
Linear models are linear in parameters.
OLS applies to linear models no matter how x and y
are defined.
But be careful about the interpretation of .
ie_Slides02
my, School of Economics, UNSW
22
2. Simple Regression Model (Ch2)
OLS estimators
A random sample, containing independent draws
from the same population, is random.
A data set is a realisation of the random sample.
OLS estimates ( 0 , 1 ) computed from a random
sample is random, called the OLS estimators.
To make inference about the population parameters
(0, 1), we need to understand the statistical
properties of the OLS estimators.
In particular, we like to know the means and
variances of the OLS estimators.
We find these under a set of assumptions about the
simple regression model.
ie_Slides02
my, School of Economics, UNSW
23
2. Simple Regression Model (Ch2)
Assumptions about simple regression model
(SLR1 to SLR4)
1. (linear in parameters) In the population model, y is
related to x by y = 0 + 1 x + u, where (0, 1) are
population parameters and u is disturbance.
2. (random sample) {(xi , yi), i = 1,2,...,n} with n > 2 is a
random sample drawn from the population model.
3. (sample variation) The sample outcomes on x are
not of the same value.
4. (zero conditional mean) The disturbance u satisfies
E(u | x) = 0 for any given value of x. For the random
sample, E(ui | xi) = 0 for i = 1,2,...,n.
ie_Slides02
my, School of Economics, UNSW
24
2. Simple Regression Model (Ch2)
Property 1 of OLS estimators
Theorem 2.1
Under SLR1 to SLR4, the OLS estimators are
unbiased: E( 1 ) 1, E( 0 ) 0 .
Unbiased estimators ( 0 , 1 )
they are centred around (0, 1).
they correctly estimate (0, 1) on average.
It is useful to note that ( y i y ) 1( xi x ) (ui u ),
1 1
ie_Slides02
n
i 1
(ui u )( x i x )
n
i 1
( xi x )
The estimation error is
entirely driven by a linear
combination of ui with
weights dependent on x.
my, School of Economics, UNSW
25
2. Simple Regression Model (Ch2)
Property 2 of OLS estimators
5. (SLR5, homoskedasticity)
Var(ui|xi) = 2 for i = 1,2,...,n. (It implies Var(ui) = 2.)
Strictly, Theorem 2.2 is about the variances
of OLS estimators, conditional on given x.
Theorem 2.2
Under SLR1 to SLR5, the variances of ( 0 , 1 ) are:
Var ( 1 )
n
i 1
( xi x )
2 n 1 i 1 x i2
n
2
2
, Var ( 0 )
n
i 1
( xi x )
the larger is 2, the greater are the variances.
the larger the variation in x, the smaller the variances.
ie_Slides02
my, School of Economics, UNSW
26
2. Simple Regression Model (Ch2)
Homoskedasticity and heteroskedasticity
ie_Slides02
my, School of Economics, UNSW
27
2. Simple Regression Model (Ch2)
Estimation of 2
As the residual approximates u, the estimator of 2 is
2
u
SSR
i
2
i 1 .
n2
n2
n
2 is the number of
estimated coefficients
2 is known as the standard error of the
regression, useful in forming the standard errors of
( 0 , 1 ).
Theorem 2.3 (unbiased estimator of 2)
Under SLR1 to SLR5, E ( 2 ) 2 .
ie_Slides02
my, School of Economics, UNSW
28
2. Simple Regression Model (Ch2)
OLS in STATA
standard
error of
regression
SSR
ie_Slides02
my, School of Economics, UNSW
29
2. Simple Regression Model (Ch2)
Summary
What is a simple regression model?
What is the ZCM assumption? Why is it crucial for
model interpretation and OLS being unbiased?
What is the OLS estimation principle?
What are PRF, SRF, error term and residual?
How is R-squared is related to SSR?
Can we describe, in a simple linear regression model,
the nonlinear relationship between x and y?
What are Assumptions SLR1 to SLR5? Why do we
need to understand them?
What are the statistical properties of OLS estimators?
How do you OLS in STATA? regress y x
ie_Slides02
my, School of Economics, UNSW
30
