CHAPTER 15: INTRODUCION TO THE ANALYSIS OF  The probability of doing an experiment with many

VARIANCE groups and analyzing the data with more than one
comparison
INTRODUCTION: THE F DISTRIBUTION
OVERVIEW OF ONE-WAY ANOVA
 Developed by R.A. Fisher
 Fobt, the ratio of 2 independent variance estimates of  Used to analyze multigroup experiments
the same population variance 𝜎 2  Using the F test allows to make one overall
comparison that tells whether there is a significant
𝒗𝒂𝒓𝒊𝒂𝒏𝒄𝒆 𝒆𝒔𝒕𝒊𝒎𝒂𝒕𝒆 𝟏 𝒐𝒇 𝝈𝟐
Fobt = 𝒗𝒂𝒓𝒊𝒂𝒏𝒄𝒆 𝒆𝒔𝒊𝒎𝒂𝒕𝒆 𝟐 𝒐𝒇 𝝈𝟐 difference between the means of the groups
 It avoids the problem of an increased probability of
 Generated empirically by Type I error that occurs when assessing many t
 Taking all possible samples of size n1 and n2 values
from the same population  Used in both independent groups and repeated
 Estimating the population variance 𝜎 2 from measure designs
each of the samples using s12 and s22  Used when one or more factors are investigated in
 Calculating Fobt for all possible combinations of the same experiment
s12 and s22
 Simple randomized-group design/one-way analysis
 Calculating p(F) for each different value of Fobt
of variance, independent groups design
 Sampling distribution of F gives all possible F values
 Single factor experiment, independent groups
along with the p(F) for each value, assuming
design, subjects are randomly sampled from the
sampling is random from the population
population and then randomly assigned to the
 Varies with 2 degrees of freedom, numerator and conditions
denominator
 There are as many independent groups as there are
 Df for numerator = df1 = n – 1
conditions
 Df for denominator = df2 = n - 1
 If the study is investigating the effect of an
 F ratio never have a negative value (s12 and s22 will
independent variable as a factor, then the conditions
always be positive)
would be the different levels of the independent
 F distribution is positively skewed variable used
 Median F value is approximately equal to 1  Scores from several independent groups are
F TEST AND THE ANALYSIS OF VARIANCE (ANOVA) analyzed
 H1 in ANOVA is nondirectional
 The 2 group study involving a control group and  One or more of the conditions have different effects
experimental group often aren’t sufficient to allow a from at least one of the others on the dependent
clear interpretation of the findings variable
 Experiments that required more than 2 groups  Ho states that the different conditions are equally
involves experiments in which the independent effective, in which case the scores in each group are
variable is varied as a factor; a predetermined range random samples from populations having the same
of the independent variable is selected, and several mean value
values spanning the range are used in the  ANOVA assumes that only the mean of the scores is
experiment affected by the independent variable, not the
 In an experiment of 2 groups, there would be just variance
one t calculation, and we would compare tobt with
tcrit to see whether tobt fell in the critical region for 𝝈𝟐 𝟏 = 𝝈𝟐 𝟐 = 𝝈 𝟐 𝟑 = ⋯ = 𝝈𝟐 𝒌
rejecting H0
 ANOVA partitions the total variablility of the data
 When involving many comparisons, the probability
(SStotal) into 2 sources:
of getting t values equal to or greater than tcrit goes
 Variability within each group, within-groups
up
sum of variances (SSwithin)
  Variability that exists between the groups, 2
Estimate of 𝜎𝑋
between-groups sum of squares (SSbetween)
 Each sum of squares is used to form an independent  𝑋 = grand mean (overall mean of all scores
estimate of the H0 population variance combined)
 Within-groups variance estimate (MSwithin)and  𝐾= number of sample means = number of groups
between-groups variance estimate (MSbetween)  Expanding it will result in conceptual equation for
𝑴𝑺𝒃𝒆𝒕
the between-groups variance estimate
F ratio = 𝑴𝑺𝒘  This equation can only be used when there is the
same number of subjects in each group
 MSbet increase with the magnitude of the
 Numerator is called the between-groups sum of
independent variable’s effect
squares
 MSw is unaffected
 Larger the F ratio, more unreasonable the H0 𝑺𝑺𝒃𝒆𝒕
MSbet = 𝒅𝒇𝒃𝒆𝒕
 Fobt equal or greater than Fcrit, reject and vice versa
 SSbet = between-groups sum of squares
WITHIN-GROUPS VARIANCE ESTIMATE, MSWITHIN
 Dfbet = between-group degrees of freedom
 Based on the variability within each group  As the effect of the independent variable increases,
 It tells how large the differences are between the the difference between the sample means increase
scores within each group and the group mean  MSbet increases with the effect of the independent
 MSw, Mean Square within variable
 Same variance used in t test for independent groups (𝑿𝟏𝟐 ) (𝐚𝐥𝐥 𝐬𝐜𝐨𝐫𝐞𝐬 ∑ 𝑿)𝟐
SSw = [ 𝒏𝟏
+ ….] - 𝑵
𝑺𝑺𝟏+𝑺𝑺𝟐+𝑺𝑺𝟑 +⋯+𝑺𝑺𝒌
MSwithin = (𝒏𝟏−𝟏)+⋯
Computational equation for betweenn-groups sum of
Conceptual equation for MSw squares
𝑺𝑺𝟏+𝑺𝑺𝟐+𝑺𝑺𝟑 +⋯+𝑺𝑺𝒌 THE F RATIO
MSwithin =
𝑵−𝒌
 F increases with the effect of the independent
 Numerator is called the within-groups sum of square
variable
𝑺𝑺𝒘
MSwithin = 𝒅𝒇𝒘  The larger the F ratio, the more reasonable it’s that
the independent variable has had a real effect
 SSw = within-groups sum of squares
𝑴𝑺𝒃𝒆𝒕 𝝈𝟐 + 𝒊𝒏𝒅𝒆𝒑𝒆𝒏𝒅𝒆𝒏𝒕 𝒗𝒂𝒓𝒊𝒂𝒃𝒍𝒆 𝒆𝒇𝒇𝒆𝒄𝒕𝒔
 Dfw = within-group degrees of freedom F ratio =
𝑴𝑺𝒘
=
𝝈𝟐

SSw = all scores ∑ 𝑿𝟐 − [

(𝑿𝟏𝟐 )
+ ….]  If equal or more than Fcrit, reject and vice versa
𝒏𝟏
 If less than 1, retain H0
Computational equation for within-groups sum of
LOGIC UNDERLYING THE ONE-WAY ANOVA
squares
 Deviation of each score from the grand mean =
BETWEEN-GROUPS VARIANCE ESTIMATE, MSBETWEEN
deviation of the score from its own group mean +
 Based on the variability between groups deviation of that group mean from the grand mean
 MSbet, between-groups variance estimate  SStotal = SSw + SSbet
 H0 states that each group is a random sample from  MSw isn’t a measure of the real effect of the
populations where 𝜇2 1 = 𝜇2 2 = 𝜇2 3 = ⋯ = 𝜇2 𝑘 independent variable
 If H0 is correct , then we can use the variability  MSw is an estimate of 𝜎 2 that’s unaffected by
between the means of the samples to estimate the treatment differences
variance of these populations, 𝜎 2  MSw isn’t sensitive to independent variable effects
𝟐
while MSbet is
𝟐 ∑(𝑿 − 𝑿𝑮)
𝑺𝑿 =
𝒌−𝟏
 The greater the effect of the independent variable,  Biased estimate of this proportion in the population
the more the means of each group will differ from  Biased estimate is usually larger than the true size of
each other and affecting MSbet the effect
 If the independent variable has no effect, then both 𝑺𝑺𝒃𝒆𝒕
MSbet and MSw are independent estimates of 𝜎 2 η2= 𝑺𝑺𝒕
and their ratio is disturbed as F with df = df bet and
POWER OF THE ANALYSIS OF VARIANCE
dfw
POWER AND N
RELATIONSHIP BETWEEN ANOVA AND THE T TEST
 Power varies directly with N
 T2 = F
 Increases in Fobt also increases power
 When 2 independent groups are involved and H0 is
tested, we can use ANOVA or t test POWER AND THE REAL EFFECT OF THE INDEPENDENT
VARIABLE
ASSUMPTIONS UNDERLYING THE ANALYSIS OF
VARIANCE  Power varies directly with the size of the real effect
of the independent variable
 Normally distributed samples from populations
 Larger the real effect of the independent variable is,
 Samples are drawn from populations of equal
the higher is the power
variances
 ANOVA assumes homogeneity of variance POWER AND SAMPLE VARIABILITY
 ANOVA is a robust test
 Power varies inversely with the sample variability
 It’s minimally affected by violations of population
 Increase in within-group variability result in
normality
decreases in power
 It’s relatively sensitive to violations of homogeneity
variance, provided the samples are equal of size MULTIPLE COMPARISONS

SIZE OF EFFECT USING 𝝎𝟐 OR η2  In one-way ANOVA, a significant F indicates all the

conditions don’t have the same effect on the
COHEN’S CRITERIA FOR INTERPRETING THE VALUE OF
dependent variable
𝝎𝟐 OR η2
𝟐 2  A significant F tells that at least one conditions
𝝎 OR η (PROPORTION INTERPRETATION
OF VARIANCE differs from at least one of the others
ACCOUNTED FOR)  To determine which conditions differ, multiple
0.01 – 0.05 SMALL EFFECT comparisons between pairs of group means are
0.06 – 0.13 MEDIUM EFFECT usually made
≥0.14 LARGE EFFECT
A PRIORI, OR PLANNED, COMPARISONS

 Specific comparisons that are planned in advance of

OMEGA SQUARED, 𝝎𝟐
the experiment and often arise from predictions
 Unbiased estimate of this proportion in the based on theory and prior research
population  Directional or non-directional comparisons
 Don’t correct for the higher probability of Type I
𝝈𝟐 𝒃𝒆𝒕 error and because of it, it’s higher than post hoc
𝝎𝟐 =
𝝈𝟐 𝒃𝒆𝒕 + 𝝈𝟐 𝒘
𝑿𝟏 − 𝑿𝟐
Conceptual equation 𝒕𝒐𝒃𝒕 =
𝑺𝑺𝟏+𝑺𝑺𝟐 𝟏 𝟏
𝑺𝑺𝒃𝒆𝒕 − (𝒌 − 𝟏)𝑴𝑺𝒘 √( )( + )
𝟐 𝒏𝟏+𝒏𝟐−𝟐 𝒏𝟏 𝒏𝟐
𝝎 =
𝑺𝑺𝒕 + 𝑴𝑺𝒘
T equation for independent groups
Computational equation

ETA SQUARED, η2
 𝑿𝟏 − 𝑿𝟐  Maintains the Type I error rate at 𝛼 when controlling
𝒕𝒐𝒃𝒕 = for all possible comparisons, not just pair-wise mean
𝟏 𝟏
√𝑴𝑺𝒘 ( + 𝒏𝟐) comparisons
𝒏𝟏
 Most conservative
General t equation for planned comparisons
 Limits the probability of making the Type I error rate
𝑿𝟏 − 𝑿𝟐 at 𝛼 when controlling for all possible post hoc
𝒕𝒐𝒃𝒕 = comparisons
𝟐𝑴𝑺𝒘
√  Safest post hoc test one can use in protecting against
𝒏
making Type I error
T equation for planned comparisons with equal n in the  Often used only with pair-wise mean comparisons
2 groups  It uses dfbet, MSw and Fcrit to protect from making
A POSTERIORI, OR POST HOC, COMPARISONS Type I error

 Comparisons aren’t planned before conducting the 𝑺𝑺𝒃𝒆𝒕 (𝒈𝒓𝒐𝒖𝒑𝒔 𝒊 𝒂𝒏𝒅 𝒋)

𝟐 𝟐
experiment (∑ 𝑿𝒊) (∑ 𝑿𝒋)
 Aren’t based on theory and prior research =[ + ]
𝒏𝒊 𝒏𝒋
 The comparisons arise after the experimenter sees
(𝒈𝒓𝒐𝒖𝒑𝒔 𝒊 𝒂𝒏𝒅 𝒋 ∑ 𝑿)𝟐
the data and picks groups with mean scores that are −
far apart 𝒏𝒊 + 𝒏𝒋
 Or they may arise from doing all mean comparisons Computational equation for SSbet groups I and j
possible with no theoretical priori basis
 Correct for the higher probability of Type I error Dfbet = k – 1
𝑺𝑺𝒃𝒆𝒕
THE TUKEY HONESTY SIGNIFICANT DIFFERENCE (HSD) 𝑴𝑺 (𝒈𝒓𝒐𝒖𝒑𝒔 𝒊 𝒂𝒏𝒅 𝒋) = 𝒅𝒇𝒃𝒆𝒕
TEST
𝑴𝑺𝒃𝒆𝒕
𝑭 𝑺𝒄𝒉𝒆𝒇𝒇𝒆 =
 Maintains the Type I error rate at 𝛼 when controlling 𝑴𝑺𝒘
for all possible comparisons between pairs of means
COMPARISON BETWEEN PLANNED COMPARISONS,
 Avoids inflated probability of making Type I error
THE TUKEY HSD, AND THE SCHEFFE TEST
that would result from making these comparisons if t
and the sampling distribution of t were used, by  Planned comparisons are more powerful than post
using a new statistic Q and the sampling distribution hoc comparisons
of Q  Tukey HSD is more powerful than Scheffe test
 Studentized range distribution, sampling  Planned comparisons is the method of choice when
distribution of Q applicable because of its greater power
 Qobt is always positive to avoid Q statistic to be like t  Planned comparisons should be relatively few and
statistic should flow meaningfully and logically from the
 Qobt ≥ Qcrit, reject and vice versa experimental design
 Tukey HSD if interest centers on getting the most
𝑿𝒊 − 𝑿𝒋
𝑸𝒐𝒃𝒕 = power while at the same time reasonably controlling
𝑴𝑺𝒘 Type I error
√
𝒏
 Controlling Type I error for all possible comparisons
 Xi = larger of the 2 means being compared reduces power of the Scheffe test
 Xj = smaller of the 2 means being compared  Pair-wise mean = Tukey HSD
 MSw = within-groups variance estimate  Otherwise = Scheffe
 N = number of subjects in each group

THE SCHEFFE TEST