Chapter 25:

Analysis of
Experiments
and
Observational
Studies

1
When to use a one-way ANOVA

The null hypothesis (H0) of ANOVA is that there is no

difference among group means. The alternative
hypothesis (Ha) is that at least one group differs
significantly from the overall mean of the dependent
variable.
Analysis of variance (ANOVA) is an inferential method
used to test the equality of three or more population
means.

1-2
 Analysis of Variance
SST=SSC+SSE
𝒄 𝒏𝒊 𝒄 𝒄 𝒏𝒊

ഥ)𝟐 = ෍ 𝒏𝒋 (ഥ
෍ ෍(𝒙𝒊𝒋 − 𝒙 ഥ)𝟐 + ෍ ෍(𝒙𝒊𝒋 − 𝒙
𝒙𝒋 − 𝒙 ഥ𝒋 )𝟐
𝒋=𝟏 𝒊=𝟏 𝒋=𝟏 𝒋=𝟏 𝒊=𝟏

Note: The formula ensures that larger groups contribute proportionally

more to the between-group variability.

1-3
Test Statistic

1-4
 Example 2

An environmentalist wanted to determine if the mean acidity of rain

differed among Alaska, Florida, and Texas. He randomly selected six rain
dates at each site obtained the following data:

1-5
Solution

1-6
 Solution

Notice that there are differences among the sample means. Are the differences small
enough to be explained solely by sampling variability? Or are they of sufficient
magnitude so that a more reasonable explanation is that the μ’s are not all equal? The
conclusion depends on how much variation among the sample means (based on their
deviations from the grand mean) compares to the variation within the three samples.

1-7
Solution

1-8
25.2 Assumptions and Conditions for
ANOVA
Independence Assumption

The groups must be independent of each other.

No test can verify this assumption.

You have to think about how the data were collected and
check that the Randomization Condition is satisfied.

25-9
25.2 Assumptions and Conditions for
ANOVA
Equal Variance Assumption

ANOVA assumes that the true variances of the treatment groups

are equal. We can check the corresponding Similar Variance
Condition in various ways:
• Look at side-by-side boxplots of the groups. Look
for differences in spreads.

• Examine the boxplots for a relationship between

the mean values and the spreads. A common
pattern is increasing spread with increasing mean.

25-10
25.2 Assumptions and
Conditions for ANOVA

1-11
25.2 Assumptions and Conditions for
ANOVA
Normal Population Assumption

Like Student’s t-tests, the F-test requires that the underlying

errors follow a Normal model. As before when we faced this
assumption, we’ll check a corresponding Nearly Normal
Condition.
• Examine a histogram of all the residuals.
• Example a Normal probability plot.

25-12
25.2 Assumptions and Conditions for
ANOVA
Normal Population Assumption
For the Tom’s Tom-Toms experiment, the residuals are not
Normal. In fact, the distribution exhibits bimodality.

25-13
25.2 Assumptions and Conditions for
ANOVA
Normal Population Assumption
The bimodality shows
up in every treatment!

This bimodality came

as no surprise to the
manager. He
responded,
“…customers …either
order a complete new
drum set,
or…accessories… or
choose not to
purchase anything.”

25-14
25.2 Assumptions and Conditions for
ANOVA
Normal Population Assumption
These data (and the residuals) clearly violate the Nearly Normal
Condition.

Does that mean that we can’t say anything about the null hypothesis?

No. Fortunately, the sample sizes are very large, and there are no
individual outliers that have undue influence on the means.

With sample sizes this large, we can appeal to the Central Limit
Theorem and still make inferences about the means.

In particular, we are safe in rejecting the null hypothesis.

25-15
 Practice Problem
Management styles differ among organizations and may potentially affect employee’s job
satisfaction. A sample of employees was randomly selected from each of three
companies with different management styles (Authoritarian, Laissez faire and
Participative) and asked to rate their level of job satisfaction on a 10-point scale (10 being
the highest level of satisfaction). The data collected and partial ANOVA results appear
below.

One-way ANOVA: Authoritarian, Laissez, Participative

Source DF SS MS F P
Factor 2 97.40
Error 27 37.40
Total 29 134.80

1-16
 Practice Problem
1. What are the null and alternative hypotheses (in words, not symbols)?

H0 : Mean employee job satisfaction ratings are equal across companies with
different management styles.
HA : Mean employee job satisfaction ratings are not all equal across companies with
different management styles.

2. Are the conditions / assumptions for ANOVA met?

Employees were randomly selected from each of the three companies. Boxplots of
the original data indicate that the equal variance assumption is reasonable. The
normal probability plot of residuals indicates that the normality assumption is
reasonable.

3. Calculate the F-statistic.

F = 35.16
F_tabulated=3.35

Calculated F> Tabulated F

1-17
.
 Practice Problem

4. The P-value for this statistic turns out to be < .001. State the conclusion.

With such a small P-value, we can reject the null hypothesis and conclude that
mean employee job satisfaction ratings are not all equal across companies with
different management styles.

1-18
1-19
 Practice Problem
A health food company wants to determine the best process for blanching dark green
vegetable material so it can be added to other food products without losing vitamin A.
They are examining three different blanching processes to determine which is best in
terms of vitamin A retention (mg/100mg). The data collected and partial ANOVA results
appear below.

One-way ANOVA: Process 1, Process 2, Process 3

Source DF SS MS F P
Factor 2 1282.8
Error 21 578.9
Total 23 1861.6

1-20
 Practice Problem
1. What are the null and alternative hypotheses (in words, not symbols)?
H0 : Mean vitamin A retention amounts are equal across the three blanching processes.
HA : Mean vitamin A retention amounts are not all equal across the three blanching
processes.

2. Are the conditions / assumptions for ANOVA met?

We need to assume that the vegetable material samples were randomly assigned to the processes.
Boxplots of the original data indicate that the distributions are symmetric but that the equal variance
assumption may not be reasonable. The normal probability plot of residuals indicates that the normality
assumption is reasonable. We proceed with caution.

3. Calculate the F-statistic.

F = 23.27
4. The P-value for this statistic turns out to be < .001. State the conclusion.
With such a small P-value, we can reject the null hypothesis and conclude that mean vitamin A retention
amounts are not all equal across the different blanching processes.

1-21
1-22
Two-Way Analysis of Variance
• In doing a study that involves a two-way
analysis of variance, the researcher is able to
test the effects of two independent variables or
factors on one dependent variable.
• In addition, the interaction effect of the two
variables can be tested.
• The groups for a two-way ANOVA are sometimes
called treatment groups.
• A two-way ANOVA has several null hypotheses.
There is one for each independent variable and
one for the interaction.
The Two-way ANOVA
• We need to test for the independent and combined effects
of multiple variables on performance. We do this with an
ANOVA that asks:

(i) How different from each other are the means for levels
of Variable A?
(ii) How different from each other are the means for levels
of Variable B?
(iii) How different from each other are the means for the
treatment combinations produced by A and B together?

Two Way ANOVA

1-25
1-26
 A company sells three items: Swimming pools, spas, and
saunas. The owner decides to see whether the age of the
Example: sales representative and the type of item affect monthly sales.
At α = 0.05, analyze the data shown, using a two-way ANOVA.
Sales are given in hundreds of dollars for a randomly selected
month, and five salespeople were selected for each group.
Column
Headings
Pool Spa Sauna
Over 30 56 43 47
23 25 43
Row 52 16 52
Headings 28 27 61
35 32 74
30 or under 16 58 15
14 62 14
18 68 22
27 72 16
31 83 27
 Examples

The blood pressure example is a 2 X 2 design

Factor A (diet modification) has two levels
Factor B (drug therapy) has two levels .

The lecture comprehension example is a 3 X 2

design Factor A (type of lecture) has three levels
Factor B (method of presentation) has two
levels

1-28
 Hypothesis Test
The hypotheses are longer than you are used too.
There will be 3 sets of Hypotheses:
1) Difference in means with respect to the ROW HEADINGS
H0: There is no difference in means with respect to RH1 and RH2
H1: There is a difference in means with respect to RH1 and RH2
2) Difference in means with respect to COLUMN HEADINGS
H0: There is no difference in means with respect to CH1 and CH2
H1: There is a difference in means with respect to CH1 and CH2
3) Interaction between the ROW headings (RH)/COLUMN headings(CH)
H0: There is no interaction effect between RH/CH
H1: There is an interaction effect between RH/CH
The Two-way ANOVA
• The first two of those questions are questions
about main effects of the respective independent
variables.

• The third question is about the interaction effect,

the effect of the two variables considered
simultaneously.

Two Way ANOVA

1-31
1-32
1-33
Example:

1-34
1-35
 Interpretation of different calculations
• SSbtw results from the sum of squares of the group means minus the
overall mean multiplied by the number of values in the groups.
• The sums of squares of the factors SSA and SSB result from the sum
of squares of the means of the factor levels minus the total mean.
• Now we can calculate the sum of squares for the interaction. These
are obtained by calculating SSbtw minus SSA minus SSB.
• Finally, we calculate the sum of squares for the error. This will
calculate similar to the total sum of squares, so again we use each
individual value. Only in this case, instead of subtracting the overall
mean from each value, we subtract the respective group mean from
each value.

1-36
1-37
Mean Squares (MS)
Together with the sums of squares and the degrees of freedom, the mean
squares can now be calculated:

MS_tot = SStot / dftot = 84.8 / 19 = 4.46

MS_btw = SSbtw / dfbtw = 7.6 / 3 = 2.53

MS_A = SSA / dfA = 5 / 1 = 5

MS_B = SSB / dfB = 0.8 / 1 = 0.8

MS_AB = SSAB / dfAB = 1.8 / 1 = 1.8

MS_Within = SSwithin / dferr = 77.2 / 16 = 4.83

1-38
 𝑀𝑆𝐴 5
𝐹𝐴 = = = 1.04
𝑀𝑆𝑤𝑖𝑡ℎ𝑖𝑛 4.83
𝑀𝑆𝐵 0.8
𝐹𝐵 = = = 0.17
𝑀𝑆𝑤𝑖𝑡ℎ𝑖𝑛 4.83
𝑀𝑆𝐴𝐵 1.8
𝐹𝐴𝐵 = = = 0.373
𝑀𝑆𝑤𝑖𝑡ℎ𝑖𝑛 4.83

1-39
 Sum of df Mean F p
Squares Squares
𝑉𝑎𝑟𝑖𝑎𝑏𝑙𝑒 𝐴 Studied 5 1 5 1.04 .324
ቐ 𝑉𝑎𝑟𝑖𝑎𝑏𝑙𝑒 𝐵 Gender 0.8 1 0.8 0.17 .689
𝐼𝑛𝑡𝑒𝑟𝑎𝑐𝑡𝑖𝑜𝑛 Studied x 1.8 1 1.8 0.37 .55
Gender
Within 77.2 16 4.83
Total 668 19

1-40
 Interpreting two-way ANOVA
• The most important in this table are the three marked rows. With these three
rows, you can test whether the 3 null hypotheses we made earlier are kept or
rejected.
• The first row tests you null hypothesis of whether studied or not studied has an
effect on attitude towards retirement planning. The second row tests whether
gender has an effect on attitude. Finally the third row tests, the interaction
between studied and gender.
• You can read the p-value in each case right at the last column. Let's say we set
the significance level at 5%. If our calculated p-value is less than 0.05, then the
null hypothesis is rejected, and if the calculated p-value is greater than 0.05, the
null hypothesis is not rejected.
• Thus, in this case, we see that all three p-values are greater than 0.05 and thus
we cannot reject any of the three null hypotheses.
• Therefore, neither whether one has studied or not nor gender has a significant
effect on attitudes toward retirement planning. And there is also no significant
interaction between studied and gender in terms of attitudes toward retirement
planning.

1-41
1-42
 Interaction effect
• The dependent variable is plotted on the y axis, in our example: the
attitude towards retirement provision. On the x axis, one of the two
factors is plotted, let's just take gender. The other factor is
represented by lines with different colors. Green is studied and red is
not studied.
• The endpoints of the lines are the mean values of the groups, e.g.
male and not studied.
• In this diagram, one can see that both gender and the variable of
having studied or not have an influence on attitudes toward
retirement planning. Females have a higher value than males and
studied have a higher value than not studied.
• But now finally to the interaction effects, for that we compare these
two graphs.

1-43
Interaction effect

1-44
 Interaction effect
In the first case, we said there is no interaction effect. If a person has studied, he has a
value that is, say, 1.5 higher than a person who has not studied.
This increase of 1.5 is independent of whether the person is male or female.
It is different in this case, here studied persons also have a higher value, but how
much higher the value is- depends on whether one is male or female. If I am male,
there is a difference of, let's say for example 0.5 and if I am female, there is a
difference of 3.5.
So in this case we clearly have an interaction between gender and study because the
two variables affect each other. It makes a difference how strong the influence from
studying is depending on whether I am male or female.
In this case, we do have an interaction effect, but the direction still remains the same.
So females have higher scores than males and studied have higher scores than non-
studied.

1-45
Assumptions for Two-Way ANOVA
1. The populations from which the samples were
obtained must be normally or approximately
normally distributed.
2. The samples must be independent.
3. The variances of the populations from which
the samples were selected must be equal.
4. The groups must be equal in sample size.
Two-way Anova – Example 2
A researcher is interested in comparing the
effectiveness of 3 different methods of teaching
reading, and also in whether the effectiveness
might vary as a function of the reading ability of
the students. Fifteen students with high reading
ability and fifteen students with low reading ability
were divided into three equal-sized group and
each group was taught by one of these methods.
Listed on the next slide are the reading
performance scores for the various groups at
school year-end.

Two Way ANOVA

Two-way Anova – Example 2
Teaching Method
Ability A B C

High X 37.6 32.4 33.2

s2 2.8 9.3 11.7

Low X 20.0 18.4 17.6

s2 10.0 4.3 4.3

Two Way ANOVA

Two-way Anova – Example 2
• (a) Do the appropriate analysis to answer the
questions posed by the researcher (all αs = .05)

• (b) The London School Board is currently using

Method B and, prior to this experiment, had been
thinking of changing to Method A because they
believed that A would be better. At α = .01,
determine whether this belief is supported by
these data.

Two Way ANOVA

Example 2 – hypothesis test for A
• H0: No difference among means for levels of A
• HA: At least one mean differs significantly

• Test statistic: F = MSA

MSE

• Rejection region: Fobt < F(2, 24, .05) = 3.40

Two Way ANOVA

Example 2 – hypothesis test for B
• H0: No difference among means for levels of B
• HA: At least one B mean differs significantly

• Test statistic: F = MSB

• MSE

• Rejection region: Fobt < F(1, 24, .05) = 4.26

Two Way ANOVA

Example 2 – hypothesis test for
interaction
• H0: A and B do not interact to affect treatment
means
• HA: A and B do interact to affect treatment means

• Test statistic: F = MSAB

MSE

• Rejection region: Fobt < F(2, 24, .05) = 3.40

Two Way ANOVA

Two-way ANOVA – Example 2
Source df SS MS F
Method 2 77.066 38.533 5.45*
Ability 1 1856.533 1856.533 262.72*
MxA 2 16.267 8.134 1.15
Within 24 169.6 7.067
Total 29

Reject HO for Method and for Ability, not for

interaction.
Two Way ANOVA