Introduction to statistical testing
Illustrated with XLSTAT
Jean Paul Maalouf
webinar@xlstat.com
Nov. 9, 2016
www.xlstat.com
1
Goal of this
webinar
Let you become
independent in using our
web stat test selection tool
(whether youre an XLSTAT user or not)
Link
2
PLAN
XLSTAT: who are we ?
Statistics: categories
Reminder on Descriptive / exploratory statistics
Statistical tests: principles, steps & practice on XLSTAT
Parametric vs non parametric tests practice on XLSTAT
Tests on independent vs paired samples
Statistical tests: Comparison vs Association
Practice on XLSTAT: Fishers exact test on a contingency table
Appendix: How to interpret p-value > alfa?
All the data in this webinar were made up unless otherwise 3
specified
XLSTAT: Who are
we?
XLSTAT is a user-friendly
statistical add-on software
for Microsoft Excel
4
XLSTAT
A growing software and team
New version,
XLSTAT realizes VBA interface, New products,
its first sale on C++ computations, new website,
the Internet 7 languages growing and
1993 2000 2009 dynamic team 2016
Thierry Fahmy The company New offers XLSTAT 365
develops a user- 1996 Addinsoft is 2006 adapted to 2015 Cloud version of
friendly solution created business needs XLSTAT for Excel
for data analysis: 365
XLSTAT is born XLSTAT Free
Free limited
Edition
5
XLSTAT in a few numbers
200+ statistical features 50k users
General or field-oriented solutions Across the world. Companies, education, research
16 employees 130k visits/month on the website
Always receptive to the needs of users Easy tutorials available in 5 languages
7 languages 400 downloads/day
6
Statistics: 4
categories
7
Statistics: 4 categories
Recording Recording
Nov. 30
Description Exploration Tests Modeling
I want to summarize I want to easily extract I want to accept / I want to understand
small data sets (1-3 information from a reject a very precise the way a phenomenon
variables) using large data set hypothesis assuming evolves according to a
simple statistics or without necessarily error risks. (t tests, set of parameters.
charts (mean, having a precise ANOVA, correlation (regression, ANOVA,
standard deviation, question to answer. tests, chi-square...) ANCOVA...)
8
boxplots...) (PCA, AHC...)
Reminder on
Descriptive /
exploratory
statistics
9
Data set: online shoe selling platform
Variables
Individuals
10
Toward exploratory data analysis: scatter plot
colored by group
- Invoice amount decreases with time spent
on the website.
- Plutonians spend more money on the website
compared to others.
- Martians and humans form a relatively
homogeneous group
- ...
Webinar Recording
11
The same kind of reasoning on a higher
number of variables... Exploratory statistics
(or Exploratory Data Analysis)
12
Principal Component Analysis
Chart 1: correlation circle ; chart 2: observations
Weight-
Height- Weight+
time on site+ Height+
time on site-
13
Webinar Recording
PCA: explorations ...
Weight increases with height Shoe size is unrelated to weight / height
Time spent on site decreases with weight & height Derrick has big feet. Shaun has small feet.
Looks like there are two clusters in the data And so on...
14
Data exploration inspired us many hypotheses. Are they valid?
Statistical tests
15
Statistical tests
I want to accept / reject a very precise
hypothesis assuming error risks.
Statistical tests usually answer yes/no
questions
16
Statistical testing: steps
Writing up the question (answer: yes/no)
Writing up the null & the alternative hypotheses
Choosing the appropriate statistical test & the alfa risk threshold (check out the guide online)
Gathering the data
Things will be added here later
Running the test
Answering the question: if p-value < alfa, we reject H0 with a risk proportional to p-value of being wrong
17
Step 1: writing up
Question: do fertilizers A & B the question
induce a difference in sugar
rate in bananas?
18
Step 2: Writing
up the null & the
H0 alternative
VS hypotheses
Ha
19
Writing up hypotheses
Question
? Do fertilizers A & B induce a difference in sugar rate
in bananas?
Null Hypothesis
H0 Generally implies an idea of equality
H0: mean sugar rate in A-fertilized bananas = mean sugar rate in B-fertilized
bananas
Alternative Hypothesis
Ha Generally implies an idea of difference
Ha: mean sugar rate in A-fertilized bananas mean sugar rate in B-fertilized
bananas
20
Statistical testing: steps where are we?
Writing up the question (answer: yes/no)
Writing up the null & the alternative hypotheses
Choosing the appropriate statistical test & the alfa risk threshold (check out the guide online)
Gathering the data
Things will be added here later
Running the test
Answering the question: if p-value < alfa, we reject H0 with a risk proportional to p-value of being wrong
21
Are we comparing means?
If yes, how many?
Step 3a:
Are we comparing proportions?
If yes, how many? choosing the
Are we comparing variances?
If yes, how many?
appropriate
Are we testing associations?
...
statistical test
In our case, we want to compare 2 means
Students t-test for two independent samples
Link: choosing the appropriate statistical
test according to your situation
22
The alfa risk threshold (0<alfa<1)
is the threshold below which we
decide to reject H0
Step 3b:
The more we want to limit the risk choosing the alfa
of taking a wrong decision, the
more we should decrease alfa risk threshold
People often set alfa at 0.05. This
is not a reason to do it
systematically
(but this is what well do in our example )
23
Experiment: 60 banana trees are
planted; 30 of them receive fertilizer A,
30 of them receive fertilizer B
Step 4: gathering
the data
24
Step 5: running
the test in
XLSTAT
25
Step 6:
interpreting the
p-value result and
VS answering the
alfa
question
26
Interpreting the result
Question The test computes a
? Do fertilizers A & B induce a difference in sugar rate number called p-value.
in bananas? 0 < p-value < 1
The p-value is the risk you take
Null Hypothesis of being wrong when rejecting
H0 Generally implies an idea of equality H0 and accepting Ha
H0: mean sugar rate in A-fertilized bananas = mean sugar rate in B-fertilized
bananas
Decision : If p-value < alfa, we
Alternative Hypothesis reject H0 and accept Ha
Ha Generally implies an idea of difference assuming a risk proportional to p-
Ha: mean sugar rate in A-fertilized bananas mean sugar rate in B-fertilized value of being wrong.
bananas
27
Interpreting the result
Decision: p-value < alfa. We reject H0 & accept Ha with a very low risk of being wrong.
Answer: The two means (fertilizer A vs fertilizer B) are significantly different
28
Parametric vs non
Power
parametric tests
VS
Robustness
29
Parametric vs non parametric tests
Differences on the way they work
A statistical test can be either parametric or non parametric
Parametric tests are reliable only under certain conditions that are linked
to the distribution of populations. These conditions can be found on our
online statistical testing guide.
Non parametric tests do not assume any underlying distribution. Most of
them are computed from the ranks of the data.
30
So why do we still use parametric tests?
Differences on their usefulness
Non parametric tests: reliable in a larger number of situations than
parametric tests they are more robust.
Parametric tests: more able to reject H0 if it is false, and if applicability
conditions are respected they are more powerful*. *Statistical power of a test is
its ability to lead to a rejection
of H0 if H0 is wrong
So, which type should you choose? Heres a proposition:
Choose an appropriate parametric test
Gather the data
Are assumptions for the parametric test met?
Yes No
Replace with a non parametric test, less powerful but more robust
Run the test
31
Tests on
independent vs
paired samples
32
Tests on independent vs paired samples
Independent samples
Two or more distinct populations
Examples : compare a treated group and a control group; compare
females and males; compare treated and untreated banana trees.
Paired samples
One single population
Examples : measuring the weight of patients before/after a treatment ;
follow up companies or surveyed individuals at different dates ; follow
photosynthetic capacities of the same banana trees at different dates/
33
Statistical tests:
comparison vs
association
34
Statistical tests: comparison & association
Comparison tests
Comparing means (Student / ANOVA)
Comparing variances (Fisher / Levene)
Comparing proportions (tests on proportions)
Variables association tests
Test the association between two qualitative variables (chi-square
& exact Fishers test)
Test the association between two quantitative variables (Pearson &
Spearman correlation coefficients)
35
Commonly used statistical tests
Parametric tests and their non parametric equivalents
Independent / paired Non parametric
Question Parametric tests
samples equivalents
Student's t-test on Mann-Whitney's
Independent
independent samples test
Compare 2 means
Student's t-test on paired
Paired Wilcoxon's test
samples
Independent ANOVA Krukal-Wallis test
Compare k means Repeated measures
Paired Friedman's test
ANOVA
Compare 2 variances Fisher's test
Independent
Compare k variances Levene's test
Independent Chi2 test Fisher's exact test
Association (qualitative var.)
Paired Cochran's Q test
Spearman
Association (quantitative var.) Independent Pearson correlation
correlation
Link: choose an appropriate test according to your situation
36
Association tests:
Fishers exact test
on two qualitative
variables
Investigating the significance of a
contingency table ( = crosstab)
37
Application: association test (qualitative
variables)
EXAMPLE: car garage, customer satisfaction survey
38
Launching the test in XLSTAT
EXAMPLE: car garage, customer satisfaction survey
39
Association test example
EXAMPLE: car garage, customer satisfaction survey
p-value > alfa. We cannot reject H0.
H0: proportions of categories a & b do not change according to categories no-yes-dk
Ha: proportions of categories a & b change according to categories no-yes-dk
40
Statistical tests: revisiting the steps, a conclusion
Writing up the question (answer: yes/no)
Writing up the null & the alternative hypotheses
Choosing the appropriate statistical test (comparison / association) & the alfa risk threshold
Gathering the data
Are assumptions for the parametric test met?
Yes No
Replacing with a non parametric test, less powerful but more robust
Running the test
Answering the question: if p-value < alfa, we reject H0 with a risk proportional to p-value of being wrong
41
Statistics: 4 categories
Recording Recording
Nov. 30
Description Exploration Tests Modeling
I want to summarize I want to easily extract I want to accept / I want to understand
small data sets (1-3 information from a reject a very precise the way a phenomenon
variables) using large data set hypothesis assuming evolves according to a
simple statistics or without necessarily error risks. (t tests, set of parameters.
charts (mean, having a precise ANOVA, correlation (regression, ANOVA,
standard deviation, question to answer. tests, chi-square...) ANCOVA...)
42
boxplots...) (PCA, AHC...)
Future webinars
Nov. 30, 2016: statistical modeling (click here)
43
Thanks for attending!
All the tools we saw are available in all XLSTAT solutions (except XLSTAT-Free)
Survey time
44
Appendix: How
to interpret p >
alfa?
45
Appendix: How to interpret p > alfa?
If p-value < threshold (often 0.05), we reject H0 and accept Ha with a risk
proportional to p-value of being wrong.
If p-value > threshold, there are two possibilities:
If Statistical power* is high (>0.95)
We accept H0 and reject Ha by taking another risk (Bta = 1 - Power) of being
wrong.
If Statistical power is low (<0.95)
The risk of being wrong when accepting H0 is too high (power is low)
The risk of being wrong when rejecting H0 is too high (p-value is high)
We are unable to take any decision. Game over.
*(statistical power being the ability of an experiment/a test to make you reject H0
when it is false)
46
Statistical power: how to increase it
Statistical power increases with:
The number of measurements
Measurement precision
Size effect
The alfa threshold
The statistical test used
47
