Guide to Experimental Design | Overview, 5
steps & Examples
• Experiments are used to study causal relationships. You manipulate
one or more independent variables and measure their effect on one
or more dependent variables.
There are five key steps in designing an experiment:
• Define your variables
• Write your hypothesis (prediction)
• Design your experimental treatments
• Assign your subjects to treatment groups
• Measure your dependent variable
• Step 1: Define your variables
You should begin with a specific research question. We will work with two research
question examples, one from health sciences and one from ecology:
Example question 1: Phone use and sleep
You want to know how phone use before bedtime affects sleep patterns.
Specifically, you ask how the number of minutes a person uses their phone
before sleep affects the number of hours they sleep.
Example question 2: Temperature and soil respiration
You want to know how temperature affects soil respiration. Specifically, you ask
how increased air temperature near the soil surface affects the amount of
carbon dioxide (CO2) respired from the soil.
• To translate your research question into an experimental hypothesis,
you need to define the main variables and make predictions about
how they are related.
• Start by simply listing the independent and dependent variables.
Independent vs. Dependent Variables
In research, variables are any characteristics that can take on different
values, such as height, age, temperature, or test scores.
Researchers often use studies to test cause-and-effect relationships to
measure independent and dependent variables.
• The independent variable is the cause. Its value is independent of
other variables in your study.
• The dependent variable is the effect. Its value depends on changes in
the independent variable.
Research question Independent variable Dependent variable
Phone use and Minutes of phone use Hours of sleep per
sleep before sleep night
Temperature and Air temperature just CO2 respired from soil
soil respiration above the soil surface
Designing an experiment means Step 2: Write your hypothesis
planning how you test your
hypothesis under a controlled setting
to reach a valid conclusion
Null hypothesis (H0) Alternate hypothesis (H1)
Phone use and sleep Phone use before sleep does not Increasing phone use before
correlate with the amount of sleep leads to a decrease in
sleep a person gets. sleep.
Temperature and soil respiration Air temperature does not Increased air temperature leads
correlate with soil respiration. to increased soil respiration.
As you design your experiment, you need to strike a balance between
internal validity and external validity.
Internal validity External validity
• How strongly you can demonstrate • How confidently you can generalization
causation your findings
• Make sure that only your manipulation • Are your study subjects representative?
explains your outcomes
• Is your study environment similar to real-
life contexts?
Experimental have higher internal validity than other types of research, but they have
lower external validity
• Step 3: Design your experimental treatments
How you manipulate the independent variable can affect the experiment’s external
validity – that is, the extent to which the results can be generalized and applied to
the broader world.
First, you may need to decide how widely to vary your independent variable.
Soil-warming experiment
You can choose to increase air temperature:
• just slightly above the natural range for your study region.
• over a wider range of temperatures to mimic future warming.
• over an extreme range that is beyond any possible natural variation.
Second, you may need to choose how finely to vary your independent variable.
Sometimes this choice is made for you by your experimental system, but often you
will need to decide, and this will affect how much you can infer from your results.
Phone-use experiment
You can choose to treat phone use as:
• a categorical variable: either as binary (yes/no) or as levels of a factor (no
phone use, low phone use, high phone use).
• a continuous variable (minutes of phone use measured every night).
• Step 4: Assign your subjects to treatment groups
How you apply your experimental treatments to your test subjects is crucial for
obtaining valid and reliable results.
First, you need to consider the study size: how many individuals will be included in
the experiment? In general, the more subjects you include, the greater your
experiment’s statistical power, which determines how much confidence you can
have in your results.
Then you need to randomly assign your subjects to treatment groups. Each group
receives a different level of treatment (e.g. no phone use, low phone use, high
phone use).
You should also include a control group, which receives no treatment. The control
group tells us what would have happened to your test subjects without any
experimental intervention.
• Step 5: Measure your dependent variable
you need to decide how you’ll collect data on your dependent variable outcomes.
Phone use experiment
In your experiment about phone use and sleep, you could measure your
dependent variable in one of two ways:
• Ask participants to record what time they go to sleep and get up each
day.
• Ask participants to wear a sleep tracker.
• What is experimental design?
• Experimental design means planning a set of procedures to investigate a
relationship between variables. To design a controlled experiment, you need:
✓A testable hypothesis
✓At least one independent variable that can be precisely manipulated
✓At least one dependent variable that can be precisely measured
• The difference between an observational study and an experiment.
The key difference between observational studies and experimental designs is that
a well-done observational study does not influence the responses of participants,
while experiments do have some sort of treatment condition applied to at least
some participants by random assignment.
• https://youtu.be/1-wc7CNeIz8
