experimental design in ap stats

Calcworkshop

Experimental Design in Statistics w/ 11 Examples!

// Last Updated: September 20, 2020 - Watch Video //

A proper experimental design is a critical skill in statistics.

Jenn, Founder Calcworkshop ® , 15+ Years Experience (Licensed & Certified Teacher)

Without proper controls and safeguards, unintended consequences can ruin our study and lead to wrong conclusions.

So let’s dive in to see what’s this is all about!

What’s the difference between an observational study and an experimental study?

An observational study is one in which investigators merely measure variables of interest without influencing the subjects.

And an experiment is a study in which investigators administer some form of treatment on one or more groups?

In other words, an observation is hands-off, whereas an experiment is hands-on.

So what’s the purpose of an experiment?

To establish causation (i.e., cause and effect).

All this means is that we wish to determine the effect an independent explanatory variable has on a dependent response variable.

The explanatory variable explains a response, similar to a child falling and skins their knee and starting to cry. The child is crying in response to falling and skinning their knee. So the explanatory variable is the fall, and the response variable is crying.

Explanatory Vs Response Variable In Everyday Life

Let’s look at another example. Suppose a medical journal describes two studies in which subjects who had a seizure were randomly assigned to two different treatments:

• No treatment.
• A high dose of vitamin C.

The subjects were observed for a year, and the number of seizures for each subject was recorded. Identify the explanatory variable (independent variable), response variable (dependent variable), and include the experimental units.

The explanatory variable is whether the subject received either no treatment or a high dose of vitamin C. The response variable is whether the subject had a seizure during the time of the study. The experimental units in this study are the subjects who recently had a seizure.

Okay, so using the example above, notice that one of the groups did not receive treatment. This group is called a control group and acts as a baseline to see how a new treatment differs from those who don’t receive treatment. Typically, the control group is given something called a placebo, a substance designed to resemble medicine but does not contain an active drug component. A placebo is a dummy treatment, and should not have a physical effect on a person.

Before we talk about the characteristics of a well-designed experiment, we need to discuss some things to look out for:

• Confounding
• Lurking variables

Confounding happens when two explanatory variables are both associated with a response variable and also associated with each other, causing the investigator not to be able to identify their effects and the response variable separately.

A lurking variable is usually unobserved at the time of the study, which influences the association between the two variables of interest. In essence, a lurking variable is a third variable that is not measured in the study but may change the response variable.

For example, a study reported a relationship between smoking and health. A study of 1430 women were asked whether they smoked. Ten years later, a follow-up survey observed whether each woman was still alive or deceased. The researchers studied the possible link between whether a woman smoked and whether she survived the 10-year study period. They reported that:

• 21% of the smokers died
• 32% of the nonsmokers died

So, is smoking beneficial to your health, or is there something that could explain how this happened?

Older women are less likely to be smokers, and older women are more likely to die. Because age is a variable that influences the explanatory and response variable, it is considered a confounding variable.

But does smoking cause death?

Notice that the lurking variable, age, can also be a contributing factor. While there is a correlation between smoking and mortality, and also a correlation between smoking and age, we aren’t 100% sure that they are the cause of the mortality rate in women.

Lurking – Confounding – Correlation – Causation Diagram

Now, something important to point out is that a lurking variable is one that is not measured in the study that could influence the results. Using the example above, some other possible lurking variables are:

• Stress Level.

These variables were not measured in the study but could influence smoking habits as well as mortality rates.

What is important to note about the difference between confounding and lurking variables is that a confounding variable is measured in a study, while a lurking variable is not.

Additionally, correlation does not imply causation!

Alright, so now it’s time to talk about blinding: single-blind, double-blind experiments, as well as the placebo effect.

A single-blind experiment is when the subjects are unaware of which treatment they are receiving, but the investigator measuring the responses knows what treatments are going to which subject. In other words, the researcher knows which individual gets the placebo and which ones receive the experimental treatment. One major pitfall for this type of design is that the researcher may consciously or unconsciously influence the subject since they know who is receiving treatment and who isn’t.

A double-blind experiment is when both the subjects and investigator do not know who receives the placebo and who receives the treatment. A double-blind model is considered the best model for clinical trials as it eliminates the possibility of bias on the part of the researcher and the possibility of producing a placebo effect from the subject.

The placebo effect is when a subject has an effect or response to a fake treatment because they “believe” that the result should occur as noted by Yale . For example, a person struggling with insomnia takes a placebo (sugar pill) but instantly falls asleep because they believe they are receiving a sleep aid like Ambien or Lunesta.

Placebo Effect – Real Life Example

So, what are the three primary requirements for a well-designed experiment?

• Randomization

In a controlled experiment , the researchers, or investigators, decide which subjects are assigned to a control group and which subjects are assigned to a treatment group. In doing so, we ensure that the control and treatment groups are as similar as possible, and limit possible confounding influences such as lurking variables. A replicated experiment that is repeated on many different subjects helps reduce the chance of variation on the results. And randomization means we randomly assign subjects into control and treatment groups.

When subjects are divided into control groups and treatment groups randomly, we can use probability to predict the differences we expect to observe. If the differences between the two groups are higher than what we would expect to see naturally (by chance), we say that the results are statistically significant.

For example, if it is surmised that a new medicine reduces the effects of illness from 72 hours to 71 hours, this would not be considered statistically significant. The difference from 72 hours to 71 hours is not substantial enough to support that the observed effect was due to something other than normal random variation.

Now there are two major types of designs:

• Completely-Randomized Design (CRD)
• Block Design

A completely randomized design is the process of assigning subjects to control and treatment groups using probability, as seen in the flow diagram below.

Completely Randomized Design Example

A block design is a research method that places subjects into groups of similar experimental units or conditions, like age or gender, and then assign subjects to control and treatment groups using probability, as shown below.

Randomized Block Design Example

Additionally, a useful and particular case of a blocking strategy is something called a matched-pair design . This is when two variables are paired to control for lurking variables.

For example, imagine we want to study if walking daily improved blood pressure. If the blood pressure for five subjects is measured at the beginning of the study and then again after participating in a walking program for one month, then the observations would be considered dependent samples because the same five subjects are used in the before and after observations; thus, a matched-pair design.

Please note that our video lesson will not focus on quasi-experiments. A quasi experimental design lacks random assignments; therefore, the independent variable can be manipulated prior to measuring the dependent variable, which may lead to confounding. For the sake of our lesson, and all future lessons, we will be using research methods where random sampling and experimental designs are used.

Together we will learn how to identify explanatory variables (independent variable) and response variables (dependent variables), understand and define confounding and lurking variables, see the effects of single-blind and double-blind experiments, and design randomized and block experiments.

Experimental Designs – Lesson & Examples (Video)

1 hr 06 min

• Introduction to Video: Experiments
• 00:00:29 – Observational Study vs Experimental Study and Response and Explanatory Variables (Examples #1-4)
• Exclusive Content for Members Only
• 00:09:15 – Identify the response and explanatory variables and the experimental units and treatment (Examples #5-6)
• 00:14:47 – Introduction of lurking variables and confounding with ice cream and homicide example
• 00:18:57 – Lurking variables, Confounding, Placebo Effect, Single Blind and Double Blind Experiments (Example #7)
• 00:27:20 – What was the placebo effect and was the experiment single or double blind? (Example #8)
• 00:30:36 – Characteristics of a well designed and constructed experiment that is statistically significant
• 00:35:08 – Overview of Complete Randomized Design, Block Design and Matched Pair Design
• 00:44:23 – Design and experiment using complete randomized design or a block design (Examples #9-10)
• 00:56:09 – Identify the response and explanatory variables, experimental units, lurking variables, and design an experiment to test a new drug (Example #11)
• Practice Problems with Step-by-Step Solutions
• Chapter Tests with Video Solutions

Get access to all the courses and over 450 HD videos with your subscription

Monthly and Yearly Plans Available

Get My Subscription Now

Still wondering if CalcWorkshop is right for you? Take a Tour and find out how a membership can take the struggle out of learning math.

Experimental Design

Experimental design refers to the process of planning and conducting an experiment to investigate cause-and-effect relationships between variables. It involves defining treatments, assigning participants to different groups, and controlling for confounding factors to ensure valid results.

Related terms

Control Group : A group in an experiment that does not receive any treatment or intervention. It serves as a baseline for comparison against the experimental group(s).

Randomization : The process of assigning participants or subjects to different treatment groups randomly. It helps minimize bias and ensures that each participant has an equal chance of being assigned to any group.

Confounding Variable : An extraneous variable that is not intentionally controlled but may influence the outcome of an experiment. It can lead to inaccurate conclusions if not properly accounted for in the design.

" Experimental Design " appears in:

Study guides ( 3 ).

• AP Statistics - 3.5 Introduction to Experimental Design
• AP Statistics - 3.6 Selecting an Experimental Design
• AP Statistics - 8.5 Setting Up a Chi-Square Test for Homogeneity or Independence

Subjects ( 2 )

• AP Physics 2
• AP Psychology

Practice Questions ( 1 )

• Which experimental design should be used in a study of college students' grades considering major and course load as factors?

Additional resources ( 3 )

• AP Statistics - Unit 3 Overview: Collecting Data
• AP Statistics - Unit 6 Overview: Inference for Categorical Data: Proportions
• AP Statistics - Unit 7 Overview: Means

© 2024 Fiveable Inc. All rights reserved.

Ap® and sat® are trademarks registered by the college board, which is not affiliated with, and does not endorse this website..

Helping math teachers bring statistics to life

Designing Experiments ( Topic 3.5 )

Chapter 4 - day 7, day 12 day 13, learning targets.

Describe the placebo effect and the purpose of blinding in an experiment.

Describe how to randomly assign treatments in an experiment using slips of paper, technology, or a table of random digits.

Explain the purpose of comparison, random assignment, control, and replication in an experiment.

Activity: Would You Fall for That? 

Answer Key:

Start by showing this video .

This activity is getting students to start thinking about the Caffeine Experiment, which we use as a review day before the Chapter 4 Test.

There are a lot of ideas in the learning targets in this lesson.  You have to be very intentional in the Debrief and Big Ideas to be sure that you are addressing all of them.

The Language of Experiments

Suppose that we thought that the temperature of the Coca Cola might also effect the pulse rate.  So not only are we using Coke and Caffeine Free Coke, we are also testing it at 35 degrees, 50 degrees, and 70 degrees.

Factor: A variable that is manipulated and may cause a change in the response variable (presence of caffeine and temperature).

Levels: The different values of the factors.  The factor “presence of caffeine” has two levels (yes and no) and the factor “temperature” has three levels (35, 50, and 70 degrees).

Treatments: A specific condition applied to the individuals in an experiment.  In this experiment, there are 6 treatments: (Yes, 35), (No, 35),  (Yes, 50), (No, 50), (Yes, 70), (No, 70).

Luke's Lesson Notes

Here is a brief video highlighting some key information to help you prepare to teach this lesson.

In order to continue enjoying our site, we ask that you confirm your identity as a human. Thank you very much for your cooperation.