control treatment meaning experiment

• Science Notes Posts
• Contact Science Notes
• Todd Helmenstine Biography
• Anne Helmenstine Biography
• Free Printable Periodic Tables (PDF and PNG)
• Periodic Table Wallpapers
• Interactive Periodic Table
• Periodic Table Posters
• Science Experiments for Kids
• How to Grow Crystals
• Chemistry Projects
• Fire and Flames Projects
• Holiday Science
• Chemistry Problems With Answers
• Physics Problems
• Unit Conversion Example Problems
• Chemistry Worksheets
• Biology Worksheets
• Periodic Table Worksheets
• Physical Science Worksheets
• Science Lab Worksheets
• My Amazon Books

Control Group Definition and Examples

The control group is the set of subjects that does not receive the treatment in a study. In other words, it is the group where the independent variable is held constant. This is important because the control group is a baseline for measuring the effects of a treatment in an experiment or study. A controlled experiment is one which includes one or more control groups.

• The experimental group experiences a treatment or change in the independent variable. In contrast, the independent variable is constant in the control group.
• A control group is important because it allows meaningful comparison. The researcher compares the experimental group to it to assess whether or not there is a relationship between the independent and dependent variable and the magnitude of the effect.
• There are different types of control groups. A controlled experiment has one more control group.

Control Group vs Experimental Group

The only difference between the control group and experimental group is that subjects in the experimental group receive the treatment being studied, while participants in the control group do not. Otherwise, all other variables between the two groups are the same.

Control Group vs Control Variable

A control group is not the same thing as a control variable. A control variable or controlled variable is any factor that is held constant during an experiment. Examples of common control variables include temperature, duration, and sample size. The control variables are the same for both the control and experimental groups.

Types of Control Groups

There are different types of control groups:

• Placebo group : A placebo group receives a placebo , which is a fake treatment that resembles the treatment in every respect except for the active ingredient. Both the placebo and treatment may contain inactive ingredients that produce side effects. Without a placebo group, these effects might be attributed to the treatment.
• Positive control group : A positive control group has conditions that guarantee a positive test result. The positive control group demonstrates an experiment is capable of producing a positive result. Positive controls help researchers identify problems with an experiment.
• Negative control group : A negative control group consists of subjects that are not exposed to a treatment. For example, in an experiment looking at the effect of fertilizer on plant growth, the negative control group receives no fertilizer.
• Natural control group : A natural control group usually is a set of subjects who naturally differ from the experimental group. For example, if you compare the effects of a treatment on women who have had children, the natural control group includes women who have not had children. Non-smokers are a natural control group in comparison to smokers.
• Randomized control group : The subjects in a randomized control group are randomly selected from a larger pool of subjects. Often, subjects are randomly assigned to either the control or experimental group. Randomization reduces bias in an experiment. There are different methods of randomly assigning test subjects.

Control Group Examples

Here are some examples of different control groups in action:

Negative Control and Placebo Group

For example, consider a study of a new cancer drug. The experimental group receives the drug. The placebo group receives a placebo, which contains the same ingredients as the drug formulation, minus the active ingredient. The negative control group receives no treatment. The reason for including the negative group is because the placebo group experiences some level of placebo effect, which is a response to experiencing some form of false treatment.

Positive and Negative Controls

For example, consider an experiment looking at whether a new drug kills bacteria. The experimental group exposes bacterial cultures to the drug. If the group survives, the drug is ineffective. If the group dies, the drug is effective.

The positive control group has a culture of bacteria that carry a drug resistance gene. If the bacteria survive drug exposure (as intended), then it shows the growth medium and conditions allow bacterial growth. If the positive control group dies, it indicates a problem with the experimental conditions. A negative control group of bacteria lacking drug resistance should die. If the negative control group survives, something is wrong with the experimental conditions.

• Bailey, R. A. (2008).  Design of Comparative Experiments . Cambridge University Press. ISBN 978-0-521-68357-9.
• Chaplin, S. (2006). “The placebo response: an important part of treatment”.  Prescriber . 17 (5): 16–22. doi: 10.1002/psb.344
• Hinkelmann, Klaus; Kempthorne, Oscar (2008).  Design and Analysis of Experiments, Volume I: Introduction to Experimental Design  (2nd ed.). Wiley. ISBN 978-0-471-72756-9.
• Pithon, M.M. (2013). “Importance of the control group in scientific research.” Dental Press J Orthod . 18 (6):13-14. doi: 10.1590/s2176-94512013000600003
• Stigler, Stephen M. (1992). “A Historical View of Statistical Concepts in Psychology and Educational Research”. American Journal of Education . 101 (1): 60–70. doi: 10.1086/444032

Related Posts

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Sweepstakes
• Guided Meditations
• Verywell Mind Insights
• 2024 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

What Is a Control Group?

Control Groups vs. Experimental Groups in Psychology Research

Doug Corrance/The Image Bank/Getty Images

Control Group vs. Experimental Group

Types of control groups.

In simple terms, the control group comprises participants who do not receive the experimental treatment. When conducting an experiment, these people are randomly assigned to this group. They also closely resemble the participants who are in the experimental group or the individuals who receive the treatment.

Experimenters utilize variables to make comparisons between an experimental group and a control group. A variable is something that researchers can manipulate, measure, and control in an experiment. The independent variable is the aspect of the experiment that the researchers manipulate (or the treatment). The dependent variable is what the researchers measure to see if the independent variable had an effect.

While they do not receive the treatment, the control group does play a vital role in the research process. Experimenters compare the experimental group to the control group to determine if the treatment had an effect.

By serving as a comparison group, researchers can isolate the independent variable and look at the impact it had.

The simplest way to determine the difference between a control group and an experimental group is to determine which group receives the treatment and which does not. To ensure that the results can then be compared accurately, the two groups should be otherwise identical.

Not exposed to the treatment (the independent variable)

Used to provide a baseline to compare results against

May receive a placebo treatment

Exposed to the treatment

Used to measure the effects of the independent variable

Identical to the control group aside from their exposure to the treatment

Why a Control Group Is Important

While the control group does not receive treatment, it does play a critical role in the experimental process. This group serves as a benchmark, allowing researchers to compare the experimental group to the control group to see what sort of impact changes to the independent variable produced.  

Because participants have been randomly assigned to either the control group or the experimental group, it can be assumed that the groups are comparable.

Any differences between the two groups are, therefore, the result of the manipulations of the independent variable. The experimenters carry out the exact same procedures with both groups with the exception of the manipulation of the independent variable in the experimental group.

There are a number of different types of control groups that might be utilized in psychology research. Some of these include:

• Positive control groups : In this case, researchers already know that a treatment is effective but want to learn more about the impact of variations of the treatment. In this case, the control group receives the treatment that is known to work, while the experimental group receives the variation so that researchers can learn more about how it performs and compares to the control.
• Negative control group : In this type of control group, the participants are not given a treatment. The experimental group can then be compared to the group that did not experience any change or results.
• Placebo control group : This type of control group receives a placebo treatment that they believe will have an effect. This control group allows researchers to examine the impact of the placebo effect and how the experimental treatment compared to the placebo treatment.
• Randomized control group : This type of control group involves using random selection to help ensure that the participants in the control group accurately reflect the demographics of the larger population.
• Natural control group : This type of control group is naturally selected, often by situational factors. For example, researchers might compare people who have experienced trauma due to war to people who have not experienced war. The people who have not experienced war-related trauma would be the control group.

Examples of Control Groups

Control groups can be used in a variety of situations. For example, imagine a study in which researchers example how distractions during an exam influence test results. The control group would take an exam in a setting with no distractions, while the experimental groups would be exposed to different distractions. The results of the exam would then be compared to see the effects that distractions had on test scores.

Experiments that look at the effects of medications on certain conditions are also examples of how a control group can be used in research. For example, researchers looking at the effectiveness of a new antidepressant might use a control group that receives a placebo and an experimental group that receives the new medication. At the end of the study, researchers would compare measures of depression for both groups to determine what impact the new medication had.

After the experiment is complete, researchers can then look at the test results and start making comparisons between the control group and the experimental group.

Uses for Control Groups

Researchers utilize control groups to conduct research in a range of different fields. Some common uses include:

• Psychology : Researchers utilize control groups to learn more about mental health, behaviors, and treatments.
• Medicine : Control groups can be used to learn more about certain health conditions, assess how well medications work to treat these conditions, and assess potential side effects that may result.
• Education : Educational researchers utilize control groups to learn more about how different curriculums, programs, or instructional methods impact student outcomes.
• Marketing : Researchers utilize control groups to learn more about how consumers respond to advertising and marketing efforts.

Malay S, Chung KC. The choice of controls for providing validity and evidence in clinical research . Plast Reconstr Surg. 2012 Oct;130(4):959-965. doi:10.1097/PRS.0b013e318262f4c8

National Cancer Institute. Control group.

Pithon MM. Importance of the control group in scientific research . Dental Press J Orthod. 2013;18(6):13-14. doi:10.1590/s2176-94512013000600003

Karlsson P, Bergmark A. Compared with what? An analysis of control-group types in Cochrane and Campbell reviews of psychosocial treatment efficacy with substance use disorders . Addiction . 2015;110(3):420-8. doi:10.1111/add.12799

Myers A, Hansen C. Experimental Psychology . Belmont, CA: Cengage Learning; 2012.

By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

• Skip to secondary menu
• Skip to main content
• Skip to primary sidebar

Statistics By Jim

Making statistics intuitive

Control Group in an Experiment

By Jim Frost 3 Comments

A control group in an experiment does not receive the treatment. Instead, it serves as a comparison group for the treatments. Researchers compare the results of a treatment group to the control group to determine the effect size, also known as the treatment effect.

Imagine that a treatment group receives a vaccine and it has an infection rate of 10%. By itself, you don’t know if that’s an improvement. However, if you also have an unvaccinated control group with an infection rate of 20%, you know the vaccine improved the outcome by 10 percentage points.

By serving as a basis for comparison, the control group reveals the treatment’s effect.

Related post : Effect Sizes in Statistics

Using Control Groups in Experiments

Most experiments include a control group and at least one treatment group. In an ideal experiment, the subjects in all groups start with the same overall characteristics except that those in the treatment groups receive a treatment. When the groups are otherwise equivalent before treatment begins, you can attribute differences after the experiment to the treatments.

Randomized controlled trials (RCTs) assign subjects to the treatment and control groups randomly. This process helps ensure the groups are comparable when treatment begins. Consequently, treatment effects are the most likely cause for differences between groups at the end of the study. Statisticians consider RCTs to be the gold standard. To learn more about this process, read my post, Random Assignment in Experiments .

Observational studies either can’t use randomized groups or don’t use them because they’re too costly or problematic. In these studies, the characteristics of the control group might be different from the treatment groups at the start of the study, making it difficult to estimate the treatment effect accurately at the end. Case-Control studies are a specific type of observational study that uses a control group.

For these types of studies, analytical methods and design choices, such as regression analysis and matching, can help statistically mitigate confounding variables. Matching involves selecting participants with similar characteristics. For each participant in the treatment group, the researchers find a subject with comparable traits to include in the control group. To learn more about this type of study and matching, read my post, Observational Studies Explained .

Control groups are key way to increase the internal validity of an experiment. To learn more, read my post about internal and external validity .

Randomized versus non-randomized control groups are just several of the different types you can have. We’ll look at more kinds later!

Related posts : When to Use Regression Analysis

Example of a Control Group

Suppose we want to determine whether regular vitamin consumption affects the risk of dying. Our experiment has the following two experimental groups:

• Control group : Does not consume vitamin supplements
• Treatment group : Regularly consumes vitamin supplements.

In this experiment, we randomly assign subjects to the two groups. Because we use random assignment, the two groups start with similar characteristics, including healthy habits, physical attributes, medical conditions, and other factors affecting the outcome. The intentional introduction of vitamin supplements in the treatment group is the only systematic difference between the groups.

After the experiment is complete, we compare the death risk between the treatment and control groups. Because the groups started roughly equal, we can reasonably attribute differences in death risk at the end of the study to vitamin consumption. By having the control group as the basis of comparison, the effect of vitamin consumption becomes clear!

Types of Control Groups

Researchers can use different types of control groups in their experiments. Earlier, you learned about the random versus non-random kinds, but there are other variations. You can use various types depending on your research goals, constraints, and ethical issues, among other things.

Negative Control Group

The group introduces a condition that the researchers expect won’t have an effect. This group typically receives no treatment. These experiments compare the effectiveness of the experimental treatment to no treatment. For example, in a vaccine study, a negative control group does not get the vaccine.

Positive Control Group

Positive control groups typically receive a standard treatment that science has already proven effective. These groups serve as a benchmark for the performance of a conventional treatment. In this vein, experiments with positive control groups compare the effectiveness of a new treatment to a standard one.

For example, an old blood pressure medicine can be the treatment in a positive control group, while the treatment group receives the new, experimental blood pressure medicine. The researchers want to determine whether the new treatment is better than the previous treatment.

In these studies, subjects can still take the standard medication for their condition, a potentially critical ethics issue.

Placebo Control Group

Placebo control groups introduce a treatment lookalike that will not affect the outcome. Standard examples of placebos are sugar pills and saline solution injections instead of genuine medicine. The key is that the placebo looks like the actual treatment. Researchers use this approach when the recipients’ belief that they’re receiving the treatment might influence their outcomes. By using placebos, the experiment controls for these psychological benefits. The researchers want to determine whether the treatment performs better than the placebo effect.

Learn more about the Placebo Effect .

Blinded Control Groups

If the subject’s awareness of their group assignment might affect their outcomes, the researchers can use a blinded experimental design that does not tell participants their group membership. Typically, blinded control groups will receive placebos, as described above. In a double-blinded control group, both subjects and researchers don’t know group assignments.

Waitlist Control Group

When there is a waitlist to receive a new treatment, those on the waitlist can serve as a control group until they receive treatment. This type of design avoids ethical concerns about withholding a better treatment until the study finishes. This design can be a variation of a positive control group because the subjects might be using conventional medicines while on the waitlist.

Historical Control Group

When historical data for a comparison group exists, it can serve as a control group for an experiment. The group doesn’t exist in the study, but the researchers compare the treatment group to the existing data. For example, the researchers might have infection rate data for unvaccinated individuals to compare to the infection rate among the vaccinated participants in their study. This approach allows everyone in the experiment to receive the new treatment. However, differences in place, time, and other circumstances can reduce the value of these comparisons. In other words, other factors might account for the apparent effects.

Share this:

Reader Interactions

December 19, 2021 at 9:17 am

Thank you very much Jim for your quick and comprehensive feedback. Extremely helpful!! Regards, Arthur

December 17, 2021 at 4:46 pm

Thank you very much Jim, very interesting article.

Can I select a control group at the end of intervention/experiment? Currently I am managing a project in rural Cambodia in five villages, however I did not select any comparison/control site at the beginning. Since I know there are other villages which have not been exposed to any type of intervention, can i select them as a control site during my end-line data collection or it will not be a legitimate control? Thank you very much, Arthur

December 18, 2021 at 1:51 am

You might be able to use that approach, but it’s not ideal. The ideal is to have control groups defined at the beginning of the study. You can use the untreated villages as a type of historical control groups that I talk about in this article. Or, if they’re awaiting to receive the intervention, it might be akin to a waitlist control group.

If you go that route, you’ll need to consider whether there was some systematic reason why these villages have not received any intervention. For example, are the villages in question more remote? And, if there is a systematic reason, would that affect your outcome variable? More generally, are they systematically different? How well do the untreated villages represent your target population?

If you had selected control villages at the beginning, you’d have been better able to ensure there weren’t any systematic differences between the villages receiving interventions and those that didn’t.

If the villages that didn’t receive any interventions are systematically different, you’ll need to incorporate that into your interpretation of the results. Are they different in ways that affect the outcomes you’re measuring? Can those differences account for the difference in outcomes between the treated and untreated villages? Hopefully, you’d be able to measure those differences between untreated/treated villages.

So, yes, you can use that approach. It’s not perfect and there will potentially be more things for you to consider and factor into your conclusions. Despite these drawbacks, it’s possible that using a pseudo control group like that is better than not doing that because at least you can make comparisons to something. Otherwise, you won’t know whether the outcomes in the intervention villages represent an improvement! Just be aware of the extra considerations!

Best of luck with your research!

Comments and Questions Cancel reply

Understanding Control Groups for Research

Introduction

What are control groups in research, examples of control groups in research, control group vs. experimental group, types of control groups, control groups in non-experimental research.

A control group is typically thought of as the baseline in an experiment. In an experiment, clinical trial, or other sort of controlled study, there are at least two groups whose results are compared against each other.

The experimental group receives some sort of treatment, and their results are compared against those of the control group, which is not given the treatment. This is important to determine whether there is an identifiable causal relationship between the treatment and the resulting effects.

As intuitive as this may sound, there is an entire methodology that is useful to understanding the role of the control group in experimental research and as part of a broader concept in research. This article will examine the particulars of that methodology so you can design your research more rigorously .

Suppose that a friend or colleague of yours has a headache. You give them some over-the-counter medicine to relieve some of the pain. Shortly after they take the medicine, the pain is gone and they feel better. In casual settings, we can assume that it must be the medicine that was the cause of their headache going away.

In scientific research, however, we don't really know if the medicine made a difference or if the headache would have gone away on its own. Maybe in the time it took for the headache to go away, they ate or drank something that might have had an effect. Perhaps they had a quick nap that helped relieve the tension from the headache. Without rigorously exploring this phenomenon , any number of confounding factors exist that can make us question the actual efficacy of any particular treatment.

Experimental research relies on observing differences between the two groups by "controlling" the independent variable , or in the case of our example above, the medicine that is given or not given depending on the group. The dependent variable in this case is the change in how the person suffering the headache feels, and the difference between taking and not taking the medicine is evidence (or lack thereof) that the treatment is effective.

The catch is that, between the control group and other groups (typically called experimental groups), it's important to ensure that all other factors are the same or at least as similar as possible. Things such as age, fitness level, and even occupation can affect the likelihood someone has a headache and whether a certain medication is effective.

Faced with this dynamic, researchers try to make sure that participants in their control group and experimental group are as similar as possible to each other, with the only difference being the treatment they receive.

Experimental research is often associated with scientists in lab coats holding beakers containing liquids with funny colors. Clinical trials that deal with medical treatments rely primarily, if not exclusively, on experimental research designs involving comparisons between control and experimental groups.

However, many studies in the social sciences also employ some sort of experimental design which calls for the use of control groups. This type of research is useful when researchers are trying to confirm or challenge an existing notion or measure the difference in effects.

Workplace efficiency research

How might a company know if an employee training program is effective? They may decide to pilot the program to a small group of their employees before they implement the training to their entire workforce.

If they adopt an experimental design, they could compare results between an experimental group of workers who participate in the training program against a control group who continues as per usual without any additional training.

Qualitative data analysis starts with ATLAS.ti

Turn data into rich insights with our powerful data analysis software. Get started with a free trial.

Mental health research

Music certainly has profound effects on psychology, but what kind of music would be most effective for concentration? Here, a researcher might be interested in having participants in a control group perform a series of tasks in an environment with no background music, and participants in multiple experimental groups perform those same tasks with background music of different genres. The subsequent analysis could determine how well people perform with classical music, jazz music, or no music at all in the background.

Educational research

Suppose that you want to improve reading ability among elementary school students, and there is research on a particular teaching method that is associated with facilitating reading comprehension. How do you measure the effects of that teaching method?

A study could be conducted on two groups of otherwise equally proficient students to measure the difference in test scores. The teacher delivers the same instruction to the control group as they have to previous students, but they teach the experimental group using the new technique. A reading test after a certain amount of instruction could determine the extent of effectiveness of the new teaching method.

As you can see from the three examples above, experimental groups are the counterbalance to control groups. A control group offers an essential point of comparison. For an experimental study to be considered credible, it must establish a baseline against which novel research is conducted.

Researchers can determine the makeup of their experimental and control groups from their literature review . Remember that the objective of a review is to establish what is known about the object of inquiry and what is not known. Where experimental groups explore the unknown aspects of scientific knowledge, a control group is a sort of simulation of what would happen if the treatment or intervention was not administered. As a result, it will benefit researchers to have a foundational knowledge of the existing research to create a credible control group against which experimental results are compared, especially in terms of remaining sensitive to relevant participant characteristics that could confound the effects of your treatment or intervention so that you can appropriately distribute participants between the experimental and control groups.

There are multiple control groups to consider depending on the study you are looking to conduct. All of them are variations of the basic control group used to establish a baseline for experimental conditions.

No-treatment control group

This kind of control group is common when trying to establish the effects of an experimental treatment against the absence of treatment. This is arguably the most straightforward approach to an experimental design as it aims to directly demonstrate how a certain change in conditions produces an effect.

Placebo control group

In this case, the control group receives some sort of treatment under the exact same procedures as those in the experimental group. The only difference in this case is that the treatment in the placebo control group has already been judged to be ineffective, except that the research participants don't know that it is ineffective.

Placebo control groups (or negative control groups) are useful for allowing researchers to account for any psychological or affective factors that might impact the outcomes. The negative control group exists to explicitly eliminate factors other than changes in the independent variable conditions as causes of the effects experienced in the experimental group.

Positive control group

Contrasted with a no-treatment control group, a positive control group employs a treatment against which the treatment in the experimental group is compared. However, unlike in a placebo group, participants in a positive control group receive treatment that is known to have an effect.

If we were to use our first example of headache medicine, a researcher could compare results between medication that is commonly known as effective against the newer medication that the researcher thinks is more effective. Positive control groups are useful for validating experimental results when compared against familiar results.

Historical control group

Rather than study participants in control group conditions, researchers may employ existing data to create historical control groups. This form of control group is useful for examining changing conditions over time, particularly when incorporating past conditions that can't be replicated in the analysis.

Qualitative research more often relies on non-experimental research such as observations and interviews to examine phenomena in their natural environments. This sort of research is more suited for inductive and exploratory inquiries, not confirmatory studies meant to test or measure a phenomenon.

That said, the broader concept of a control group is still present in observational and interview research in the form of a comparison group. Comparison groups are used in qualitative research designs to show differences between phenomena, with the exception being that there is no baseline against which data is analyzed.

Comparison groups are useful when an experimental environment cannot produce results that would be applicable to real-world conditions. Research inquiries examining the social world face challenges of having too many variables to control, making observations and interviews across comparable groups more appropriate for data collection than clinical or sterile environments.

Analyze data and generate rich results with ATLAS.ti

Try out a free trial of ATLAS.ti to see how you can make the most of your qualitative data.

Control Group vs Experimental Group

Julia Simkus

Editor at Simply Psychology

BA (Hons) Psychology, Princeton University

Julia Simkus is a graduate of Princeton University with a Bachelor of Arts in Psychology. She is currently studying for a Master's Degree in Counseling for Mental Health and Wellness in September 2023. Julia's research has been published in peer reviewed journals.

Learn about our Editorial Process

Saul McLeod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

In a controlled experiment , scientists compare a control group, and an experimental group is identical in all respects except for one difference – experimental manipulation.

Differences

Unlike the experimental group, the control group is not exposed to the independent variable under investigation. So, it provides a baseline against which any changes in the experimental group can be compared.

Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between the two are due to experimental manipulation rather than chance.

Almost all experimental studies are designed to include a control group and one or more experimental groups. In most cases, participants are randomly assigned to either a control or experimental group.

Because participants are randomly assigned to either group, we can assume that the groups are identical except for manipulating the independent variable in the experimental group.

It is important that every aspect of the experimental environment is the same and that the experimenters carry out the exact same procedures with both groups so researchers can confidently conclude that any differences between groups are actually due to the difference in treatments.

Control Group

A control group consists of participants who do not receive any experimental treatment. The control participants serve as a comparison group.

The control group is matched as closely as possible to the experimental group, including age, gender, social class, ethnicity, etc.

The difference between the control and experimental groups is that the control group is not exposed to the independent variable , which is thought to be the cause of the behavior being investigated.

Researchers will compare the individuals in the control group to those in the experimental group to isolate the independent variable and examine its impact.

The control group is important because it serves as a baseline, enabling researchers to see what impact changes to the independent variable produce and strengthening researchers’ ability to draw conclusions from a study.

Without the presence of a control group, a researcher cannot determine whether a particular treatment truly has an effect on an experimental group.

Control groups are critical to the scientific method as they help ensure the internal validity of a study.

Assume you want to test a new medication for ADHD . One group would receive the new medication, and the other group would receive a pill that looked exactly the same as the one that the others received, but it would be a placebo. The group that takes the placebo would be the control group.

Types of Control Groups

Positive control group.

• A positive control group is an experimental control that will produce a known response or the desired effect.
• A positive control is used to ensure a test’s success and confirm an experiment’s validity.
• For example, when testing for a new medication, an already commercially available medication could serve as the positive control.

Negative Control Group

• A negative control group is an experimental control that does not result in the desired outcome of the experiment.
• A negative control is used to ensure that there is no response to the treatment and help identify the influence of external factors on the test.
• An example of a negative control would be using a placebo when testing for a new medication.

Experimental Group

An experimental group consists of participants exposed to a particular manipulation of the independent variable. These are the participants who receive the treatment of interest.

Researchers will compare the responses of the experimental group to those of a control group to see if the independent variable impacted the participants.

An experiment must have at least one control group and one experimental group; however, a single experiment can include multiple experimental groups, which are all compared against the control group.

Having multiple experimental groups enables researchers to vary different levels of an experimental variable and compare the effects of these changes to the control group and among each other.

Assume you want to study to determine if listening to different types of music can help with focus while studying.

You randomly assign participants to one of three groups: one group that listens to music with lyrics, one group that listens to music without lyrics, and another group that listens to no music.

The group of participants listening to no music while studying is the control group, and the groups listening to music, whether with or without lyrics, are the two experimental groups.

Frequently Asked Questions

1. what is the difference between the control group and the experimental group in an experimental study.

Put simply; an experimental group is a group that receives the variable, or treatment, that the researchers are testing, whereas the control group does not. These two groups should be identical in all other aspects.

2. What is the purpose of a control group in an experiment

A control group is essential in experimental research because it:

Provides a baseline against which the effects of the manipulated variable (the independent variable) can be measured.

Helps to ensure that any changes observed in the experimental group are indeed due to the manipulation of the independent variable and not due to other extraneous or confounding factors.

Helps to account for the placebo effect, where participants’ beliefs about the treatment can influence their behavior or responses.

In essence, it increases the internal validity of the results and the confidence we can have in the conclusions.

3. Do experimental studies always need a control group?

Not all experiments require a control group, but a true “controlled experiment” does require at least one control group. For example, experiments that use a within-subjects design do not have a control group.

In  within-subjects designs , all participants experience every condition and are tested before and after being exposed to treatment.

These experimental designs tend to have weaker internal validity as it is more difficult for a researcher to be confident that the outcome was caused by the experimental treatment and not by a confounding variable.

4. Can a study include more than one control group?

Yes, studies can include multiple control groups. For example, if several distinct groups of subjects do not receive the treatment, these would be the control groups.

5. How is the control group treated differently from the experimental groups?

The control group and the experimental group(s) are treated identically except for one key difference: exposure to the independent variable, which is the factor being tested. The experimental group is subjected to the independent variable, whereas the control group is not.

This distinction allows researchers to measure the effect of the independent variable on the experimental group by comparing it to the control group, which serves as a baseline or standard.

Bailey, R. A. (2008). Design of Comparative Experiments. Cambridge University Press. ISBN 978-0-521-68357-9.

Hinkelmann, Klaus; Kempthorne, Oscar (2008). Design and Analysis of Experiments, Volume I: Introduction to Experimental Design (2nd ed.). Wiley. ISBN 978-0-471-72756-9.

• History & Society
• Science & Tech
• Biographies
• Animals & Nature
• Geography & Travel
• Arts & Culture
• Games & Quizzes
• On This Day
• One Good Fact
• New Articles
• Lifestyles & Social Issues
• Philosophy & Religion
• Politics, Law & Government
• World History
• Health & Medicine
• Browse Biographies
• Birds, Reptiles & Other Vertebrates
• Bugs, Mollusks & Other Invertebrates
• Environment
• Fossils & Geologic Time
• Entertainment & Pop Culture
• Sports & Recreation
• Visual Arts
• Demystified
• Image Galleries
• Infographics
• Top Questions
• Britannica Kids
• Saving Earth
• Space Next 50
• Student Center
• When did science begin?
• Where was science invented?

control group

Our editors will review what you’ve submitted and determine whether to revise the article.

• Verywell Mind - What Is a Control Group?
• National Center for Biotechnology Information - PubMed Central - Control Group Design: Enhancing Rigor in Research of Mind-Body Therapies for Depression

control group , the standard to which comparisons are made in an experiment. Many experiments are designed to include a control group and one or more experimental groups; in fact, some scholars reserve the term experiment for study designs that include a control group. Ideally, the control group and the experimental groups are identical in every way except that the experimental groups are subjected to treatments or interventions believed to have an effect on the outcome of interest while the control group is not. Inclusion of a control group greatly strengthens researchers’ ability to draw conclusions from a study. Indeed, only in the presence of a control group can a researcher determine whether a treatment under investigation truly has a significant effect on an experimental group, and the possibility of making an erroneous conclusion is reduced. See also scientific method .

A typical use of a control group is in an experiment in which the effect of a treatment is unknown and comparisons between the control group and the experimental group are used to measure the effect of the treatment. For instance, in a pharmaceutical study to determine the effectiveness of a new drug on the treatment of migraines , the experimental group will be administered the new drug and the control group will be administered a placebo (a drug that is inert, or assumed to have no effect). Each group is then given the same questionnaire and asked to rate the effectiveness of the drug in relieving symptoms . If the new drug is effective, the experimental group is expected to have a significantly better response to it than the control group. Another possible design is to include several experimental groups, each of which is given a different dosage of the new drug, plus one control group. In this design, the analyst will compare results from each of the experimental groups to the control group. This type of experiment allows the researcher to determine not only if the drug is effective but also the effectiveness of different dosages. In the absence of a control group, the researcher’s ability to draw conclusions about the new drug is greatly weakened, due to the placebo effect and other threats to validity. Comparisons between the experimental groups with different dosages can be made without including a control group, but there is no way to know if any of the dosages of the new drug are more or less effective than the placebo.

It is important that every aspect of the experimental environment be as alike as possible for all subjects in the experiment. If conditions are different for the experimental and control groups, it is impossible to know whether differences between groups are actually due to the difference in treatments or to the difference in environment. For example, in the new migraine drug study, it would be a poor study design to administer the questionnaire to the experimental group in a hospital setting while asking the control group to complete it at home. Such a study could lead to a misleading conclusion, because differences in responses between the experimental and control groups could have been due to the effect of the drug or could have been due to the conditions under which the data were collected. For instance, perhaps the experimental group received better instructions or was more motivated by being in the hospital setting to give accurate responses than the control group.

In non-laboratory and nonclinical experiments, such as field experiments in ecology or economics , even well-designed experiments are subject to numerous and complex variables that cannot always be managed across the control group and experimental groups. Randomization, in which individuals or groups of individuals are randomly assigned to the treatment and control groups, is an important tool to eliminate selection bias and can aid in disentangling the effects of the experimental treatment from other confounding factors. Appropriate sample sizes are also important.

A control group study can be managed in two different ways. In a single-blind study, the researcher will know whether a particular subject is in the control group, but the subject will not know. In a double-blind study , neither the subject nor the researcher will know which treatment the subject is receiving. In many cases, a double-blind study is preferable to a single-blind study, since the researcher cannot inadvertently affect the results or their interpretation by treating a control subject differently from an experimental subject.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Control Groups and Treatment Groups | Uses & Examples

Control Groups & Treatment Groups | Uses & Examples

Published on 6 May 2022 by Lauren Thomas . Revised on 13 April 2023.

In a scientific study, a control group is used to establish a cause-and-effect relationship by isolating the effect of an independent variable .

Researchers change the independent variable in the treatment group and keep it constant in the control group. Then they compare the results of these groups.

Using a control group means that any change in the dependent variable can be attributed to the independent variable.

Table of contents

Control groups in experiments, control groups in non-experimental research, importance of control groups, frequently asked questions about control groups.

Control groups are essential to experimental design . When researchers are interested in the impact of a new treatment, they randomly divide their study participants into at least two groups:

• The treatment group (also called the experimental group ) receives the treatment whose effect the researcher is interested in.
• The control group receives either no treatment, a standard treatment whose effect is already known, or a placebo (a fake treatment).

The treatment is any independent variable manipulated by the experimenters, and its exact form depends on the type of research being performed. In a medical trial, it might be a new drug or therapy. In public policy studies, it could be a new social policy that some receive and not others.

In a well-designed experiment, all variables apart from the treatment should be kept constant between the two groups. This means researchers can correctly measure the entire effect of the treatment without interference from confounding variables .

• You pay the students in the treatment group for achieving high grades.
• Students in the control group do not receive any money.

Studies can also include more than one treatment or control group. Researchers might want to examine the impact of multiple treatments at once, or compare a new treatment to several alternatives currently available.

• The treatment group gets the new pill.
• Control group 1 gets an identical-looking sugar pill (a placebo).
• Control group 2 gets a pill already approved to treat high blood pressure.

Since the only variable that differs between the three groups is the type of pill, any differences in average blood pressure between the three groups can be credited to the type of pill they received.

• The difference between the treatment group and control group 1 demonstrates the effectiveness of the pill as compared to no treatment.
• The difference between the treatment group and control group 2 shows whether the new pill improves on treatments already available on the market.

Prevent plagiarism, run a free check.

Although control groups are more common in experimental research, they can be used in other types of research too. Researchers generally rely on non-experimental control groups in two cases: quasi-experimental or matching design.

Control groups in quasi-experimental design

While true experiments rely on random assignment to the treatment or control groups, quasi-experimental design uses some criterion other than randomisation to assign people.

Often, these assignments are not controlled by researchers, but are pre-existing groups that have received different treatments. For example, researchers could study the effects of a new teaching method that was applied in some classes in a school but not others, or study the impact of a new policy that is implemented in one region but not in the neighbouring region.

In these cases, the classes that did not use the new teaching method, or the region that did not implement the new policy, is the control group.

Control groups in matching design

In correlational research , matching represents a potential alternate option when you cannot use either true or quasi-experimental designs.

In matching designs, the researcher matches individuals who received the ‘treatment’, or independent variable under study, to others who did not – the control group.

Each member of the treatment group thus has a counterpart in the control group identical in every way possible outside of the treatment. This ensures that the treatment is the only source of potential differences in outcomes between the two groups.

Control groups help ensure the internal validity of your research. You might see a difference over time in your dependent variable in your treatment group. However, without a control group, it is difficult to know whether the change has arisen from the treatment. It is possible that the change is due to some other variables.

If you use a control group that is identical in every other way to the treatment group, you know that the treatment – the only difference between the two groups – must be what has caused the change.

For example, people often recover from illnesses or injuries over time regardless of whether they’ve received effective treatment or not. Thus, without a control group, it’s difficult to determine whether improvements in medical conditions come from a treatment or just the natural progression of time.

Risks from invalid control groups

If your control group differs from the treatment group in ways that you haven’t accounted for, your results may reflect the interference of confounding variables instead of your independent variable.

Minimising this risk

A few methods can aid you in minimising the risk from invalid control groups.

• Ensure that all potential confounding variables are accounted for , preferably through an experimental design if possible, since it is difficult to control for all the possible confounders outside of an experimental environment.
• Use double-blinding . This will prevent the members of each group from modifying their behavior based on whether they were placed in the treatment or control group, which could then lead to biased outcomes.
• Randomly assign your subjects into control and treatment groups. This method will allow you to not only minimise the differences between the two groups on confounding variables that you can directly observe, but also those you cannot.

An experimental group, also known as a treatment group, receives the treatment whose effect researchers wish to study, whereas a control group does not. They should be identical in all other ways.

A true experiment (aka a controlled experiment) always includes at least one control group that doesn’t receive the experimental treatment.

However, some experiments use a within-subjects design to test treatments without a control group. In these designs, you usually compare one group’s outcomes before and after a treatment (instead of comparing outcomes between different groups).

For strong internal validity , it’s usually best to include a control group if possible. Without a control group, it’s harder to be certain that the outcome was caused by the experimental treatment and not by other variables.

In a controlled experiment , all extraneous variables are held constant so that they can’t influence the results. Controlled experiments require:

• A control group that receives a standard treatment, a fake treatment, or no treatment
• Random assignment of participants to ensure the groups are equivalent

Depending on your study topic, there are various other methods of controlling variables .

A confounding variable , also called a confounder or confounding factor, is a third variable in a study examining a potential cause-and-effect relationship.

A confounding variable is related to both the supposed cause and the supposed effect of the study. It can be difficult to separate the true effect of the independent variable from the effect of the confounding variable.

In your research design , it’s important to identify potential confounding variables and plan how you will reduce their impact.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Thomas, L. (2023, April 13). Control Groups & Treatment Groups | Uses & Examples. Scribbr. Retrieved 3 September 2024, from https://www.scribbr.co.uk/research-methods/control-groups/

Is this article helpful?

Lauren Thomas

Other students also liked, controlled experiments | methods & examples of control, a quick guide to experimental design | 5 steps & examples, correlation vs causation | differences, designs & examples.

• COVID-19 Tracker
• Biochemistry
• Anatomy & Physiology
• Microbiology
• Neuroscience
• Animal Kingdom
• NGSS High School
• Latest News
• Editors’ Picks
• Weekly Digest
• Quotes about Biology

Control Group

Reviewed by: BD Editors

Control Group Definition

In scientific experiments, the control group is the group of subject that receive no treatment or a standardized treatment. Without the control group, there would be nothing to compare the treatment group to. When statistics refer to something being “X times more likely to happen” they are referring to the difference in the measurement between the treatment and control group. The control group provides a baseline in the experiment. The variable that is being studied in the experiment is not changed or is limited to zero in the control group. This insures that the effects of the variable are being studied. Most experiments try to add the variable back in increments to different treatment groups, to really begin to discern the effects of the variable in the system.

Ideally, the control group is subject to the same exact conditions as the treatment groups. This insures that only the effects produced by the variable are being measured. In a study of plants, for instance, all the plants would ideally be in the same room, with the same light and air conditions. In biological studies, it is also important that the organisms in the treatment and control groups come from the same population. Ideally, the organisms would all be clones of each other, to reduce genetic differences. This is the case in many artificially selected lab species, which have been selected to be very similar to each other. This ensures that the results obtained are valid.

Examples of Control Group

Testing enzyme strength.

In a simple biological lab experiment, students can test the effects of different concentrations of enzyme. The student can prepare a stock solution of enzyme by spitting into a beaker. Human spit contains the enzyme amylase, which breaks down starches. The concentration of enzyme can be varied by dividing the stock solution and adding in various amounts of water. Once various solutions of different strength enzyme have been produced, the experiment can begin.

In several treatment beakers are placed the following ingredients: starch, iodine, and the different solutions of enzyme. In the control group, a beaker is filled with starch and iodine, but no enzyme. When iodine is in the presence of starch, it turns black. As the enzyme depletes the starch in each beaker, the solution clears up and is a lighter yellow or brown color. In this way, the student can tell how long the enzymes in each beaker take to completely process the same amount of substrate. The control group is important because it will tell the student if the starch breaks down without the enzyme, which it will, given enough time.

Testing Drugs and the Placebo Effect

When drugs are tested on humans, control groups are also used. Although control groups were just considered good science, they have found an interesting phenomena in drug trials. Oftentimes, control groups in drug trials consist of people who also have the disease or ailment, but who don’t receive the medicine being tested. Instead, to keep the control group the same as the treatment groups, the patients in the control group are also given a pill. This is a sugar pill usually and contains no medicine. This practice of having a control group is important for drug trial, because it validates the results obtained. However, the control groups have also demonstrated an interesting effect, known as the placebo effect

In some drug trials, where the control group is given a fake medicine, patients start to see results. Scientists call this the placebo effect, and as of yet it is mostly unexplained. Some scientists have suggested that people get better simply because they believed they were going to get better, but this theory remains untested. Other scientists claim that unknown variables in the experiment caused the patients to get better. This theory remains unproven, as well.

Related Biology Terms

• Treatment Group – The group that receives the variable, or altered amounts of the variable.
• Variable – The part of the experiment being studied which is changed, or altered, throughout the experiment.
• Scientific Method – The steps scientist follow to ensure their results are valid and reproducible.
• Placebo Effect – A phenomenon when patients in the control group experience the same effects as those in the treatment group, though no treatment was given.

Cite This Article

Subscribe to our newsletter, privacy policy, terms of service, scholarship, latest posts, white blood cell, t cell immunity, satellite cells, embryonic stem cells, popular topics, natural selection, hermaphrodite, digestive system, scientific method, horticulture.

Positive Control vs Negative Control: Differences & Examples

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

Learn about our Editorial Process

A positive control is designed to confirm a known response in an experimental design , while a negative control ensures there’s no effect, serving as a baseline for comparison.

The two terms are defined as below:

• Positive control refers to a group in an experiment that receives a procedure or treatment known to produce a positive result. It serves the purpose of affirming the experiment’s capability to produce a positive outcome.
• Negative control refers to a group that does not receive the procedure or treatment and is expected not to yield a positive result. Its role is to ensure that a positive result in the experiment is due to the treatment or procedure.

The experimental group is then compared to these control groups, which can help demonstrate efficacy of the experimental treatment in comparison to the positive and negative controls.

Positive Control vs Negative Control: Key Terms

Control groups.

A control group serves as a benchmark in an experiment. Typically, it is a subset of participants, subjects, or samples that do not receive the experimental treatment (as in negative control).

This could mean assigning a placebo to a human subject or leaving a sample unaltered in chemical experiments. By comparing the results obtained from the experimental group to the control, you can ascertain whether any differences are due to the treatment or random variability.

A well-configured experimental control is critical for drawing valid conclusions from an experiment. Correct use of control groups permits specificity of findings, ensuring the integrity of experimental data.

See More: Control Variables Examples

The Negative Control

Negative control is a group or condition in an experiment that ought to show no effect from the treatment.

It is useful in ensuring that the outcome isn’t accidental or influenced by an external cause. Imagine a medical test, for instance. You use distilled water, anticipating no reaction, as a negative control.

If a significant result occurs, it warns you of a possible contamination or malfunction during the testing. Failure of negative controls to stay ‘negative’ risks misinterpretation of the experiment’s result, and could undermine the validity of the findings.

The Positive Control

A positive control, on the other hand, affirms an experiment’s functionality by demonstrating a known reaction.

This might be a group or condition where the expected output is known to occur, which you include to ensure that the experiment can produce positive results when they are present. For instance, in testing an antibiotic, a well-known pathogen, susceptible to the medicine, could be the positive control.

Positive controls affirm that under appropriate conditions your experiment can produce a result. Without this reference, experiments could fail to detect true positive results, leading to false negatives. These two controls, used judiciously, are backbones of effective experimental practice.

Experimental Groups

Experimental groups are primarily characterized by their exposure to the examined variable.

That is, these are the test subjects that receive the treatment or intervention under investigation. The performance of the experimental group is then compared against the well-established markers – our positive and negative controls.

For example, an experimental group may consist of rats undergoing a pharmaceutical testing regime, or students learning under a new educational method. Fundamentally, this unit bears the brunt of the investigation and their response powers the outcomes.

However, without positive and negative controls, gauging the results of the experimental group could become erratic. Both control groups exist to highlight what outcomes are expected with and without the application of the variable in question. By comparing results, a clearer connection between the experiment variables and the observed changes surfaces, creating robust and indicative scientific conclusions.

Positive and Negative Control Examples

1. a comparative study of old and new pesticides’ effectiveness.

This hypothetical study aims to evaluate the effectiveness of a new pesticide by comparing its pest-killing potential with old pesticides and an untreated set. The investigation involves three groups: an untouched space (negative control), another treated with an established pesticide believed to kill pests (positive control), and a third area sprayed with the new pesticide (experimental group).

• Negative Control: This group consists of a plot of land infested by pests and not subjected to any pesticide treatment. It acts as the negative control. You expect no decline in pest populations in this area. Any unexpected decrease could signal external influences (i.e. confounding variables ) on the pests unrelated to pesticides, affecting the experiment’s validity.
• Positive Control: Another similar plot, this time treated with a well-established pesticide known to reduce pest populations, constitutes the positive control. A significant reduction in pests in this area would affirm that the experimental conditions are conducive to detect pest-killing effects when a pesticide is applied.
• Experimental Group: This group consists of the third plot impregnated with the new pesticide. Carefully monitoring the pest level in this research area against the backdrop of the control groups will reveal whether the new pesticide is effective or not. Through comparison with the other groups, any difference observed can be attributed to the new pesticide.

2. Evaluating the Effectiveness of a Newly Developed Weight Loss Pill

In this hypothetical study, the effectiveness of a newly formulated weight loss pill is scrutinized. The study involves three groups: a negative control group given a placebo with no weight-reducing effect, a positive control group provided with an approved weight loss pill known to cause a decrease in weight, and an experimental group given the newly developed pill.

• Negative Control: The negative control is comprised of participants who receive a placebo with no known weight loss effect. A significant reduction in weight in this group would indicate confounding factors such as dietary changes or increased physical activity, which may invalidate the study’s results.
• Positive Control: Participants in the positive control group receive an FDA-approved weight loss pill, anticipated to induce weight loss. The success of this control would prove that the experiment conditions are apt to detect the effects of weight loss pills.
• Experimental Group: This group contains individuals receiving the newly developed weight loss pill. Comparing the weight change in this group against both the positive and negative control, any difference observed would offer evidence about the effectiveness of the new pill.

3. Testing the Efficiency of a New Solar Panel Design

This hypothetical study focuses on assessing the efficiency of a new solar panel design. The study involves three sets of panels: a set that is shaded to yield no solar energy (negative control), a set with traditional solar panels that are known to produce an expected level of solar energy (positive control), and a set fitted with the new solar panel design (experimental group).

• Negative Control: The negative control involves a set of solar panels that are deliberately shaded, thus expecting no solar energy output. Any unexpected energy output from this group could point towards measurement errors, needed to be rectified for a valid experiment.
• Positive Control: The positive control set up involves traditional solar panels known to produce a specific amount of energy. If these panels produce the expected energy, it validates that the experiment conditions are capable of measuring solar energy effectively.
• Experimental Group: The experimental group features the new solar panel design. By comparing the energy output from this group against both the controls, any significant output variation would indicate the efficiency of the new design.

4. Investigating the Efficacy of a New Fertilizer on Plant Growth

This hypothetical study investigates the efficacy of a newly formulated fertilizer on plant growth. The study involves three sets of plants: a set without any fertilizer (negative control), a set treated with an established fertilizer known to promote plant growth (positive control), and a third set fed with the new fertilizer (experimental group).

• Negative Control: The negative control involves a set of plants not receiving any fertilizer. Lack of significant growth in this group will confirm that any observed growth in other groups is due to the applied fertilizer rather than other uncontrolled factors.
• Positive Control: The positive control involves another set of plants treated with a well-known fertilizer, expected to promote plant growth. Adequate growth in these plants will validate that the experimental conditions are suitable to detect the influence of a good fertilizer on plant growth.
• Experimental Group: The experimental group consists of the plants subjected to the newly formulated fertilizer. Investigating the growth in this group against the growth in the control groups will provide ascertained evidence whether the new fertilizer is efficient or not.

5. Evaluating the Impact of a New Teaching Method on Student Performance

This hypothetical study aims to evaluate the impact of a new teaching method on students’ performance. This study involves three groups, a group of students taught through traditional methods (negative control), another group taught through an established effective teaching strategy (positive control), and one more group of students taught through the new teaching method (experimental group).

• Negative Control: The negative control comprises students taught by standard teaching methods, where you expect satisfactory but not top-performing results. Any unexpected high results in this group could signal external factors such as private tutoring or independent study, which in turn may distort the experimental outcome.
• Positive Control: The positive control consists of students taught by a known efficient teaching strategy. High performance in this group would prove that the experimental conditions are competent to detect the efficiency of a teaching method.
• Experimental Group: This group consists of students receiving instruction via the new teaching method. By analyzing their performance against both control groups, any difference in results could be attributed to the new teaching method, determining its efficacy.

Table Summary

• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 10 Reasons you’re Perpetually Single
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 20 Montessori Toddler Bedrooms (Design Inspiration)
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 21 Montessori Homeschool Setups
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 101 Hidden Talents Examples

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

The Difference Between Control Group and Experimental Group

• Chemical Laws
• Periodic Table
• Projects & Experiments
• Scientific Method
• Biochemistry
• Physical Chemistry
• Medical Chemistry
• Chemistry In Everyday Life
• Famous Chemists
• Activities for Kids
• Abbreviations & Acronyms
• Weather & Climate

In an experiment , data from an experimental group is compared with data from a control group. These two groups should be identical in every respect except one: the difference between a control group and an experimental group is that the independent variable is changed for the experimental group, but is held constant in the control group.

Key Takeaways: Control vs. Experimental Group

• The control group and experimental group are compared against each other in an experiment. The only difference between the two groups is that the independent variable is changed in the experimental group. The independent variable is "controlled", or held constant, in the control group.
• A single experiment may include multiple experimental groups, which may all be compared against the control group.
• The purpose of having a control is to rule out other factors which may influence the results of an experiment. Not all experiments include a control group, but those that do are called "controlled experiments."
• A placebo may also be used in an experiment. A placebo isn't a substitute for a control group because subjects exposed to a placebo may experience effects from the belief they are being tested; this itself is known as the placebo effect.

What Are Is an Experimental Group in Experiment Design?

An experimental group is a test sample or the group that receives an experimental procedure. This group is exposed to changes in the independent variable being tested. The values of the independent variable and the impact on the dependent variable are recorded. An experiment may include multiple experimental groups at one time.

A control group is a group separated from the rest of the experiment such that the independent variable being tested cannot influence the results. This isolates the independent variable's effects on the experiment and can help rule out alternative explanations of the experimental results.

While all experiments have an experimental group, not all experiments require a control group. Controls are extremely useful where the experimental conditions are complex and difficult to isolate. Experiments that use control groups are called controlled experiments .

A Simple Example of a Controlled Experiment

A simple example of a controlled experiment may be used to determine whether or not plants need to be watered to live. The control group would be plants that are not watered. The experimental group would consist of plants that receive water. A clever scientist would wonder whether too much watering might kill the plants and would set up several experimental groups, each receiving a different amount of water.

Sometimes setting up a controlled experiment can be confusing. For example, a scientist may wonder whether or not a species of bacteria needs oxygen in order to live. To test this, cultures of bacteria may be left in the air, while other cultures are placed in a sealed container of nitrogen (the most common component of air) or deoxygenated air (which likely contained extra carbon dioxide). Which container is the control? Which is the experimental group?

Control Groups and Placebos

The most common type of control group is one held at ordinary conditions so it doesn't experience a changing variable. For example, If you want to explore the effect of salt on plant growth, the control group would be a set of plants not exposed to salt, while the experimental group would receive the salt treatment. If you want to test whether the duration of light exposure affects fish reproduction, the control group would be exposed to a "normal" number of hours of light, while the duration would change for the experimental group.

Experiments involving human subjects can be much more complex. If you're testing whether a drug is effective or not, for example, members of a control group may expect they will not be unaffected. To prevent skewing the results, a placebo may be used. A placebo is a substance that doesn't contain an active therapeutic agent. If a control group takes a placebo, participants don't know whether they are being treated or not, so they have the same expectations as members of the experimental group.

However, there is also the placebo effect to consider. Here, the recipient of the placebo experiences an effect or improvement because she believes there should be an effect. Another concern with a placebo is that it's not always easy to formulate one that truly free of active ingredients. For example, if a sugar pill is given as a placebo, there's a chance the sugar will affect the outcome of the experiment.

Positive and Negative Controls

Positive and negative controls are two other types of control groups:

• Positive control groups are control groups in which the conditions guarantee a positive result. Positive control groups are effective to show the experiment is functioning as planned.
• Negative control groups are control groups in which conditions produce a negative outcome. Negative control groups help identify outside influences which may be present that were not unaccounted for, such as contaminants.
• Bailey, R. A. (2008). Design of Comparative Experiments . Cambridge University Press. ISBN 978-0-521-68357-9.
• Chaplin, S. (2006). "The placebo response: an important part of treatment". Prescriber : 16–22. doi: 10.1002/psb.344
• Hinkelmann, Klaus; Kempthorne, Oscar (2008). Design and Analysis of Experiments, Volume I: Introduction to Experimental Design (2nd ed.). Wiley. ISBN 978-0-471-72756-9.
• What is the Difference Between Molarity and Molality?
• The Difference Between Homogeneous and Heterogeneous Mixtures
• The Difference Between Intensive and Extensive Properties
• Examples of Polar and Nonpolar Molecules
• How to Draw a Lewis Structure
• Ionic vs. Covalent Bonds: How Are They Different?
• How to Calculate Density of a Gas
• What Is Alum and How Is It Used?
• The Visible Spectrum: Wavelengths and Colors
• Examples of Physical Changes
• Chemistry Glassware Types, Names and Uses
• Fun and Interesting Chemistry Facts
• Table of Electrical Resistivity and Conductivity
• Molarity Definition in Chemistry
• Chemical Properties of Matter
• What Is a Control Group?

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Control Variables | What Are They & Why Do They Matter?

Control Variables | What Are They & Why Do They Matter?

Published on March 1, 2021 by Pritha Bhandari . Revised on June 22, 2023.

A control variable is anything that is held constant or limited in a research study. It’s a variable that is not of interest to the study’s objectives , but is controlled because it could influence the outcomes.

Variables may be controlled directly by holding them constant throughout a study (e.g., by controlling the room temperature in an experiment), or they may be controlled indirectly through methods like randomization or statistical control (e.g., to account for participant characteristics like age in statistical tests). Control variables can help prevent research biases like omitted variable bias from affecting your results.

Table of contents

Why do control variables matter, how do you control a variable, control variable vs. control group, other interesting articles, frequently asked questions about control variables.

Control variables enhance the internal validity of a study by limiting the influence of confounding and other extraneous variables . This helps you establish a correlational or causal relationship between your variables of interest and helps avoid research bias .

Aside from the independent and dependent variables , all variables that can impact the results should be controlled. If you don’t control relevant variables, you may not be able to demonstrate that they didn’t influence your results. Uncontrolled variables are alternative explanations for your results and affect the reliability of your arguments.

Control variables in experiments

In an experiment , a researcher is interested in understanding the effect of an independent variable on a dependent variable. Control variables help you ensure that your results are solely caused by your experimental manipulation.

The independent variable is whether the vitamin D supplement is added to a diet, and the dependent variable is the level of alertness.

To make sure any change in alertness is caused by the vitamin D supplement and not by other factors, you control these variables that might affect alertness:

• Timing of meals
• Caffeine intake
• Screen time

Control variables in non-experimental research

In an observational study or other types of non-experimental research, a researcher can’t manipulate the independent variable (often due to practical or ethical considerations ). Instead, control variables are measured and taken into account to infer relationships between the main variables of interest.

To account for other factors that are likely to influence the results, you also measure these control variables:

• Marital status

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

There are several ways to control extraneous variables in experimental designs, and some of these can also be used in observational studies or quasi-experimental designs.

Random assignment

In experimental studies with multiple groups, participants should be randomly assigned to the different conditions. Random assignment helps you balance the characteristics of groups so that there are no systematic differences between them.

This method of assignment controls participant variables that might otherwise differ between groups and skew your results.

It’s possible that the participants who found the study through Facebook use more screen time during the day, and this might influence how alert they are in your study.

Standardized procedures

It’s important to use the same procedures across all groups in an experiment. The groups should only differ in the independent variable manipulation so that you can isolate its effect on the dependent variable (the results).

To control variables , you can hold them constant at a fixed level using a protocol that you design and use for all participant sessions. For example, the instructions and time spent on an experimental task should be the same for all participants in a laboratory setting.

• To control for diet, fresh and frozen meals are delivered to participants three times a day.
• To control meal timings, participants are instructed to eat breakfast at 9:30, lunch at 13:00, and dinner at 18:30.
• To control caffeine intake, participants are asked to consume a maximum of one cup of coffee a day.

Statistical controls

You can measure and control for extraneous variables statistically to remove their effects on other types of variables .

“Controlling for a variable” means modelling control variable data along with independent and dependent variable data in regression analyses and ANCOVAs . That way, you can isolate the control variable’s effects from the relationship between the variables of interest.

A control variable isn’t the same as a control group . Control variables are held constant or measured throughout a study for both control and experimental groups, while an independent variable varies between control and experimental groups.

A control group doesn’t undergo the experimental treatment of interest, and its outcomes are compared with those of the experimental group. A control group usually has either no treatment, a standard treatment that’s already widely used, or a placebo (a fake treatment).

Aside from the experimental treatment, everything else in an experimental procedure should be the same between an experimental and control group.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Ecological validity

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

A control variable is any variable that’s held constant in a research study. It’s not a variable of interest in the study, but it’s controlled because it could influence the outcomes.

Control variables help you establish a correlational or causal relationship between variables by enhancing internal validity .

If you don’t control relevant extraneous variables , they may influence the outcomes of your study, and you may not be able to demonstrate that your results are really an effect of your independent variable .

Internal validity is the extent to which you can be confident that a cause-and-effect relationship established in a study cannot be explained by other factors.

“Controlling for a variable” means measuring extraneous variables and accounting for them statistically to remove their effects on other variables.

Researchers often model control variable data along with independent and dependent variable data in regression analyses and ANCOVAs . That way, you can isolate the control variable’s effects from the relationship between the variables of interest.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 22). Control Variables | What Are They & Why Do They Matter?. Scribbr. Retrieved September 7, 2024, from https://www.scribbr.com/methodology/control-variable/

Is this article helpful?

Pritha Bhandari

Other students also liked, what is a controlled experiment | definitions & examples, independent vs. dependent variables | definition & examples, extraneous variables | examples, types & controls, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• Open access
• Published: 07 September 2024

A case study of the informative value of risk of bias and reporting quality assessments for systematic reviews

• Cathalijn H. C. Leenaars   ORCID: orcid.org/0000-0002-8212-7632 1 ,
• Frans R. Stafleu 2 ,
• Christine Häger 1 &
• André Bleich 1  

Systematic Reviews volume  13 , Article number:  230 ( 2024 ) Cite this article

Metrics details

While undisputedly important, and part of any systematic review (SR) by definition, evaluation of the risk of bias within the included studies is one of the most time-consuming parts of performing an SR. In this paper, we describe a case study comprising an extensive analysis of risk of bias (RoB) and reporting quality (RQ) assessment from a previously published review (CRD42021236047). It included both animal and human studies, and the included studies compared baseline diseased subjects with controls, assessed the effects of investigational treatments, or both. We compared RoB and RQ between the different types of included primary studies. We also assessed the “informative value” of each of the separate elements for meta-researchers, based on the notion that variation in reporting may be more interesting for the meta-researcher than consistently high/low or reported/non-reported scores. In general, reporting of experimental details was low. This resulted in frequent unclear risk-of-bias scores. We observed this both for animal and for human studies and both for disease-control comparisons and investigations of experimental treatments. Plots and explorative chi-square tests showed that reporting was slightly better for human studies of investigational treatments than for the other study types. With the evidence reported as is, risk-of-bias assessments for systematic reviews have low informative value other than repeatedly showing that reporting of experimental details needs to improve in all kinds of in vivo research. Particularly for reviews that do not directly inform treatment decisions, it could be efficient to perform a thorough but partial assessment of the quality of the included studies, either of a random subset of the included publications or of a subset of relatively informative elements, comprising, e.g. ethics evaluation, conflicts of interest statements, study limitations, baseline characteristics, and the unit of analysis. This publication suggests several potential procedures.

Peer Review reports

Introduction

Researchers performing systematic reviews (SRs) face bias at two potential levels: first, at the level of the SR methods themselves, and second, at the level of the included primary studies [ 1 ]. To safeguard correct interpretation of the review’s results, transparency is required at both levels. For bias at the level of the SR methods, this is ensured by transparent reporting of the full SR methods, at least to the level of detail as required by the PRISMA statement [ 2 ]. For bias at the level of the included studies, study reporting quality (RQ) and/or risk of bias (RoB) are evaluated at the level of the individual included study. Specific tools are available to evaluate RoB in different study types [ 3 ]. Also, for reporting of primary studies, multiple guidelines and checklists are available to prevent missing important experimental details and more become available for different types of studies over time [ 4 , 5 ]. Journal endorsement of these types of guidelines has been shown to improve study reporting quality [ 6 ].

While undisputedly important, evaluation of the RoB and/or RQ of the included studies is one of the most time-consuming parts of an SR. Experienced reviewers need 10 min to an hour to complete an individual RoB assessment [ 7 ], and every included study needs to be evaluated by two reviewers. Besides spending substantial amounts of time on RoB or RQ assessments, reviewers tend to become frustrated because of the scores frequently being unclear or not reported (personal experience from the authors, colleagues and students). While automation of RoB seems to be possible without loss of accuracy [ 8 , 9 ], so far, this automation has not had significant impact on the speed; in a noninferiority randomised controlled trial of the effect of automation on person-time spent on RoB assessment, the confidence interval for the time saved ranged from − 5.20 to + 2.41 min [ 8 ].

In any scientific endeavour, there is a balance between reliability and speed; to guarantee reliability of a study, time investments are necessary. RoB or RQ assessment is generally considered to be an essential part of the systematic review process to warrant correct interpretation of the findings, but with so many studies scoring “unclear” or “not reported”, we wondered if all this time spent on RoB assessments is resulting in increased reliability of reviews.

Overall unclear risk of bias in the included primary studies is a conclusion of multiple reviews, and these assessments are useful in pinpointing problems in reporting, thereby potentially improving the quality of future publications of primary studies. However, the direct goal of most SRs is to answer a specific review question, and in that respect, unclear RoB/not reported RQ scores contribute little to the validity of the review’s results. If all included studies score “unclear” or “high” RoB on at least one of the analysed elements, the overall effect should be interpreted as inconclusive.

While it is challenging to properly evaluate the added validity value of a methodological step, we had data available allowing for an explorative case study to assess the informative value of various RoB and RQ elements in different types of studies. We previously performed an SR of the nasal potential difference (nPD) for cystic fibrosis (CF) in animals and humans, aiming to quantify the predictive value of animal models for people with CF [ 10 , 11 ]. That review comprised between-subject comparisons of both baseline versus disease-control and treatment versus treatment control. For that review, we performed full RoB and RQ analyses. This resulted in data allowing for comparisons of RoB and RQ between animal and human studies, but also between baseline and treatment studies, which are both presented in this manuscript. RoB evaluations were based on the Cochrane collaboration’s tool [ 12 ] for human studies and SYRCLE’s tool [ 13 ] for animal studies. RQ was tested based on the ARRIVE guidelines [ 14 ] for animal studies and the 2010 CONSORT guidelines [ 15 ] for human studies. Brief descriptions of these tools are provided in Table  1 .

All these tools are focussed on interventional studies. Lacking more specific tools for baseline disease-control comparisons, we applied them as far as relevant for the baseline comparisons. We performed additional analyses on our RQ and RoB assessments to assess the amount of distinctive information gained from them.

The analyses described in this manuscript are based on a case study SR of the nPD related to cystic fibrosis (CF). That review was preregistered on PROSPERO (CRD42021236047) on 5 March 2021 [ 16 ]. Part of the results were published previously [ 10 ]. The main review questions are answered in a manuscript that has more recently been published [ 11 ]. Both publications show a simple RoB plot corresponding to the publication-specific results.

For the ease of the reader, we provide a brief summary of the overall review methods. The full methods have been described in our posted protocol [ 16 ] and the earlier publications [ 10 , 11 ]. Comprehensive searches were performed in PubMed and Embase, unrestricted for publication date or language, on 23 March 2021. Title-abstract screening and full-text screening were performed by two independent reviewers blinded to the other’s decision (FS and CL) using Rayyan [ 17 ]. We included animal and/or human studies describing nPD in CF patients and/or CF animal models. We restricted to between-subject comparisons, either CF versus healthy controls or experimental CF treatments versus CF controls. Reference lists of relevant reviews and included studies were screened (single level) for snowballing. Discrepancies were all resolved by discussions between the reviewers.

Data were extracted by two independent reviewers per reference in several distinct phases. Relevant to this manuscript, FS and CL extracted RoB and RQ data in Covidence [ 18 ], in two separate projects using the same list of 48 questions for studies assessing treatment effects and studies assessing CF-control differences. The k  = 11 studies that were included in both parts of the overarching SR were included twice in the current data set, as RoB was separately scored for each comparison. Discrepancies were all resolved by discussions between the reviewers. In violation of the protocol, no third reviewer was involved.

RoB and SQ data extraction followed our review protocol, which states the following: “For human studies, risk of bias will be assessed with the Cochrane Collaboration’s tool for assessing risk of bias. For animal studies, risk of bias will be assessed with SYRCLE’s RoB tool. Besides, we will check compliance with the ARRIVE and CONSORT guidelines for reporting quality”. The four tools contain overlapping questions. To prevent unnecessary repetition of our own work, we created a single list of 48 items, which were ordered by topic for ease of extraction. For RoB, this list contains the same elements as the original tools, with the same response options (high/unclear/low RoB). For RQ, we created checklists with all elements as listed in the original tools, with the response options reported yes/no. For (RQ and RoB) elements specific to some of the included studies, the response option “irrelevant” was added. We combined these lists, only changing the order and merging duplicate elements. We do not intend this list to replace the individual tools; it was created for this specific study only.

In our list, each question was preceded by a short code indicating the tool it was derived from (A for ARRIVE, C for CONSORT, and S for SYRCLE’s) to aid later analyses. When setting up, we started with the animal-specific tools, with which the authors are more familiar. After preparing data extraction for those, we observed that all elements from the Cochrane tool had already been addressed. Therefore, this list was not explicit in our extractions. The extraction form always allowed free text to support the response. Our extraction list is provided with our supplementary data.

For RoB, the tools provide relatively clear suggestions for which level to score and when, with signalling questions and examples [ 12 , 13 ]. However, this still leaves some room for interpretation, and while the signalling questions are very educative, there are situations where the response would in our opinion not correspond to the actual bias. The RQ tools have been developed as guidelines on what to report when writing a manuscript, and not as a tool to assess RQ [ 14 , 15 ]. This means we had to operationalise upfront which level we would find sufficient to score “reported”. Our operationalisations and corrections of the tools are detailed in Table  2 .

Data were exported from Covidence into Microsoft’s Excel, where the two projects were merged and spelling and capitalisation were harmonised. Subsequent analyses were performed in R [ 21 ] version 4.3.1 (“Beagle Scouts”) via RStudio [ 22 ], using the following packages: readxl [ 23 ], dplyr [ 24 ], tidyr [ 25 ], ggplot2 [ 26 ], and crosstable [ 27 ].

Separate analyses were performed for RQ (with two levels per element) and RoB (with three levels per element). For both RoB and RQ, we first counted the numbers of irrelevant scores overall and per item. Next, irrelevant scores were deleted from further analyses. We then ranked the items by percentages for reported/not reported, or for high/unclear/low scores, and reported the top and bottom 3 (RoB) or 5 (RQ) elements.

While 100% reported is most informative to understand what actually happened in the included studies, if all authors continuously report a specific element, scoring of this element for an SR is not the most informative for meta-researchers. If an element is not reported at all, this is bad news for the overall level of confidence in an SR, but evaluating it per included study is also not too efficient except for highlighting problems in reporting, which may help to improve the quality of future (publications of) primary studies. For meta-researchers, elements with variation in reporting may be considered most interesting because these elements highlight differences between the included studies. Subgroup analyses based on specific RQ/RoB scores can help to estimate the effects of specific types of bias on the overall effect size observed in meta-analyses, as has been done for example randomisation and blinding [ 28 ]. However, these types of subgroup analyses are only possible if there is some variation in the reporting. Based on this idea, we defined a “distinctive informative value” (DIV) for RQ elements, based on the optimal variation being 50% reported and either 0% or 100% reporting being minimally informative. Thus, this “DIV” was calculated as follows:

Thus, the DIV could range from 0 (no informative value) to 50 (maximally informative), visualised in Fig.  1 .

Visual explanation of the DIV value

The DIV value was only used for ranking. The results were visualised in a heatmap, in which the intermediate shades correspond to high DIV values.

For RoB, no comparable measure was calculated. With only 10 elements but at 3 distinct levels, we thought a comparable measure would sooner hinder interpretation of informative value than help it. Instead, we show the results in an RoB plot split by population and study design type.

Because we are interested in quantifying the predictive value of animal models for human patients, we commonly perform SRs including both animal and human data (e.g. [ 29 , 30 ]). The dataset described in the current manuscript contained baseline and intervention studies in animals and humans. Because animal studies are often held responsible for the reproducibility crisis, but also to increase the external validity of this work, explorative chi-square tests (the standard statistical test for comparing percentages for binary variables) were performed to compare RQ and RoB between animal and human studies and between studies comparing baselines and treatment effects. They were performed with the base R “chisq.test” function. No power calculations were performed, as these analyses were not planned.

Literature sample

We extracted RoB and RQ data from 164 studies that were described in 151 manuscripts. These manuscripts were published from 1981 through 2020. Overall, 164 studies comprised 78 animal studies and 86 human studies, 130 comparisons of CF versus non-CF control, and 34 studies assessing experimental treatments. These numbers are detailed in a crosstable (Table  3 ).

The 48 elements in our template were completed for these 164 studies, which results in 7872 assessed elements. In total, 954 elements (12.1%) were irrelevant for various reasons (mainly for noninterventional studies and for human studies). The 7872 individual scores per study are available from the data file on OSF.

Of the 48 questions in our extraction template, 38 addressed RQ, and 10 addressed RoB.

Overall reporting quality

Of the 6232 elements related to RQ, 611 (9.8%) were deemed irrelevant. Of the remainder, 1493 (26.6% of 5621) were reported. The most reported elements were background of the research question (100% reported), objectives (98.8% reported), interpretation of the results (98.2% reported), generalisability (86.0% reported), and the experimental groups (83.5% reported). The least-reported elements were protocol violations, interim analyses + stopping rules and when the experiments were performed (all 0% reported), where the experiments were performed (0.6% reported), and all assessed outcome measures (1.2% reported).

The elements with most distinctive variation in reporting (highest DIV, refer to the “ Methods ” section for further information) were as follows: ethics evaluation (64.6% reported), conflicts of interest (34.8% reported), study limitations (29.3% reported), baseline characteristics (26.2% reported), and the unit of analysis (26.2% reported). RQ elements with DIV values over 10 are shown in Table  4 .

Overall risk of bias

Of the 1640 elements related to RoB, 343 (20.9%) were deemed irrelevant. Of the remainder, 219 (16.9%) scored high RoB, and 68 (5.2%) scored low RoB. The overall RoB scores were highest for selective outcome reporting (97.6% high), baseline group differences (19.5% high), and other biases (9.8% high); lowest for blinding of participants, caregivers, and investigators (13.4% low); blinding of outcome assessors (11.6% low) and baseline group differences (8.5% low); and most unclear for bias due to animal housing (100% unclear), detection bias due to the order of outcome measurements (99.4% unclear), and selection bias in sequence generation (97.1% unclear). The baseline group differences being both in the highest and the lowest RoB score are explained by the baseline values being reported better than the other measures, resulting in fewer unclear scores.

Variation in reporting is relatively high for most of the elements scoring high or low. Overall distinctive value of the RoB elements is low, with most scores being unclear (or, for selective outcome reporting, most scores being high).

Animal versus human studies

For RQ, the explorative chi-square tests indicated differences in reporting between animal and human studies for baseline values ( Χ 1  = 50.3, p  < 0.001), ethical review ( Χ 1  = 5.1, p  = 0.02), type of study ( Χ 1  = 11.2, p  < 0.001), experimental groups ( Χ 1  = 3.9, p  = 0.050), inclusion criteria ( Χ 1  = 24.6, p  < 0.001), the exact n value per group and in total ( Χ 1  = 26.0, p  < 0.001), (absence of) excluded datapoints ( Χ 1  = 4.5, p  = 0.03), adverse events ( Χ 1  = 5.5, p  = 0.02), and study limitations ( Χ 1  = 8.2, p  = 0.004). These explorative findings are visualised in a heatmap (Fig.  2 ).

Heatmap of reporting by type of study. Refer to Table  3 for absolute numbers of studies per category

For RoB, the explorative chi-square tests indicated differences in risk of bias between animal and human studies for baseline differences between the groups ( Χ 2  = 34.6, p  < 0.001) and incomplete outcome data ( Χ 2  = 7.6, p  = 0.02). These explorative findings are visualised in Fig.  3 .

Risk of bias by type of study. Refer to Table  3 for absolute numbers of studies per category. Note that the data shown in these plots overlap with those in the two preceding publications [ 10 , 11 ]

Studies assessing treatment effects versus studies assessing baseline differences

For RQ, the explorative chi-square tests indicated differences in reporting between comparisons of disease with control versus comparisons of treatment effects for the title listing the type of study ( X 1  = 5.0, p  = 0.03), the full paper explicitly mentioning the type of study ( X 1  = 14.0, p  < 0.001), explicit reporting of the primary outcome ( X 1  = 11.7, p  < 0.001), and reporting of adverse events X 1  = 25.4, p  < 0.001). These explorative findings are visualised in Fig.  2 .

For RoB, the explorative chi-square tests indicated differences in risk of bias between comparisons of disease with control versus comparisons of treatment effects for baseline differences between the groups ( Χ 2  = 11.4, p  = 0.003), blinding of investigators and caretakers ( Χ 2  = 29.1, p  < 0.001), blinding of outcome assessors ( Χ 2  = 6.2, p  = 0.046), and selective outcome reporting ( Χ 2  = 8.9, p  = 0.01). These explorative findings are visualised in Fig.  3 .

Overall, our results suggest lower RoB and higher RQ for human treatment studies compared to the other study types.

This literature study shows that reporting of experimental details is low, frequently resulting in unclear risk-of-bias assessments. We observed this both for animal and for human studies, with two main study designs: disease-control comparisons and, in a smaller sample, investigations of experimental treatments. Overall reporting is slightly better for elements that contribute to the “story” of a publication, such as the background of the research question, interpretation of the results and generalisability, and worst for experimental details that relate to differences between what was planned and what was actually done, such as protocol violations, interim analyses, and assessed outcome measures. The latter also results in overall high RoB scores for selective outcome reporting.

Of note, we scored this more stringently than SYRCLE’s RoB tool [ 13 ] suggests and always scored a high RoB if no protocol was posted, because only comparing the “Methods” and “Results” sections within a publication would, in our opinion, result in an overly optimistic view. Within this sample, only human treatment studies reported posting protocols upfront [ 31 , 32 ]. In contrast to selective outcome reporting, we would have scored selection, performance, and detection bias due to sequence generation more liberally for counterbalanced designs (Table  2 ), because randomisation is not the only appropriate method for preventing these types of bias. Particularly when blinding is not possible, counterbalancing [ 33 , 34 ] and Latin-square like designs [ 35 ] can decrease these biases, while randomisation would risk imbalance between groups due to “randomisation failure” [ 36 , 37 ]. We would have scored high risk of bias for blinding for these types of designs, because of increased sequence predictability. However, in practice, we did not include any studies reporting Latin-square-like or other counterbalancing designs.

One of the “non-story” elements that is reported relatively well, particularly for human treatment studies, is the blinding of participants, investigators, and caretakers. This might relate to scientists being more aware of potential bias of participants; they may consider themselves to be more objective than the general population, while the risk of influencing patients could be considered more relevant.

The main strength of this work is that it is a full formal analysis of RoB and RQ in different study types: animal and human, baseline comparisons, and treatment studies. The main limitation is that it is a single case study from a specific topic: the nPD test in CF. The results shown in this paper are not necessarily valid for other fields, particularly as we hypothesise that differences in scientific practice between medical fields relate to differences in translational success [ 38 ]. Thus, it is worth to investigate field-specific informative values before selecting which elements to score and analyse in detail.

Our comparisons of different study and population types show lower RoB and higher RQ for human treatment studies compared to the other study types for certain elements. Concerning RQ, the effects were most pronounced for the type of experimental design being explicitly mentioned and the reporting of adverse events. Concerning RoB, the effects were most pronounced for baseline differences between the groups, blinding of investigators and caretakers, and selective outcome reporting. Note, however, that the number of included treatment studies is a lot lower than the number of included baseline studies, and that the comparisons were based on only k  = 12 human treatment studies. Refer to Table  3 for absolute numbers of studies per category. Besides, our comparisons may be confounded to some extent by the publication date. The nPD was originally developed for human diagnostics [ 39 , 40 ], and animal studies only started to be reported at a later date [ 41 ]. Also, the use of the nPD as an outcome in (pre)clinical trials of investigational treatments originated at a later date [ 42 , 43 ].

Because we did not collect our data to assess time effects, we did not formally analyse them. However, we had an informal look at the publication dates by RoB score for blinding of the investigators and caretakers, and by RQ score for ethics evaluation (in box plots with dot overlay), showing more reported and fewer unclear scores in the more recent publications (data not shown). While we thus cannot rule out confounding of our results by publication date, the results are suggestive of mildly improved reporting of experimental details over time.

This study is a formal comparison of RoB and RQ scoring for two main study types (baseline comparisons and investigational treatment studies), for both animals and humans. Performing these comparisons within the context of a single SR [ 16 ] resulted in a small, but relatively homogeneous sample of primary studies about the nPD in relation to CF. On conferences and from colleagues in the animal SR field, we heard that reporting would be worse for animal than for human studies. Our comparisons allowed us to show that particularly for baseline comparisons of the nPD in CF versus control, this is not the case.

The analysed tools [ 12 , 13 , 15 ] were developed for experimental interventional studies. While some of the elements are less appropriate for other types of studies, such as animal model comparisons, our results show that many of the elements can be used and could still be useful, particularly if the reporting quality of the included studies would be better.

Implications

To correctly interpret the findings of a meta-analysis, awareness of the RoB in the included studies is more relevant than the RQ on its own. However, it is impossible to evaluate the RoB if the experimental details have not been reported, resulting in many unclear scores. With at least one unclear or high RoB score per included study, the overall conclusions of the review become inconclusive. For SRs of overall treatment effects that are performed to inform evidence-based treatment guidelines, RoB analyses remain crucial, even though the scores will often be unclear. Ideally, especially for SRs that will be used to plan future experiments/develop treatment guidelines, analyses should only include those studies consistently showing low risk of bias (i.e. low risk on all elements). However, in practice, consistently low RoB studies in our included literature samples (> 20 SRs to date) are too scarce for meaningful analyses. For other types of reviews, we think it is time to consider if complete RoB assessment is the most efficient use of limited resources. While these assessments regularly show problems in reporting, which may help to improve the quality of future primary studies, the unclear scores do not contribute much to understanding the effects observed in meta-analyses.

With PubMed already indexing nearly 300,000 mentioning the term “systematic review” in the title, abstract, or keywords, we can assume that many scientists are spending substantial amounts of time and resources on RoB and RQ assessments. Particularly for larger reviews, it could be worthwhile to restrict RoB assessment to either a random subset of the included publications or a subset of relatively informative elements. Even a combination of these two strategies may be sufficiently informative if the results of the review are not directly used to guide treatment decisions. The subset could give a reasonable indication of the overall level of evidence of the SR while saving resources. Different suggested procedures are provided in Table  5 . The authors of this work would probably have changed to such a strategy during their early data extraction phase, if the funder would not have stipulated full RoB assessment in their funding conditions.

We previously created a brief and simple taxonomy of systematised review types [ 44 ], in which we advocate RoB assessments to be a mandatory part of any SR. We would still urge anyone calling their review “systematic” to stick to this definition and perform some kind of RoB and/or RQ assessment, but two independent scientists following a lengthy and complex tool for all included publications, resulting in 74.6% of the assessed elements not being reported, or 77.9% unclear RoB, can, in our opinion, in most cases be considered inefficient and unnecessary.

Our results show that there is plenty of room for improvement in the reporting of experimental details in medical scientific literature, both for animal and for human studies. With the current status of the primary literature as it is, full RoB assessment may not be the most efficient use of limited resources, particularly for SRs that are not directly used as the basis for treatment guidelines or future experiments.

Availability of data and materials

The data described in this study are available from the Open Science Platform ( https://osf.io/fmhcq/ ) in the form of a spreadsheet file. In the data file, the first tab shows the list of questions that were used for data extraction with their respective short codes. The second tab shows the full individual study-level scores, with lines per study and columns per short code.

Abbreviations

• Cystic fibrosis

High risk of bias

Low risk of bias

No, not reported

• Nasal potential difference
• Risk of bias
• Reporting quality

Systematic review

Unclear risk of bias

Yes, reported

Drucker AM, Fleming P, Chan AW. Research techniques made simple: assessing risk of bias in systematic reviews. J Invest Dermatol. 2016;136(11):e109–14.

Article   PubMed   CAS   Google Scholar  

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.

Article   PubMed   PubMed Central   Google Scholar  

Page MJ, McKenzie JE, Higgins JPT. Tools for assessing risk of reporting biases in studies and syntheses of studies: a systematic review. BMJ Open. 2018;8(3):e019703.

Wang X, Chen Y, Yang N, Deng W, Wang Q, Li N, et al. Methodology and reporting quality of reporting guidelines: systematic review. BMC Med Res Methodol. 2015;15:74.

Zeng X, Zhang Y, Kwong JS, Zhang C, Li S, Sun F, et al. The methodological quality assessment tools for preclinical and clinical studies, systematic review and meta-analysis, and clinical practice guideline: a systematic review. J Evid Based Med. 2015;8(1):2–10.

Article   PubMed   Google Scholar  

Turner L, Shamseer L, Altman DG, Schulz KF, Moher D. Does use of the CONSORT statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review Syst Rev. 2012;1:60.

Savovic J, Weeks L, Sterne JA, Turner L, Altman DG, Moher D, et al. Evaluation of the Cochrane collaboration’s tool for assessing the risk of bias in randomized trials: focus groups, online survey, proposed recommendations and their implementation. Syst Rev. 2014;3:37.

Arno A, Thomas J, Wallace B, Marshall IJ, McKenzie JE, Elliott JH. Accuracy and efficiency of machine learning-assisted risk-of-bias assessments in “real-world” systematic reviews : a noninferiority randomized controlled trial. Ann Intern Med. 2022;175(7):1001–9.

Jardim PSJ, Rose CJ, Ames HM, Echavez JFM, Van de Velde S, Muller AE. Automating risk of bias assessment in systematic reviews: a real-time mixed methods comparison of human researchers to a machine learning system. BMC Med Res Methodol. 2022;22(1):167.

Leenaars C, Hager C, Stafleu F, Nieraad H, Bleich A. A systematic review of the effect of cystic fibrosis treatments on the nasal potential difference test in animals and humans. Diagnostics (Basel). 2023;13(19):3098.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Leenaars CHC, Stafleu FR, Hager C, Nieraad H, Bleich A. A systematic review of animal and human data comparing the nasal potential difference test between cystic fibrosis and control. Sci Rep. 2024;14(1):9664.

Higgins JPT, Savović J, Page MJ, Elbers RG, Sterne JAC. Chapter 8: Assessing risk of bias in a randomized trial. Cochrane Handbook for Systematic Reviews of Interventions. 2022.

Google Scholar  

Hooijmans CR, Rovers MM, de Vries RB, Leenaars M, Ritskes-Hoitinga M, Langendam MW. SYRCLE’s risk of bias tool for animal studies. BMC Med Res Methodol. 2014;14:43.

Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG. Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. PLoS Biol. 2010;8(6):e1000412.

Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement JAMA. 1996;276(8):637–9.

CAS   Google Scholar  

Leenaars C, Stafleu F, Bleich A. The nasal potential difference test for diagnosing cystic fibrosis and assessing disease severity: a systematic review. 2021.

Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210.

Covidence systematic review software Melbourne, Australia: Veritas Health Innovation. Available from: www.covidence.org .

Percie du Sert N, Hurst V, Ahluwalia A, Alam S, Avey MT, Baker M, et al. The ARRIVE guidelines 20: updated guidelines for reporting animal research. J Cereb Blood Flow Metab. 2020;40(9):1769–77.

Knowles MR, Gatzy JT, Boucher RC. Aldosterone metabolism and transepithelial potential difference in normal and cystic fibrosis subjects. Pediatr Res. 1985;19(7):676–9.

Team RC. a language and environment for statistical computing. R Foundation for Statistical Computing. 2021.

RStudio_Team. RStudio: integrated development for R. Boston, MA.: RStudio, Inc.; 2019. Available from: http://www.rstudio.com/ .

Wickham H, Bryan J. readxl: read Excel files. R package version 1.3.1. 2019.

Wickham H, François R, Henry L, Müller K. dplyr: a grammar of data manipulation. R package version 1.0.3. 2021.

Wickham H, Girlich M. tidyr: tidy messy data. R package version 1.2.0. 2022.

Wickham H. ggplot2: elegant graphics for data analysis. New York: Springer-Verlag; 2016.

Book   Google Scholar  

Chaltiel D. Crosstable: crosstables for descriptive analyses. R package version 0.5.0. 2022.

Macleod MR, van der Worp HB, Sena ES, Howells DW, Dirnagl U, Donnan GA. Evidence for the efficacy of NXY-059 in experimental focal cerebral ischaemia is confounded by study quality. Stroke. 2008;39(10):2824–9.

Leenaars C, Stafleu F, de Jong D, van Berlo M, Geurts T, Coenen-de Roo T, et al. A systematic review comparing experimental design of animal and human methotrexate efficacy studies for rheumatoid arthritis: lessons for the translational value of animal studies. Animals (Basel). 2020;10(6):1047.

Leenaars CHC, Kouwenaar C, Stafleu FR, Bleich A, Ritskes-Hoitinga M, De Vries RBM, et al. Animal to human translation: a systematic scoping review of reported concordance rates. J Transl Med. 2019;17(1):223.

Kerem E, Konstan MW, De Boeck K, Accurso FJ, Sermet-Gaudelus I, Wilschanski M, et al. Ataluren for the treatment of nonsense-mutation cystic fibrosis: a randomised, double-blind, placebo-controlled phase 3 trial. Lancet Respir Med. 2014;2(7):539–47.

Rowe SM, Liu B, Hill A, Hathorne H, Cohen M, Beamer JR, et al. Optimizing nasal potential difference analysis for CFTR modulator development: assessment of ivacaftor in CF subjects with the G551D-CFTR mutation. PLoS ONE. 2013;8(7): e66955.

Reese HW. Counterbalancing and other uses of repeated-measures Latin-square designs: analyses and interpretations. J Exp Child Psychol. 1997;64(1):137–58.

Article   Google Scholar  

Zeelenberg R, Pecher D. A method for simultaneously counterbalancing condition order and assignment of stimulus materials to conditions. Behav Res Methods. 2015;47(1):127–33.

Richardson JTE. The use of Latin-square designs in educational and psychological research. Educ Res Rev. 2018;24:84–97.

King G, Nielsen R, Coberley C, Pope JE, Wells A. Avoiding randomization failure in program evaluation, with application to the Medicare Health Support program. Popul Health Manag. 2011;14(Suppl 1):S11-22.

Meier B, Nietlispach F. Fallacies of evidence-based medicine in cardiovascular medicine. Am J Cardiol. 2019;123(4):690–4.

Van de Wall G, Van Hattem A, Timmermans J, Ritskes-Hoitinga M, Bleich A, Leenaars C. Comparing translational success rates across medical research fields - a combined analysis of literature and clinical trial data. Altex. 2023;40(4):584–94.

PubMed   Google Scholar  

Knowles MR, Gatzy JT, Boucher RC. Increased bioelectric potential differences across respiratory epithelia in cystic fibrosis. N Engl Med. 1981;305:1489–95.

Article   CAS   Google Scholar  

Unal-Maelger OH, Urbanek R. Status of determining the transepithelial potential difference (PD) of the respiratory epithelium in the diagnosis of mucoviscidosis. Monatsschr Kinderheilkd. 1988;136(2):76–80.

PubMed   CAS   Google Scholar  

Dorin JR, Dickinson P, Alton EW, Smith SN, Geddes DM, Stevenson BJ, et al. Cystic fibrosis in the mouse by targeted insertional mutagenesis. Nature. 1992;359(6392):211–5.

Alton EW, Middleton PG, Caplen NJ, Smith SN, Steel DM, Munkonge FM, et al. Non-invasive liposome-mediated gene delivery can correct the ion transport defect in cystic fibrosis mutant mice. Nat Genet. 1993;5(2):135–42.

Caplen NJ, Alton EW, Middleton PG, Dorin JR, Stevenson BJ, Gao X, et al. Liposome-mediated CFTR gene transfer to the nasal epithelium of patients with cystic fibrosis. Nat Med. 1995;1(1):39–46.

Leenaars C, Tsaioun K, Stafleu F, Rooney K, Meijboom F, Ritskes-Hoitinga M, et al. Reviewing the animal literature: how to describe and choose between different types of literature reviews. Lab Anim. 2021;55(2):129–41.

Download references

Acknowledgements

The authors kindly acknowledge Dr. Hendrik Nieraad for his help in study classification.

Open Access funding enabled and organized by Projekt DEAL. This research was funded by the BMBF, grant number 01KC1904. During grant review, the BMBF asked for changes in the review design which we incorporated. Publication of the review results was a condition of the call. Otherwise, the BMBF had no role in the collection, analysis and interpretation of data, or in writing the manuscript.

Author information

Authors and affiliations.

Institute for Laboratory Animal Science, Hannover Medical School, Carl Neubergstrasse 1, 30625, Hannover, Germany

Cathalijn H. C. Leenaars, Christine Häger & André Bleich

Department of Animals in Science and Society, Utrecht University, Yalelaan 2, Utrecht, 3584 CM, the Netherlands

Frans R. Stafleu

You can also search for this author in PubMed   Google Scholar

Contributions

CL and AB acquired the grant to perform this work and designed the study. CL performed the searches. FS and CL extracted the data. CL performed the analyses. CH performed quality controls for the data and analyses. CL drafted the manuscript. All authors revised the manuscript and approved of the final version.

Corresponding author

Correspondence to Cathalijn H. C. Leenaars .

Ethics declarations

Declarations.

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Leenaars, C.H.C., Stafleu, F.R., Häger, C. et al. A case study of the informative value of risk of bias and reporting quality assessments for systematic reviews. Syst Rev 13 , 230 (2024). https://doi.org/10.1186/s13643-024-02650-w

Download citation

Received : 10 April 2024

Accepted : 28 August 2024

Published : 07 September 2024

DOI : https://doi.org/10.1186/s13643-024-02650-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Systematic reviews
• Informative value

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]