in an experimental design the variable the researcher has control

Controlled Experiment

Saul McLeod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

This is when a hypothesis is scientifically tested.

In a controlled experiment, an independent variable (the cause) is systematically manipulated, and the dependent variable (the effect) is measured; any extraneous variables are controlled.

The researcher can operationalize (i.e., define) the studied variables so they can be objectively measured. The quantitative data can be analyzed to see if there is a difference between the experimental and control groups.

What is the control group?

In experiments scientists compare a control group and an experimental group that are identical in all respects, except for one difference – experimental manipulation.

Unlike the experimental group, the control group is not exposed to the independent variable under investigation and so provides a baseline against which any changes in the experimental group can be compared.

Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between the two are due to experimental manipulation rather than chance.

Randomly allocating participants to independent variable groups means that all participants should have an equal chance of participating in each condition.

The principle of random allocation is to avoid bias in how the experiment is carried out and limit the effects of participant variables.

What are extraneous variables?

The researcher wants to ensure that the manipulation of the independent variable has changed the changes in the dependent variable.

Hence, all the other variables that could affect the dependent variable to change must be controlled. These other variables are called extraneous or confounding variables.

Extraneous variables should be controlled were possible, as they might be important enough to provide alternative explanations for the effects.

In practice, it would be difficult to control all the variables in a child’s educational achievement. For example, it would be difficult to control variables that have happened in the past.

A researcher can only control the current environment of participants, such as time of day and noise levels.

Why conduct controlled experiments?

Scientists use controlled experiments because they allow for precise control of extraneous and independent variables. This allows a cause-and-effect relationship to be established.

Controlled experiments also follow a standardized step-by-step procedure. This makes it easy for another researcher to replicate the study.

Key Terminology

Experimental group.

The group being treated or otherwise manipulated for the sake of the experiment.

Control Group

They receive no treatment and are used as a comparison group.

Ecological validity

The degree to which an investigation represents real-life experiences.

Experimenter effects

These are the ways that the experimenter can accidentally influence the participant through their appearance or behavior.

Demand characteristics

The clues in an experiment lead the participants to think they know what the researcher is looking for (e.g., the experimenter’s body language).

Independent variable (IV)

The variable the experimenter manipulates (i.e., changes) – is assumed to have a direct effect on the dependent variable.

Dependent variable (DV)

Variable the experimenter measures. This is the outcome (i.e., the result) of a study.

Extraneous variables (EV)

All variables that are not independent variables but could affect the results (DV) of the experiment. Extraneous variables should be controlled where possible.

Confounding variables

Variable(s) that have affected the results (DV), apart from the IV. A confounding variable could be an extraneous variable that has not been controlled.

Random Allocation

Randomly allocating participants to independent variable conditions means that all participants should have an equal chance of participating in each condition.

Order effects

Changes in participants’ performance due to their repeating the same or similar test more than once. Examples of order effects include:

(i) practice effect: an improvement in performance on a task due to repetition, for example, because of familiarity with the task;

(ii) fatigue effect: a decrease in performance of a task due to repetition, for example, because of boredom or tiredness.

What is the control in an experiment?

In an experiment , the control is a standard or baseline group not exposed to the experimental treatment or manipulation. It serves as a comparison group to the experimental group, which does receive the treatment or manipulation.

The control group helps to account for other variables that might influence the outcome, allowing researchers to attribute differences in results more confidently to the experimental treatment.

Establishing a cause-and-effect relationship between the manipulated variable (independent variable) and the outcome (dependent variable) is critical in establishing a cause-and-effect relationship between the manipulated variable.

What is the purpose of controlling the environment when testing a hypothesis?

Controlling the environment when testing a hypothesis aims to eliminate or minimize the influence of extraneous variables. These variables other than the independent variable might affect the dependent variable, potentially confounding the results.

By controlling the environment, researchers can ensure that any observed changes in the dependent variable are likely due to the manipulation of the independent variable, not other factors.

This enhances the experiment’s validity, allowing for more accurate conclusions about cause-and-effect relationships.

It also improves the experiment’s replicability, meaning other researchers can repeat the experiment under the same conditions to verify the results.

Why are hypotheses important to controlled experiments?

Hypotheses are crucial to controlled experiments because they provide a clear focus and direction for the research. A hypothesis is a testable prediction about the relationship between variables.

It guides the design of the experiment, including what variables to manipulate (independent variables) and what outcomes to measure (dependent variables).

The experiment is then conducted to test the validity of the hypothesis. If the results align with the hypothesis, they provide evidence supporting it.

The hypothesis may be revised or rejected if the results do not align. Thus, hypotheses are central to the scientific method, driving the iterative inquiry, experimentation, and knowledge advancement process.

What is the experimental method?

The experimental method is a systematic approach in scientific research where an independent variable is manipulated to observe its effect on a dependent variable, under controlled conditions.

• Skip to secondary menu
• Skip to main content
• Skip to primary sidebar

Statistics By Jim

Making statistics intuitive

Experimental Design: Definition and Types

By Jim Frost 3 Comments

What is Experimental Design?

An experimental design is a detailed plan for collecting and using data to identify causal relationships. Through careful planning, the design of experiments allows your data collection efforts to have a reasonable chance of detecting effects and testing hypotheses that answer your research questions.

An experiment is a data collection procedure that occurs in controlled conditions to identify and understand causal relationships between variables. Researchers can use many potential designs. The ultimate choice depends on their research question, resources, goals, and constraints. In some fields of study, researchers refer to experimental design as the design of experiments (DOE). Both terms are synonymous.

Ultimately, the design of experiments helps ensure that your procedures and data will evaluate your research question effectively. Without an experimental design, you might waste your efforts in a process that, for many potential reasons, can’t answer your research question. In short, it helps you trust your results.

Learn more about Independent and Dependent Variables .

Design of Experiments: Goals & Settings

Experiments occur in many settings, ranging from psychology, social sciences, medicine, physics, engineering, and industrial and service sectors. Typically, experimental goals are to discover a previously unknown effect , confirm a known effect, or test a hypothesis.

Effects represent causal relationships between variables. For example, in a medical experiment, does the new medicine cause an improvement in health outcomes? If so, the medicine has a causal effect on the outcome.

An experimental design’s focus depends on the subject area and can include the following goals:

• Understanding the relationships between variables.
• Identifying the variables that have the largest impact on the outcomes.
• Finding the input variable settings that produce an optimal result.

For example, psychologists have conducted experiments to understand how conformity affects decision-making. Sociologists have performed experiments to determine whether ethnicity affects the public reaction to staged bike thefts. These experiments map out the causal relationships between variables, and their primary goal is to understand the role of various factors.

Conversely, in a manufacturing environment, the researchers might use an experimental design to find the factors that most effectively improve their product’s strength, identify the optimal manufacturing settings, and do all that while accounting for various constraints. In short, a manufacturer’s goal is often to use experiments to improve their products cost-effectively.

In a medical experiment, the goal might be to quantify the medicine’s effect and find the optimum dosage.

Developing an Experimental Design

Developing an experimental design involves planning that maximizes the potential to collect data that is both trustworthy and able to detect causal relationships. Specifically, these studies aim to see effects when they exist in the population the researchers are studying, preferentially favor causal effects, isolate each factor’s true effect from potential confounders, and produce conclusions that you can generalize to the real world.

To accomplish these goals, experimental designs carefully manage data validity and reliability , and internal and external experimental validity. When your experiment is valid and reliable, you can expect your procedures and data to produce trustworthy results.

An excellent experimental design involves the following:

• Lots of preplanning.
• Developing experimental treatments.
• Determining how to assign subjects to treatment groups.

The remainder of this article focuses on how experimental designs incorporate these essential items to accomplish their research goals.

Learn more about Data Reliability vs. Validity and Internal and External Experimental Validity .

Preplanning, Defining, and Operationalizing for Design of Experiments

This phase of the design of experiments helps you identify critical variables, know how to measure them while ensuring reliability and validity, and understand the relationships between them. The review can also help you find ways to reduce sources of variability, which increases your ability to detect treatment effects. Notably, the literature review allows you to learn how similar studies designed their experiments and the challenges they faced.

Operationalizing a study involves taking your research question, using the background information you gathered, and formulating an actionable plan.

This process should produce a specific and testable hypothesis using data that you can reasonably collect given the resources available to the experiment.

• Null hypothesis : The jumping exercise intervention does not affect bone density.
• Alternative hypothesis : The jumping exercise intervention affects bone density.

To learn more about this early phase, read Five Steps for Conducting Scientific Studies with Statistical Analyses .

Formulating Treatments in Experimental Designs

In an experimental design, treatments are variables that the researchers control. They are the primary independent variables of interest. Researchers administer the treatment to the subjects or items in the experiment and want to know whether it causes changes in the outcome.

As the name implies, a treatment can be medical in nature, such as a new medicine or vaccine. But it’s a general term that applies to other things such as training programs, manufacturing settings, teaching methods, and types of fertilizers. I helped run an experiment where the treatment was a jumping exercise intervention that we hoped would increase bone density. All these treatment examples are things that potentially influence a measurable outcome.

Even when you know your treatment generally, you must carefully consider the amount. How large of a dose? If you’re comparing three different temperatures in a manufacturing process, how far apart are they? For my bone mineral density study, we had to determine how frequently the exercise sessions would occur and how long each lasted.

How you define the treatments in the design of experiments can affect your findings and the generalizability of your results.

Assigning Subjects to Experimental Groups

A crucial decision for all experimental designs is determining how researchers assign subjects to the experimental conditions—the treatment and control groups. The control group is often, but not always, the lack of a treatment. It serves as a basis for comparison by showing outcomes for subjects who don’t receive a treatment. Learn more about Control Groups .

How your experimental design assigns subjects to the groups affects how confident you can be that the findings represent true causal effects rather than mere correlation caused by confounders. Indeed, the assignment method influences how you control for confounding variables. This is the difference between correlation and causation .

Imagine a study finds that vitamin consumption correlates with better health outcomes. As a researcher, you want to be able to say that vitamin consumption causes the improvements. However, with the wrong experimental design, you might only be able to say there is an association. A confounder, and not the vitamins, might actually cause the health benefits.

Let’s explore some of the ways to assign subjects in design of experiments.

Completely Randomized Designs

A completely randomized experimental design randomly assigns all subjects to the treatment and control groups. You simply take each participant and use a random process to determine their group assignment. You can flip coins, roll a die, or use a computer. Randomized experiments must be prospective studies because they need to be able to control group assignment.

Random assignment in the design of experiments helps ensure that the groups are roughly equivalent at the beginning of the study. This equivalence at the start increases your confidence that any differences you see at the end were caused by the treatments. The randomization tends to equalize confounders between the experimental groups and, thereby, cancels out their effects, leaving only the treatment effects.

For example, in a vitamin study, the researchers can randomly assign participants to either the control or vitamin group. Because the groups are approximately equal when the experiment starts, if the health outcomes are different at the end of the study, the researchers can be confident that the vitamins caused those improvements.

Statisticians consider randomized experimental designs to be the best for identifying causal relationships.

If you can’t randomly assign subjects but want to draw causal conclusions about an intervention, consider using a quasi-experimental design .

Learn more about Randomized Controlled Trials and Random Assignment in Experiments .

Randomized Block Designs

Nuisance factors are variables that can affect the outcome, but they are not the researcher’s primary interest. Unfortunately, they can hide or distort the treatment results. When experimenters know about specific nuisance factors, they can use a randomized block design to minimize their impact.

This experimental design takes subjects with a shared “nuisance” characteristic and groups them into blocks. The participants in each block are then randomly assigned to the experimental groups. This process allows the experiment to control for known nuisance factors.

Blocking in the design of experiments reduces the impact of nuisance factors on experimental error. The analysis assesses the effects of the treatment within each block, which removes the variability between blocks. The result is that blocked experimental designs can reduce the impact of nuisance variables, increasing the ability to detect treatment effects accurately.

Suppose you’re testing various teaching methods. Because grade level likely affects educational outcomes, you might use grade level as a blocking factor. To use a randomized block design for this scenario, divide the participants by grade level and then randomly assign the members of each grade level to the experimental groups.

A standard guideline for an experimental design is to “Block what you can, randomize what you cannot.” Use blocking for a few primary nuisance factors. Then use random assignment to distribute the unblocked nuisance factors equally between the experimental conditions.

You can also use covariates to control nuisance factors. Learn about Covariates: Definition and Uses .

Observational Studies

In some experimental designs, randomly assigning subjects to the experimental conditions is impossible or unethical. The researchers simply can’t assign participants to the experimental groups. However, they can observe them in their natural groupings, measure the essential variables, and look for correlations. These observational studies are also known as quasi-experimental designs. Retrospective studies must be observational in nature because they look back at past events.

Imagine you’re studying the effects of depression on an activity. Clearly, you can’t randomly assign participants to the depression and control groups. But you can observe participants with and without depression and see how their task performance differs.

Observational studies let you perform research when you can’t control the treatment. However, quasi-experimental designs increase the problem of confounding variables. For this design of experiments, correlation does not necessarily imply causation. While special procedures can help control confounders in an observational study, you’re ultimately less confident that the results represent causal findings.

Learn more about Observational Studies .

For a good comparison, learn about the differences and tradeoffs between Observational Studies and Randomized Experiments .

Between-Subjects vs. Within-Subjects Experimental Designs

When you think of the design of experiments, you probably picture a treatment and control group. Researchers assign participants to only one of these groups, so each group contains entirely different subjects than the other groups. Analysts compare the groups at the end of the experiment. Statisticians refer to this method as a between-subjects, or independent measures, experimental design.

In a between-subjects design , you can have more than one treatment group, but each subject is exposed to only one condition, the control group or one of the treatment groups.

A potential downside to this approach is that differences between groups at the beginning can affect the results at the end. As you’ve read earlier, random assignment can reduce those differences, but it is imperfect. There will always be some variability between the groups.

In a  within-subjects experimental design , also known as repeated measures, subjects experience all treatment conditions and are measured for each. Each subject acts as their own control, which reduces variability and increases the statistical power to detect effects.

In this experimental design, you minimize pre-existing differences between the experimental conditions because they all contain the same subjects. However, the order of treatments can affect the results. Beware of practice and fatigue effects. Learn more about Repeated Measures Designs .

Design of Experiments Examples

For example, a bone density study has three experimental groups—a control group, a stretching exercise group, and a jumping exercise group.

In a between-subjects experimental design, scientists randomly assign each participant to one of the three groups.

In a within-subjects design, all subjects experience the three conditions sequentially while the researchers measure bone density repeatedly. The procedure can switch the order of treatments for the participants to help reduce order effects.

Matched Pairs Experimental Design

A matched pairs experimental design is a between-subjects study that uses pairs of similar subjects. Researchers use this approach to reduce pre-existing differences between experimental groups. It’s yet another design of experiments method for reducing sources of variability.

Researchers identify variables likely to affect the outcome, such as demographics. When they pick a subject with a set of characteristics, they try to locate another participant with similar attributes to create a matched pair. Scientists randomly assign one member of a pair to the treatment group and the other to the control group.

On the plus side, this process creates two similar groups, and it doesn’t create treatment order effects. While matched pairs do not produce the perfectly matched groups of a within-subjects design (which uses the same subjects in all conditions), it aims to reduce variability between groups relative to a between-subjects study.

On the downside, finding matched pairs is very time-consuming. Additionally, if one member of a matched pair drops out, the other subject must leave the study too.

Learn more about Matched Pairs Design: Uses & Examples .

Another consideration is whether you’ll use a cross-sectional design (one point in time) or use a longitudinal study to track changes over time .

A case study is a research method that often serves as a precursor to a more rigorous experimental design by identifying research questions, variables, and hypotheses to test. Learn more about What is a Case Study? Definition & Examples .

In conclusion, the design of experiments is extremely sensitive to subject area concerns and the time and resources available to the researchers. Developing a suitable experimental design requires balancing a multitude of considerations. A successful design is necessary to obtain trustworthy answers to your research question and to have a reasonable chance of detecting treatment effects when they exist.

Share this:

Reader Interactions

March 23, 2024 at 2:35 pm

Dear Jim You wrote a superb document, I will use it in my Buistatistics course, along with your three books. Thank you very much! Miguel

March 23, 2024 at 5:43 pm

Thanks so much, Miguel! Glad this post was helpful and I trust the books will be as well.

April 10, 2023 at 4:36 am

What are the purpose and uses of experimental research design?

Comments and Questions Cancel reply

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Controlled Experiments | Methods & Examples of Control

Controlled Experiments | Methods & Examples of Control

Published on 19 April 2022 by Pritha Bhandari . Revised on 10 October 2022.

In experiments , researchers manipulate independent variables to test their effects on dependent variables. In a controlled experiment , all variables other than the independent variable are controlled or held constant so they don’t influence the dependent variable.

Controlling variables can involve:

• Holding variables at a constant or restricted level (e.g., keeping room temperature fixed)
• Measuring variables to statistically control for them in your analyses
• Balancing variables across your experiment through randomisation (e.g., using a random order of tasks)

Table of contents

Why does control matter in experiments, methods of control, problems with controlled experiments, frequently asked questions about controlled experiments.

Control in experiments is critical for internal validity , which allows you to establish a cause-and-effect relationship between variables.

• Your independent variable is the colour used in advertising.
• Your dependent variable is the price that participants are willing to pay for a standard fast food meal.

Extraneous variables are factors that you’re not interested in studying, but that can still influence the dependent variable. For strong internal validity, you need to remove their effects from your experiment.

• Design and description of the meal
• Study environment (e.g., temperature or lighting)
• Participant’s frequency of buying fast food
• Participant’s familiarity with the specific fast food brand
• Participant’s socioeconomic status

Prevent plagiarism, run a free check.

You can control some variables by standardising your data collection procedures. All participants should be tested in the same environment with identical materials. Only the independent variable (e.g., advert colour) should be systematically changed between groups.

Other extraneous variables can be controlled through your sampling procedures . Ideally, you’ll select a sample that’s representative of your target population by using relevant inclusion and exclusion criteria (e.g., including participants from a specific income bracket, and not including participants with colour blindness).

By measuring extraneous participant variables (e.g., age or gender) that may affect your experimental results, you can also include them in later analyses.

After gathering your participants, you’ll need to place them into groups to test different independent variable treatments. The types of groups and method of assigning participants to groups will help you implement control in your experiment.

Control groups

Controlled experiments require control groups . Control groups allow you to test a comparable treatment, no treatment, or a fake treatment, and compare the outcome with your experimental treatment.

You can assess whether it’s your treatment specifically that caused the outcomes, or whether time or any other treatment might have resulted in the same effects.

• A control group that’s presented with red advertisements for a fast food meal
• An experimental group that’s presented with green advertisements for the same fast food meal

Random assignment

To avoid systematic differences between the participants in your control and treatment groups, you should use random assignment .

This helps ensure that any extraneous participant variables are evenly distributed, allowing for a valid comparison between groups .

Random assignment is a hallmark of a ‘true experiment’ – it differentiates true experiments from quasi-experiments .

Masking (blinding)

Masking in experiments means hiding condition assignment from participants or researchers – or, in a double-blind study , from both. It’s often used in clinical studies that test new treatments or drugs.

Sometimes, researchers may unintentionally encourage participants to behave in ways that support their hypotheses. In other cases, cues in the study environment may signal the goal of the experiment to participants and influence their responses.

Using masking means that participants don’t know whether they’re in the control group or the experimental group. This helps you control biases from participants or researchers that could influence your study results.

Although controlled experiments are the strongest way to test causal relationships, they also involve some challenges.

Difficult to control all variables

Especially in research with human participants, it’s impossible to hold all extraneous variables constant, because every individual has different experiences that may influence their perception, attitudes, or behaviors.

But measuring or restricting extraneous variables allows you to limit their influence or statistically control for them in your study.

Risk of low external validity

Controlled experiments have disadvantages when it comes to external validity – the extent to which your results can be generalised to broad populations and settings.

The more controlled your experiment is, the less it resembles real world contexts. That makes it harder to apply your findings outside of a controlled setting.

There’s always a tradeoff between internal and external validity . It’s important to consider your research aims when deciding whether to prioritise control or generalisability in your experiment.

Experimental designs are a set of procedures that you plan in order to examine the relationship between variables that interest you.

To design a successful experiment, first identify:

• A testable hypothesis
• One or more independent variables that you will manipulate
• One or more dependent variables that you will measure

When designing the experiment, first decide:

• How your variable(s) will be manipulated
• How you will control for any potential confounding or lurking variables
• How many subjects you will include
• How you will assign treatments to your subjects

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Bhandari, P. (2022, October 10). Controlled Experiments | Methods & Examples of Control. Scribbr. Retrieved 16 September 2024, from https://www.scribbr.co.uk/research-methods/controlled-experiments/

Is this article helpful?

Pritha Bhandari

Instant insights, infinite possibilities

Experimental design: Guide, steps, examples

Last updated

27 April 2023

Reviewed by

Miroslav Damyanov

Short on time? Get an AI generated summary of this article instead

Experimental research design is a scientific framework that allows you to manipulate one or more variables while controlling the test environment. 

When testing a theory or new product, it can be helpful to have a certain level of control and manipulate variables to discover different outcomes. You can use these experiments to determine cause and effect or study variable associations. 

This guide explores the types of experimental design, the steps in designing an experiment, and the advantages and limitations of experimental design. 

Make research less tedious

Dovetail streamlines research to help you uncover and share actionable insights

• What is experimental research design?

You can determine the relationship between each of the variables by: 

Manipulating one or more independent variables (i.e., stimuli or treatments)

Applying the changes to one or more dependent variables (i.e., test groups or outcomes)

With the ability to analyze the relationship between variables and using measurable data, you can increase the accuracy of the result. 

What is a good experimental design?

A good experimental design requires: 

Significant planning to ensure control over the testing environment

Sound experimental treatments

Properly assigning subjects to treatment groups

Without proper planning, unexpected external variables can alter an experiment's outcome. 

To meet your research goals, your experimental design should include these characteristics:

Provide unbiased estimates of inputs and associated uncertainties

Enable the researcher to detect differences caused by independent variables

Include a plan for analysis and reporting of the results

Provide easily interpretable results with specific conclusions

What's the difference between experimental and quasi-experimental design?

The major difference between experimental and quasi-experimental design is the random assignment of subjects to groups. 

A true experiment relies on certain controls. Typically, the researcher designs the treatment and randomly assigns subjects to control and treatment groups. 

However, these conditions are unethical or impossible to achieve in some situations.

When it's unethical or impractical to assign participants randomly, that’s when a quasi-experimental design comes in. 

This design allows researchers to conduct a similar experiment by assigning subjects to groups based on non-random criteria. 

Another type of quasi-experimental design might occur when the researcher doesn't have control over the treatment but studies pre-existing groups after they receive different treatments.

When can a researcher conduct experimental research?

Various settings and professions can use experimental research to gather information and observe behavior in controlled settings. 

Basically, a researcher can conduct experimental research any time they want to test a theory with variable and dependent controls. 

Experimental research is an option when the project includes an independent variable and a desire to understand the relationship between cause and effect. 

• The importance of experimental research design

Experimental research enables researchers to conduct studies that provide specific, definitive answers to questions and hypotheses. 

Researchers can test Independent variables in controlled settings to:

Test the effectiveness of a new medication

Design better products for consumers

Answer questions about human health and behavior

Developing a quality research plan means a researcher can accurately answer vital research questions with minimal error. As a result, definitive conclusions can influence the future of the independent variable. 

Types of experimental research designs

There are three main types of experimental research design. The research type you use will depend on the criteria of your experiment, your research budget, and environmental limitations. 

Pre-experimental research design

A pre-experimental research study is a basic observational study that monitors independent variables’ effects. 

During research, you observe one or more groups after applying a treatment to test whether the treatment causes any change. 

The three subtypes of pre-experimental research design are:

One-shot case study research design

This research method introduces a single test group to a single stimulus to study the results at the end of the application. 

After researchers presume the stimulus or treatment has caused changes, they gather results to determine how it affects the test subjects. 

One-group pretest-posttest design

This method uses a single test group but includes a pretest study as a benchmark. The researcher applies a test before and after the group’s exposure to a specific stimulus. 

Static group comparison design

This method includes two or more groups, enabling the researcher to use one group as a control. They apply a stimulus to one group and leave the other group static. 

A posttest study compares the results among groups. 

True experimental research design

A true experiment is the most common research method. It involves statistical analysis to prove or disprove a specific hypothesis . 

Under completely experimental conditions, researchers expose participants in two or more randomized groups to different stimuli. 

Random selection removes any potential for bias, providing more reliable results. 

These are the three main sub-groups of true experimental research design:

Posttest-only control group design

This structure requires the researcher to divide participants into two random groups. One group receives no stimuli and acts as a control while the other group experiences stimuli.

Researchers perform a test at the end of the experiment to observe the stimuli exposure results.

Pretest-posttest control group design

This test also requires two groups. It includes a pretest as a benchmark before introducing the stimulus. 

The pretest introduces multiple ways to test subjects. For instance, if the control group also experiences a change, it reveals that taking the test twice changes the results.

Solomon four-group design

This structure divides subjects into two groups, with two as control groups. Researchers assign the first control group a posttest only and the second control group a pretest and a posttest. 

The two variable groups mirror the control groups, but researchers expose them to stimuli. The ability to differentiate between groups in multiple ways provides researchers with more testing approaches for data-based conclusions. 

Quasi-experimental research design

Although closely related to a true experiment, quasi-experimental research design differs in approach and scope. 

Quasi-experimental research design doesn’t have randomly selected participants. Researchers typically divide the groups in this research by pre-existing differences. 

Quasi-experimental research is more common in educational studies, nursing, or other research projects where it's not ethical or practical to use randomized subject groups.

• 5 steps for designing an experiment

Experimental research requires a clearly defined plan to outline the research parameters and expected goals. 

Here are five key steps in designing a successful experiment:

Step 1: Define variables and their relationship

Your experiment should begin with a question: What are you hoping to learn through your experiment? 

The relationship between variables in your study will determine your answer.

Define the independent variable (the intended stimuli) and the dependent variable (the expected effect of the stimuli). After identifying these groups, consider how you might control them in your experiment. 

Could natural variations affect your research? If so, your experiment should include a pretest and posttest. 

Step 2: Develop a specific, testable hypothesis

With a firm understanding of the system you intend to study, you can write a specific, testable hypothesis. 

What is the expected outcome of your study? 

Develop a prediction about how the independent variable will affect the dependent variable. 

How will the stimuli in your experiment affect your test subjects? 

Your hypothesis should provide a prediction of the answer to your research question . 

Step 3: Design experimental treatments to manipulate your independent variable

Depending on your experiment, your variable may be a fixed stimulus (like a medical treatment) or a variable stimulus (like a period during which an activity occurs). 

Determine which type of stimulus meets your experiment’s needs and how widely or finely to vary your stimuli. 

Step 4: Assign subjects to groups

When you have a clear idea of how to carry out your experiment, you can determine how to assemble test groups for an accurate study. 

When choosing your study groups, consider: 

The size of your experiment

Whether you can select groups randomly

Your target audience for the outcome of the study

You should be able to create groups with an equal number of subjects and include subjects that match your target audience. Remember, you should assign one group as a control and use one or more groups to study the effects of variables. 

Step 5: Plan how to measure your dependent variable

This step determines how you'll collect data to determine the study's outcome. You should seek reliable and valid measurements that minimize research bias or error. 

You can measure some data with scientific tools, while you’ll need to operationalize other forms to turn them into measurable observations.

• Advantages of experimental research

Experimental research is an integral part of our world. It allows researchers to conduct experiments that answer specific questions. 

While researchers use many methods to conduct different experiments, experimental research offers these distinct benefits:

Researchers can determine cause and effect by manipulating variables.

It gives researchers a high level of control.

Researchers can test multiple variables within a single experiment.

All industries and fields of knowledge can use it. 

Researchers can duplicate results to promote the validity of the study .

Replicating natural settings rapidly means immediate research.

Researchers can combine it with other research methods.

It provides specific conclusions about the validity of a product, theory, or idea.

• Disadvantages (or limitations) of experimental research

Unfortunately, no research type yields ideal conditions or perfect results. 

While experimental research might be the right choice for some studies, certain conditions could render experiments useless or even dangerous. 

Before conducting experimental research, consider these disadvantages and limitations:

Required professional qualification

Only competent professionals with an academic degree and specific training are qualified to conduct rigorous experimental research. This ensures results are unbiased and valid. 

Limited scope

Experimental research may not capture the complexity of some phenomena, such as social interactions or cultural norms. These are difficult to control in a laboratory setting.

Resource-intensive

Experimental research can be expensive, time-consuming, and require significant resources, such as specialized equipment or trained personnel.

Limited generalizability

The controlled nature means the research findings may not fully apply to real-world situations or people outside the experimental setting.

Practical or ethical concerns

Some experiments may involve manipulating variables that could harm participants or violate ethical guidelines . 

Researchers must ensure their experiments do not cause harm or discomfort to participants. 

Sometimes, recruiting a sample of people to randomly assign may be difficult. 

• Experimental research design example

Experiments across all industries and research realms provide scientists, developers, and other researchers with definitive answers. These experiments can solve problems, create inventions, and heal illnesses. 

Product design testing is an excellent example of experimental research. 

A company in the product development phase creates multiple prototypes for testing. With a randomized selection, researchers introduce each test group to a different prototype. 

When groups experience different product designs , the company can assess which option most appeals to potential customers. 

Experimental research design provides researchers with a controlled environment to conduct experiments that evaluate cause and effect. 

Using the five steps to develop a research plan ensures you anticipate and eliminate external variables while answering life’s crucial questions.

Should you be using a customer insights hub?

Do you want to discover previous research faster?

Do you share your research findings with others?

Do you analyze research data?

Start for free today, add your research, and get to key insights faster

Editor’s picks

Last updated: 18 April 2023

Last updated: 27 February 2023

Last updated: 22 August 2024

Last updated: 5 February 2023

Last updated: 16 August 2024

Last updated: 9 March 2023

Last updated: 30 April 2024

Last updated: 12 December 2023

Last updated: 11 March 2024

Last updated: 4 July 2024

Last updated: 6 March 2024

Last updated: 5 March 2024

Last updated: 13 May 2024

Latest articles

Related topics, .css-je19u9{-webkit-align-items:flex-end;-webkit-box-align:flex-end;-ms-flex-align:flex-end;align-items:flex-end;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-box-flex-wrap:wrap;-webkit-flex-wrap:wrap;-ms-flex-wrap:wrap;flex-wrap:wrap;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;row-gap:0;text-align:center;max-width:671px;}@media (max-width: 1079px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}}@media (max-width: 799px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}} decide what to .css-1kiodld{max-height:56px;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}@media (max-width: 1079px){.css-1kiodld{display:none;}} build next, decide what to build next.

• Types of experimental

Log in or sign up

Get started for free

• Activities, Experiments, Online Games, Visual Aids
• Activities, Experiments, and Investigations
• Experimental Design and the Scientific Method

Experimental Design - Independent, Dependent, and Controlled Variables

To view these resources with no ads, please login or subscribe to help support our content development. school subscriptions can access more than 175 downloadable unit bundles in our store for free (a value of $1,500). district subscriptions provide huge group discounts for their schools. email for a quote: [email protected] ..

Scientific experiments are meant to show cause and effect of a phenomena (relationships in nature).  The “ variables ” are any factor, trait, or condition that can be changed in the experiment and that can have an effect on the outcome of the experiment.

An experiment can have three kinds of variables: i ndependent, dependent, and controlled .

• The independent variable is one single factor that is changed by the scientist followed by observation to watch for changes. It is important that there is just one independent variable, so that results are not confusing.
• The dependent variable is the factor that changes as a result of the change to the independent variable.
• The controlled variables (or constant variables) are factors that the scientist wants to remain constant if the experiment is to show accurate results. To be able to measure results, each of the variables must be able to be measured.

For example, let’s design an experiment with two plants sitting in the sun side by side. The controlled variables (or constants) are that at the beginning of the experiment, the plants are the same size, get the same amount of sunlight, experience the same ambient temperature and are in the same amount and consistency of soil (the weight of the soil and container should be measured before the plants are added). The independent variable is that one plant is getting watered (1 cup of water) every day and one plant is getting watered (1 cup of water) once a week. The dependent variables are the changes in the two plants that the scientist observes over time.

Can you describe the dependent variable that may result from this experiment? After four weeks, the dependent variable may be that one plant is taller, heavier and more developed than the other. These results can be recorded and graphed by measuring and comparing both plants’ height, weight (removing the weight of the soil and container recorded beforehand) and a comparison of observable foliage.

Using What You Learned: Design another experiment using the two plants, but change the independent variable. Can you describe the dependent variable that may result from this new experiment?

Think of another simple experiment and name the independent, dependent, and controlled variables. Use the graphic organizer included in the PDF below to organize your experiment's variables.

Please Login or Subscribe to access downloadable content.

Citing Research References

When you research information you must cite the reference. Citing for websites is different from citing from books, magazines and periodicals. The style of citing shown here is from the MLA Style Citations (Modern Language Association).

When citing a WEBSITE the general format is as follows. Author Last Name, First Name(s). "Title: Subtitle of Part of Web Page, if appropriate." Title: Subtitle: Section of Page if appropriate. Sponsoring/Publishing Agency, If Given. Additional significant descriptive information. Date of Electronic Publication or other Date, such as Last Updated. Day Month Year of access < URL >.

Here is an example of citing this page:

Amsel, Sheri. "Experimental Design - Independent, Dependent, and Controlled Variables" Exploring Nature Educational Resource ©2005-2024. March 25, 2024 < http://www.exploringnature.org/db/view/Experimental-Design-Independent-Dependent-and-Controlled-Variables >

Exploringnature.org has more than 2,000 illustrated animals. Read about them, color them, label them, learn to draw them.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

14.1 What is experimental design and when should you use it?

Learning objectives.

Learners will be able to…

• Describe the purpose of experimental design research
• Describe nomethetic causality and the logic of experimental design
• Identify the characteristics of a basic experiment
• Discuss the relationship between dependent and independent variables in experiments
• Identify the three major types of experimental designs

Pre-awareness check (Knowledge)

What are your thoughts on the phrase ‘experiment’ in the realm of social sciences? In an experiment, what is the independent variable?

The basics of experiments

In social work research, experimental design is used to test the effects of treatments, interventions, programs, or other conditions to which individuals, groups, organizations, or communities may be exposed to. There are a lot of experiments social work researchers can use to explore topics such as treatments for depression, impacts of school-based mental health on student outcomes, or prevention of abuse of people with disabilities. The American Psychological Association defines an experiment   as:

a series of observations conducted under controlled conditions to study a relationship with the purpose of drawing causal inferences about that relationship. An experiment involves the manipulation of an independent variable , the measurement of a dependent variable , and the exposure of various participants to one or more of the conditions being studied. Random selection of participants and their random assignment to conditions also are necessary in experiments .

In experimental design, the independent variable is the intervention, treatment, or condition that is being investigated as a potential cause of change (i.e., the experimental condition ). The effect, or outcome, of the experimental condition is the dependent variable. Trying out a new restaurant, dating a new person – we often call these things “experiments.” However, a true social science experiment would include recruitment of a large enough sample, random assignment to control and experimental groups, exposing those in the experimental group to an experimental condition, and collecting observations at the end of the experiment.

Social scientists use this level of rigor and control to maximize the internal validity of their research. Internal validity is the confidence researchers have about whether the independent variable (e.g, treatment) truly produces a change in the dependent, or outcome, variable. The logic and features of experimental design are intended to help establish causality and to reduce threats to internal validity , which we will discuss in Section 14.5 .

Experiments attempt to establish a nomothetic causal relationship between two variables—the treatment and its intended outcome.  We discussed the four criteria for establishing nomothetic causality in Section 4.3 :

• plausibility,
• covariation,
• temporality, and
• nonspuriousness.

Experiments should establish plausibility , having a plausible reason why their intervention would cause changes in the dependent variable. Usually, a theory framework or previous empirical evidence will indicate the plausibility of a causal relationship.

Covariation can be established for causal explanations by showing that the “cause” and the “effect” change together.  In experiments, the cause is an intervention, treatment, or other experimental condition. Whether or not a research participant is exposed to the experimental condition is the independent variable. The effect in an experiment is the outcome being assessed and is the dependent variable in the study. When the independent and dependent variables covary, they can have a positive association (e.g., those exposed to the intervention have increased self-esteem) or a negative association (e.g., those exposed to the intervention have reduced anxiety).

Since researcher controls when the intervention is administered, they can be assured that changes in the independent variable (the treatment) happens before changes in the dependent variable (the outcome). In this way, experiments assure temporality .

Finally, one of the most important features of experiments is that they allow researchers to eliminate spurious variables to support the criterion of nonspuriousness . True experiments are usually conducted under strictly controlled conditions. The intervention is given in the same way to each person, with a minimal number of other variables that might cause their post-test scores to change.

The logic of experimental design

How do we know that one phenomenon causes another? The complexity of the social world in which we practice and conduct research means that causes of social problems are rarely cut and dry. Uncovering explanations for social problems is key to helping clients address them, and experimental research designs are one road to finding answers.

Just because two phenomena are related in some way doesn’t mean that one causes the other. Ice cream sales increase in the summer, and so does the rate of violent crime; does that mean that eating ice cream is going to make me violent? Obviously not, because ice cream is great. The reality of that association is far more complex—it could be that hot weather makes people more irritable and, at times, violent, while also making people want ice cream. More likely, though, there are other social factors not accounted for in the way we just described this association.

As we have discussed, experimental designs can help clear up at least some of this fog by allowing researchers to isolate the effect of interventions on dependent variables by controlling extraneous variables . In true experimental design (discussed in the next section) and quasi-experimental design, researchers accomplish this w ith a control group or comparison group and the experimental group . The experimental group is sometimes called the treatment group because people in the experimental group receive the treatment or are exposed to the experimental condition (but we will call it the experimental group in this chapter.) The control/comparison group does not receive the treatment or intervention. Instead they may receive what is known as “treatment as usual” or perhaps no treatment at all.

In a well-designed experiment, the control group should look almost identical to the experimental group in terms of demographics and other relevant factors. What if we want to know the effect of CBT on social anxiety, but we have learned in prior research that men tend to have a more difficult time overcoming social anxiety? We would want our control and experimental groups to have a similar portions of men, since ostensibly, both groups’ results would be affected by the men in the group. If your control group has 5 women, 6 men, and 4 non-binary people, then your experimental group should be made up of roughly the same gender balance to help control for the influence of gender on the outcome of your intervention. (In reality, the groups should be similar along other dimensions, as well, and your group will likely be much larger.) The researcher will use the same outcome measures for both groups and compare them, and assuming the experiment was designed correctly, get a pretty good answer about whether the intervention had an effect on social anxiety.

Random assignment [/pb_glossary], also called randomization, entails using a random process to decide which participants are put into the control or experimental group (which participants receive an intervention and which do not). By randomly assigning participants to a group, you can reduce the effect of extraneous variables on your research because there won’t be a systematic difference between the groups.

Do not confuse random assignment with random sampling . Random sampling is a method for selecting a sample from a population and is rarely used in psychological research. Random assignment is a method for assigning participants in a sample to the different conditions, and it is an important element of all experimental research in psychology and other related fields. Random sampling helps a great deal with external validity, or generalizability , whereas random assignment increases internal validity .

Other Features of Experiments that Help Establish Causality

To control for spuriousness (as well as meeting the three other criteria for establishing causality), experiments try to control as many aspects of the research process as possible: using control groups, having large enough sample sizes, standardizing the treatment, etc. Researchers in large experiments often employ clinicians or other research staff to help them. Researchers train their staff members exhaustively, provide pre-scripted responses to common questions, and control the physical environment of the experiment so each person who participates receives the exact same treatment. Experimental researchers also document their procedures, so that others can review them and make changes in future research if they think it will improve on the ability to control for spurious variables.

An interesting example is Bruce Alexander’s (2010) Rat Park experiments. Much of the early research conducted on addictive drugs, like heroin and cocaine, was conducted on animals other than humans, usually mice or rats. The scientific consensus up until Alexander’s experiments was that cocaine and heroin were so addictive that rats, if offered the drugs, would consume them repeatedly until they perished. Researchers claimed this behavior explained how addiction worked in humans, but Alexander was not so sure. He knew rats were social animals and the experimental procedure from previous experiments did not allow them to socialize. Instead, rats were kept isolated in small cages with only food, water, and metal walls. To Alexander, social isolation was a spurious variable, causing changes in addictive behavior not due to the drug itself. Alexander created an experiment of his own, in which rats were allowed to run freely in an interesting environment, socialize and mate with other rats, and of course, drink from a solution that contained an addictive drug. In this environment, rats did not become hopelessly addicted to drugs. In fact, they had little interest in the substance. To Alexander, the results of his experiment demonstrated that social isolation was more of a causal factor for addiction than the drug itself.

One challenge with Alexander’s findings is that subsequent researchers have had mixed success replicating his findings (e.g., Petrie, 1996; Solinas, Thiriet, El Rawas, Lardeux, & Jaber, 2009). Replication involves conducting another researcher’s experiment in the same manner and seeing if it produces the same results. If the causal relationship is real, it should occur in all (or at least most) rigorous replications of the experiment.

Replicability

[INSERT A PARAGRAPH ABOUT REPLICATION/REPRODUCTION HERE. CAN USE/REFERENCE THIS   IF IT’S HELPFUL; include glossary definition as well as other general info]

To allow for easier replication, researchers should describe their experimental methods diligently. Researchers with the Open Science Collaboration (2015) [1] conducted the Reproducibility Project , which caused a significant controversy regarding the validity of psychological studies. The researchers with the project attempted to reproduce the results of 100 experiments published in major psychology journals since 2008. What they found was shocking. Although 97% of the original studies reported significant results, only 36% of the replicated studies had significant findings. The average effect size in the replication studies was half that of the original studies. The implications of the Reproducibility Project are potentially staggering, and encourage social scientists to carefully consider the validity of their reported findings and that the scientific community take steps to ensure researchers do not cherry-pick data or change their hypotheses simply to get published.

Generalizability

Let’s return to Alexander’s Rat Park study and consider the implications of his experiment for substance use professionals.  The conclusions he drew from his experiments on rats were meant to be generalized to the population. If this could be done, the experiment would have a high degree of external validity , which is the degree to which conclusions generalize to larger populations and different situations. Alexander argues his conclusions about addiction and social isolation help us understand why people living in deprived, isolated environments may become addicted to drugs more often than those in more enriching environments. Similarly, earlier rat researchers argued their results showed these drugs were instantly addictive to humans, often to the point of death.

Neither study’s results will match up perfectly with real life. There are clients in social work practice who may fit into Alexander’s social isolation model, but social isolation is complex. Clients can live in environments with other sociable humans, work jobs, and have romantic relationships; does this mean they are not socially isolated? On the other hand, clients may face structural racism, poverty, trauma, and other challenges that may contribute to their social environment. Alexander’s work helps understand clients’ experiences, but the explanation is incomplete. Human existence is more complicated than the experimental conditions in Rat Park.

Effectiveness versus Efficacy

Social workers are especially attentive to how social context shapes social life. This consideration points out a potential weakness of experiments. They can be rather artificial. When an experiment demonstrates causality under ideal, controlled circumstances, it establishes the efficacy of an intervention.

How often do real-world social interactions occur in the same way that they do in a controlled experiment? Experiments that are conducted in community settings by community practitioners are less easily controlled than those conducted in a lab or with researchers who adhere strictly to research protocols delivering the intervention. When an experiment demonstrates causality in a real-world setting that is not tightly controlled, it establishes the effectiveness of the intervention.

The distinction between efficacy and effectiveness demonstrates the tension between internal and external validity. Internal validity and external validity are conceptually linked. Internal validity refers to the degree to which the intervention causes its intended outcomes, and external validity refers to how well that relationship applies to different groups and circumstances than the experiment. However, the more researchers tightly control the environment to ensure internal validity, the more they may risk external validity for generalizing their results to different populations and circumstances. Correspondingly, researchers whose settings are just like the real world will be less able to ensure internal validity, as there are many factors that could pollute the research process. This is not to suggest that experimental research findings cannot have high levels of both internal and external validity, but that experimental researchers must always be aware of this potential weakness and clearly report limitations in their research reports.

Types of Experimental Designs

Experimental design is an umbrella term for a research method that is designed to test hypotheses related to causality under controlled conditions. Table 14.1 describes the three major types of experimental design (pre-experimental, quasi-experimental, and true experimental) and presents subtypes for each. As we will see in the coming sections, some types of experimental design are better at establishing causality than others. It’s also worth considering that true experiments, which most effectively establish causality , are often difficult and expensive to implement. Although the other experimental designs aren’t perfect, they still produce useful, valid evidence and may be more feasible to carry out.

Key Takeaways

• Experimental designs are useful for establishing causality, but some types of experimental design do this better than others.
• Experiments help researchers isolate the effect of the independent variable on the dependent variable by controlling for the effect of extraneous variables .
• Experiments use a control/comparison group and an experimental group to test the effects of interventions. These groups should be as similar to each other as possible in terms of demographics and other relevant factors.
• True experiments have control groups with randomly assigned participants; quasi-experimental types of experiments have comparison groups to which participants are not randomly assigned; pre-experimental designs do not have a comparison group.

TRACK 1 (IF YOU  ARE  CREATING A RESEARCH PROPOSAL FOR THIS CLASS):

• Think about the research project you’ve been designing so far. How might you use a basic experiment to answer your question? If your question isn’t explanatory, try to formulate a new explanatory question and consider the usefulness of an experiment.
• Why is establishing a simple relationship between two variables not indicative of one causing the other?

TRACK 2 (IF YOU  AREN’T  CREATING A RESEARCH PROPOSAL FOR THIS CLASS):

Imagine you are interested in studying child welfare practice. You are interested in learning more about community-based programs aimed to prevent child maltreatment and to prevent out-of-home placement for children.

• Think about the research project stated above. How might you use a basic experiment to look more into this research topic? Try to formulate an explanatory question and consider the usefulness of an experiment.
• Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349 (6251), aac4716. Doi: 10.1126/science.aac4716 ↵

an operation or procedure carried out under controlled conditions in order to discover an unknown effect or law, to test or establish a hypothesis, or to illustrate a known law.

treatment, intervention, or experience that is being tested in an experiment (the independent variable) that is received by the experimental group and not by the control group.

Ability to say that one variable "causes" something to happen to another variable. Very important to assess when thinking about studies that examine causation such as experimental or quasi-experimental designs.

circumstances or events that may affect the outcome of an experiment, resulting in changes in the research participants that are not a result of the intervention, treatment, or experimental condition being tested

causal explanations that can be universally applied to groups, such as scientific laws or universal truths

as a criteria for causal relationship, the relationship must make logical sense and seem possible

when the values of two variables change at the same time

as a criteria for causal relationship, the cause must come before the effect

an association between two variables that is NOT caused by a third variable

variables and characteristics that have an effect on your outcome, but aren't the primary variable whose influence you're interested in testing.

the group of participants in our study who do not receive the intervention we are researching in experiments with random assignment

the group of participants in our study who do not receive the intervention we are researching in experiments without random assignment

in experimental design, the group of participants in our study who do receive the intervention we are researching

The ability to apply research findings beyond the study sample to some broader population,

This is a synonymous term for generalizability - the ability to apply the findings of a study beyond the sample to a broader population.

performance of an intervention under ideal and controlled circumstances, such as in a lab or delivered by trained researcher-interventionists

The performance of an intervention under "real-world" conditions that are not closely controlled and ideal

the idea that one event, behavior, or belief will result in the occurrence of another, subsequent event, behavior, or belief

Doctoral Research Methods in Social Work Copyright © by Mavs Open Press. All Rights Reserved.

Share This Book

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

10 Experimental research

Experimental research—often considered to be the ‘gold standard’ in research designs—is one of the most rigorous of all research designs. In this design, one or more independent variables are manipulated by the researcher (as treatments), subjects are randomly assigned to different treatment levels (random assignment), and the results of the treatments on outcomes (dependent variables) are observed. The unique strength of experimental research is its internal validity (causality) due to its ability to link cause and effect through treatment manipulation, while controlling for the spurious effect of extraneous variable.

Experimental research is best suited for explanatory research—rather than for descriptive or exploratory research—where the goal of the study is to examine cause-effect relationships. It also works well for research that involves a relatively limited and well-defined set of independent variables that can either be manipulated or controlled. Experimental research can be conducted in laboratory or field settings. Laboratory experiments , conducted in laboratory (artificial) settings, tend to be high in internal validity, but this comes at the cost of low external validity (generalisability), because the artificial (laboratory) setting in which the study is conducted may not reflect the real world. Field experiments are conducted in field settings such as in a real organisation, and are high in both internal and external validity. But such experiments are relatively rare, because of the difficulties associated with manipulating treatments and controlling for extraneous effects in a field setting.

Experimental research can be grouped into two broad categories: true experimental designs and quasi-experimental designs. Both designs require treatment manipulation, but while true experiments also require random assignment, quasi-experiments do not. Sometimes, we also refer to non-experimental research, which is not really a research design, but an all-inclusive term that includes all types of research that do not employ treatment manipulation or random assignment, such as survey research, observational research, and correlational studies.

Basic concepts

Treatment and control groups. In experimental research, some subjects are administered one or more experimental stimulus called a treatment (the treatment group ) while other subjects are not given such a stimulus (the control group ). The treatment may be considered successful if subjects in the treatment group rate more favourably on outcome variables than control group subjects. Multiple levels of experimental stimulus may be administered, in which case, there may be more than one treatment group. For example, in order to test the effects of a new drug intended to treat a certain medical condition like dementia, if a sample of dementia patients is randomly divided into three groups, with the first group receiving a high dosage of the drug, the second group receiving a low dosage, and the third group receiving a placebo such as a sugar pill (control group), then the first two groups are experimental groups and the third group is a control group. After administering the drug for a period of time, if the condition of the experimental group subjects improved significantly more than the control group subjects, we can say that the drug is effective. We can also compare the conditions of the high and low dosage experimental groups to determine if the high dose is more effective than the low dose.

Treatment manipulation. Treatments are the unique feature of experimental research that sets this design apart from all other research methods. Treatment manipulation helps control for the ‘cause’ in cause-effect relationships. Naturally, the validity of experimental research depends on how well the treatment was manipulated. Treatment manipulation must be checked using pretests and pilot tests prior to the experimental study. Any measurements conducted before the treatment is administered are called pretest measures , while those conducted after the treatment are posttest measures .

Random selection and assignment. Random selection is the process of randomly drawing a sample from a population or a sampling frame. This approach is typically employed in survey research, and ensures that each unit in the population has a positive chance of being selected into the sample. Random assignment, however, is a process of randomly assigning subjects to experimental or control groups. This is a standard practice in true experimental research to ensure that treatment groups are similar (equivalent) to each other and to the control group prior to treatment administration. Random selection is related to sampling, and is therefore more closely related to the external validity (generalisability) of findings. However, random assignment is related to design, and is therefore most related to internal validity. It is possible to have both random selection and random assignment in well-designed experimental research, but quasi-experimental research involves neither random selection nor random assignment.

Threats to internal validity. Although experimental designs are considered more rigorous than other research methods in terms of the internal validity of their inferences (by virtue of their ability to control causes through treatment manipulation), they are not immune to internal validity threats. Some of these threats to internal validity are described below, within the context of a study of the impact of a special remedial math tutoring program for improving the math abilities of high school students.

History threat is the possibility that the observed effects (dependent variables) are caused by extraneous or historical events rather than by the experimental treatment. For instance, students’ post-remedial math score improvement may have been caused by their preparation for a math exam at their school, rather than the remedial math program.

Maturation threat refers to the possibility that observed effects are caused by natural maturation of subjects (e.g., a general improvement in their intellectual ability to understand complex concepts) rather than the experimental treatment.

Testing threat is a threat in pre-post designs where subjects’ posttest responses are conditioned by their pretest responses. For instance, if students remember their answers from the pretest evaluation, they may tend to repeat them in the posttest exam.

Not conducting a pretest can help avoid this threat.

Instrumentation threat , which also occurs in pre-post designs, refers to the possibility that the difference between pretest and posttest scores is not due to the remedial math program, but due to changes in the administered test, such as the posttest having a higher or lower degree of difficulty than the pretest.

Mortality threat refers to the possibility that subjects may be dropping out of the study at differential rates between the treatment and control groups due to a systematic reason, such that the dropouts were mostly students who scored low on the pretest. If the low-performing students drop out, the results of the posttest will be artificially inflated by the preponderance of high-performing students.

Regression threat —also called a regression to the mean—refers to the statistical tendency of a group’s overall performance to regress toward the mean during a posttest rather than in the anticipated direction. For instance, if subjects scored high on a pretest, they will have a tendency to score lower on the posttest (closer to the mean) because their high scores (away from the mean) during the pretest were possibly a statistical aberration. This problem tends to be more prevalent in non-random samples and when the two measures are imperfectly correlated.

Two-group experimental designs

Pretest-posttest control group design . In this design, subjects are randomly assigned to treatment and control groups, subjected to an initial (pretest) measurement of the dependent variables of interest, the treatment group is administered a treatment (representing the independent variable of interest), and the dependent variables measured again (posttest). The notation of this design is shown in Figure 10.1.

Statistical analysis of this design involves a simple analysis of variance (ANOVA) between the treatment and control groups. The pretest-posttest design handles several threats to internal validity, such as maturation, testing, and regression, since these threats can be expected to influence both treatment and control groups in a similar (random) manner. The selection threat is controlled via random assignment. However, additional threats to internal validity may exist. For instance, mortality can be a problem if there are differential dropout rates between the two groups, and the pretest measurement may bias the posttest measurement—especially if the pretest introduces unusual topics or content.

Posttest -only control group design . This design is a simpler version of the pretest-posttest design where pretest measurements are omitted. The design notation is shown in Figure 10.2.

The treatment effect is measured simply as the difference in the posttest scores between the two groups:

The appropriate statistical analysis of this design is also a two-group analysis of variance (ANOVA). The simplicity of this design makes it more attractive than the pretest-posttest design in terms of internal validity. This design controls for maturation, testing, regression, selection, and pretest-posttest interaction, though the mortality threat may continue to exist.

Because the pretest measure is not a measurement of the dependent variable, but rather a covariate, the treatment effect is measured as the difference in the posttest scores between the treatment and control groups as:

Due to the presence of covariates, the right statistical analysis of this design is a two-group analysis of covariance (ANCOVA). This design has all the advantages of posttest-only design, but with internal validity due to the controlling of covariates. Covariance designs can also be extended to pretest-posttest control group design.

Factorial designs

Two-group designs are inadequate if your research requires manipulation of two or more independent variables (treatments). In such cases, you would need four or higher-group designs. Such designs, quite popular in experimental research, are commonly called factorial designs. Each independent variable in this design is called a factor , and each subdivision of a factor is called a level . Factorial designs enable the researcher to examine not only the individual effect of each treatment on the dependent variables (called main effects), but also their joint effect (called interaction effects).

In a factorial design, a main effect is said to exist if the dependent variable shows a significant difference between multiple levels of one factor, at all levels of other factors. No change in the dependent variable across factor levels is the null case (baseline), from which main effects are evaluated. In the above example, you may see a main effect of instructional type, instructional time, or both on learning outcomes. An interaction effect exists when the effect of differences in one factor depends upon the level of a second factor. In our example, if the effect of instructional type on learning outcomes is greater for three hours/week of instructional time than for one and a half hours/week, then we can say that there is an interaction effect between instructional type and instructional time on learning outcomes. Note that the presence of interaction effects dominate and make main effects irrelevant, and it is not meaningful to interpret main effects if interaction effects are significant.

Hybrid experimental designs

Hybrid designs are those that are formed by combining features of more established designs. Three such hybrid designs are randomised bocks design, Solomon four-group design, and switched replications design.

Randomised block design. This is a variation of the posttest-only or pretest-posttest control group design where the subject population can be grouped into relatively homogeneous subgroups (called blocks ) within which the experiment is replicated. For instance, if you want to replicate the same posttest-only design among university students and full-time working professionals (two homogeneous blocks), subjects in both blocks are randomly split between the treatment group (receiving the same treatment) and the control group (see Figure 10.5). The purpose of this design is to reduce the ‘noise’ or variance in data that may be attributable to differences between the blocks so that the actual effect of interest can be detected more accurately.

Solomon four-group design . In this design, the sample is divided into two treatment groups and two control groups. One treatment group and one control group receive the pretest, and the other two groups do not. This design represents a combination of posttest-only and pretest-posttest control group design, and is intended to test for the potential biasing effect of pretest measurement on posttest measures that tends to occur in pretest-posttest designs, but not in posttest-only designs. The design notation is shown in Figure 10.6.

Switched replication design . This is a two-group design implemented in two phases with three waves of measurement. The treatment group in the first phase serves as the control group in the second phase, and the control group in the first phase becomes the treatment group in the second phase, as illustrated in Figure 10.7. In other words, the original design is repeated or replicated temporally with treatment/control roles switched between the two groups. By the end of the study, all participants will have received the treatment either during the first or the second phase. This design is most feasible in organisational contexts where organisational programs (e.g., employee training) are implemented in a phased manner or are repeated at regular intervals.

Quasi-experimental designs

Quasi-experimental designs are almost identical to true experimental designs, but lacking one key ingredient: random assignment. For instance, one entire class section or one organisation is used as the treatment group, while another section of the same class or a different organisation in the same industry is used as the control group. This lack of random assignment potentially results in groups that are non-equivalent, such as one group possessing greater mastery of certain content than the other group, say by virtue of having a better teacher in a previous semester, which introduces the possibility of selection bias . Quasi-experimental designs are therefore inferior to true experimental designs in interval validity due to the presence of a variety of selection related threats such as selection-maturation threat (the treatment and control groups maturing at different rates), selection-history threat (the treatment and control groups being differentially impacted by extraneous or historical events), selection-regression threat (the treatment and control groups regressing toward the mean between pretest and posttest at different rates), selection-instrumentation threat (the treatment and control groups responding differently to the measurement), selection-testing (the treatment and control groups responding differently to the pretest), and selection-mortality (the treatment and control groups demonstrating differential dropout rates). Given these selection threats, it is generally preferable to avoid quasi-experimental designs to the greatest extent possible.

In addition, there are quite a few unique non-equivalent designs without corresponding true experimental design cousins. Some of the more useful of these designs are discussed next.

Regression discontinuity (RD) design . This is a non-equivalent pretest-posttest design where subjects are assigned to the treatment or control group based on a cut-off score on a preprogram measure. For instance, patients who are severely ill may be assigned to a treatment group to test the efficacy of a new drug or treatment protocol and those who are mildly ill are assigned to the control group. In another example, students who are lagging behind on standardised test scores may be selected for a remedial curriculum program intended to improve their performance, while those who score high on such tests are not selected from the remedial program.

Because of the use of a cut-off score, it is possible that the observed results may be a function of the cut-off score rather than the treatment, which introduces a new threat to internal validity. However, using the cut-off score also ensures that limited or costly resources are distributed to people who need them the most, rather than randomly across a population, while simultaneously allowing a quasi-experimental treatment. The control group scores in the RD design do not serve as a benchmark for comparing treatment group scores, given the systematic non-equivalence between the two groups. Rather, if there is no discontinuity between pretest and posttest scores in the control group, but such a discontinuity persists in the treatment group, then this discontinuity is viewed as evidence of the treatment effect.

Proxy pretest design . This design, shown in Figure 10.11, looks very similar to the standard NEGD (pretest-posttest) design, with one critical difference: the pretest score is collected after the treatment is administered. A typical application of this design is when a researcher is brought in to test the efficacy of a program (e.g., an educational program) after the program has already started and pretest data is not available. Under such circumstances, the best option for the researcher is often to use a different prerecorded measure, such as students’ grade point average before the start of the program, as a proxy for pretest data. A variation of the proxy pretest design is to use subjects’ posttest recollection of pretest data, which may be subject to recall bias, but nevertheless may provide a measure of perceived gain or change in the dependent variable.

Separate pretest-posttest samples design . This design is useful if it is not possible to collect pretest and posttest data from the same subjects for some reason. As shown in Figure 10.12, there are four groups in this design, but two groups come from a single non-equivalent group, while the other two groups come from a different non-equivalent group. For instance, say you want to test customer satisfaction with a new online service that is implemented in one city but not in another. In this case, customers in the first city serve as the treatment group and those in the second city constitute the control group. If it is not possible to obtain pretest and posttest measures from the same customers, you can measure customer satisfaction at one point in time, implement the new service program, and measure customer satisfaction (with a different set of customers) after the program is implemented. Customer satisfaction is also measured in the control group at the same times as in the treatment group, but without the new program implementation. The design is not particularly strong, because you cannot examine the changes in any specific customer’s satisfaction score before and after the implementation, but you can only examine average customer satisfaction scores. Despite the lower internal validity, this design may still be a useful way of collecting quasi-experimental data when pretest and posttest data is not available from the same subjects.

An interesting variation of the NEDV design is a pattern-matching NEDV design , which employs multiple outcome variables and a theory that explains how much each variable will be affected by the treatment. The researcher can then examine if the theoretical prediction is matched in actual observations. This pattern-matching technique—based on the degree of correspondence between theoretical and observed patterns—is a powerful way of alleviating internal validity concerns in the original NEDV design.

Perils of experimental research

Experimental research is one of the most difficult of research designs, and should not be taken lightly. This type of research is often best with a multitude of methodological problems. First, though experimental research requires theories for framing hypotheses for testing, much of current experimental research is atheoretical. Without theories, the hypotheses being tested tend to be ad hoc, possibly illogical, and meaningless. Second, many of the measurement instruments used in experimental research are not tested for reliability and validity, and are incomparable across studies. Consequently, results generated using such instruments are also incomparable. Third, often experimental research uses inappropriate research designs, such as irrelevant dependent variables, no interaction effects, no experimental controls, and non-equivalent stimulus across treatment groups. Findings from such studies tend to lack internal validity and are highly suspect. Fourth, the treatments (tasks) used in experimental research may be diverse, incomparable, and inconsistent across studies, and sometimes inappropriate for the subject population. For instance, undergraduate student subjects are often asked to pretend that they are marketing managers and asked to perform a complex budget allocation task in which they have no experience or expertise. The use of such inappropriate tasks, introduces new threats to internal validity (i.e., subject’s performance may be an artefact of the content or difficulty of the task setting), generates findings that are non-interpretable and meaningless, and makes integration of findings across studies impossible.

The design of proper experimental treatments is a very important task in experimental design, because the treatment is the raison d’etre of the experimental method, and must never be rushed or neglected. To design an adequate and appropriate task, researchers should use prevalidated tasks if available, conduct treatment manipulation checks to check for the adequacy of such tasks (by debriefing subjects after performing the assigned task), conduct pilot tests (repeatedly, if necessary), and if in doubt, use tasks that are simple and familiar for the respondent sample rather than tasks that are complex or unfamiliar.

In summary, this chapter introduced key concepts in the experimental design research method and introduced a variety of true experimental and quasi-experimental designs. Although these designs vary widely in internal validity, designs with less internal validity should not be overlooked and may sometimes be useful under specific circumstances and empirical contingencies.

Social Science Research: Principles, Methods and Practices (Revised edition) Copyright © 2019 by Anol Bhattacherjee is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

6.1 Experiment Basics

Learning objectives.

• Explain what an experiment is and recognize examples of studies that are experiments and studies that are not experiments.
• Explain what internal validity is and why experiments are considered to be high in internal validity.
• Explain what external validity is and evaluate studies in terms of their external validity.
• Distinguish between the manipulation of the independent variable and control of extraneous variables and explain the importance of each.
• Recognize examples of confounding variables and explain how they affect the internal validity of a study.

What Is an Experiment?

As we saw earlier in the book, an experiment is a type of study designed specifically to answer the question of whether there is a causal relationship between two variables. Do changes in an independent variable cause changes in a dependent variable? Experiments have two fundamental features. The first is that the researchers manipulate, or systematically vary, the level of the independent variable. The different levels of the independent variable are called conditions. For example, in Darley and Latané’s experiment, the independent variable was the number of witnesses that participants believed to be present. The researchers manipulated this independent variable by telling participants that there were either one, two, or five other students involved in the discussion, thereby creating three conditions. The second fundamental feature of an experiment is that the researcher controls, or minimizes the variability in, variables other than the independent and dependent variable. These other variables are called extraneous variables. Darley and Latané tested all their participants in the same room, exposed them to the same emergency situation, and so on. They also randomly assigned their participants to conditions so that the three groups would be similar to each other to begin with. Notice that although the words manipulation and control have similar meanings in everyday language, researchers make a clear distinction between them. They manipulate the independent variable by systematically changing its levels and control other variables by holding them constant.

Internal and External Validity

Internal validity.

Recall that the fact that two variables are statistically related does not necessarily mean that one causes the other. “Correlation does not imply causation.” For example, if it were the case that people who exercise regularly are happier than people who do not exercise regularly, this would not necessarily mean that exercising increases people’s happiness. It could mean instead that greater happiness causes people to exercise (the directionality problem) or that something like better physical health causes people to exercise and be happier (the third-variable problem).

The purpose of an experiment, however, is to show that two variables are statistically related and to do so in a way that supports the conclusion that the independent variable caused any observed differences in the dependent variable. The basic logic is this: If the researcher creates two or more highly similar conditions and then manipulates the independent variable to produce just one difference between them, then any later difference between the conditions must have been caused by the independent variable. For example, because the only difference between Darley and Latané’s conditions was the number of students that participants believed to be involved in the discussion, this must have been responsible for differences in helping between the conditions.

An empirical study is said to be high in internal validity if the way it was conducted supports the conclusion that the independent variable caused any observed differences in the dependent variable. Thus experiments are high in internal validity because the way they are conducted—with the manipulation of the independent variable and the control of extraneous variables—provides strong support for causal conclusions.

External Validity

At the same time, the way that experiments are conducted sometimes leads to a different kind of criticism. Specifically, the need to manipulate the independent variable and control extraneous variables means that experiments are often conducted under conditions that seem artificial or unlike “real life” (Stanovich, 2010). In many psychology experiments, the participants are all college undergraduates and come to a classroom or laboratory to fill out a series of paper-and-pencil questionnaires or to perform a carefully designed computerized task. Consider, for example, an experiment in which researcher Barbara Fredrickson and her colleagues had college students come to a laboratory on campus and complete a math test while wearing a swimsuit (Fredrickson, Roberts, Noll, Quinn, & Twenge, 1998). At first, this might seem silly. When will college students ever have to complete math tests in their swimsuits outside of this experiment?

The issue we are confronting is that of external validity. An empirical study is high in external validity if the way it was conducted supports generalizing the results to people and situations beyond those actually studied. As a general rule, studies are higher in external validity when the participants and the situation studied are similar to those that the researchers want to generalize to. Imagine, for example, that a group of researchers is interested in how shoppers in large grocery stores are affected by whether breakfast cereal is packaged in yellow or purple boxes. Their study would be high in external validity if they studied the decisions of ordinary people doing their weekly shopping in a real grocery store. If the shoppers bought much more cereal in purple boxes, the researchers would be fairly confident that this would be true for other shoppers in other stores. Their study would be relatively low in external validity, however, if they studied a sample of college students in a laboratory at a selective college who merely judged the appeal of various colors presented on a computer screen. If the students judged purple to be more appealing than yellow, the researchers would not be very confident that this is relevant to grocery shoppers’ cereal-buying decisions.

We should be careful, however, not to draw the blanket conclusion that experiments are low in external validity. One reason is that experiments need not seem artificial. Consider that Darley and Latané’s experiment provided a reasonably good simulation of a real emergency situation. Or consider field experiments that are conducted entirely outside the laboratory. In one such experiment, Robert Cialdini and his colleagues studied whether hotel guests choose to reuse their towels for a second day as opposed to having them washed as a way of conserving water and energy (Cialdini, 2005). These researchers manipulated the message on a card left in a large sample of hotel rooms. One version of the message emphasized showing respect for the environment, another emphasized that the hotel would donate a portion of their savings to an environmental cause, and a third emphasized that most hotel guests choose to reuse their towels. The result was that guests who received the message that most hotel guests choose to reuse their towels reused their own towels substantially more often than guests receiving either of the other two messages. Given the way they conducted their study, it seems very likely that their result would hold true for other guests in other hotels.

A second reason not to draw the blanket conclusion that experiments are low in external validity is that they are often conducted to learn about psychological processes that are likely to operate in a variety of people and situations. Let us return to the experiment by Fredrickson and colleagues. They found that the women in their study, but not the men, performed worse on the math test when they were wearing swimsuits. They argued that this was due to women’s greater tendency to objectify themselves—to think about themselves from the perspective of an outside observer—which diverts their attention away from other tasks. They argued, furthermore, that this process of self-objectification and its effect on attention is likely to operate in a variety of women and situations—even if none of them ever finds herself taking a math test in her swimsuit.

Manipulation of the Independent Variable

Again, to manipulate an independent variable means to change its level systematically so that different groups of participants are exposed to different levels of that variable, or the same group of participants is exposed to different levels at different times. For example, to see whether expressive writing affects people’s health, a researcher might instruct some participants to write about traumatic experiences and others to write about neutral experiences. The different levels of the independent variable are referred to as conditions , and researchers often give the conditions short descriptive names to make it easy to talk and write about them. In this case, the conditions might be called the “traumatic condition” and the “neutral condition.”

Notice that the manipulation of an independent variable must involve the active intervention of the researcher. Comparing groups of people who differ on the independent variable before the study begins is not the same as manipulating that variable. For example, a researcher who compares the health of people who already keep a journal with the health of people who do not keep a journal has not manipulated this variable and therefore not conducted an experiment. This is important because groups that already differ in one way at the beginning of a study are likely to differ in other ways too. For example, people who choose to keep journals might also be more conscientious, more introverted, or less stressed than people who do not. Therefore, any observed difference between the two groups in terms of their health might have been caused by whether or not they keep a journal, or it might have been caused by any of the other differences between people who do and do not keep journals. Thus the active manipulation of the independent variable is crucial for eliminating the third-variable problem.

Of course, there are many situations in which the independent variable cannot be manipulated for practical or ethical reasons and therefore an experiment is not possible. For example, whether or not people have a significant early illness experience cannot be manipulated, making it impossible to do an experiment on the effect of early illness experiences on the development of hypochondriasis. This does not mean it is impossible to study the relationship between early illness experiences and hypochondriasis—only that it must be done using nonexperimental approaches. We will discuss this in detail later in the book.

In many experiments, the independent variable is a construct that can only be manipulated indirectly. For example, a researcher might try to manipulate participants’ stress levels indirectly by telling some of them that they have five minutes to prepare a short speech that they will then have to give to an audience of other participants. In such situations, researchers often include a manipulation check in their procedure. A manipulation check is a separate measure of the construct the researcher is trying to manipulate. For example, researchers trying to manipulate participants’ stress levels might give them a paper-and-pencil stress questionnaire or take their blood pressure—perhaps right after the manipulation or at the end of the procedure—to verify that they successfully manipulated this variable.

Control of Extraneous Variables

An extraneous variable is anything that varies in the context of a study other than the independent and dependent variables. In an experiment on the effect of expressive writing on health, for example, extraneous variables would include participant variables (individual differences) such as their writing ability, their diet, and their shoe size. They would also include situation or task variables such as the time of day when participants write, whether they write by hand or on a computer, and the weather. Extraneous variables pose a problem because many of them are likely to have some effect on the dependent variable. For example, participants’ health will be affected by many things other than whether or not they engage in expressive writing. This can make it difficult to separate the effect of the independent variable from the effects of the extraneous variables, which is why it is important to control extraneous variables by holding them constant.

Extraneous Variables as “Noise”

Extraneous variables make it difficult to detect the effect of the independent variable in two ways. One is by adding variability or “noise” to the data. Imagine a simple experiment on the effect of mood (happy vs. sad) on the number of happy childhood events people are able to recall. Participants are put into a negative or positive mood (by showing them a happy or sad video clip) and then asked to recall as many happy childhood events as they can. The two leftmost columns of Table 6.1 “Hypothetical Noiseless Data and Realistic Noisy Data” show what the data might look like if there were no extraneous variables and the number of happy childhood events participants recalled was affected only by their moods. Every participant in the happy mood condition recalled exactly four happy childhood events, and every participant in the sad mood condition recalled exactly three. The effect of mood here is quite obvious. In reality, however, the data would probably look more like those in the two rightmost columns of Table 6.1 “Hypothetical Noiseless Data and Realistic Noisy Data” . Even in the happy mood condition, some participants would recall fewer happy memories because they have fewer to draw on, use less effective strategies, or are less motivated. And even in the sad mood condition, some participants would recall more happy childhood memories because they have more happy memories to draw on, they use more effective recall strategies, or they are more motivated. Although the mean difference between the two groups is the same as in the idealized data, this difference is much less obvious in the context of the greater variability in the data. Thus one reason researchers try to control extraneous variables is so their data look more like the idealized data in Table 6.1 “Hypothetical Noiseless Data and Realistic Noisy Data” , which makes the effect of the independent variable is easier to detect (although real data never look quite that good).

Table 6.1 Hypothetical Noiseless Data and Realistic Noisy Data

One way to control extraneous variables is to hold them constant. This can mean holding situation or task variables constant by testing all participants in the same location, giving them identical instructions, treating them in the same way, and so on. It can also mean holding participant variables constant. For example, many studies of language limit participants to right-handed people, who generally have their language areas isolated in their left cerebral hemispheres. Left-handed people are more likely to have their language areas isolated in their right cerebral hemispheres or distributed across both hemispheres, which can change the way they process language and thereby add noise to the data.

In principle, researchers can control extraneous variables by limiting participants to one very specific category of person, such as 20-year-old, straight, female, right-handed, sophomore psychology majors. The obvious downside to this approach is that it would lower the external validity of the study—in particular, the extent to which the results can be generalized beyond the people actually studied. For example, it might be unclear whether results obtained with a sample of younger straight women would apply to older gay men. In many situations, the advantages of a diverse sample outweigh the reduction in noise achieved by a homogeneous one.

Extraneous Variables as Confounding Variables

The second way that extraneous variables can make it difficult to detect the effect of the independent variable is by becoming confounding variables. A confounding variable is an extraneous variable that differs on average across levels of the independent variable. For example, in almost all experiments, participants’ intelligence quotients (IQs) will be an extraneous variable. But as long as there are participants with lower and higher IQs at each level of the independent variable so that the average IQ is roughly equal, then this variation is probably acceptable (and may even be desirable). What would be bad, however, would be for participants at one level of the independent variable to have substantially lower IQs on average and participants at another level to have substantially higher IQs on average. In this case, IQ would be a confounding variable.

To confound means to confuse, and this is exactly what confounding variables do. Because they differ across conditions—just like the independent variable—they provide an alternative explanation for any observed difference in the dependent variable. Figure 6.1 “Hypothetical Results From a Study on the Effect of Mood on Memory” shows the results of a hypothetical study, in which participants in a positive mood condition scored higher on a memory task than participants in a negative mood condition. But if IQ is a confounding variable—with participants in the positive mood condition having higher IQs on average than participants in the negative mood condition—then it is unclear whether it was the positive moods or the higher IQs that caused participants in the first condition to score higher. One way to avoid confounding variables is by holding extraneous variables constant. For example, one could prevent IQ from becoming a confounding variable by limiting participants only to those with IQs of exactly 100. But this approach is not always desirable for reasons we have already discussed. A second and much more general approach—random assignment to conditions—will be discussed in detail shortly.

Figure 6.1 Hypothetical Results From a Study on the Effect of Mood on Memory

Because IQ also differs across conditions, it is a confounding variable.

Key Takeaways

• An experiment is a type of empirical study that features the manipulation of an independent variable, the measurement of a dependent variable, and control of extraneous variables.
• Studies are high in internal validity to the extent that the way they are conducted supports the conclusion that the independent variable caused any observed differences in the dependent variable. Experiments are generally high in internal validity because of the manipulation of the independent variable and control of extraneous variables.
• Studies are high in external validity to the extent that the result can be generalized to people and situations beyond those actually studied. Although experiments can seem “artificial”—and low in external validity—it is important to consider whether the psychological processes under study are likely to operate in other people and situations.
• Practice: List five variables that can be manipulated by the researcher in an experiment. List five variables that cannot be manipulated by the researcher in an experiment.

Practice: For each of the following topics, decide whether that topic could be studied using an experimental research design and explain why or why not.

• Effect of parietal lobe damage on people’s ability to do basic arithmetic.
• Effect of being clinically depressed on the number of close friendships people have.
• Effect of group training on the social skills of teenagers with Asperger’s syndrome.
• Effect of paying people to take an IQ test on their performance on that test.

Cialdini, R. (2005, April). Don’t throw in the towel: Use social influence research. APS Observer . Retrieved from http://www.psychologicalscience.org/observer/getArticle.cfm?id=1762 .

Fredrickson, B. L., Roberts, T.-A., Noll, S. M., Quinn, D. M., & Twenge, J. M. (1998). The swimsuit becomes you: Sex differences in self-objectification, restrained eating, and math performance. Journal of Personality and Social Psychology, 75 , 269–284.

Stanovich, K. E. (2010). How to think straight about psychology (9th ed.). Boston, MA: Allyn & Bacon.

Research Methods in Psychology Copyright © 2016 by University of Minnesota is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Control Variables | What Are They & Why Do They Matter?

Control Variables | What Are They & Why Do They Matter?

Published on March 1, 2021 by Pritha Bhandari . Revised on June 22, 2023.

A control variable is anything that is held constant or limited in a research study. It’s a variable that is not of interest to the study’s objectives , but is controlled because it could influence the outcomes.

Variables may be controlled directly by holding them constant throughout a study (e.g., by controlling the room temperature in an experiment), or they may be controlled indirectly through methods like randomization or statistical control (e.g., to account for participant characteristics like age in statistical tests). Control variables can help prevent research biases like omitted variable bias from affecting your results.

Table of contents

Why do control variables matter, how do you control a variable, control variable vs. control group, other interesting articles, frequently asked questions about control variables.

Control variables enhance the internal validity of a study by limiting the influence of confounding and other extraneous variables . This helps you establish a correlational or causal relationship between your variables of interest and helps avoid research bias .

Aside from the independent and dependent variables , all variables that can impact the results should be controlled. If you don’t control relevant variables, you may not be able to demonstrate that they didn’t influence your results. Uncontrolled variables are alternative explanations for your results and affect the reliability of your arguments.

Control variables in experiments

In an experiment , a researcher is interested in understanding the effect of an independent variable on a dependent variable. Control variables help you ensure that your results are solely caused by your experimental manipulation.

The independent variable is whether the vitamin D supplement is added to a diet, and the dependent variable is the level of alertness.

To make sure any change in alertness is caused by the vitamin D supplement and not by other factors, you control these variables that might affect alertness:

• Timing of meals
• Caffeine intake
• Screen time

Control variables in non-experimental research

In an observational study or other types of non-experimental research, a researcher can’t manipulate the independent variable (often due to practical or ethical considerations ). Instead, control variables are measured and taken into account to infer relationships between the main variables of interest.

To account for other factors that are likely to influence the results, you also measure these control variables:

• Marital status

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

There are several ways to control extraneous variables in experimental designs, and some of these can also be used in observational studies or quasi-experimental designs.

Random assignment

In experimental studies with multiple groups, participants should be randomly assigned to the different conditions. Random assignment helps you balance the characteristics of groups so that there are no systematic differences between them.

This method of assignment controls participant variables that might otherwise differ between groups and skew your results.

It’s possible that the participants who found the study through Facebook use more screen time during the day, and this might influence how alert they are in your study.

Standardized procedures

It’s important to use the same procedures across all groups in an experiment. The groups should only differ in the independent variable manipulation so that you can isolate its effect on the dependent variable (the results).

To control variables , you can hold them constant at a fixed level using a protocol that you design and use for all participant sessions. For example, the instructions and time spent on an experimental task should be the same for all participants in a laboratory setting.

• To control for diet, fresh and frozen meals are delivered to participants three times a day.
• To control meal timings, participants are instructed to eat breakfast at 9:30, lunch at 13:00, and dinner at 18:30.
• To control caffeine intake, participants are asked to consume a maximum of one cup of coffee a day.

Statistical controls

You can measure and control for extraneous variables statistically to remove their effects on other types of variables .

“Controlling for a variable” means modelling control variable data along with independent and dependent variable data in regression analyses and ANCOVAs . That way, you can isolate the control variable’s effects from the relationship between the variables of interest.

A control variable isn’t the same as a control group . Control variables are held constant or measured throughout a study for both control and experimental groups, while an independent variable varies between control and experimental groups.

A control group doesn’t undergo the experimental treatment of interest, and its outcomes are compared with those of the experimental group. A control group usually has either no treatment, a standard treatment that’s already widely used, or a placebo (a fake treatment).

Aside from the experimental treatment, everything else in an experimental procedure should be the same between an experimental and control group.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Ecological validity

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Prevent plagiarism. Run a free check.

A control variable is any variable that’s held constant in a research study. It’s not a variable of interest in the study, but it’s controlled because it could influence the outcomes.

Control variables help you establish a correlational or causal relationship between variables by enhancing internal validity .

If you don’t control relevant extraneous variables , they may influence the outcomes of your study, and you may not be able to demonstrate that your results are really an effect of your independent variable .

Internal validity is the extent to which you can be confident that a cause-and-effect relationship established in a study cannot be explained by other factors.

“Controlling for a variable” means measuring extraneous variables and accounting for them statistically to remove their effects on other variables.

Researchers often model control variable data along with independent and dependent variable data in regression analyses and ANCOVAs . That way, you can isolate the control variable’s effects from the relationship between the variables of interest.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 22). Control Variables | What Are They & Why Do They Matter?. Scribbr. Retrieved September 16, 2024, from https://www.scribbr.com/methodology/control-variable/

Is this article helpful?

Pritha Bhandari

Other students also liked, what is a controlled experiment | definitions & examples, independent vs. dependent variables | definition & examples, extraneous variables | examples, types & controls, what is your plagiarism score.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

1.4 Designed Experiments

Observational studies vs. experiments.

Ignoring anecdotal evidence, there are two primary types of data collection: observational studies and controlled (designed) experiments . Remember, we typically cannot make claims of causality from observation studies because of the potential presence of confounding factors. However, making causal conclusions based on experiments is often reasonable if we control for those factors.

Suppose you want to investigate the effectiveness of vitamin D in preventing disease. You recruit a group of subjects and ask them if they regularly take vitamin D. You notice that the subjects who take vitamin D exhibit better health on average than those who do not. Does this prove that vitamin D is effective in disease prevention? It does not. There are many differences between the two groups beyond just vitamin D consumption. People who take vitamin D regularly often take other steps to improve their health: exercise, diet, other vitamin supplements, choosing not to smoke. Any one of these factors could influence health. As described, this study does not necessarily prove that vitamin D is the key to disease prevention.

Experiments ultimately aim to provide evidence for use in decision-making, so how could we narrow our focus and make claims of causality? In this section, you will learn important aspects of experimental design.

Designed Experiments

The purpose of an experiment is to investigate the relationship between two variables. When one variable causes change in another, we call the first variable the explanatory variable . The affected variable is called the response variable . In a randomized experiment, the researcher manipulates values of the explanatory variable and measures the resulting changes in the response variable. The different values of the explanatory variable may be called treatments . An experimental unit is a single object or individual being measured.

The main principles to follow in experimental design are:

Randomization

Replication.

In order to provide evidence that the explanatory variable is indeed causing the changes in the response variable, it is necessary to isolate the explanatory variable. The researcher must design the experiment in such a way that there is only one difference between groups being compared: the planned treatments. This is accomplished by randomizing the experimental units placed into treatment groups. When subjects are assigned treatments randomly, all of the potential lurking variables are spread equally among the groups. At this point, the only difference between groups is the one imposed by the researcher. As a result, different outcomes measured in the response variable must be a direct result of the different treatments. In this way, an experiment can show an apparent cause-and-effect connection between the explanatory and response variables.

Recall our previous example of investigating the effectiveness of vitamin D in preventing disease. Individuals in our trial could be randomly assigned, perhaps by flipping a coin, into one of two groups: the control group (no treatment) and the experimental group (extra doses of vitamin D).

The more cases researchers observe, the more accurately they can estimate the effect of the explanatory variable on the response. In a single study, we replicate by collecting a sufficiently large sample. Additionally, a group of scientists may replicate an entire study to verify an earlier finding. It is also helpful to subject individuals to the same treatment more than once, which is known as repeated measures .

The power of suggestion can have an important influence on the outcome of an experiment. Studies have shown that the expectations of the study participant can be as important as the actual medication. In one study of performance-enhancing drugs, researchers noted, “ Results showed that believing one had taken the substance resulted in [performance] times almost as fast as those associated with consuming the drug itself. In contrast, taking the drug without knowledge yielded no significant performance increment.” [1]

It is often difficult to isolate the effects of the explanatory variable. To counter the power of suggestion, researchers set aside one treatment group as a control group . This group is given a placebo treatment—a treatment that cannot influence the response variable. The control group helps researchers balance the effects of being in an experiment with the effects of the active treatments. Of course, if you are participating in a study and you know that you are receiving a pill that contains no actual medication, then the power of suggestion is no longer a factor. Blinding in a randomized experiment preserves the power of suggestion. When a person involved in a research study is blinded, he does not know who is receiving the active treatment(s) and who is receiving the placebo treatment. A double-blind experiment is one in which both the subjects and the researchers involved with the subjects are unaware.

Randomized experiments are an essential tool in research. The U.S. Food and Drug Administration typically requires that a new drug can only be marketed after two independently conducted randomized trials confirm its safety and efficacy; the European Medicines Agency has a similar policy. Large randomized experiments in medicine have provided the basis for major public health initiatives. In 1954, approximately 750,000 children participated in a randomized study comparing the polio vaccine with a placebo. In the United States, the results of the study quickly led to the widespread and successful use of the vaccine for polio prevention.

How does sleep deprivation affect your ability to drive? A recent study measured its effects on 19 professional drivers. Each driver participated in two experimental sessions: one after normal sleep and one after 27 hours of total sleep deprivation. The treatments were assigned in random order. In each session, performance was measured on a variety of tasks including a driving simulation.

The Smell & Taste Treatment and Research Foundation conducted a study to investigate whether smell can affect learning. Subjects completed pencil-and-paper mazes multiple times while wearing masks. They completed the mazes three times wearing floral-scented masks and three times with unscented masks. Participants were assigned at random to wear the floral mask during either the first three or last three trials. For each trial, researchers recorded the time it took to complete the maze and whether the subject’s impression of the mask’s scent was positive, negative, or neutral.

• Describe the explanatory and response variables in this study.
• What are the treatments?
• Identify any lurking variables that could interfere with this study.
• Is it possible to use blinding in this study?
• The explanatory variable is scent, and the response variable is the time it takes to complete the maze.
• There are two treatments: a floral-scented mask and an unscented mask.
• All subjects experienced both treatments. The order of treatments was randomly assigned so there were no differences between the treatment groups. Random assignment eliminates the problem of lurking variables.
• Subjects will clearly know whether they can smell flowers or not, so subjects cannot be blinded in this study. Researchers timing the mazes can be blinded, though. The researcher who is observing a subject will not know which mask is being worn.

More Experimental Design

There are many different experimental designs from the most basic—a single treatment and control group—to some very complicated designs. When working with more than one treatment in an experimental design setting, these variables are often called factors , especially if they are categorical.   The values of factors are are often called levels . When there are multiple factors, the combinations of each of the levels are called treatment combinations , or interactions. Some basic types of interactions you may see are:

• Completely randomized
• Block design
• Matched pairs design

Completely Randomized

This essential research tool does not require much explanation. It involves figuring out how many treatments will be administered and randomly assigning participants to their respective groups.

Block Design

Researchers sometimes know or suspect that variables outside of the treatment influence the response. Based on this, they may first group individuals into blocks and then randomly draw cases from each block for the treatment groups. This strategy is often referred to as blocking . For instance, if we are looking at the effect of a drug on heart attacks, we might first split patients in the study into low-risk and high-risk blocks, then randomly assign half the patients from each block to the control group and the other half to the treatment group, as shown in the figure below. This strategy ensures each treatment group has an equal number of low-risk and high-risk patients.

Matched Pairs

A matched pairs design is one where very similar individuals (or even the same individual) receive two different treatments (or treatment vs. control) and the results are compared. Though this design is very effective, it can be hard to find many suitably similar individuals. Some common forms of a matched pairs design are twin studies, before-and-after measurements, pre- and post-test situations, and crossover studies.

Was the use of a new wetsuit design responsible for an observed increase in swim velocities at the 2000 Summer Olympics? In a matched pairs study designed to investigate this question, twelve competitive swimmers swam 1,500 meters at maximal speed, once wearing a wetsuit and once wearing a regular swimsuit. The order of wetsuit and swimsuit trials was randomized for each of the 12 swimmers. Figure 1.6 shows the average velocity recorded for each swimmer, measured in meters per second (m/s).

In this data, two sets of observations are uniquely paired so that an observation in one set matches an observation in the other; in this case, each swimmer has two measured velocities, one with a wetsuit and one with a swimsuit. A natural measure of the effect of the wetsuit on swim velocity is the difference between the measured maximum velocities (velocity.diff = wet.suit.velocit – swim.suit.velocity). Even though there are two measurements per individual, using the difference in observations as the variable of interest allows for the problem to be analyzed.

A new windshield treatment claims to repel water more effectively. Ten windshields are tested by simulating rain without the new treatment. The same windshields are then treated, and the experiment is run again. What experiment design is being implemented here?

Matched pairs

A new medicine is said to help improve sleep. Eight subjects are picked at random and given the medicine. The mean hours slept for each person were recorded before and after stating the medication. What experiment design is being implemented here?

Click here for more multimedia resources, including podcasts, videos, lecture notes, and worked examples.

Figure References

Figure 1.5: Kindred Grey (2020). Block design. CC BY-SA 4.0.

• McClung, Mary, and Dave Collins, ""Because I know it will!": Placebo Effects of an Ergogenic Acid on Athletic Performance," Journal of Sport & Exercise Psychology, 29 , no. 3 (2007): 382-394. ↵

Data collection where no variables are manipulated

Type of experiment where variables are manipulated and data is collected in a controlled setting

The independent variable in an experiment; the value controlled by researchers

The dependent variable in an experiment; the value that is measured for change at the end of an experiment

Different values or components of the explanatory variable applied in an experiment

Any individual or object to be measured

When an individual goes through a single treatment more than once

A group in a randomized experiment that receives no (or inactive) treatment but is otherwise managed exactly as the other groups

An inactive treatment that has no real effect on the explanatory variable

Not telling participants which treatment they are receiving

The act of blinding both the subjects of an experiment and the researchers who work with the subjects

Variables in an experiment

Certain values of variables in an experiment

Combinations of levels of variables in an experiment

Dividing participants into treatment groups randomly

Grouping individuals based on a variable into "blocks" and then randomizing cases within each block to the treatment groups

Very similar individuals (or even the same individual) receive two different treatments (or treatment vs. control), then the results are compared

Significant Statistics Copyright © 2024 by John Morgan Russell, OpenStaxCollege, OpenIntro is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 6: Experimental Research

Experimental Design

Learning Objectives

• Explain the difference between between-subjects and within-subjects experiments, list some of the pros and cons of each approach, and decide which approach to use to answer a particular research question.
• Define random assignment, distinguish it from random sampling, explain its purpose in experimental research, and use some simple strategies to implement it.
• Define what a control condition is, explain its purpose in research on treatment effectiveness, and describe some alternative types of control conditions.
• Define several types of carryover effect, give examples of each, and explain how counterbalancing helps to deal with them.

In this section, we look at some different ways to design an experiment. The primary distinction we will make is between approaches in which each participant experiences one level of the independent variable and approaches in which each participant experiences all levels of the independent variable. The former are called between-subjects experiments and the latter are called within-subjects experiments.

Between-Subjects Experiments

In a  between-subjects experiment , each participant is tested in only one condition. For example, a researcher with a sample of 100 university  students might assign half of them to write about a traumatic event and the other half write about a neutral event. Or a researcher with a sample of 60 people with severe agoraphobia (fear of open spaces) might assign 20 of them to receive each of three different treatments for that disorder. It is essential in a between-subjects experiment that the researcher assign participants to conditions so that the different groups are, on average, highly similar to each other. Those in a trauma condition and a neutral condition, for example, should include a similar proportion of men and women, and they should have similar average intelligence quotients (IQs), similar average levels of motivation, similar average numbers of health problems, and so on. This matching is a matter of controlling these extraneous participant variables across conditions so that they do not become confounding variables.

Random Assignment

The primary way that researchers accomplish this kind of control of extraneous variables across conditions is called  random assignment , which means using a random process to decide which participants are tested in which conditions. Do not confuse random assignment with random sampling. Random sampling is a method for selecting a sample from a population, and it is rarely used in psychological research. Random assignment is a method for assigning participants in a sample to the different conditions, and it is an important element of all experimental research in psychology and other fields too.

In its strictest sense, random assignment should meet two criteria. One is that each participant has an equal chance of being assigned to each condition (e.g., a 50% chance of being assigned to each of two conditions). The second is that each participant is assigned to a condition independently of other participants. Thus one way to assign participants to two conditions would be to flip a coin for each one. If the coin lands heads, the participant is assigned to Condition A, and if it lands tails, the participant is assigned to Condition B. For three conditions, one could use a computer to generate a random integer from 1 to 3 for each participant. If the integer is 1, the participant is assigned to Condition A; if it is 2, the participant is assigned to Condition B; and if it is 3, the participant is assigned to Condition C. In practice, a full sequence of conditions—one for each participant expected to be in the experiment—is usually created ahead of time, and each new participant is assigned to the next condition in the sequence as he or she is tested. When the procedure is computerized, the computer program often handles the random assignment.

One problem with coin flipping and other strict procedures for random assignment is that they are likely to result in unequal sample sizes in the different conditions. Unequal sample sizes are generally not a serious problem, and you should never throw away data you have already collected to achieve equal sample sizes. However, for a fixed number of participants, it is statistically most efficient to divide them into equal-sized groups. It is standard practice, therefore, to use a kind of modified random assignment that keeps the number of participants in each group as similar as possible. One approach is block randomization . In block randomization, all the conditions occur once in the sequence before any of them is repeated. Then they all occur again before any of them is repeated again. Within each of these “blocks,” the conditions occur in a random order. Again, the sequence of conditions is usually generated before any participants are tested, and each new participant is assigned to the next condition in the sequence.  Table 6.2  shows such a sequence for assigning nine participants to three conditions. The Research Randomizer website will generate block randomization sequences for any number of participants and conditions. Again, when the procedure is computerized, the computer program often handles the block randomization.

Random assignment is not guaranteed to control all extraneous variables across conditions. It is always possible that just by chance, the participants in one condition might turn out to be substantially older, less tired, more motivated, or less depressed on average than the participants in another condition. However, there are some reasons that this possibility is not a major concern. One is that random assignment works better than one might expect, especially for large samples. Another is that the inferential statistics that researchers use to decide whether a difference between groups reflects a difference in the population takes the “fallibility” of random assignment into account. Yet another reason is that even if random assignment does result in a confounding variable and therefore produces misleading results, this confound is likely to be detected when the experiment is replicated. The upshot is that random assignment to conditions—although not infallible in terms of controlling extraneous variables—is always considered a strength of a research design.

Treatment and Control Conditions

Between-subjects experiments are often used to determine whether a treatment works. In psychological research, a  treatment  is any intervention meant to change people’s behaviour for the better. This  intervention  includes psychotherapies and medical treatments for psychological disorders but also interventions designed to improve learning, promote conservation, reduce prejudice, and so on. To determine whether a treatment works, participants are randomly assigned to either a  treatment condition , in which they receive the treatment, or a control condition , in which they do not receive the treatment. If participants in the treatment condition end up better off than participants in the control condition—for example, they are less depressed, learn faster, conserve more, express less prejudice—then the researcher can conclude that the treatment works. In research on the effectiveness of psychotherapies and medical treatments, this type of experiment is often called a randomized clinical trial .

There are different types of control conditions. In a  no-treatment control condition , participants receive no treatment whatsoever. One problem with this approach, however, is the existence of placebo effects. A  placebo  is a simulated treatment that lacks any active ingredient or element that should make it effective, and a  placebo effect  is a positive effect of such a treatment. Many folk remedies that seem to work—such as eating chicken soup for a cold or placing soap under the bedsheets to stop nighttime leg cramps—are probably nothing more than placebos. Although placebo effects are not well understood, they are probably driven primarily by people’s expectations that they will improve. Having the expectation to improve can result in reduced stress, anxiety, and depression, which can alter perceptions and even improve immune system functioning (Price, Finniss, & Benedetti, 2008) [1] .

Placebo effects are interesting in their own right (see  Note “The Powerful Placebo” ), but they also pose a serious problem for researchers who want to determine whether a treatment works.  Figure 6.2  shows some hypothetical results in which participants in a treatment condition improved more on average than participants in a no-treatment control condition. If these conditions (the two leftmost bars in  Figure 6.2 ) were the only conditions in this experiment, however, one could not conclude that the treatment worked. It could be instead that participants in the treatment group improved more because they expected to improve, while those in the no-treatment control condition did not.

Fortunately, there are several solutions to this problem. One is to include a placebo control condition , in which participants receive a placebo that looks much like the treatment but lacks the active ingredient or element thought to be responsible for the treatment’s effectiveness. When participants in a treatment condition take a pill, for example, then those in a placebo control condition would take an identical-looking pill that lacks the active ingredient in the treatment (a “sugar pill”). In research on psychotherapy effectiveness, the placebo might involve going to a psychotherapist and talking in an unstructured way about one’s problems. The idea is that if participants in both the treatment and the placebo control groups expect to improve, then any improvement in the treatment group over and above that in the placebo control group must have been caused by the treatment and not by participants’ expectations. This  difference  is what is shown by a comparison of the two outer bars in  Figure 6.2 .

Of course, the principle of informed consent requires that participants be told that they will be assigned to either a treatment or a placebo control condition—even though they cannot be told which until the experiment ends. In many cases the participants who had been in the control condition are then offered an opportunity to have the real treatment. An alternative approach is to use a waitlist control condition , in which participants are told that they will receive the treatment but must wait until the participants in the treatment condition have already received it. This disclosure allows researchers to compare participants who have received the treatment with participants who are not currently receiving it but who still expect to improve (eventually). A final solution to the problem of placebo effects is to leave out the control condition completely and compare any new treatment with the best available alternative treatment. For example, a new treatment for simple phobia could be compared with standard exposure therapy. Because participants in both conditions receive a treatment, their expectations about improvement should be similar. This approach also makes sense because once there is an effective treatment, the interesting question about a new treatment is not simply “Does it work?” but “Does it work better than what is already available?

The Powerful Placebo

Many people are not surprised that placebos can have a positive effect on disorders that seem fundamentally psychological, including depression, anxiety, and insomnia. However, placebos can also have a positive effect on disorders that most people think of as fundamentally physiological. These include asthma, ulcers, and warts (Shapiro & Shapiro, 1999) [2] . There is even evidence that placebo surgery—also called “sham surgery”—can be as effective as actual surgery.

Medical researcher J. Bruce Moseley and his colleagues conducted a study on the effectiveness of two arthroscopic surgery procedures for osteoarthritis of the knee (Moseley et al., 2002) [3] . The control participants in this study were prepped for surgery, received a tranquilizer, and even received three small incisions in their knees. But they did not receive the actual arthroscopic surgical procedure. The surprising result was that all participants improved in terms of both knee pain and function, and the sham surgery group improved just as much as the treatment groups. According to the researchers, “This study provides strong evidence that arthroscopic lavage with or without débridement [the surgical procedures used] is not better than and appears to be equivalent to a placebo procedure in improving knee pain and self-reported function” (p. 85).

Within-Subjects Experiments

In a within-subjects experiment , each participant is tested under all conditions. Consider an experiment on the effect of a defendant’s physical attractiveness on judgments of his guilt. Again, in a between-subjects experiment, one group of participants would be shown an attractive defendant and asked to judge his guilt, and another group of participants would be shown an unattractive defendant and asked to judge his guilt. In a within-subjects experiment, however, the same group of participants would judge the guilt of both an attractive and an unattractive defendant.

The primary advantage of this approach is that it provides maximum control of extraneous participant variables. Participants in all conditions have the same mean IQ, same socioeconomic status, same number of siblings, and so on—because they are the very same people. Within-subjects experiments also make it possible to use statistical procedures that remove the effect of these extraneous participant variables on the dependent variable and therefore make the data less “noisy” and the effect of the independent variable easier to detect. We will look more closely at this idea later in the book.  However, not all experiments can use a within-subjects design nor would it be desirable to.

Carryover Effects and Counterbalancing

The primary disad vantage of within-subjects designs is that they can result in carryover effects. A  carryover effect  is an effect of being tested in one condition on participants’ behaviour in later conditions. One type of carryover effect is a  practice effect , where participants perform a task better in later conditions because they have had a chance to practice it. Another type is a fatigue effect , where participants perform a task worse in later conditions because they become tired or bored. Being tested in one condition can also change how participants perceive stimuli or interpret their task in later conditions. This  type of effect  is called a  context effect . For example, an average-looking defendant might be judged more harshly when participants have just judged an attractive defendant than when they have just judged an unattractive defendant. Within-subjects experiments also make it easier for participants to guess the hypothesis. For example, a participant who is asked to judge the guilt of an attractive defendant and then is asked to judge the guilt of an unattractive defendant is likely to guess that the hypothesis is that defendant attractiveness affects judgments of guilt. This  knowledge  could lead the participant to judge the unattractive defendant more harshly because he thinks this is what he is expected to do. Or it could make participants judge the two defendants similarly in an effort to be “fair.”

Carryover effects can be interesting in their own right. (Does the attractiveness of one person depend on the attractiveness of other people that we have seen recently?) But when they are not the focus of the research, carryover effects can be problematic. Imagine, for example, that participants judge the guilt of an attractive defendant and then judge the guilt of an unattractive defendant. If they judge the unattractive defendant more harshly, this might be because of his unattractiveness. But it could be instead that they judge him more harshly because they are becoming bored or tired. In other words, the order of the conditions is a confounding variable. The attractive condition is always the first condition and the unattractive condition the second. Thus any difference between the conditions in terms of the dependent variable could be caused by the order of the conditions and not the independent variable itself.

There is a solution to the problem of order effects, however, that can be used in many situations. It is  counterbalancing , which means testing different participants in different orders. For example, some participants would be tested in the attractive defendant condition followed by the unattractive defendant condition, and others would be tested in the unattractive condition followed by the attractive condition. With three conditions, there would be six different orders (ABC, ACB, BAC, BCA, CAB, and CBA), so some participants would be tested in each of the six orders. With counterbalancing, participants are assigned to orders randomly, using the techniques we have already discussed. Thus random assignment plays an important role in within-subjects designs just as in between-subjects designs. Here, instead of randomly assigning to conditions, they are randomly assigned to different orders of conditions. In fact, it can safely be said that if a study does not involve random assignment in one form or another, it is not an experiment.

An efficient way of counterbalancing is through a Latin square design which randomizes through having equal rows and columns. For example, if you have four treatments, you must have four versions. Like a Sudoku puzzle, no treatment can repeat in a row or column. For four versions of four treatments, the Latin square design would look like:

There are two ways to think about what counterbalancing accomplishes. One is that it controls the order of conditions so that it is no longer a confounding variable. Instead of the attractive condition always being first and the unattractive condition always being second, the attractive condition comes first for some participants and second for others. Likewise, the unattractive condition comes first for some participants and second for others. Thus any overall difference in the dependent variable between the two conditions cannot have been caused by the order of conditions. A second way to think about what counterbalancing accomplishes is that if there are carryover effects, it makes it possible to detect them. One can analyze the data separately for each order to see whether it had an effect.

When 9 is “larger” than 221

Researcher Michael Birnbaum has argued that the lack of context provided by between-subjects designs is often a bigger problem than the context effects created by within-subjects designs. To demonstrate this problem, he asked participants to rate two numbers on how large they were on a scale of 1-to-10 where 1 was “very very small” and 10 was “very very large”.  One group of participants were asked to rate the number 9 and another group was asked to rate the number 221 (Birnbaum, 1999) [4] . Participants in this between-subjects design gave the number 9 a mean rating of 5.13 and the number 221 a mean rating of 3.10. In other words, they rated 9 as larger than 221! According to Birnbaum, this difference is because participants spontaneously compared 9 with other one-digit numbers (in which case it is relatively large) and compared 221 with other three-digit numbers (in which case it is relatively small) .

Simultaneous Within-Subjects Designs

So far, we have discussed an approach to within-subjects designs in which participants are tested in one condition at a time. There is another approach, however, that is often used when participants make multiple responses in each condition. Imagine, for example, that participants judge the guilt of 10 attractive defendants and 10 unattractive defendants. Instead of having people make judgments about all 10 defendants of one type followed by all 10 defendants of the other type, the researcher could present all 20 defendants in a sequence that mixed the two types. The researcher could then compute each participant’s mean rating for each type of defendant. Or imagine an experiment designed to see whether people with social anxiety disorder remember negative adjectives (e.g., “stupid,” “incompetent”) better than positive ones (e.g., “happy,” “productive”). The researcher could have participants study a single list that includes both kinds of words and then have them try to recall as many words as possible. The researcher could then count the number of each type of word that was recalled. There are many ways to determine the order in which the stimuli are presented, but one common way is to generate a different random order for each participant.

Between-Subjects or Within-Subjects?

Almost every experiment can be conducted using either a between-subjects design or a within-subjects design. This possibility means that researchers must choose between the two approaches based on their relative merits for the particular situation.

Between-subjects experiments have the advantage of being conceptually simpler and requiring less testing time per participant. They also avoid carryover effects without the need for counterbalancing. Within-subjects experiments have the advantage of controlling extraneous participant variables, which generally reduces noise in the data and makes it easier to detect a relationship between the independent and dependent variables.

A good rule of thumb, then, is that if it is possible to conduct a within-subjects experiment (with proper counterbalancing) in the time that is available per participant—and you have no serious concerns about carryover effects—this design is probably the best option. If a within-subjects design would be difficult or impossible to carry out, then you should consider a between-subjects design instead. For example, if you were testing participants in a doctor’s waiting room or shoppers in line at a grocery store, you might not have enough time to test each participant in all conditions and therefore would opt for a between-subjects design. Or imagine you were trying to reduce people’s level of prejudice by having them interact with someone of another race. A within-subjects design with counterbalancing would require testing some participants in the treatment condition first and then in a control condition. But if the treatment works and reduces people’s level of prejudice, then they would no longer be suitable for testing in the control condition. This difficulty is true for many designs that involve a treatment meant to produce long-term change in participants’ behaviour (e.g., studies testing the effectiveness of psychotherapy). Clearly, a between-subjects design would be necessary here.

Remember also that using one type of design does not preclude using the other type in a different study. There is no reason that a researcher could not use both a between-subjects design and a within-subjects design to answer the same research question. In fact, professional researchers often take exactly this type of mixed methods approach.

Key Takeaways

• Experiments can be conducted using either between-subjects or within-subjects designs. Deciding which to use in a particular situation requires careful consideration of the pros and cons of each approach.
• Random assignment to conditions in between-subjects experiments or to orders of conditions in within-subjects experiments is a fundamental element of experimental research. Its purpose is to control extraneous variables so that they do not become confounding variables.
• Experimental research on the effectiveness of a treatment requires both a treatment condition and a control condition, which can be a no-treatment control condition, a placebo control condition, or a waitlist control condition. Experimental treatments can also be compared with the best available alternative.
• You want to test the relative effectiveness of two training programs for running a marathon.
• Using photographs of people as stimuli, you want to see if smiling people are perceived as more intelligent than people who are not smiling.
• In a field experiment, you want to see if the way a panhandler is dressed (neatly vs. sloppily) affects whether or not passersby give him any money.
• You want to see if concrete nouns (e.g.,  dog ) are recalled better than abstract nouns (e.g.,  truth ).
• Discussion: Imagine that an experiment shows that participants who receive psychodynamic therapy for a dog phobia improve more than participants in a no-treatment control group. Explain a fundamental problem with this research design and at least two ways that it might be corrected.
• Price, D. D., Finniss, D. G., & Benedetti, F. (2008). A comprehensive review of the placebo effect: Recent advances and current thought. Annual Review of Psychology, 59 , 565–590. ↵
• Shapiro, A. K., & Shapiro, E. (1999). The powerful placebo: From ancient priest to modern physician . Baltimore, MD: Johns Hopkins University Press. ↵
• Moseley, J. B., O’Malley, K., Petersen, N. J., Menke, T. J., Brody, B. A., Kuykendall, D. H., … Wray, N. P. (2002). A controlled trial of arthroscopic surgery for osteoarthritis of the knee. The New England Journal of Medicine, 347 , 81–88. ↵
• Birnbaum, M.H. (1999). How to show that 9>221: Collect judgments in a between-subjects design. Psychological Methods, 4(3), 243-249. ↵

An experiment in which each participant is only tested in one condition.

A method of controlling extraneous variables across conditions by using a random process to decide which participants will be tested in the different conditions.

All the conditions of an experiment occur once in the sequence before any of them is repeated.

Any intervention meant to change people’s behaviour for the better.

A condition in a study where participants receive treatment.

A condition in a study that the other condition is compared to. This group does not receive the treatment or intervention that the other conditions do.

A type of experiment to research the effectiveness of psychotherapies and medical treatments.

A type of control condition in which participants receive no treatment.

A simulated treatment that lacks any active ingredient or element that should make it effective.

A positive effect of a treatment that lacks any active ingredient or element to make it effective.

Participants receive a placebo that looks like the treatment but lacks the active ingredient or element thought to be responsible for the treatment’s effectiveness.

Participants are told that they will receive the treatment but must wait until the participants in the treatment condition have already received it.

Each participant is tested under all conditions.

An effect of being tested in one condition on participants’ behaviour in later conditions.

Participants perform a task better in later conditions because they have had a chance to practice it.

Participants perform a task worse in later conditions because they become tired or bored.

Being tested in one condition can also change how participants perceive stimuli or interpret their task in later conditions.

Testing different participants in different orders.

Research Methods in Psychology - 2nd Canadian Edition Copyright © 2015 by Paul C. Price, Rajiv Jhangiani, & I-Chant A. Chiang is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

5.1 Experiment Basics

Learning objectives.

• Explain what an experiment is and recognize examples of studies that are experiments and studies that are not experiments.
• Distinguish between the manipulation of the independent variable and control of extraneous variables and explain the importance of each.
• Recognize examples of confounding variables and explain how they affect the internal validity of a study.

What Is an Experiment?

As we saw earlier in the book, an  experiment  is a type of study designed specifically to answer the question of whether there is a causal relationship between two variables. In other words, whether changes in an independent variable  cause  a change in a dependent variable. Experiments have two fundamental features. The first is that the researchers manipulate, or systematically vary, the level of the independent variable. The different levels of the independent variable are called conditions . For example, in Darley and Latané’s experiment, the independent variable was the number of witnesses that participants believed to be present. The researchers manipulated this independent variable by telling participants that there were either one, two, or five other students involved in the discussion, thereby creating three conditions. For a new researcher, it is easy to confuse  these terms by believing there are three independent variables in this situation: one, two, or five students involved in the discussion, but there is actually only one independent variable (number of witnesses) with three different levels or conditions (one, two or five students). The second fundamental feature of an experiment is that the researcher controls, or minimizes the variability in, variables other than the independent and dependent variable. These other variables are called extraneous variables . Darley and Latané tested all their participants in the same room, exposed them to the same emergency situation, and so on. They also randomly assigned their participants to conditions so that the three groups would be similar to each other to begin with. Notice that although the words  manipulation  and  control  have similar meanings in everyday language, researchers make a clear distinction between them. They manipulate  the independent variable by systematically changing its levels and control  other variables by holding them constant.

Manipulation of the Independent Variable

Again, to  manipulate  an independent variable means to change its level systematically so that different groups of participants are exposed to different levels of that variable, or the same group of participants is exposed to different levels at different times. For example, to see whether expressive writing affects people’s health, a researcher might instruct some participants to write about traumatic experiences and others to write about neutral experiences. As discussed earlier in this chapter, the different levels of the independent variable are referred to as  conditions , and researchers often give the conditions short descriptive names to make it easy to talk and write about them. In this case, the conditions might be called the “traumatic condition” and the “neutral condition.”

Notice that the manipulation of an independent variable must involve the active intervention of the researcher. Comparing groups of people who differ on the independent variable before the study begins is not the same as manipulating that variable. For example, a researcher who compares the health of people who already keep a journal with the health of people who do not keep a journal has not manipulated this variable and therefore has not conducted an experiment. This distinction  is important because groups that already differ in one way at the beginning of a study are likely to differ in other ways too. For example, people who choose to keep journals might also be more conscientious, more introverted, or less stressed than people who do not. Therefore, any observed difference between the two groups in terms of their health might have been caused by whether or not they keep a journal, or it might have been caused by any of the other differences between people who do and do not keep journals. Thus the active manipulation of the independent variable is crucial for eliminating potential alternative explanations for the results.

Of course, there are many situations in which the independent variable cannot be manipulated for practical or ethical reasons and therefore an experiment is not possible. For example, whether or not people have a significant early illness experience cannot be manipulated, making it impossible to conduct an experiment on the effect of early illness experiences on the development of hypochondriasis. This caveat does not mean it is impossible to study the relationship between early illness experiences and hypochondriasis—only that it must be done using nonexperimental approaches. We will discuss this type of methodology in detail later in the book.

Independent variables can be manipulated to create two conditions and experiments involving a single independent variable with two conditions is often referred to as a  single factor two-level design.  However, sometimes greater insights can be gained by adding more conditions to an experiment. When an experiment has one independent variable that is manipulated to produce more than two conditions it is referred to as a single factor multi level design.  So rather than comparing a condition in which there was one witness to a condition in which there were five witnesses (which would represent a single-factor two-level design), Darley and Latané’s used a single factor multi-level design, by manipulating the independent variable to produce three conditions (a one witness, a two witnesses, and a five witnesses condition).

Control of Extraneous Variables

As we have seen previously in the chapter, an  extraneous variable  is anything that varies in the context of a study other than the independent and dependent variables. In an experiment on the effect of expressive writing on health, for example, extraneous variables would include participant variables (individual differences) such as their writing ability, their diet, and their gender. They would also include situational or task variables such as the time of day when participants write, whether they write by hand or on a computer, and the weather. Extraneous variables pose a problem because many of them are likely to have some effect on the dependent variable. For example, participants’ health will be affected by many things other than whether or not they engage in expressive writing. This influencing factor can make it difficult to separate the effect of the independent variable from the effects of the extraneous variables, which is why it is important to  control  extraneous variables by holding them constant.

Extraneous Variables as “Noise”

Extraneous variables make it difficult to detect the effect of the independent variable in two ways. One is by adding variability or “noise” to the data. Imagine a simple experiment on the effect of mood (happy vs. sad) on the number of happy childhood events people are able to recall. Participants are put into a negative or positive mood (by showing them a happy or sad video clip) and then asked to recall as many happy childhood events as they can. The two leftmost columns of  Table 5.1 show what the data might look like if there were no extraneous variables and the number of happy childhood events participants recalled was affected only by their moods. Every participant in the happy mood condition recalled exactly four happy childhood events, and every participant in the sad mood condition recalled exactly three. The effect of mood here is quite obvious. In reality, however, the data would probably look more like those in the two rightmost columns of  Table 5.1 . Even in the happy mood condition, some participants would recall fewer happy memories because they have fewer to draw on, use less effective recall strategies, or are less motivated. And even in the sad mood condition, some participants would recall more happy childhood memories because they have more happy memories to draw on, they use more effective recall strategies, or they are more motivated. Although the mean difference between the two groups is the same as in the idealized data, this difference is much less obvious in the context of the greater variability in the data. Thus one reason researchers try to control extraneous variables is so their data look more like the idealized data in  Table 5.1 , which makes the effect of the independent variable easier to detect (although real data never look quite  that  good).

One way to control extraneous variables is to hold them constant. This technique can mean holding situation or task variables constant by testing all participants in the same location, giving them identical instructions, treating them in the same way, and so on. It can also mean holding participant variables constant. For example, many studies of language limit participants to right-handed people, who generally have their language areas isolated in their left cerebral hemispheres. Left-handed people are more likely to have their language areas isolated in their right cerebral hemispheres or distributed across both hemispheres, which can change the way they process language and thereby add noise to the data.

In principle, researchers can control extraneous variables by limiting participants to one very specific category of person, such as 20-year-old, heterosexual, female, right-handed psychology majors. The obvious downside to this approach is that it would lower the external validity of the study—in particular, the extent to which the results can be generalized beyond the people actually studied. For example, it might be unclear whether results obtained with a sample of younger heterosexual women would apply to older homosexual men. In many situations, the advantages of a diverse sample (increased external validity) outweigh the reduction in noise achieved by a homogeneous one.

Extraneous Variables as Confounding Variables

The second way that extraneous variables can make it difficult to detect the effect of the independent variable is by becoming confounding variables. A confounding variable  is an extraneous variable that differs on average across  levels of the independent variable (i.e., it is an extraneous variable that varies systematically with the independent variable). For example, in almost all experiments, participants’ intelligence quotients (IQs) will be an extraneous variable. But as long as there are participants with lower and higher IQs in each condition so that the average IQ is roughly equal across the conditions, then this variation is probably acceptable (and may even be desirable). What would be bad, however, would be for participants in one condition to have substantially lower IQs on average and participants in another condition to have substantially higher IQs on average. In this case, IQ would be a confounding variable.

To confound means to confuse , and this effect is exactly why confounding variables are undesirable. Because they differ systematically across conditions—just like the independent variable—they provide an alternative explanation for any observed difference in the dependent variable.  Figure 5.1  shows the results of a hypothetical study, in which participants in a positive mood condition scored higher on a memory task than participants in a negative mood condition. But if IQ is a confounding variable—with participants in the positive mood condition having higher IQs on average than participants in the negative mood condition—then it is unclear whether it was the positive moods or the higher IQs that caused participants in the first condition to score higher. One way to avoid confounding variables is by holding extraneous variables constant. For example, one could prevent IQ from becoming a confounding variable by limiting participants only to those with IQs of exactly 100. But this approach is not always desirable for reasons we have already discussed. A second and much more general approach—random assignment to conditions—will be discussed in detail shortly.

Figure 5.1 Hypothetical Results From a Study on the Effect of Mood on Memory. Because IQ also differs across conditions, it is a confounding variable.

Key Takeaways

• An experiment is a type of empirical study that features the manipulation of an independent variable, the measurement of a dependent variable, and control of extraneous variables.
• An extraneous variable is any variable other than the independent and dependent variables. A confound is an extraneous variable that varies systematically with the independent variable.
• Practice: List five variables that can be manipulated by the researcher in an experiment. List five variables that cannot be manipulated by the researcher in an experiment.
• Effect of parietal lobe damage on people’s ability to do basic arithmetic.
• Effect of being clinically depressed on the number of close friendships people have.
• Effect of group training on the social skills of teenagers with Asperger’s syndrome.
• Effect of paying people to take an IQ test on their performance on that test.

Share This Book

• Increase Font Size

Home » Blog » Comprehensive Guide to Research Methodology – Design | Methods | Best Practices

Comprehensive Guide to Research Methodology – Design | Methods | Best Practices

• Other Laws | Blog |
• 19 Min Read
• Last Updated on 16 September, 2024

Recent Posts

Other Laws, Blog

Responsible Business Conduct and ESG in India – Best Practices | Emerging Regulation

News, Blog, Account & Audit

[Opinion] Internal Financial Control—Need to Focus?

Latest from taxmann.

Table of Contents

• Introduction
• Steps in Research Process
• Classification of Research Design

1. Introduction

This article describes the research process and different research designs in detail. Management and social science research, like other forms of scientific inquiry, require a structured sequence of highly interrelated steps (Zigmund et al., 2010). The research process involves a series of steps or actions essential for the smooth conduct of any research. The figure below illustrates the sequence of the research process. It is to be noted that these steps are not a road map to all kinds of research. Basically, it is applicable for deductive or functionalist research, and it can or needs to be revised to suit the requirements of a specific project. The research process doesn’t need to be followed successively; rather, the steps overlap frequently and are interrelated. The research process offers a comprehensive guideline that can be referred to for any management and social science research. It may happen that later stages can be accomplished before the earlier stages.

The steps involved in the research process are neither mutually exclusive nor separate and distinct. The selection of a research topic at the outset, defining the research problem and objectives, influences the selection of a sample and data collection. The sample selection may affect the design of questionnaire items. For example, suppose an organization wants to know the cause of attrition among lower-category employees with low educational qualifications. In that case, the wording for the questionnaire will be easier than for people in top management positions with professional educational qualifications. The steps may differ based on the objectives of the research. However, research based on deductive logic should follow the steps outlined below:

2. Steps in Research Process

• Problem Identification
• Literature Review
• Formulating Research Questions
• Research Design
• Data Collection
• Data Analysis
• Conclusions and Report Writing.

The quest for research must always be triggered by the longing to explore and gain more knowledge and understanding. The management dilemma encourages the need for a decision. The need may arise owing to the cause that the researchers want to discover or reestablish certain relationships. The orientation might be to solve immediate management issues, discover something new, or have purely academic intentions. For instance, in an organization, the manager may want to know the reason for high attrition and lack of job satisfaction, or a retail store may survey the post-purchase satisfaction among the customers.

2.1 Research Problem Identification

Defining the research problem is the first step in the research process. The researchers get the proper direction to conduct their research by first understanding the research problems. Hence, a well-defined research problem is crucial. When the problem is discovered, researchers and management can take further steps to define the problem clearly and precisely. A problem defined with accuracy and conscience helps the researchers utilize the available resources effectively. It is imperative for researchers to explore what exactly is the problem and what are the objectives of the research. The rule generally followed to define the research problem is that the definition should permit the researchers to acquire all details required to address the managerial issues and show guidelines for finding a solution. The researcher should be careful not to define the problem too broadly or narrowly. Examples of broad managerial problems are defining a strategy for enhancing organizational performance and a strategy to elevate the organization’s brand equity. An example of a narrow definition of a problem is how to match competitors’ recruitment strategies. To overcome the possibility of both errors while defining the research problem, the researchers must define the problem with broad, popular terms and devise its components. The broad general statement helps the researchers get a sound perspective on the research problem and avoid the error of defining the problem narrowly. On the other side, the specific component helps to identify the key aspects of the research problem, extend a transparent guideline to proceed further and avoid the error of defining the problem too broadly. In management and social science research, broad management problems need to be converted to information-oriented research problems that focus more on the cause than the symptoms. Some examples of managerial problems converted to research problems are presented in Table below. The conversion of management dilemma to managerial questions and further to research questions can be carried out through exploratory research. Such research incorporates an examination of past research studies, a review of extant literature and organizational records and interviewing experts (Cooper et al., 2016).

2.2 Literature Review

Exploring the existing literature is critical in the research process. Researchers must explore and investigate extant literature to observe whether other researchers have already addressed the identified research problem. A literature review is a systematic search of published work, including periodicals, books, journal papers (conceptual and empirical), and reports, representing theory and empirical work about the research problem and topic at hand. A survey of existing literature is customary in applied research and is an elementary requirement of a basic research report. The internet, electronic databases, websites, and e-library help the researcher to carry out literature surveys systematically and easily.

The literature review aims to study the existing state of knowledge in the domain of interest, to picture key authors, theories, methods, topics, and findings in that domain, and to explore the gaps in knowledge in that domain. A literature review conducted systematically reveals whether initial research questions have already gained substantial attention in the extant literature, whether more interesting newer research questions are available, whether past studies have consistent findings or contradictions exist, flaws in the body of research that the researchers can address, and whether the initial research questions need to be revised as per the findings of the literature review. Furthermore, the review can answer the proposed research questions and help identify theories used in previous studies to address similar research questions. For example, for an organization interested in determining the true cause of turnover, the researcher will study extensively the existing literature on attrition and its causes. By studying relevant journal articles, books, and book chapters, the researcher will discover the causes of attrition in general, find out the existing gaps, and suggest the management carry forward the research to find causes specific to the organization.

As deductive research primarily involves theory testing, the researchers must identify one or more theories that can illuminate the proposed research questions. Through an extensive literature review, researchers may uncover various concepts and constructs related to the phenomenon of interest. A theory will extend support to constructs/variables that are logically relevant to the chosen phenomenon. In the deductive approach, researchers use theory/theories as the logical basis for hypothesis testing. However, researchers must carefully select the theories appropriate for the identified problem to be studied. The hypotheses need to be logically formulated and connected to the research objectives.

2.3 Formulating Research Questions

After problem identification and clarification, with or without an exploratory research approach, the researchers should derive the research objectives. Cautious attention to problem definition helps the researchers devise proper research objectives. Research objectives are the goal to be achieved through research. The research objective drives the research process further. A well-devised research objective enhances the possibility of gathering, relevant information and avoiding unwanted information. The research objectives can be properly developed with the consensus of the researchers and management on the actual managerial and business problems. The researcher should ensure that the research objectives are clearly stated, appropriate, and will yield germane information. The research objective may involve exploring the likelihood of venturing into a new market or may necessitate examining the effect of a new organizational policy on employee performance. The nature and types of objectives lead to choosing an appropriate research design.

Research Objectives:  Research objectives represent the goal of the research the researchers want to accomplish.

2.3.1 Suitable Research Questions

Research questions are important to conduct effective research. Without a clear research question, the researcher may face the risk of unfocused research and will not be sure of what the research is about. Research questions are refined descriptions of the components of the research problem. These are questions related to behavior, events or phenomena of interest that the researchers search for answers in their research. Examples include what factors motivate the employees in an organization to apply the gained knowledge back to their jobs or what needs to be done to enhance the creativity of school-going students. Research questions can best state the objectives of the research. Each component of the research problem needs to be broken down into sub-parts or research questions. Research questions inquire about the information essential concerning the problem components. Properly answered research questions will lead to effective decision-making. While formulating research questions, researchers should be guided by the problem statement, theoretical background, and analytical framework.

Sources of Research Questions

• Extant Literature
• Personal experience
• Societal issues
• Managerial problems
• New theories
• Technological advancement
• Empirical cases
• Contradictory finding

2.3.1.1 Significance of Research Questions

Research questions are critical because they guide scientific and systematic literature search, the decision about appropriate research design, the decision about data collection and target audience, data analysis, selection of right tools and techniques and overall to move in the right direction.

The researcher can utilize different sources for formulating research questions, such as extant literature, personal experience, societal issues, managerial problems, new theories, technological advancement, and contradictory findings. The research question must portray certain attributes. Research questions in quantitative research are more specific compared to qualitative research. Sometimes, some qualitative research follows an open approach without any research questions. The main steps involved in formulating research questions are illustrated in Figure below.

Criteria of Effective Research Questions

• Rateability
• Systematic and logical
• Significant
• Fascinating
• Logical association among variables

The sequence in selecting research questions suggests that the researchers are engrossed in a process of progressive focusing down when developing the research questions. It helps them to slide down from the general research area to research questions. While formulating the research questions, the researchers should understand that ending a research question with a question mark is essential. Without a question mark, a statement cannot be considered as a research question. It is quite possible that the researchers may not get answers to all research questions. The research questions need to be related to each other.

2.4 Planning the Research Design

After formulating research problems and literature surveys, the next stage in the research process is to develop the research design. Research design is the blueprint of research activities to answer research questions. It is a master plan that includes research methods and procedures for gathering and analyzing the relevant information with minimum cost, time, and effort. A research design extends a plan for carrying out the research. The researchers need to decide the source to collect information, the techniques of research design (survey or experiment), sampling techniques, and the cost and schedule of the research. The success of these objectives depends on the purpose of the research. Usually, research purposes are segregated into four types: exploration, description, diagnosis, and experimentation.

There are varied designs, such as experimental or non-experimental hypotheses testing (details of different research designs are outlined in section 2.3 in this chapter). There are four primary research methods for descriptive and causal research: survey, experiments, secondary data, and observations. The selection of an appropriate research method relies on the research objectives, available data sources, the cost and effort of collecting data, and the importance of managerial decisions. If the research objective is exploration, a flexible research design can extend better opportunities to investigate different aspects of the research problem. On the other hand, if the intention is simply to describe any situation or phenomena of interest to examine the relationship between two or more variables, the appropriate design should prioritize minimizing bias and maximizing reliability in data collection and analysis. For example, suppose a researcher wants to conduct exploratory research to know the different types of arthritis common in India. In that case, it may require a flexible design relying on secondary data from hospital records or discussions with doctors or other experts to reach conclusions. However, to invent COVID-vaccination and medicine for the COVID-19 virus, the researchers conducted varied experiments to reach a conclusion.

2.4.1 Hypotheses Development

Exploratory research helps the researchers define the research questions, key variables, and theoretical underpinnings and formulate hypotheses if required in the research. The hypotheses must be logically derived based on the research questions and linked to research objectives. A hypothesis is a tentative proposition regarding a research phenomenon. It may be a tentative statement that indicates an association between two or more variables, guided by any supportive theory, theoretical framework, or analytical model. It is a viable answer to the research questions framed by the researchers. Hypotheses are statements of relationships or propositions that are declarative and can be tested with empirical data. Some examples are:

H 1 : Training influences organizational performance.

H 2 : Training enhances employee performance.

For two more research questions i.e., “to what extent does brand love determine purchase intention?” and “does age and family background moderate the relationship?”, the hypotheses are:

H 1 : Brand love is related to purchase intention.

H 2 : Age and Family status moderate the association between brand love and purchase intention. Figure below provides a pictorial representation of the hypotheses drawn.

However, it is not always feasible for researchers to formulate hypotheses in all situations. Sometimes, researchers may lack all relevant information, and theoretical support may not be available to formulate the hypotheses.

2.5 Sampling Design

This stage of the research process involves an investigation of the population under study. A complete investigation of the population under study is known as a census inquiry. Usually, in census investigation, all units or items of the population are studied with high accuracy and reliability. However, it is usually not practicable and feasible for the researchers to study the entire population. Researchers usually prefer to investigate small, representative subgroups from the population known as sample. The procedure to select the sub-groups/samples is called sampling design. Sampling entails the process of drawing conclusions based on a subgroup of the population. Hence, the sample is a subset of the population. The first question that needs to be addressed in sampling is “who is to be included in the sample?” and this requires the identification of the target population under study. It is difficult for the researcher to define the population and sampling unit. For example, if a researcher wants to investigate the financial savings and vehicle loan association survey. In that case, individuals with existing accounts will be taken, and this sample unit represents the existing customers and not the potential customers. Hence, it is critical in sampling design to determine the specific target population.

Secondly, the issue that concerns the researchers in sampling design is selecting an appropriate sample size, and the third concern is selecting the sampling units. Researchers need to address these concerns to justify the research. Samples can be selected either using probability sampling techniques or non-probability sampling techniques. There are four types of probability sampling such as simple random, systematic, stratified, and cluster sampling. Non-probability sampling includes convenience, judgmental, quota, and snowball sampling. Depending on the objective, researchers should select the appropriate sampling techniques for their study.

2.6 Fieldwork and Gathering Data

After the formalization of the sampling plan, the fieldwork and data-gathering stage begins. The researcher gathers data after finalizing what to research, among whom, and which method to use. Data gathering involves the process of information collection. Different data collection instruments are available for researchers to collect information or data. Broadly, there are two ways to collect data, such as primary and secondary data collection methods. Primary data include data collected firsthand and are original. Varied methods are available for primary data collection, such as structured and unstructured interviews, focused group discussion, observation, and survey using a structured questionnaire. The data can be collected offline or online. Secondary data included information collected from published or unpublished sources that were already available. Some secondary data collection sources are articles, magazines, company records, expert opinion survey data, feedback of customers, government data, and past research on the subject. For example, to conduct a survey of job satisfaction in an organization, the researcher may circulate a printed questionnaire offline or mail the questionnaire to the selected respondents following an appropriate sampling technique.

Another example could be a study that investigates the purchase preference for luxury cars, and the base model demands primary and empirical information. However, another study that intended to describe the financial investment behavior of existing customers will use secondary data. At this stage, the researchers need to ensure the reliability and validity of the data obtained for the study.

2.7 Data Processing and Analysis

After data gathering, the data needs to be converted or properly coded to answer the research question under study. The information gathered in the data collection phase should be mined from the primary raw data. Data processing starts with data editing, coding, and tabulation. First, it is vital for the researchers to check the data collection forms for missing data, clarity, and consistency in categorization. The editing process involves problems associated with data, such as respondents’ response errors. Editing improves the quality of the data and makes the data usable for tabulation, analysis, and interpretation. Tabulation is a technical process in which classified data are presented in tables. Researchers use computers to feed data to a computer spreadsheet for data analysis. The preparation of a spreadsheet also requires lots of expertise and experience.

After coding the data, the next step is to analyze the data. Data analysis is the utilization of reasoning to make sense of data gathered. Ample statistical techniques are available for the researchers to analyze the data. Based on the research questions, objectives, study types, sampling framework used, data types, and degree of accuracy involved in the research, one can choose from parametric or non-parametric techniques for data analysis. Researchers may adopt univariate, bi-variate or multi-variate methods for data analysis. The analysis may include simple frequency analysis, multiple regression, or structural equation modeling. Different techniques are available for qualitative data, presented in Part 3 of this book.

2.8 Drawing Conclusion and Preparing a Report

After data analysis, the final stage in the research process is the interpretation of the results. The researcher requires analytical skills to interpret the statistical results, link the output with the research objectives, and state the implications of the result.

Research Design:  Research design is the blueprint/systematic steps to carry out research smoothly

Finally, researchers must communicate the result in the form of a report. The preparation of the final report needs to be done with the utmost care. The final report should include the identified research questions, research approach, data collection method, data analysis techniques, study findings, and implications for theory and practice. The structure of the report will be discussed in the last section of this book. The report should be prepared comprehensively to be usable by management or organizations for decision-making.

3. Classification of Research Design

This section highlights the classification of research design. As mentioned in the previous section, research design is the framework for carrying out management and other research. After the identification of a problem, the researchers formulate the research design. A good research design ensures the effectiveness of the research work. The choice of selecting an appropriate design relies on the research objectives. The broad categorization of research design with sub-categorization is detailed in various sub-sections.

3.1 Exploratory Research Design

Methods to Conduct Exploratory Research

• Literature survey
• Secondary sources of data
• Experience survey
• Focused group discussions
• Observations
• Structured and unstructured interviews
• Pilot surveys
• Case Studies

Exploratory research design is the simplest form of research design. The researchers explore the true nature of the problem. When researchers aim to study a new area or examine a new interest, exploratory design is a good option. This research design is flexible and versatile in approach. The information required by the researchers is defined loosely and unstructured. Researchers carrying out qualitative research usually adopt exploratory research design. Exploratory research design serves three purposes (a) it helps the researchers to address their inquisitiveness and quest for better understanding (b) to assess the practicality of carrying out border research (c) and devise methods for further studies.

Methods to Conduct Descriptive Research

• Self-administered survey
• Phone survey
• Mail survey/online survey
• Observation
• Personal interview
• Telephone interview

Exploratory research design has paramount significance in management and social science research. They are crucial for researchers who want to study something new. To cite an example, during the COVID-19 pandemic, physical health, mental health, and safety of school and college-going children were a concern for most people. The online education system was the new normal at that time. Research studying the impact of digitalization, long time spent in online studies on students’ health and mental well-being during the COVID-19 pandemic, is of an exploratory kind. One of the disadvantages of exploratory research design is that researchers rarely get specific answers to the research questions.

3.2 Descriptive Research Design

The prime objective of descriptive research design is to describe certain situations or events. This type of design provides an extensive explanation of the research phenomena under study. In descriptive research, the researchers possess prior knowledge about the problem situations. The information is defined with clarity. This kind of research is preplanned and more structured than exploratory research. Researchers must formulate research questions properly and have clarity regarding the types of data needed and the procedure to be followed to achieve the research objectives. Researchers have the luxury of covering a large representative sample. Researchers must answer five Ws and one H – what, who, when, where, why, and how of research issues. What kind of information is required for the research, who are the target respondents, when the information will be collected, where to interact with the respondents, why information is collected from the respondents and how to collect data from the respondents. Descriptive research studies can be cross-sectional or longitudinal. The major objectives for the following descriptive research are given below.

• To explain the characteristics of certain groups such as the Indian population, employees, students, marketing personnel, organizations, sales persons. For example, a university to design a customized online higher studies course for working professionals needs a holistic profile of the interested population.
• To evaluate the portion of individuals in a specific population portraying a typical behavior. For instance, when a researcher is inclined to know the percentage of employees not interested in an online platform introduced for them in their organization.
• To predict for future. For instance, to know the future of physical retail stores due to the widespread expansion of online stores.
• To examine the extent to which management research variables relate to each other. For example, to what extent does work-life balance, salary, and conducive work environment enhance employee job satisfaction?

3.3 Causal Research Design

Usually, causal research design is adopted by researchers to explain causal relationships among phenomena under study. Causal research examines cause-and-effect relationships among variables. Causal research has certain criteria, as already discussed in Chapter 1. Causal research follows a planned and structured design like descriptive research. Though the magnitude of the relationship among variables is examined in descriptive research, the causal association cannot be explained through such research. Experimentation is one of the methods for carrying out causal research.

In causal research, the researchers usually examine the impact of one variable on another. The researchers try to explore the cause-and-effect relationship (nomothetic explanation). How can the researcher know whether cause and effect are associated? There are three criteria for a nomothetic causal relationship when (1) two or more variables are correlated, (2) the cause precedes the effect and (3) the absence of a plausible alternative explanation for the effect other than the proposed cause (Babbie, 2020). First, without establishing a correlation among two or more variables, causation cannot exist. Second, the cause should happen before the effect in time. For instance, it is more sensible to say that children’s religious affiliation is caused by their parents than to reflect that parents’ religious affiliation is due to children; even in some cases, it is plausible that children may convert to other religions later with their parent’s permission. The third significant condition for a causal relationship is that the effect cannot be attributed to any external third variable for establishing causation.

To cite one classic example, there is a causal association between sales of ice cream and death owing to drowning. Intake of more ice creams in summer does lead to a higher death rate due to drowning. The third intervening variable that causes higher death is season or temperature. In summer, higher deaths occur due to swimming and not because of taking ice-creams. The intervening variable season or temperature causes a higher death rate.

To establish a reliable causal relationship among two or more variables, other influencing variables must be controlled to neutralize their impact on the studied variables. For example, to study the effect of factors influencing training transfer in soft skill training, the other intervening variables such as age, gender, and educational qualification need to be controlled. This kind of research sometimes demands experimentation to establish causality. In most cases, causal research is quantitative and needs statistical hypothesis testing.

3.4 Experimental Research Design

Experimental research aims to examine the cause-effect relationship in a controlled setting by isolating the cause from the effect in time. The three criteria suggested by John Stuart Mill mirror in experimental research. In experimental research, the cause is administered to one group of subjects, known as the treatment group and not to the control group, and the researchers observe the difference in mean effect among the subjects of both groups. Whether variation in the cause is connected to variation in effect is observed. To be more specific, the researcher manipulates the independent variable and examines the change in the dependent variable, keeping other variables constant. Researchers used varied methods during the experiments to reduce the plausible effect of other explanations for the effect, along with ancillary methods to investigate the plausibility of those that cannot be ruled out. It is vital in experimental studies to control the extraneous and confounding variables while carrying out the experiments. Ignorance of such variables may lead to spurious relationships among studied variables. However, bringing many of the variables under experimental control is impossible. For example, personal characteristics of the subject like age, sex, intelligence, beliefs and persona. In such cases, the researchers must observe natural variations in the variables of concern. Then, statistical procedures are used to rule out the plausible impact of uncontrolled factors.

Experimental Research Design:  An experiment is a method of collecting evidence to indicate the effect of one variable on another.

Experimental research design can be conducted in a laboratory setting (laboratory experiment) or in a field setting (field experiments) where the phenomena of research interest happen. As an example, one of the most talked about and controversial experiments conducted on understanding human behavior has been the Stanford Prison Experiments, which took place at Stanford University in 1971. The experiments were funded by the US Office of Naval Research, and the principal investigator for the same was Prof Phillip Zimbardo. The major purpose of these experiments was to understand how norms develop and social expectations about roles shape group behavior. Experimental studies are segregated into four categories such as pre-experimental, true-experimental, quasi-experimental and statistical design.

3.4.1 Correlation, Causation and Cofounds

Correlation cannot be treated as causation, and correlation does not always prove causation. In correlation, it is unclear which variable comes first or whether any alternative explanation exists for the assumed effect. Two variables may be correlated due to chance. Correlation is symmetric, while causation is asymmetric. Two variables may be co-related, but their relationship may be affected by a third variable called cofounds. For example, let’s say that high salary and high educational qualifications are correlated. It is difficult to say with confirmation which comes first. Whether a high educational qualification leads to a high salary, or a high salary leads to a high educational qualification. Both possibilities can hold true and necessitate further investigation. Until researchers conclude through their investigation, a mere correlation among these two variables will not give a clear picture of their causal relationship. There is also the possibility of an alternative explanation for the relationship between high salary and high educational qualifications. The link may be due to a third variable called intellect, which results in high salary and high educational qualifications.

In management research, social science, and natural science, three significant pairs of components are required for experimentation: Experimental and control group, independent and dependent variable, and pre-test and post-test.

3.4.1.1 Experimental and Control Group

The group in which an experimental treatment is administered is known as the experimental or treatment group. In contrast, the group in which no experiment is administered is known as the control group. Using control groups enables the researchers to assess the experiment’s effects. For example, suppose a researcher wants to study the impact of rewards on employee productivity in an organization. In that case, the researcher can experiment with two groups of employees. One group will be given external rewards, known as the experimental group, and the other group (control group) will provide no external rewards. Then, the researcher can investigate the causal association between rewards on employees’ productivity through this experiment. The use of a control group is quite common in medical science research. In social science and management research, the use of control groups and experimental studies became popular with several experiments conducted in the late 1920s and early 1930s by F. J. Roethlisberger and W. J. Dickson (1939) to discover the changes required in working conditions to enhance employee satisfaction and productivity. Their series of experiments resulted in the Hawthorne effect.

3.4.1.2 Independent and Dependent Variables

In experimental research, the researchers study the impact of an independent variable on the dependent variable. Usually, experimental stimuli, whether present or absent, are considered independent variables. Independent variables are manipulated in the study, and their effects are assessed and compared. The researchers compare outcomes when the stimulus is present and not present. Hence, the independent variable is the cause, and the dependent variable is the presumed effect. It is to be noted that the independent variable in one study may serve as a dependent variable in another study. For example, an experiment intends to explore the causality between high salary and job satisfaction, job satisfaction is the dependent variable. However, in another experiment designed to explore the causality between job satisfaction and employee productivity, job satisfaction is the independent variable.

3.4.1.3 Pre- and Post-test

In an experiment, the experimenters measure the variable before conducting the experiment on the group known as the pre-test and measure the variable after conducting the experiments is called as post-test. Hence, subjects are exposed to a stimulus called a dependent variable (pre-testing), then exposed to a stimulus, i.e., independent variable, and again assessed with a dependent variable (post-testing). Any discrepancies between the two measurements of dependent variables are ascribed to the independent variable.

Disclaimer: The content/information published on the website is only for general information of the user and shall not be construed as legal advice. While the Taxmann has exercised reasonable efforts to ensure the veracity of information/content published, Taxmann shall be under no liability in any manner whatsoever for incorrect information, if any.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

PREVIOUS POST

To subscribe to our weekly newsletter please log in/register on Taxmann.com

Latest books.

R.K. Jain's Customs Tariff of India | Set of 2 Volumes

R.K. Jain's Customs Law Manual | 2023-24 | Set of 2 Volumes

R.K. Jain's GST Law Manual | 2023-24

R.K. Jain's GST Tariff of India | 2023-24

Everything on Tax and Corporate Laws of India

Everything you need on Tax & Corporate Laws. Authentic Databases, Books, Journals, Practice Modules, Exam Platforms, and More.

• Express Delivery | Secured Payment
• Free Shipping in India on order(s) above ₹500
• +91 8688939939
• About Company
• Media Coverage
• Union Budget 2024-25

• Business & Support
• Sell with Taxmann
• Locate Dealers
• Locate Representatives
• CD Key Activation
• Privacy Policy
• Return Policy
• Payment Terms

The impact of mining projects on household livelihoods in the Benishangul-Gumuz Region, Ethiopia

• Original Paper
• Published: 12 September 2024

Cite this article

• Netsanet Zeleke Haile   ORCID: orcid.org/0009-0006-5831-2762 1 ,
• Admassu Tesso Huluka   ORCID: orcid.org/0000-0002-1946-0977 2 &
• Amsalu Bedemo Beyene   ORCID: orcid.org/0000-0002-0317-0844 3  

The recent changes in mining policy in Ethiopia have led to a significant increase in mining investments in regions rich in minerals. One such region is the Benishangul Gumuz regional (BGR) state, which has attracted both domestic and foreign investors interested in extracting coal, marble, and gold. However, there is a lack of studies on how mining projects in Ethiopia, particularly in the BGR state, affect the livelihoods of households. This study aims to address this gap by examining the impact of mining projects on household livelihoods in the BGR. To achieve this, we employed quasi-experimental and cross-sectional research designs and collected primary data from 333 households. This included 162 households located near the mining projects (treated households) and 171 households in areas without mining projects (control households). For the analysis, we utilized principal component analysis (PCA) and propensity score matching (PSM) techniques. We used the sustainable livelihood framework (SLF) as the conceptual and analytical tool, focusing on the five livelihood assets as the outcome variable of the study. The results of the Average Treatment Effect on the Treated (ATT) revealed that mining has a significantly positive impact on the financial and physical assets of households. However, it also has a notable negative impact on human, natural, and social capitals, which are crucial for the sustainable development of rural households. Therefore, the research recommends that government institutions and policymakers should address these challenges using a comprehensive approach that balances the benefits of mining with the preservation and enhancement of necessary community assets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Impacts of mining projects in Papua New Guinea on livelihoods and poverty in indigenous mining communities

Sustainable livelihoods in artisanal small-scale mining communities: a case study of tarkwa-nsuaem municipality of ghana.

Mining-induced displacement and resettlement in Ghana: an assessment of the prospects and challenges in selected mining communities

Adach-Pawelus K, Gogolewska A, Górniak-Zimroz J, Herbert JH, Hidalgo A, Kiełczawa B, Krupa-Kurzynowska J, Lampinen M, Mamelkina MA, Paszkowska G, Szyszka D, Tuunila R, Worsa-Kozak M, Woźniak J (2020) Towards sustainable mining in the didactic process—meitim project as an opportunity to increase the attractiveness of mining courses (a case study of Poland). Sustain (Switzerland) 12(23):1–18. https://doi.org/10.3390/su122310138

Article   Google Scholar  

Agariga F, Abugre S, Siabi EK, Appiah M (2021) Mining impact on Livelihoods of Farmers of Asutifi North District, Ghana. Environ Manage Sustainable Dev 10(4):29. https://doi.org/10.5296/emsd.v10i4.19066

Alemu Y, Tolossa D (2022) Livelihood impacts of large-Scale Agricultural Investments using empirical evidence from Shashamane Rural District of Oromia Region, Ethiopia. Sustain (Switzerland) 14(15). https://doi.org/10.3390/su14159082

Andrews T, Elizalde B, Billon P, Le, Hoon OC, Reyes D, Thomson L (2016) The rise in Conflict Associated with Mining Operations. What Lies Beneath?

Asare KY, Mensah JV, Agyenim JB (2024) Sustainability of alternative livelihood strategies in selected sand mining communities in the Ga South Municipality and Gomoa East District of Ghana. Cogent Social Sci 10(1). https://doi.org/10.1080/23311886.2024.2340436

Ascher W (2022) Claremont McKenna College Rural Development in Papua New Guinea: Mining, Logging, Agriculture, and Alternatives Submitted to

Assan JK, Muhammed A-R (2018) The impact of mining on farming as a livelihood strategy and its implications for poverty reduction and household well-being in Ghana. Int J Dev Sustain 7(1):1–20. https://isdsnet.com/ijds-v7n1-01.pdf

Baddianaah I, Tuu GN, Baatuuwie BN (2021) Livelihood implications of artisanal gold mining in farming communities: insight from the Wa East District, Ghana. Ghana J Geogr 13(3). https://doi.org/10.4314/gjg.v13i3.5

Bajwa SK (2015) A Study of Status of Livelihood Assets at Household Level: Evidence from Saidpur Village . December , 1–29. https://assets.publishing.service.gov.uk/media/57a08d3c40f0b64974001734/R7850_Majale_RGUU1_Review.pdf

Balcha D, Oyda S (2021) Assessing the impact of Stone Quarrying on the Landscape, Crop and Grazing Land Degradation in Loma Bosa District, Dawuro Zone, Southern Ethiopia. Int J Sci Res Civil Eng 5(2):79–91

Google Scholar  

Béné C, Riba A, Wilson D (2018) Measuring Changes in Resilience as a Result of the SUR1M Project in Niger . 1–47. https://itad.com/wp-content/uploads/2018/05/DFID-BRACED_EA3-Impact-Evaluation_Niger_Final-for-DFID.pdf

Béné C, Riba A, Wilson D (2020) Impacts of resilience interventions– evidence from a quasi-experimental assessment in Niger. Int J Disaster Risk Reduct 43(February):101390. https://doi.org/10.1016/j.ijdrr.2019.101390

Benishangul Gumuz Region Bureau of Finance and Economic Development (BGR, & BoFED) (2022) Benishangul Gumuz Regional State Physical and Socio-Economic Profile . May , 1–130

Benishangul Gumuz Finance Economy and Developemnt Office (2020) The population Projection of the Benishangul Gumuz Region for Budglocation

Benishangul Gumuz Region Mining Agency (BGRMA) (2022) Mining Anual Report

Bro R, Smilde AK (2014) Principal component analysis. Anal Methods 6(9):2812–2831. https://doi.org/10.1039/c3ay41907j

Caliendo M, Kopeing S (2005) Some practical Guidance for the implementation of propensity score matching. J Agric Sci 1588(4):313–325. https://doi.org/10.1017/S0021859620000593

Central Statistical Authority (2016) Demographic and Health Survey. In Central Statistical Agency Population, Ethiopia . https://doi.org/10.1109/ipfa.2004.1345625

Chambers R, Conway G (2014) Sustainable rural livelihoods: practical concepts for the 21st century . September . https://www.researchgate.net/publication/248535825%0ASustainable

Chen Y, Hu Z, Li P, Li G, Yuan D, Guo J (2022) Assessment and Effect of Mining Subsidence on Farmland in coal–crop overlapped areas: a case of Shandong Province, China. Agric (Switzerland) 12(8):1–17. https://doi.org/10.3390/agriculture12081235

Cochran WG (1977) Sampling techniques . John Wiley & Sons. https://entermediadb.sumitomodrive.com/assets/mediadb/services/module/asset/downloads/originals/2018/11/23/c3137ff81/AGMA_Technical_Paper_Sumitomo_Cycloidal_Vibration_Analysis.pdf/AGMA_Technical_Paper_Sumitomo_Cycloidal_Vibration_Analysis.pdf

Cordes KY, Östensson O, Toledano P, CCSI (2016) Employment from Mining and Agricultural Investments how much myth, how much reality? Issue July. https://scholarship.law.columbia.edu/sustainable_investment_staffpubs/88/

Department for International Development (DFID) (1999) Sustainable Livelihoods Guidance Sheets 2 . https://www.ennonline.net/attachments/872/section2.pdf

Department for International Development (DfID) (1999) Sustainable livelihoods Guidance sheets introduction: overview. Sustainable Livelihoods Guidance Sheets, 10

Dinc I, Sigdel M, Dinc S, Sigdel MS, Pusey ML, Aygun RS (2014) Evaluation of normalization and PCA on the performance of classifiers for protein crystallization images. Conference Proceedings - IEEE SOUTHEASTCON , 1–19. https://doi.org/10.1109/SECON.2014.6950744

Djigsa WK (2021) Towards socially responsible mining investment in Ethiopia: imagining a New Moral Economy. Rev Public Adminstration Manage 9(6):1–6. https://www.walshmedicalmedia.com/open-access/towards-socially-responsible-mining-investment-in-ethiopia-imagining-a-new-moral-economy.pdf

Doso SJ, Cieem G, Ayensu-ntim A, Twumasi-ankrah B, Barimah PT (2015) Effects of loss of Agricultural Land due to large-scale gold mining on Agriculture in Ghana: the case of the Western Region. Br J Res 2(6):196–221

Federal Democratic Republic of Ethiopia Planning and Development Commission (FDREPDC) (2021) Ten years development plan: a pathway to prosperity. 86 pp. https://www.ircwash.org/sites/default/files/ten_year_development_plan_a_pathway_to_prosperity.2021-2030_version.pdf

Gbedzi DD, Ofosu EA, Mortey EM, Obiri-Yeboah A, Nyantakyi EK, Siabi EK, Abdallah F, Domfeh MK, Amankwah-Minkah A (2022) Impact of mining on land use land cover change and water quality in the Asutifi North District of Ghana, West Africa. Environ Challenges 6(August 2021). https://doi.org/10.1016/j.envc.2022.100441

Gharbi I, Kammoun A (2023) Developing a multidimensional financial inclusion index: a comparison based on income groups. J Risk Financial Manage 16(6):296. https://doi.org/10.3390/jrfm16060296

Guyalo AK, Alemu EA, Degaga DT (2022) Impact of large-scale agricultural investments on the food security status of local community in Gambella region, Ethiopia. Agric Food Secur 11(1):1–29. https://doi.org/10.1186/s40066-022-00381-6

Hair J, Anderson R, Babin B, Black W (2010) Multivariate Data Analysis. Australia: Cengage: Vol. 7 edition. Multivariate Data Analysis.pdf. https://www.drnishikantjha.com/papersCollection/

Hassen AE (2022) Competency oriented management for sustainable artisanal and small-scale mining sector development in Ethiopia. Nat Resour Forum 46(1):116–135. https://doi.org/10.1111/1477-8947.12246

Hilson G (2019) Why is there a large-scale mining ‘bias’ in sub-saharan Africa? Land Use Policy 81:852–861. https://doi.org/10.1016/j.landusepol.2017.02.013

Horsley J, Prout S, Tonts M, Ali SH (2015) Sustainable livelihoods and indicators for regional development in mining economies. Extractive Industries Soc 2(2):368–380. https://doi.org/10.1016/j.exis.2014.12.001

Idemudia U, Tuokuu FXD, Essah M (2022) The extractive industry and human rights in Africa: lessons from the past and future directions. Resour Policy 78(March):102838. https://doi.org/10.1016/j.resourpol.2022.102838

International Growth Centre (IGC) (2018) The Mining Debate: Let’s Talk About Inclusive Growth (Vol. 1, Issue 1). http://www.ghana.gov.gh/index.php/media-center/features/1031-the-mining-debate-let-s-talk-about-inclusive-growth

Irene WM, Raphael GW, Daniel WI (2021) Impact of mining on environment: a case study of Taita Taveta County, Kenya. Afr J Environ Sci Technol 15(5):202–213. https://doi.org/10.5897/ajest2020.2926

Jima AO (2021) Maintaining mutual benefits between Investor and Local Community: mechanisms to Reopen Lega Dambi Gold Mining, Ethiopia. PanAfrican J Gov Dev, 2 (1). https://journals.ju.edu.et/index.php/panjogov/article/view/2913

Jolliffe PM (2002) Principal Component Anal. In Prisons and Forced Labour in Japan (Second). Springer. https://doi.org/10.4324/9781351206358-1

Krantz L (2001) Sustainable Livelihood Approach to Poverty Reduction . January 2001 , 1–38. https://doi.org/10.1007/978-3-319-45623-2_9

Kuir-Ayius DD (2016) Building community resilience in mine impacted communities: A study on delivery of health services in Papua New Guinea . April , 1–461. https://docs.google.com/open?id=0B2PU844i_g5sYzIyNzMzMzItOWI2ZS00M2U3LWE2MTQtM2MxN2M3MzUzZjBh

Leuenberger A, Winkler MS, Cambaco O, Cossa H, Kihwele F, Lyatuu I, Zabre HR, Farnham A, MacEte E, Munguambe K (2021) Health impacts of industrial mining on surrounding communities: local perspectives from three sub-saharan African countries. PLoS ONE 16(6 June):1–23. https://doi.org/10.1371/journal.pone.0252433

Lin Y, He C (2021) Evaluation of livelihood sustainability in the context of natural forest land degradation vulnerability: a case study of five counties in China. Sustain (Switzerland) 13(12):1–16. https://doi.org/10.3390/su13126580

Ma J, Zhang J, Li L, Zeng Z, Sun J, Zhou Q, Bill, Zhang Y (2018) Study on livelihood assets-based spatial differentiation of the income of natural tourism communities. Sustain (Switzerland) 10(2):1–20. https://doi.org/10.3390/su10020353

Maciejewska A, Kuzak Ł, Sobieraj J, Metelski D (2022) The impact of Opencast Lignite Mining on Rural Development: A literature review and selected Case studies using desk research, Panel Data and GIS-Based analysis. Energies 15(15). https://doi.org/10.3390/en15155402

Ministry of Mines and Energy [MME] (2009) National Report on Mining to the United Nations Commission on Sustainable Development (UNCSD). In Ministry of Mines and Energy . http://www.mme.gov.na/directorates/energies/

Mishra A (2017) Methods of impact evaluation: a review. SSRN Electron J May 2018 https://doi.org/10.2139/ssrn.2943601

Mojarradi G, Rezaei R, Ketabi A (2016) Negative impacts of mine exploitations on rural regions of Tekab Township. JME J Min Environ 7(1):57–66

Mononen T, Sihvonen J, Sairinen R, Tiainen H (2023) Local governance of the mining industry—five Finnish examples. Resour Policy 82(March):103478. https://doi.org/10.1016/j.resourpol.2023.103478

Moreda T (2017) Large-scale land acquisitions, state authority and indigenous local communities: insights from Ethiopia. Third World Q 38(3):698–716. https://doi.org/10.1080/01436597.2016.1191941

Moritz T, Ejdemo T, Söderholm P, Wårell L (2017) The local employment impacts of mining: an econometric analysis of job multipliers in northern Sweden. Mineral Econ 30(1):53–65. https://doi.org/10.1007/s13563-017-0103-1

Moyo S, Jha P, Yeros P (2019) The Scramble for Land and Natural resources in Africa. Advances in African Economic, Social and Political Development. Springer Singapore. https://doi.org/10.1007/978-981-10-5840-0_1

Mtero F (2017) Rural livelihoods, large-scale mining and agrarian change in Mapela, Limpopo, South Africa. Resour Policy 53(June):190–200. https://doi.org/10.1016/j.resourpol.2017.06.015

Muñoz-Duque LA, Pérez Osorno MM, Betancur Vargas A (2020) Spoliation, socio-environmental conflicts and human rights violation implications of large scale mining in Latin America. Revista U D C Actualidad Divulgación Científica 23(1):1–10. https://doi.org/10.31910/rudca.v23.n1.2020.988

Mwakesi I, Wahome R, Ichang’i D (2020) Mining impact on communities’ livelihoods: a case study of Taita Taveta county, Kenya. AIMS Environ Sci 7(3):286–301. https://doi.org/10.3934/environsci.2020018

Nalule VR (2019) Mining and the law in Africa: Exploring the social and environmental impacts. In Mining and the Law in Africa: Exploring the Social and Environmental Impacts . https://doi.org/10.1007/978-3-030-33008-8

Narula SA, Magray MA, Desore A (2021) A sustainable livelihood framework to implement CSR project in coal mining sector a sustainable livelihood framework to implement CSR project in coal mining sector. J Sustainable Min 16(3):83–93. https://doi.org/10.1016/j.jsm.2017.10.001

Nunoo S, Manu J, Owusu-Akyaw FKB, Nyame FK (2022) Impact of artisanal small-scale (gold and diamond) mining activities on the Offin, Oda and Pra rivers in Southern Ghana, West Africa: a scientific response to public concern. Heliyon 8(12):e12323. https://doi.org/10.1016/j.heliyon.2022.e12323

Okyere M, Ayitey JZ, Ajabuin BA (2021) Large scale mining in Ghana: a review of the implications on the host communities. J Degraded Min Lands Manage 9(1):3193–3199. https://doi.org/10.15243/JDMLM.2021.091.3193

Perry M, Rowe JE (2015) Fly-in, fly-out, drive-out: the Australian mining boom and its impacts on the local economy. Local Econ 30(1):139–148. https://doi.org/10.1177/0269094214564957

Regassa A (2021) Frontiers of extraction and Contestation: dispossession, exclusion and local resistance against MIDROC Laga-Dambi Gold Mine, southern Ethiopia. Extractive Industries Soc xxxx:100980. https://doi.org/10.1016/j.exis.2021.100980

Rela IZ, Awang AH, Ramli Z, Taufik Y, Md. Sum S, Muhammad M (2020) Effect of corporate social responsibility on community resilience: empirical evidence in the nickel mining industry in Southeast Sulawesi, Indonesia. Sustain (Switzerland) 12(4). https://doi.org/10.3390/su12041395

Rogers J, Révész A (2020) Experimental and quasi-experimental designs. Phys Rev B (Issue 111:133–143. https://www.researchgate.net/publication/334250281%0AExperimental

Scoones I (1999) Sustainable Rural Livelihoods Aframework for Analysis. In IDS Working Paper 72 (Vol. 42, Issue 2). https://doi.org/10.1057/palgrave.development.1110037

Serrat O (2008) The sustainable livelihoods Approach. Knowl Solutions 15(November):21–26. https://doi.org/10.1007/978-981-10-0983-9_5

Sherrieb K, Norris FH, Galea S (2016) Measuring Capacities for Community Resilience Author (s): Kathleen Sherrieb, Fran H. Norris and Sandro Galea Published by: Springer Stable URL : http://www.jstor.org/stable/40927589 REFERENCES Linked references are available on JSTOR for this article . 99 (2), 227–247

Singh S, Nayak S (2020) Development of sustainable livelihood security index for different agro-climatic zones of Uttar Pradesh, India. J Rural Dev 39(1):110–129. https://doi.org/10.25175/jrd/2020/v39/i1/125991

Stewart AG (2020) Mining is bad for health: a voyage of discovery. Environ Geochem Health 42(4):1153–1165. https://doi.org/10.1007/s10653-019-00367-7

Stokes-Walters R, Fofana ML, Songbono JL, Barry AO, Diallo S, Nordhagen S, Zhang LX, Klemm RD, Winch PJ (2021) If you don’t find anything, you can’t eat - Mining livelihoods and income, gender roles, and food choices in northern Guinea. Resour Policy 70(December 2020). https://doi.org/10.1016/j.resourpol.2020.101939

Teklemariam D, Azadi H, Nyssen J, Haile M, Witlox F (2016) How sustainable is transnational farmland acquisition in Ethiopia? Lessons learned from the Benishangul-Gumuz Region. Sustain (Switzerland) 8(3):1–27. https://doi.org/10.3390/su8030213

Thavaneswaran A, Lix L (2008) Propensity score matching in Observational studies. Centre for Manitoba Health Policy

The World Bank Group (2017) Mining in Africa Are local Communities Better off? In Africa Development forunm . https://doi.org/10.1596/978-1-4648-0819-7

Tura HA (2017) Linking Land rights and the right to adequate food in Ethiopia: normative and implementation gaps. Nordic J Hum Rights 35(2):85–105. https://doi.org/10.1080/18918131.2017.1312860

United Nations Developemnt Programme [UNDP] (2015) Application of the Sustainable Livelihoods Framework in Developemnt Projects

van Rijn F, Burger K, den Belder E (2012) Impact assessment in the sustainable Livelihood Framework. Dev Pract 22(7):1019–1035. https://doi.org/10.1080/09614524.2012.696586

Vasco B-K, Amos M, Victor O (2018) Impact of small-scale mining activities on the livelihoods assets of Rural households in the Bekwai Municipality, Ghana. Help Ensure Our Sustain 1(3):1–22. https://ageconsearch.umn.edu/record/277792?ln=en

Weldegiorgis FS, Dietsche E, Ahmad S (2023) Inter-sectoral economic linkages in the Mining industries of Botswana and Tanzania: analysis using partial hypothetical extraction method. Resources 12(7):78. https://doi.org/10.3390/resources12070078

White H, Raitzer DA (2017) Impact Evaluation of Development Interventions: A Practical Guide. In Asian Development Bank . https://www.adb.org/sites/default/files/publication/392376/impact-evaluation-development-interventions-guide.pdf

White H, Sabarwal S (2014) Quasi-Experimental Design and Methods . 1–14. https://www.unicef-irc.org/KM/IE/img/downloads/Quasi-Experimental_Design_and_Methods_ENG.pdf

Yamarak L, Parton KA (2021) Impacts of mining projects in Papua New Guinea on livelihoods and poverty in indigenous mining communities. Mineral Econ 2001. https://doi.org/10.1007/s13563-021-00284-1

Zainuddin Rela I, Firihu MZ, Awang AH, Iswandi M, Malek JA, Nikoyan A, Nalepo L, Batoa H, Salahuddin S (2021) Formation of farming community resilience models for sustainable agricultural development at the mining neighborhood in Southeast Sulawesi Indonesia. Sustain (Switzerland) 13(2):1–17. https://doi.org/10.3390/su13020878

Download references

Acknowledgements

We would like to acknowledge the Ethiopian Civil Service University for funding us our data collection.

The authors declare that no funds, grants, or other supports were received during the preparation of this manuscript.

Author information

Authors and affiliations.

Department of Public Management, Ethiopian Civil Service University, Addis Ababa, Ethiopia

Netsanet Zeleke Haile

Department of Development Management, Ethiopian Civil Service University, Addis Ababa, Ethiopia

Admassu Tesso Huluka

Department of Policy Studies, Ethiopian Civil Service University, Addis Ababa, Ethiopia

Amsalu Bedemo Beyene

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed equally to this work. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Netsanet Zeleke Haile .

Ethics declarations

Conflict of interest.

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Haile, N.Z., Huluka, A.T. & Beyene, A.B. The impact of mining projects on household livelihoods in the Benishangul-Gumuz Region, Ethiopia. Miner Econ (2024). https://doi.org/10.1007/s13563-024-00471-w

Download citation

Received : 14 November 2023

Accepted : 02 September 2024

Published : 12 September 2024

DOI : https://doi.org/10.1007/s13563-024-00471-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Mining projects
• Sustainable livelihood framework
• Propensity score matching
• Benishangul-Gumuz region
• Find a journal
• Publish with us
• Track your research