non experimental research papers

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

7.1 Overview of Nonexperimental Research

Learning objectives.

Define nonexperimental research, distinguish it clearly from experimental research, and give several examples.
Explain when a researcher might choose to conduct nonexperimental research as opposed to experimental research.

What Is Nonexperimental Research?

Nonexperimental research is research that lacks the manipulation of an independent variable, random assignment of participants to conditions or orders of conditions, or both.

In a sense, it is unfair to define this large and diverse set of approaches collectively by what they are not . But doing so reflects the fact that most researchers in psychology consider the distinction between experimental and nonexperimental research to be an extremely important one. This is because while experimental research can provide strong evidence that changes in an independent variable cause differences in a dependent variable, nonexperimental research generally cannot. As we will see, however, this does not mean that nonexperimental research is less important than experimental research or inferior to it in any general sense.

When to Use Nonexperimental Research

As we saw in Chapter 6 “Experimental Research” , experimental research is appropriate when the researcher has a specific research question or hypothesis about a causal relationship between two variables—and it is possible, feasible, and ethical to manipulate the independent variable and randomly assign participants to conditions or to orders of conditions. It stands to reason, therefore, that nonexperimental research is appropriate—even necessary—when these conditions are not met. There are many ways in which this can be the case.

The research question or hypothesis can be about a single variable rather than a statistical relationship between two variables (e.g., How accurate are people’s first impressions?).
The research question can be about a noncausal statistical relationship between variables (e.g., Is there a correlation between verbal intelligence and mathematical intelligence?).
The research question can be about a causal relationship, but the independent variable cannot be manipulated or participants cannot be randomly assigned to conditions or orders of conditions (e.g., Does damage to a person’s hippocampus impair the formation of long-term memory traces?).
The research question can be broad and exploratory, or it can be about what it is like to have a particular experience (e.g., What is it like to be a working mother diagnosed with depression?).

Again, the choice between the experimental and nonexperimental approaches is generally dictated by the nature of the research question. If it is about a causal relationship and involves an independent variable that can be manipulated, the experimental approach is typically preferred. Otherwise, the nonexperimental approach is preferred. But the two approaches can also be used to address the same research question in complementary ways. For example, nonexperimental studies establishing that there is a relationship between watching violent television and aggressive behavior have been complemented by experimental studies confirming that the relationship is a causal one (Bushman & Huesmann, 2001). Similarly, after his original study, Milgram conducted experiments to explore the factors that affect obedience. He manipulated several independent variables, such as the distance between the experimenter and the participant, the participant and the confederate, and the location of the study (Milgram, 1974).

Types of Nonexperimental Research

Nonexperimental research falls into three broad categories: single-variable research, correlational and quasi-experimental research, and qualitative research. First, research can be nonexperimental because it focuses on a single variable rather than a statistical relationship between two variables. Although there is no widely shared term for this kind of research, we will call it single-variable research . Milgram’s original obedience study was nonexperimental in this way. He was primarily interested in one variable—the extent to which participants obeyed the researcher when he told them to shock the confederate—and he observed all participants performing the same task under the same conditions. The study by Loftus and Pickrell described at the beginning of this chapter is also a good example of single-variable research. The variable was whether participants “remembered” having experienced mildly traumatic childhood events (e.g., getting lost in a shopping mall) that they had not actually experienced but that the research asked them about repeatedly. In this particular study, nearly a third of the participants “remembered” at least one event. (As with Milgram’s original study, this study inspired several later experiments on the factors that affect false memories.)

As these examples make clear, single-variable research can answer interesting and important questions. What it cannot do, however, is answer questions about statistical relationships between variables. This is a point that beginning researchers sometimes miss. Imagine, for example, a group of research methods students interested in the relationship between children’s being the victim of bullying and the children’s self-esteem. The first thing that is likely to occur to these researchers is to obtain a sample of middle-school students who have been bullied and then to measure their self-esteem. But this would be a single-variable study with self-esteem as the only variable. Although it would tell the researchers something about the self-esteem of children who have been bullied, it would not tell them what they really want to know, which is how the self-esteem of children who have been bullied compares with the self-esteem of children who have not. Is it lower? Is it the same? Could it even be higher? To answer this question, their sample would also have to include middle-school students who have not been bullied.

Research can also be nonexperimental because it focuses on a statistical relationship between two variables but does not include the manipulation of an independent variable, random assignment of participants to conditions or orders of conditions, or both. This kind of research takes two basic forms: correlational research and quasi-experimental research. In correlational research , the researcher measures the two variables of interest with little or no attempt to control extraneous variables and then assesses the relationship between them. A research methods student who finds out whether each of several middle-school students has been bullied and then measures each student’s self-esteem is conducting correlational research. In quasi-experimental research , the researcher manipulates an independent variable but does not randomly assign participants to conditions or orders of conditions. For example, a researcher might start an antibullying program (a kind of treatment) at one school and compare the incidence of bullying at that school with the incidence at a similar school that has no antibullying program.

The final way in which research can be nonexperimental is that it can be qualitative. The types of research we have discussed so far are all quantitative, referring to the fact that the data consist of numbers that are analyzed using statistical techniques. In qualitative research , the data are usually nonnumerical and are analyzed using nonstatistical techniques. Rosenhan’s study of the experience of people in a psychiatric ward was primarily qualitative. The data were the notes taken by the “pseudopatients”—the people pretending to have heard voices—along with their hospital records. Rosenhan’s analysis consists mainly of a written description of the experiences of the pseudopatients, supported by several concrete examples. To illustrate the hospital staff’s tendency to “depersonalize” their patients, he noted, “Upon being admitted, I and other pseudopatients took the initial physical examinations in a semipublic room, where staff members went about their own business as if we were not there” (Rosenhan, 1973, p. 256).

Internal Validity Revisited

Recall that internal validity is the extent to which the design of a study supports the conclusion that changes in the independent variable caused any observed differences in the dependent variable. Figure 7.1 shows how experimental, quasi-experimental, and correlational research vary in terms of internal validity. Experimental research tends to be highest because it addresses the directionality and third-variable problems through manipulation and the control of extraneous variables through random assignment. If the average score on the dependent variable in an experiment differs across conditions, it is quite likely that the independent variable is responsible for that difference. Correlational research is lowest because it fails to address either problem. If the average score on the dependent variable differs across levels of the independent variable, it could be that the independent variable is responsible, but there are other interpretations. In some situations, the direction of causality could be reversed. In others, there could be a third variable that is causing differences in both the independent and dependent variables. Quasi-experimental research is in the middle because the manipulation of the independent variable addresses some problems, but the lack of random assignment and experimental control fails to address others. Imagine, for example, that a researcher finds two similar schools, starts an antibullying program in one, and then finds fewer bullying incidents in that “treatment school” than in the “control school.” There is no directionality problem because clearly the number of bullying incidents did not determine which school got the program. However, the lack of random assignment of children to schools could still mean that students in the treatment school differed from students in the control school in some other way that could explain the difference in bullying.

Experiments are generally high in internal validity, quasi-experiments lower, and correlational studies lower still.

Notice also in Figure 7.1 that there is some overlap in the internal validity of experiments, quasi-experiments, and correlational studies. For example, a poorly designed experiment that includes many confounding variables can be lower in internal validity than a well designed quasi-experiment with no obvious confounding variables.

Key Takeaways

Nonexperimental research is research that lacks the manipulation of an independent variable, control of extraneous variables through random assignment, or both.
There are three broad types of nonexperimental research. Single-variable research focuses on a single variable rather than a relationship between variables. Correlational and quasi-experimental research focus on a statistical relationship but lack manipulation or random assignment. Qualitative research focuses on broader research questions, typically involves collecting large amounts of data from a small number of participants, and analyzes the data nonstatistically.
In general, experimental research is high in internal validity, correlational research is low in internal validity, and quasi-experimental research is in between.

Discussion: For each of the following studies, decide which type of research design it is and explain why.

A researcher conducts detailed interviews with unmarried teenage fathers to learn about how they feel and what they think about their role as fathers and summarizes their feelings in a written narrative.
A researcher measures the impulsivity of a large sample of drivers and looks at the statistical relationship between this variable and the number of traffic tickets the drivers have received.
A researcher randomly assigns patients with low back pain either to a treatment involving hypnosis or to a treatment involving exercise. She then measures their level of low back pain after 3 months.
A college instructor gives weekly quizzes to students in one section of his course but no weekly quizzes to students in another section to see whether this has an effect on their test performance.

Bushman, B. J., & Huesmann, L. R. (2001). Effects of televised violence on aggression. In D. Singer & J. Singer (Eds.), Handbook of children and the media (pp. 223–254). Thousand Oaks, CA: Sage.

Milgram, S. (1974). Obedience to authority: An experimental view . New York, NY: Harper & Row.

Rosenhan, D. L. (1973). On being sane in insane places. Science, 179 , 250–258.

Research Methods in Psychology Copyright © 2016 by University of Minnesota is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Skip to main content
Skip to primary sidebar
Skip to footer
QuestionPro

Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case AskWhy Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Investigación de mercado

Non-experimental research: What it is, overview & advantages

Non-experimental research is the type of research that lacks an independent variable. Instead, the researcher observes the context in which the phenomenon occurs and analyzes it to obtain information.

Unlike experimental research , where the variables are held constant, non-experimental research happens during the study when the researcher cannot control, manipulate or alter the subjects but relies on interpretation or observations to conclude.

This means that the method must not rely on correlations, surveys , or case studies and cannot demonstrate an actual cause and effect relationship.

Characteristics of non-experimental research

Some of the essential characteristics of non-experimental research are necessary for the final results. Let’s talk about them to identify the most critical parts of them.

Most studies are based on events that occurred previously and are analyzed later.
In this method, controlled experiments are not performed for reasons such as ethics or morality.
No study samples are created; on the contrary, the samples or participants already exist and develop in their environment.
The researcher does not intervene directly in the environment of the sample.
This method studies the phenomena exactly as they occurred.

Types of non-experimental research

Non-experimental research can take the following forms:

Cross-sectional research : Cross-sectional research is used to observe and analyze the exact time of the research to cover various study groups or samples. This type of research is divided into:

Descriptive: When values are observed where one or more variables are presented.
Causal: It is responsible for explaining the reasons and relationship that exists between variables in a given time.

Longitudinal research: In a longitudinal study , researchers aim to analyze the changes and development of the relationships between variables over time. Longitudinal research can be divided into:

Trend: When they study the changes faced by the study group in general.
Group evolution: When the study group is a smaller sample.
Panel: It is in charge of analyzing individual and group changes to discover the factor that produces them.

LEARN ABOUT: Quasi-experimental Research

When to use non-experimental research

Non-experimental research can be applied in the following ways:

When the research question may be about one variable rather than a statistical relationship about two variables.
There is a non-causal statistical relationship between variables in the research question.
The research question has a causal research relationship, but the independent variable cannot be manipulated.
In exploratory or broad research where a particular experience is confronted.

Advantages and disadvantages

Some advantages of non-experimental research are:

It is very flexible during the research process
The cause of the phenomenon is known, and the effect it has is investigated.
The researcher can define the characteristics of the study group.

Among the disadvantages of non-experimental research are:

The groups are not representative of the entire population.
Errors in the methodology may occur, leading to research biases .

Non-experimental research is based on the observation of phenomena in their natural environment. In this way, they can be studied later to reach a conclusion.

Difference between experimental and non-experimental research

Experimental research involves changing variables and randomly assigning conditions to participants. As it can determine the cause, experimental research designs are used for research in medicine, biology, and social science.

Experimental research designs have strict standards for control and establishing validity. Although they may need many resources, they can lead to very interesting results.

Non-experimental research, on the other hand, is usually descriptive or correlational without any explicit changes done by the researcher. You simply describe the situation as it is, or describe a relationship between variables. Without any control, it is difficult to determine causal effects. The validity remains a concern in this type of research. However, it’s’ more regarding the measurements instead of the effects.

LEARN MORE: Descriptive Research vs Correlational Research

Whether you should choose experimental research or non-experimental research design depends on your goals and resources. If you need any help with how to conduct research and collect relevant data, or have queries regarding the best approach for your research goals, contact us today! You can create an account with our survey software and avail of 88+ features including dashboard and reporting for free.

Create a free account

MORE LIKE THIS

Was The Experience Memorable? (Part II) — Tuesday CX Thoughts

Sep 17, 2024

Data Discovery: What it is, Importance, Process + Use Cases

Sep 16, 2024

Competitive Insights : Importance, How to Get + Usage

Sep 13, 2024

Participant Engagement: Strategies + Improving Interaction

Sep 12, 2024

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 7: Nonexperimental Research

Overview of Nonexperimental Research

Learning Objectives

Define nonexperimental research, distinguish it clearly from experimental research, and give several examples.
Explain when a researcher might choose to conduct nonexperimental research as opposed to experimental research.

What Is Nonexperimental Research?

Nonexperimental research is research that lacks the manipulation of an independent variable, random assignment of participants to conditions or orders of conditions, or both.

In a sense, it is unfair to define this large and diverse set of approaches collectively by what they are not . But doing so reflects the fact that most researchers in psychology consider the distinction between experimental and nonexperimental research to be an extremely important one. This distinction is because although experimental research can provide strong evidence that changes in an independent variable cause differences in a dependent variable, nonexperimental research generally cannot. As we will see, however, this inability does not mean that nonexperimental research is less important than experimental research or inferior to it in any general sense.

When to Use Nonexperimental Research

As we saw in Chapter 6 , experimental research is appropriate when the researcher has a specific research question or hypothesis about a causal relationship between two variables—and it is possible, feasible, and ethical to manipulate the independent variable and randomly assign participants to conditions or to orders of conditions. It stands to reason, therefore, that nonexperimental research is appropriate—even necessary—when these conditions are not met. There are many ways in which preferring nonexperimental research can be the case.

The research question or hypothesis can be about a single variable rather than a statistical relationship between two variables (e.g., How accurate are people’s first impressions?).
The research question can be about a noncausal statistical relationship between variables (e.g., Is there a correlation between verbal intelligence and mathematical intelligence?).
The research question can be about a causal relationship, but the independent variable cannot be manipulated or participants cannot be randomly assigned to conditions or orders of conditions (e.g., Does damage to a person’s hippocampus impair the formation of long-term memory traces?).
The research question can be broad and exploratory, or it can be about what it is like to have a particular experience (e.g., What is it like to be a working mother diagnosed with depression?).

Again, the choice between the experimental and nonexperimental approaches is generally dictated by the nature of the research question. If it is about a causal relationship and involves an independent variable that can be manipulated, the experimental approach is typically preferred. Otherwise, the nonexperimental approach is preferred. But the two approaches can also be used to address the same research question in complementary ways. For example, nonexperimental studies establishing that there is a relationship between watching violent television and aggressive behaviour have been complemented by experimental studies confirming that the relationship is a causal one (Bushman & Huesmann, 2001) [1] . Similarly, after his original study, Milgram conducted experiments to explore the factors that affect obedience. He manipulated several independent variables, such as the distance between the experimenter and the participant, the participant and the confederate, and the location of the study (Milgram, 1974) [2] .

Types of Nonexperimental Research

As these examples make clear, single-variable research can answer interesting and important questions. What it cannot do, however, is answer questions about statistical relationships between variables. This detail is a point that beginning researchers sometimes miss. Imagine, for example, a group of research methods students interested in the relationship between children’s being the victim of bullying and the children’s self-esteem. The first thing that is likely to occur to these researchers is to obtain a sample of middle-school students who have been bullied and then to measure their self-esteem. But this design would be a single-variable study with self-esteem as the only variable. Although it would tell the researchers something about the self-esteem of children who have been bullied, it would not tell them what they really want to know, which is how the self-esteem of children who have been bullied compares with the self-esteem of children who have not. Is it lower? Is it the same? Could it even be higher? To answer this question, their sample would also have to include middle-school students who have not been bullied thereby introducing another variable.

The final way in which research can be nonexperimental is that it can be qualitative. The types of research we have discussed so far are all quantitative, referring to the fact that the data consist of numbers that are analyzed using statistical techniques. In qualitative research , the data are usually nonnumerical and therefore cannot be analyzed using statistical techniques. Rosenhan’s study of the experience of people in a psychiatric ward was primarily qualitative. The data were the notes taken by the “pseudopatients”—the people pretending to have heard voices—along with their hospital records. Rosenhan’s analysis consists mainly of a written description of the experiences of the pseudopatients, supported by several concrete examples. To illustrate the hospital staff’s tendency to “depersonalize” their patients, he noted, “Upon being admitted, I and other pseudopatients took the initial physical examinations in a semipublic room, where staff members went about their own business as if we were not there” (Rosenhan, 1973, p. 256). [3] Qualitative data has a separate set of analysis tools depending on the research question. For example, thematic analysis would focus on themes that emerge in the data or conversation analysis would focus on the way the words were said in an interview or focus group.

Internal Validity Revisited

Key Takeaways

Nonexperimental research is research that lacks the manipulation of an independent variable, control of extraneous variables through random assignment, or both.
There are three broad types of nonexperimental research. Single-variable research focuses on a single variable rather than a relationship between variables. Correlational and quasi-experimental research focus on a statistical relationship but lack manipulation or random assignment. Qualitative research focuses on broader research questions, typically involves collecting large amounts of data from a small number of participants, and analyses the data nonstatistically.
In general, experimental research is high in internal validity, correlational research is low in internal validity, and quasi-experimental research is in between.

Discussion: For each of the following studies, decide which type of research design it is and explain why.

A researcher conducts detailed interviews with unmarried teenage fathers to learn about how they feel and what they think about their role as fathers and summarizes their feelings in a written narrative.
A researcher measures the impulsivity of a large sample of drivers and looks at the statistical relationship between this variable and the number of traffic tickets the drivers have received.
A researcher randomly assigns patients with low back pain either to a treatment involving hypnosis or to a treatment involving exercise. She then measures their level of low back pain after 3 months.
A college instructor gives weekly quizzes to students in one section of his course but no weekly quizzes to students in another section to see whether this has an effect on their test performance.
Bushman, B. J., & Huesmann, L. R. (2001). Effects of televised violence on aggression. In D. Singer & J. Singer (Eds.), Handbook of children and the media (pp. 223–254). Thousand Oaks, CA: Sage. ↵
Milgram, S. (1974). Obedience to authority: An experimental view . New York, NY: Harper & Row. ↵
Rosenhan, D. L. (1973). On being sane in insane places. Science, 179 , 250–258. ↵

Research that lacks the manipulation of an independent variable, random assignment of participants to conditions or orders of conditions, or both.

Research that focuses on a single variable rather than a statistical relationship between two variables.

The researcher measures the two variables of interest with little or no attempt to control extraneous variables and then assesses the relationship between them.

The researcher manipulates an independent variable but does not randomly assign participants to conditions or orders of conditions.

Research Methods in Psychology - 2nd Canadian Edition Copyright © 2015 by Paul C. Price, Rajiv Jhangiani, & I-Chant A. Chiang is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings
My Bibliography
Collections
Citation manager

Save citation to file

Email citation, add to collections.

Create a new collection
Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

Search in PubMed
Search in NLM Catalog
Add to Search

Determining the level of evidence: Nonexperimental research designs

Affiliation.

1 Amy Glasofer is a nurse scientist at Virtua Center for Learning in Mt. Laurel, N.J., and Ann B. Townsend is an adult NP with The Nurse Practitioner Group, LLC.
PMID: 33953103
DOI: 10.1097/01.NURSE.0000731852.39123.e1

To support evidence-based nursing practice, the authors provide guidelines for appraising research based on quality, quantity, and consistency. This article, the second of a three-part series, focuses on nonexperimental research designs.

PubMed Disclaimer

Related information

Linkout - more resources, full text sources.

Ovid Technologies, Inc.
Wolters Kluwer

Other Literature Sources

scite Smart Citations

Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

To read this content please select one of the options below:

Please note you do not have access to teaching notes, nonexperimental research: strengths, weaknesses and issues of precision.

European Journal of Training and Development

ISSN : 2046-9012

Article publication date: 6 September 2016

Nonexperimental research, defined as any kind of quantitative or qualitative research that is not an experiment, is the predominate kind of research design used in the social sciences. How to unambiguously and correctly present the results of nonexperimental research, however, remains decidedly unclear and possibly detrimental to applied disciplines such as human resource development. To clarify issues about the accurate reporting and generalization of nonexperimental research results, this paper aims to present information about the relative strength of research designs, followed by the strengths and weaknesses of nonexperimental research. Further, some possible ways to more precisely report nonexperimental findings without using causal language are explored. Next, the researcher takes the position that the results of nonexperimental research can be used cautiously, yet appropriately, for making practice recommendations. Finally, some closing thoughts about nonexperimental research and the appropriate use of causal language are presented.

Design/methodology/approach

A review of the extant social science literature was consulted to inform this paper.

Nonexperimental research, when reported accurately, makes a tremendous contribution because it can be used for conducting research when experimentation is not feasible or desired. It can be used also to make tentative recommendations for practice.

Originality/value

This article presents useful means to more accurately report nonexperimental findings through avoiding causal language. Ways to link nonexperimental results to making practice recommendations are explored.

Research design
Experimental design
Causal inference
Nonexperimental
Social science research
Triangulation

Reio, T.G. (2016), "Nonexperimental research: strengths, weaknesses and issues of precision", European Journal of Training and Development , Vol. 40 No. 8/9, pp. 676-690. https://doi.org/10.1108/EJTD-07-2015-0058

Emerald Group Publishing Limited

All feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 6: Data Collection Strategies

6.2 Nonexperimental Research

Nonexperimental research is research that lacks manipulation of an independent variable and/or random assignment of participants to conditions. While the distinction between experimental and nonexperimental research is considered important, it does not mean that nonexperimental research is less important or inferior to experimental research (Price, Jhangiani & Chiang, 2015).

When to Use Nonexperimental Research

Often it is not possible, feasible, and/or ethical to manipulate the independent variable, nor to randomly assign participants to conditions or to orders of conditions. In such cases, nonexperimental research is more appropriate and often necessary. Price, et al. (2015) provide the following examples that demonstrate when the research question is better answered with non-experimental methods:

The research question or hypothesis contains a single variable rather than a statistical relationship between two variables (e.g., How accurate are people’s first impressions?).
The research question involves a non-causal statistical relationship between variables (e.g., is there a correlation between verbal intelligence and mathematical intelligence?).
The research question involves a causal relationship, but the independent variable cannot be manipulated, or participants cannot be randomly assigned to conditions or orders of conditions (e.g., Does damage to a person’s hippocampus impair the formation of long-term memory traces?).
The research question is broad and exploratory, or explores a particular experience (e.g., what is it like to be a working mother diagnosed with depression?).

As demonstrated above, it is the nature of the research question that guides the choice between experimental and non-experimental approaches. However, this is not to suggest that a research project cannot contain elements of both an experiment and a non-experiment. For example, nonexperimental studies that establish a relationship between two variables can be explored further in an experimental study to confirm or refute the causal nature of the relationship (Price, Jhangiani & Chiang, 2015).

Types of Nonexperimental Research

In social sciences it is often the case that a true experimental approach is inappropriate and unethical. For example, conducting a true experiment may require the researcher to deny needed treatment to a patient, which is clearly an ethical issue. Furthermore, it might not be equitable or ethical to provide a large financial or other reward to members of an experimental group, as can occur in a true experiment.

There are three types of non-experimental research: cross-sectional, correlational, and observational. In the following sections we explore each of three types of nonexperimental research.

Research Methods for the Social Sciences: An Introduction Copyright © 2020 by Valerie Sheppard is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Overview of Non-Experimental Research

Rajiv S. Jhangiani; I-Chant A. Chiang; Carrie Cuttler; and Dana C. Leighton

Learning Objectives

Define non-experimental research, distinguish it clearly from experimental research, and give several examples.
Explain when a researcher might choose to conduct non-experimental research as opposed to experimental research.

What Is Non-Experimental Research?

Non-experimental research is research that lacks the manipulation of an independent variable. Rather than manipulating an independent variable, researchers conducting non-experimental research simply measure variables as they naturally occur (in the lab or real world).

Most researchers in psychology consider the distinction between experimental and non-experimental research to be an extremely important one. This is because although experimental research can provide strong evidence that changes in an independent variable cause differences in a dependent variable, non-experimental research generally cannot. As we will see, however, this inability to make causal conclusions does not mean that non-experimental research is less important than experimental research. It is simply used in cases where experimental research is not able to be carried out.

When to Use Non-Experimental Research

As we saw in the last chapter , experimental research is appropriate when the researcher has a specific research question or hypothesis about a causal relationship between two variables—and it is possible, feasible, and ethical to manipulate the independent variable. It stands to reason, therefore, that non-experimental research is appropriate—even necessary—when these conditions are not met. There are many times in which non-experimental research is preferred, including when:

the research question or hypothesis relates to a single variable rather than a statistical relationship between two variables (e.g., how accurate are people’s first impressions?).
the research question pertains to a non-causal statistical relationship between variables (e.g., is there a correlation between verbal intelligence and mathematical intelligence?).
the research question is about a causal relationship, but the independent variable cannot be manipulated or participants cannot be randomly assigned to conditions or orders of conditions for practical or ethical reasons (e.g., does damage to a person’s hippocampus impair the formation of long-term memory traces?).
the research question is broad and exploratory, or is about what it is like to have a particular experience (e.g., what is it like to be a working mother diagnosed with depression?).

Again, the choice between the experimental and non-experimental approaches is generally dictated by the nature of the research question. Recall the three goals of science are to describe, to predict, and to explain. If the goal is to explain and the research question pertains to causal relationships, then the experimental approach is typically preferred. If the goal is to describe or to predict, a non-experimental approach is appropriate. But the two approaches can also be used to address the same research question in complementary ways. For example, in Milgram’s original (non-experimental) obedience study, he was primarily interested in one variable—the extent to which participants obeyed the researcher when he told them to shock the confederate—and he observed all participants performing the same task under the same conditions. However, Milgram subsequently conducted experiments to explore the factors that affect obedience. He manipulated several independent variables, such as the distance between the experimenter and the participant, the participant and the confederate, and the location of the study (Milgram, 1974) [1] .

Types of Non-Experimental Research

Non-experimental research falls into two broad categories: correlational research and observational research.

The most common type of non-experimental research conducted in psychology is correlational research. Correlational research is considered non-experimental because it focuses on the statistical relationship between two variables but does not include the manipulation of an independent variable. More specifically, in correlational research , the researcher measures two variables with little or no attempt to control extraneous variables and then assesses the relationship between them. As an example, a researcher interested in the relationship between self-esteem and school achievement could collect data on students’ self-esteem and their GPAs to see if the two variables are statistically related.

Observational research is non-experimental because it focuses on making observations of behavior in a natural or laboratory setting without manipulating anything. Milgram’s original obedience study was non-experimental in this way. He was primarily interested in the extent to which participants obeyed the researcher when he told them to shock the confederate and he observed all participants performing the same task under the same conditions. The study by Loftus and Pickrell described at the beginning of this chapter is also a good example of observational research. The variable was whether participants “remembered” having experienced mildly traumatic childhood events (e.g., getting lost in a shopping mall) that they had not actually experienced but that the researchers asked them about repeatedly. In this particular study, nearly a third of the participants “remembered” at least one event. (As with Milgram’s original study, this study inspired several later experiments on the factors that affect false memories).

Cross-Sectional, Longitudinal, and Cross-Sequential Studies

When psychologists wish to study change over time (for example, when developmental psychologists wish to study aging) they usually take one of three non-experimental approaches: cross-sectional, longitudinal, or cross-sequential. Cross-sectional studies involve comparing two or more pre-existing groups of people (e.g., children at different stages of development). What makes this approach non-experimental is that there is no manipulation of an independent variable and no random assignment of participants to groups. Using this design, developmental psychologists compare groups of people of different ages (e.g., young adults spanning from 18-25 years of age versus older adults spanning 60-75 years of age) on various dependent variables (e.g., memory, depression, life satisfaction). Of course, the primary limitation of using this design to study the effects of aging is that differences between the groups other than age may account for differences in the dependent variable. For instance, differences between the groups may reflect the generation that people come from (a cohort effect ) rather than a direct effect of age. For this reason, longitudinal studies , in which one group of people is followed over time as they age, offer a superior means of studying the effects of aging. However, longitudinal studies are by definition more time consuming and so require a much greater investment on the part of the researcher and the participants. A third approach, known as cross-sequential studies , combines elements of both cross-sectional and longitudinal studies. Rather than measuring differences between people in different age groups or following the same people over a long period of time, researchers adopting this approach choose a smaller period of time during which they follow people in different age groups. For example, they might measure changes over a ten year period among participants who at the start of the study fall into the following age groups: 20 years old, 30 years old, 40 years old, 50 years old, and 60 years old. This design is advantageous because the researcher reaps the immediate benefits of being able to compare the age groups after the first assessment. Further, by following the different age groups over time they can subsequently determine whether the original differences they found across the age groups are due to true age effects or cohort effects.

The types of research we have discussed so far are all quantitative, referring to the fact that the data consist of numbers that are analyzed using statistical techniques. But as you will learn in this chapter, many observational research studies are more qualitative in nature. In qualitative research , the data are usually nonnumerical and therefore cannot be analyzed using statistical techniques. Rosenhan’s observational study of the experience of people in psychiatric wards was primarily qualitative. The data were the notes taken by the “pseudopatients”—the people pretending to have heard voices—along with their hospital records. Rosenhan’s analysis consists mainly of a written description of the experiences of the pseudopatients, supported by several concrete examples. To illustrate the hospital staff’s tendency to “depersonalize” their patients, he noted, “Upon being admitted, I and other pseudopatients took the initial physical examinations in a semi-public room, where staff members went about their own business as if we were not there” (Rosenhan, 1973, p. 256) [2] . Qualitative data has a separate set of analysis tools depending on the research question. For example, thematic analysis would focus on themes that emerge in the data or conversation analysis would focus on the way the words were said in an interview or focus group.

Internal Validity Revisited

Recall that internal validity is the extent to which the design of a study supports the conclusion that changes in the independent variable caused any observed differences in the dependent variable. Figure 6.1 shows how experimental, quasi-experimental, and non-experimental (correlational) research vary in terms of internal validity. Experimental research tends to be highest in internal validity because the use of manipulation (of the independent variable) and control (of extraneous variables) help to rule out alternative explanations for the observed relationships. If the average score on the dependent variable in an experiment differs across conditions, it is quite likely that the independent variable is responsible for that difference. Non-experimental (correlational) research is lowest in internal validity because these designs fail to use manipulation or control. Quasi-experimental research (which will be described in more detail in a subsequent chapter) falls in the middle because it contains some, but not all, of the features of a true experiment. For instance, it may fail to use random assignment to assign participants to groups or fail to use counterbalancing to control for potential order effects. Imagine, for example, that a researcher finds two similar schools, starts an anti-bullying program in one, and then finds fewer bullying incidents in that “treatment school” than in the “control school.” While a comparison is being made with a control condition, the inability to randomly assign children to schools could still mean that students in the treatment school differed from students in the control school in some other way that could explain the difference in bullying (e.g., there may be a selection effect).

Figure 6.1 Internal Validity of Correlational, Quasi-Experimental, and Experimental Studies. Experiments are generally high in internal validity, quasi-experiments lower, and correlational studies lower still.

Notice also in Figure 6.1 that there is some overlap in the internal validity of experiments, quasi-experiments, and correlational (non-experimental) studies. For example, a poorly designed experiment that includes many confounding variables can be lower in internal validity than a well-designed quasi-experiment with no obvious confounding variables. Internal validity is also only one of several validities that one might consider, as noted in Chapter 5.

Milgram, S. (1974). Obedience to authority: An experimental view . New York, NY: Harper & Row. ↵
Rosenhan, D. L. (1973). On being sane in insane places. Science, 179 , 250–258. ↵

A research that lacks the manipulation of an independent variable.

Research that is non-experimental because it focuses on the statistical relationship between two variables but does not include the manipulation of an independent variable.

Research that is non-experimental because it focuses on recording systemic observations of behavior in a natural or laboratory setting without manipulating anything.

Studies that involve comparing two or more pre-existing groups of people (e.g., children at different stages of development).

Differences between the groups may reflect the generation that people come from rather than a direct effect of age.

Studies in which one group of people are followed over time as they age.

Studies in which researchers follow people in different age groups in a smaller period of time.

Overview of Non-Experimental Research Copyright © by Rajiv S. Jhangiani; I-Chant A. Chiang; Carrie Cuttler; and Dana C. Leighton is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Experimental Vs Non-Experimental Research: 15 Key Differences

There is a general misconception around research that once the research is non-experimental, then it is non-scientific, making it more important to understand what experimental and experimental research entails. Experimental research is the most common type of research, which a lot of people refer to as scientific research.

Non experimental research, on the other hand, is easily used to classify research that is not experimental. It clearly differs from experimental research, and as such has different use cases.

In this article, we will be explaining these differences in detail so as to ensure proper identification during the research process.

What is Experimental Research?

Experimental research is the type of research that uses a scientific approach towards manipulating one or more control variables of the research subject(s) and measuring the effect of this manipulation on the subject. It is known for the fact that it allows the manipulation of control variables.

This research method is widely used in various physical and social science fields, even though it may be quite difficult to execute. Within the information field, they are much more common in information systems research than in library and information management research.

Experimental research is usually undertaken when the goal of the research is to trace cause-and-effect relationships between defined variables. However, the type of experimental research chosen has a significant influence on the results of the experiment.

Therefore bringing us to the different types of experimental research. There are 3 main types of experimental research, namely; pre-experimental, quasi-experimental, and true experimental research.

Pre-experimental Research

Pre-experimental research is the simplest form of research, and is carried out by observing a group or groups of dependent variables after the treatment of an independent variable which is presumed to cause change on the group(s). It is further divided into three types.

One-shot case study research
One-group pretest-posttest research
Static-group comparison

Quasi-experimental Research

The Quasi type of experimental research is similar to true experimental research, but uses carefully selected rather than randomized subjects. The following are examples of quasi-experimental research:

Time series
No equivalent control group design
Counterbalanced design.

True Experimental Research

True experimental research is the most accurate type, and may simply be called experimental research. It manipulates a control group towards a group of randomly selected subjects and records the effect of this manipulation.

True experimental research can be further classified into the following groups:

The posttest-only control group
The pretest-posttest control group
Solomon four-group

Pros of True Experimental Research

Researchers can have control over variables.
It can be combined with other research methods.
The research process is usually well structured.
It provides specific conclusions.
The results of experimental research can be easily duplicated.

Cons of True Experimental Research

It is highly prone to human error.
Exerting control over extraneous variables may lead to the personal bias of the researcher.
It is time-consuming.
It is expensive.
Manipulating control variables may have ethical implications.
It produces artificial results.

What is Non-Experimental Research?

Non-experimental research is the type of research that does not involve the manipulation of control or independent variable. In non-experimental research, researchers measure variables as they naturally occur without any further manipulation.

This type of research is used when the researcher has no specific research question about a causal relationship between 2 different variables, and manipulation of the independent variable is impossible. They are also used when:

subjects cannot be randomly assigned to conditions.
the research subject is about a causal relationship but the independent variable cannot be manipulated.
the research is broad and exploratory
the research pertains to a non-causal relationship between variables.
limited information can be accessed about the research subject.

There are 3 main types of non-experimental research , namely; cross-sectional research, correlation research, and observational research.

Cross-sectional Research

Cross-sectional research involves the comparison of two or more pre-existing groups of people under the same criteria. This approach is classified as non-experimental because the groups are not randomly selected and the independent variable is not manipulated.

For example, an academic institution may want to reward its first-class students with a scholarship for their academic excellence. Therefore, each faculty places students in the eligible and ineligible group according to their class of degree.

In this case, the student’s class of degree cannot be manipulated to qualify him or her for a scholarship because it is an unethical thing to do. Therefore, the placement is cross-sectional.

Correlational Research

Correlational type of research compares the statistical relationship between two variables .Correlational research is classified as non-experimental because it does not manipulate the independent variables.

For example, a researcher may wish to investigate the relationship between the class of family students come from and their grades in school. A questionnaire may be given to students to know the average income of their family, then compare it with CGPAs.

The researcher will discover whether these two factors are positively correlated, negatively corrected, or have zero correlation at the end of the research.

Observational Research

Observational research focuses on observing the behavior of a research subject in a natural or laboratory setting. It is classified as non-experimental because it does not involve the manipulation of independent variables.

A good example of observational research is an investigation of the crowd effect or psychology in a particular group of people. Imagine a situation where there are 2 ATMs at a place, and only one of the ATMs is filled with a queue, while the other is abandoned.

The crowd effect infers that the majority of newcomers will also abandon the other ATM.

You will notice that each of these non-experimental research is descriptive in nature. It then suffices to say that descriptive research is an example of non-experimental research.

Pros of Observational Research

The research process is very close to a real-life situation.
It does not allow for the manipulation of variables due to ethical reasons.
Human characteristics are not subject to experimental manipulation.

Cons of Observational Research

The groups may be dissimilar and nonhomogeneous because they are not randomly selected, affecting the authenticity and generalizability of the study results.
The results obtained cannot be absolutely clear and error-free.

What Are The Differences Between Experimental and Non-Experimental Research?

Definitions

Experimental research is the type of research that uses a scientific approach towards manipulating one or more control variables and measuring their defect on the dependent variables, while non-experimental research is the type of research that does not involve the manipulation of control variables.

The main distinction in these 2 types of research is their attitude towards the manipulation of control variables. Experimental allows for the manipulation of control variables while non-experimental research doesn’t.

Examples of experimental research are laboratory experiments that involve mixing different chemical elements together to see the effect of one element on the other while non-experimental research examples are investigations into the characteristics of different chemical elements.

Consider a researcher carrying out a laboratory test to determine the effect of adding Nitrogen gas to Hydrogen gas. It may be discovered that using the Haber process, one can create Nitrogen gas.

Non-experimental research may further be carried out on Ammonia, to determine its characteristics, behaviour, and nature.

There are 3 types of experimental research, namely; experimental research, quasi-experimental research, and true experimental research. Although also 3 in number, non-experimental research can be classified into cross-sectional research, correlational research, and observational research.

The different types of experimental research are further divided into different parts, while non-experimental research types are not further divided. Clearly, these divisions are not the same in experimental and non-experimental research.

Characteristics

Experimental research is usually quantitative, controlled, and multivariable. Non-experimental research can be both quantitative and qualitative , has an uncontrolled variable, and also a cross-sectional research problem.

The characteristics of experimental research are the direct opposite of that of non-experimental research. The most distinct characteristic element is the ability to control or manipulate independent variables in experimental research and not in non-experimental research.

In experimental research, a level of control is usually exerted on extraneous variables, therefore tampering with the natural research setting. Experimental research settings are usually more natural with no tampering with the extraneous variables.

Data Collection/Tools

The data used during experimental research is collected through observational study, simulations, and surveys while non-experimental data is collected through observations, surveys, and case studies. The main distinction between these data collection tools is case studies and simulations.

Even at that, similar tools are used differently. For example, an observational study may be used during a laboratory experiment that tests how the effect of a control variable manifests over a period of time in experimental research.

However, when used in non-experimental research, data is collected based on the researcher’s discretion and not through a clear scientific reaction. In this case, we see a difference in the level of objectivity.

The goal of experimental research is to measure the causes and effects of variables present in research, while non-experimental research provides very little to no information about causal agents.

Experimental research answers the question of why something is happening. This is quite different in non-experimental research, as they are more descriptive in nature with the end goal being to describe what .

Experimental research is mostly used to make scientific innovations and find major solutions to problems while non-experimental research is used to define subject characteristics, measure data trends, compare situations and validate existing conditions.

For example, if experimental research results in an innovative discovery or solution, non-experimental research will be conducted to validate this discovery. This research is done for a period of time in order to properly study the subject of research.

Experimental research process is usually well structured and as such produces results with very little to no errors, while non-experimental research helps to create real-life related experiments. There are a lot more advantages of experimental and non-experimental research , with the absence of each of these advantages in the other leaving it at a disadvantage.

For example, the lack of a random selection process in non-experimental research leads to the inability to arrive at a generalizable result. Similarly, the ability to manipulate control variables in experimental research may lead to the personal bias of the researcher.

Disadvantage

Experimental research is highly prone to human error while the major disadvantage of non-experimental research is that the results obtained cannot be absolutely clear and error-free. In the long run, the error obtained due to human error may affect the results of the experimental research.

Some other disadvantages of experimental research include the following; extraneous variables cannot always be controlled, human responses can be difficult to measure, and participants may also cause bias.

In experimental research, researchers can control and manipulate control variables, while in non-experimental research, researchers cannot manipulate these variables. This cannot be done due to ethical reasons.

For example, when promoting employees due to how well they did in their annual performance review, it will be unethical to manipulate the results of the performance review (independent variable). That way, we can get impartial results of those who deserve a promotion and those who don’t.

Experimental researchers may also decide to eliminate extraneous variables so as to have enough control over the research process. Once again, this is something that cannot be done in non-experimental research because it relates more to real-life situations.

Experimental research is carried out in an unnatural setting because most of the factors that influence the setting are controlled while the non-experimental research setting remains natural and uncontrolled. One of the things usually tampered with during research is extraneous variables.

In a bid to get a perfect and well-structured research process and results, researchers sometimes eliminate extraneous variables. Although sometimes seen as insignificant, the elimination of these variables may affect the research results.

Consider the optimization problem whose aim is to minimize the cost of production of a car, with the constraints being the number of workers and the number of hours they spend working per day.

In this problem, extraneous variables like machine failure rates or accidents are eliminated. In the long run, these things may occur and may invalidate the result.

Cause-Effect Relationship

The relationship between cause and effect is established in experimental research while it cannot be established in non-experimental research. Rather than establish a cause-effect relationship, non-experimental research focuses on providing descriptive results.

Although it acknowledges the causal variable and its effect on the dependent variables, it does not measure how or the extent to which these dependent variables change. It, however, observes these changes, compares the changes in 2 variables, and describes them.

Experimental research does not compare variables while non-experimental research does. It compares 2 variables and describes the relationship between them.

The relationship between these variables can be positively correlated, negatively correlated or not correlated at all. For example, consider a case whereby the subject of research is a drum, and the control or independent variable is the drumstick.

Experimental research will measure the effect of hitting the drumstick on the drum, where the result of this research will be sound. That is, when you hit a drumstick on a drum, it makes a sound.

Non-experimental research, on the other hand, will investigate the correlation between how hard the drum is hit and the loudness of the sound that comes out. That is, if the sound will be higher with a harder bang, lower with a harder bang, or will remain the same no matter how hard we hit the drum.

Quantitativeness

Experimental research is a quantitative research method while non-experimental research can be both quantitative and qualitative depending on the time and the situation where it is been used. An example of a non-experimental quantitative research method is correlational research .

Researchers use it to correlate two or more variables using mathematical analysis methods. The original patterns, relationships, and trends between variables are observed, then the impact of one of these variables on the other is recorded along with how it changes the relationship between the two variables.

Observational research is an example of non-experimental research, which is classified as a qualitative research method.

Cross-section

Experimental research is usually single-sectional while non-experimental research is cross-sectional. That is, when evaluating the research subjects in experimental research, each group is evaluated as an entity.

For example, let us consider a medical research process investigating the prevalence of breast cancer in a certain community. In this community, we will find people of different ages, ethnicities, and social backgrounds.

If a significant amount of women from a particular age are found to be more prone to have the disease, the researcher can conduct further studies to understand the reason behind it. A further study into this will be experimental and the subject won’t be a cross-sectional group.

A lot of researchers consider the distinction between experimental and non-experimental research to be an extremely important one. This is partly due to the fact that experimental research can accommodate the manipulation of independent variables, which is something non-experimental research can not.

Therefore, as a researcher who is interested in using any one of experimental and non-experimental research, it is important to understand the distinction between these two. This helps in deciding which method is better for carrying out particular research.

Connect to Formplus, Get Started Now - It's Free!

examples of experimental research
non experimental research
busayo.longe

Experimental Research Designs: Types, Examples & Methods

Ultimate guide to experimental research. It’s definition, types, characteristics, uses, examples and methodolgy

Simpson’s Paradox & How to Avoid it in Experimental Research

In this article, we are going to look at Simpson’s Paradox from its historical point and later, we’ll consider its effect in...

What is Experimenter Bias? Definition, Types & Mitigation

In this article, we will look into the concept of experimental bias and how it can be identified in your research

Response vs Explanatory Variables: Definition & Examples

In this article, we’ll be comparing the two types of variables, what they both mean and see some of their real-life applications in research

Formplus - For Seamless Data Collection

Collect data the right way with a versatile data collection tool. try formplus and transform your work productivity today..

Warning: The NCBI web site requires JavaScript to function. more...

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings
Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Bennett WL, Cheskin LJ, Wilson RF, et al. Methods for Evaluating Natural Experiments in Obesity: Systematic Evidence Review [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2017 Dec. (Comparative Effectiveness Reviews, No. 204.)

Cover of Methods for Evaluating Natural Experiments in Obesity: Systematic Evidence Review

Methods for Evaluating Natural Experiments in Obesity: Systematic Evidence Review [Internet].

Key question 4 experimental and non-experimental methods, key question (kq) 4. which experimental and non-experimental methods have been used in studies of how programs, policies, or built environment changes affect or are associated with obesity prevention and control outcomes.

While randomized controlled trials (RCTs) and other experimental approaches can provide strong evidence of the effectiveness of some interventions, not all interventions are amenable to randomization. An alternative approach involves use of natural experiment designs, which aim to approximate casual effects by capitalizing on existing variation in an exposure or intervention. Natural experiment studies use a wide range of analytic approaches, which are further described in this section. The goal of KQ 4 was to describe experimental and non-experimental methods in terms of study design and analytic approach.

Key Findings

Of the 294 studies included in this review, 156 were natural experiment studies (53%), 118 were experimental studies (40%), and 20 provided insufficient information to determine if a natural experiment took place (7%) and were classified as “other study designs.”
Natural experiment studies evaluating policies, programs, and built environment changes most commonly used cross-sectional comparisons of exposed and unexposed groups (n=55, 35%).
Difference-in-differences approaches that compared exposed and unexposed groups before and after an exposure were used in 45 studies (29%), and pre/post designs that compared one group before and after an exposure were used in 48 studies (31%).
A small number of natural experiment studies in this review used other non-experimental designs including 4 instrumental variable approaches, 1 regression discontinuity approach, and 4 interrupted time series analyses with more than 2 time points pre- and post-intervention.

Data Abstraction

To address KQ 4, we abstracted details of the natural experiment studies, experimental studies, and other study designs and analytic approaches. After determining whether a study met the UK’s Medical Research Council criteria for a “natural experiment study,” we further classified study designs using the Effective Public Health Practice Project Quality Assessment (EPHPP) tool: 91 RCT, controlled clinical trial, cohort, case-control, interrupted time series, cross-sectional, and “other”. Definitions of these designs are provided in Appendix B .

Given that some study designs (such as cohort studies) could be used in multiple ways to estimate effects, we further distinguished the analytic approach of each study. While the EPHPP tool collects information regarding the data collection structure, our risk of bias forms additionally focused on the analytic tools used to analyze data. For each study, we classified the analytic approach as an RCT, instrumental variables design, regression discontinuity, interrupted time series, cross-sectional comparison of exposed and unexposed groups, controlled clinical trial, pre/post comparison, difference in differences, or “other” approach.

Data Synthesis

We described the types of study designs and analytic methods being used, and their frequency of use, by study design.

Description of Methods Used in Natural Experiment Studies

Over half of the studies (n=156) included in this review met the main criterion for natural experiment studies, based on the MRC Report, 41 clearly indicating that the researchers were not in control of the exposure allocation (see Methods for KQ4). Eighty-eight of the natural experiment studies took place in community settings (56%) and 49 (31%) were in other institutional settings (e.g., schools or worksites). Nineteen studies (12%) evaluated an intervention or exposure at the level of the individual. Analysis was performed at the individual level in 90 percent of these studies. For example, Project Healthy Schools, a diet and physical activity intervention, was implemented at 23 schools in Michigan, and its success was evaluated by comparing individual changes in body mass index (BMI) and diet among 6 th graders before and after the program was implemented. 92 Most studies evaluating community or school-level programs used methods such as multilevel modeling or robust standard errors to account for the hierarchical structure of the data, but 20 studies (13%) did not account for nested data structure in the analysis.

The most common analytic approach in natural experiment studies was cross-sectional comparisons of exposed and unexposed groups (n=55, 35%). For example, Taber and colleagues used regression models to compare dietary intake among high school students in California, one of the first states to regulate the nutrition content of competitive foods in schools, to students in states without competitive food laws. 93 The second most common design was pre/post with the pre-intervention period serving as the control for the post-intervention period (n= 48, 31%). For example, one study evaluated changes in physical activity after construction of a bus line and car-free walking and cycling route in Cambridge using a pre/post analysis embedded in an existing cohort study. The study compared activity levels collected from an annual survey one year before and one year after the construction was complete. 55 Forty-five studies used difference-in-difference approaches (29%) looking at changes before and after the intervention compared to an external control group. This was the third most common study design. This category includes studies such as an evaluation of a new supermarket that opened in a food desert through the Pennsylvania Fresh Food Financing Initiative. 55 Researchers collected BMI and fruit and vegetable intake from residents of intervention and control neighborhoods at one point before and one point after the construction of the supermarket.

For the 48 pre/post studies, the average number of time points pre- and post-intervention was 1 and 1.6, respectively. For the 45 difference-in-difference studies, which used changes in time and across groups, there were an average of 1 and 1.2 time points pre- and post-intervention. Among the studies included in this review, all pre/post studies measured variables at a single time point pre-intervention, and 80 percent had a single measure post-intervention. Difference-in-difference studies also all had a single measure pre-intervention, and 83 percent had only a single post-intervention measure. We use the terms pre/post and difference-in-difference for studies with multiple time points post-intervention rather than interrupted time series due to the small number of time points (maximum 6 points) and the lack of formal interrupted time series methods such as autoregressive integrated moving average.

Description of Methods Used in Experimental Studies

One hundred eighteen studies (40%) included in this review met the criterion for experimental studies, defined as evaluations of researcher-controlled programs, policies, or built environment changes (see Methods for KQ 4). Experimental methods included 74 RCTs (63% of the experimental studies, 25% of all studies), and 44 controlled trials (37% of experimental studies, 15% of all studies) in which the investigator assigned the exposure by means other than randomization ( Table 10 ). Shape Up Somerville is an example of a controlled clinical trial that took place in one intervention and two control communities in Massachusetts. Researchers selected Somerville as the intervention site due to ongoing relationships with that community, so this would not be considered a natural experiment. This community-wide, multi-level diet and physical activity program included a Walk to School Campaign, changes to school breakfast and lunch programs, school and afterschool education programming, enhanced playgrounds for recess, and a restaurant initiative. 94

Overview of study design or data collection structure (N=294 studies).

Of the experimental studies, the unit of intervention allocation was most often at the organization level (66%) or the community level (23%). Analyses were most often conducted at the individual level (93%).

Description of Methods Used in Other Study Approaches

Twenty studies in this category (7%) did not provide sufficient detail to determine whether the research team was in control of assigning the intervention, whether the intervention was originally intended to be research, or otherwise would be included as either a natural experiment or experiment. Most often, the exposure was at the community (40%) or organizational (45%) level, but the analysis was carried out at the individual level (90% of studies). Thirty-five percent of these studies did not take the hierarchical nature of the data structure into account in the analysis. These studies were most often pre/post (45%) or difference-in–differences (40%) designs.

Cite this Page Bennett WL, Cheskin LJ, Wilson RF, et al. Methods for Evaluating Natural Experiments in Obesity: Systematic Evidence Review [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2017 Dec. (Comparative Effectiveness Reviews, No. 204.) Key Question 4, Experimental and Non-Experimental Methods.
PDF version of this title (6.1M)

Other titles in these collections

AHRQ Comparative Effectiveness Reviews
Health Services/Technology Assessment Texts (HSTAT)

Recent Activity

Experimental and Non-Experimental Methods - Methods for Evaluating Natural Exper... Experimental and Non-Experimental Methods - Methods for Evaluating Natural Experiments in Obesity: Systematic Evidence Review

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Non-experimental research designs:: Investigating the spatial distribution and social ecology of male homicide

In book: Transforming Research Methods in the Social Sciences (pp.19-35)

University of South Africa

University of the Witwatersrand

Stellenbosch University

Discover the world's research

25+ million members
160+ million publication pages
2.3+ billion citations
CHILD INDIC RES
Amon Exavery
Peter Josephat Kirigiti
Ramkumar T. Balan
John Charles

Nick Malherbe
Muhammad Daniel Suzuki

nathaniel coakley

B WORLD HEALTH ORGAN
Richard Matzopoulos

Victoria Pillay-van Wyk

J RES CRIME DELINQ

Pam Groenewald
CITY COMMUNITY
Jacob H. Becker

Recruit researchers
Join for free
Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

COMMENTS

6.1 Overview of Non-Experimental Research
Non-experimental research is research that lacks the manipulation of an independent variable. Rather than manipulating an independent variable, researchers conducting non-experimental research simply measure variables as they naturally occur (in the lab or real world). Most researchers in psychology consider the distinction between experimental ...
Quantitative Research with Nonexperimental Designs
Leung and Shek (2018) explain: Experimental research design utilizes the principle of manipulation of the independent variables and examines its cause-and-effect relationship on the dependent variables by controlling the effects of other variables. Usually, the experimenter assigns two or more groups with similar characteristics.
7.1 Overview of Nonexperimental Research
When to Use Nonexperimental Research. As we saw in Chapter 6 "Experimental Research", experimental research is appropriate when the researcher has a specific research question or hypothesis about a causal relationship between two variables—and it is possible, feasible, and ethical to manipulate the independent variable and randomly assign participants to conditions or to orders of ...
Non-experimental research: What it is, Types & Tips
Non-experimental research is the type of research that lacks an independent variable. Instead, the researcher observes the context in which the phenomenon occurs and analyzes it to obtain information. Unlike experimental research, where the variables are held constant, non-experimental research happens during the study when the researcher ...
Overview of Nonexperimental Research
Key Takeaways. Nonexperimental research is research that lacks the manipulation of an independent variable, control of extraneous variables through random assignment, or both. There are three broad types of nonexperimental research. Single-variable research focuses on a single variable rather than a relationship between variables.
Determining the level of evidence: Nonexperimental research ...
Humans. Research Design*. To support evidence-based nursing practice, the authors provide guidelines for appraising research based on quality, quantity, and consistency. This article, the second of a three-part series, focuses on nonexperimental research designs.
Nonexperimental research: strengths, weaknesses and ...
Finally, some closing thoughts about nonexperimental research and the appropriate use of causal language are presented.,A review of the extant social science literature was consulted to inform this paper.,Nonexperimental research, when reported accurately, makes a tremendous contribution because it can be used for conducting research when ...
1.6: Non-Experimental Research
Non-experimental research is research that lacks the manipulation of an independent variable. Rather than manipulating an independent variable, researchers conducting non-experimental research simply measure variables as they naturally occur (in the lab or real world). Most researchers in social sciences consider the distinction between ...
A Primer to Experimental and Nonexperimental Quantitative Research: The
There are two broad areas of inquiry in quantitative research (Fig 1).There is research that aims to describe or explore diseases, conditions, or variables; and there is research that tries to find a causal relationship among variables, conditions, and real-life situations. 2 These two fields of research do not exist independently; they talk to each other.
6.2 Nonexperimental Research
6.2 Nonexperimental Research Nonexperimental research is research that lacks manipulation of an independent variable and/or random assignment of participants to conditions.While the distinction between experimental and nonexperimental research is considered important, it does not mean that nonexperimental research is less important or inferior to experimental research (Price, Jhangiani ...
Overview of Non-Experimental Research
Non-experimental research is research that lacks the manipulation of an independent variable. Rather than manipulating an independent variable, researchers conducting non-experimental research simply measure variables as they naturally occur (in the lab or real world). Most researchers in psychology consider the distinction between experimental ...
Nonexperimental research: strengths, weaknesses and issues of precision
The research design is non-experimental. A non-experimental research design as defined by Reio (2016) is the predominant method researchers use in the social sciences. The design is useful when ...
Toward a New Classification of Nonexperimental Quantitative Research
In this paper, the present treatment of nonexperimental methods is reviewed and critiqued, and a new, two-dimensional classification of nonexperimental quantitative research is proposed. The first dimension is based on the primary "research objective" (i.e., description, prediction, and explanation), and the second dimension is called the ...
(PDF) Nonexperimental Quantitative Research and Its Role in Guiding
Qualitative research is. one type of nonexperimental research commonly found. in special education journals. However, as McDuff ie. and Scruggs (2008 [this issue]) conclude, qualitative. research ...
Designs of Empirical Evaluations of Nonexperimental Methods in Field
Assessing correspondence between experimental and non-experimental estimates in within-study-comparisons. Evaluation Review, 42(2), 214-247. Crossref. Google Scholar. ... MDRC working paper in research methodology. New York, NY. Google Scholar. Tang Y., Cook T. D. (2018). Statistical power for the comparative regression discontinuity design ...
Research study designs: Non-experimental
The category of non-experimental designs is the most heterogeneous of the three classification categories (experimental, quasi-experimental, and non-experimental). Examples of the most common non-experimental designs are listed in Table 1. Although, in general, this category has the lowest level of scientific rigor, each design within this category varies as to its own individual level of ...
Experimental Vs Non-Experimental Research: 15 Key Differences
Definitions. Experimental research is the type of research that uses a scientific approach towards manipulating one or more control variables and measuring their defect on the dependent variables, while non-experimental research is the type of research that does not involve the manipulation of control variables.
Planning and Conducting Clinical Research: The Whole Process
Experimental or non-experimental: Non-experimental research (i.e., "observational"), in contrast to experimental, involves data collection of the study participants in their natural or real-world environments. Non-experimental researches are usually the diagnostic and prognostic studies with cross-sectional in data collection.
Experimental and Non-Experimental Methods
The goal of KQ 4 was to describe experimental and non-experimental methods in terms of study design and analytic approach. Key Findings Of the 294 studies included in this review, 156 were natural experiment studies (53%), 118 were experimental studies (40%), and 20 provided insufficient information to determine if a natural experiment took ...
Non-experimental research designs:: Investigating the spatial
Besides, Non-Experimental research incorporates strategies used to perform quantitative research in which no variables in the study are manipulated [14]. The Non-Experimental research comprises: ...

7.1 Overview of Nonexperimental Research

What Is Nonexperimental Research?

When to Use Nonexperimental Research

Types of Nonexperimental Research

Internal Validity Revisited

Key Takeaways

Non-experimental research: What it is, overview & advantages

Characteristics of non-experimental research

Types of non-experimental research

When to use non-experimental research

Advantages and disadvantages

Difference between experimental and non-experimental research

MORE LIKE THIS

Was The Experience Memorable? (Part II) — Tuesday CX Thoughts

Data Discovery: What it is, Importance, Process + Use Cases

Competitive Insights : Importance, How to Get + Usage

Participant Engagement: Strategies + Improving Interaction

Other categories

Overview of Nonexperimental Research

What Is Nonexperimental Research?

When to Use Nonexperimental Research

Types of Nonexperimental Research

Internal Validity Revisited

Share This Book

Save citation to file

Add to My Bibliography

Determining the level of evidence: Nonexperimental research designs

Similar articles

Related information

Other Literature Sources

To read this content please select one of the options below:

Design/methodology/approach

Originality/value

Related articles

Report an issue or find answers to frequently asked questions

6.2 Nonexperimental Research

When to Use Nonexperimental Research

Types of Nonexperimental Research

Share This Book

Overview of Non-Experimental Research

Learning Objectives

What Is Non-Experimental Research?

When to Use Non-Experimental Research

Types of Non-Experimental Research

Cross-Sectional, Longitudinal, and Cross-Sequential Studies

Internal Validity Revisited

Share This Book

What is Experimental Research?

Pre-experimental Research

Quasi-experimental Research

True Experimental Research

Pros of True Experimental Research

Cons of True Experimental Research

What is Non-Experimental Research?

Cross-sectional Research

Correlational Research

Observational Research

Pros of Observational Research

Cons of Observational Research

What Are The Differences Between Experimental and Non-Experimental Research?

Experimental Research Designs: Types, Examples & Methods

Simpson’s Paradox & How to Avoid it in Experimental Research

What is Experimenter Bias? Definition, Types & Mitigation

Response vs Explanatory Variables: Definition & Examples

Formplus - For Seamless Data Collection

Methods for Evaluating Natural Experiments in Obesity: Systematic Evidence Review [Internet].

Key Findings

Data Abstraction

Data Synthesis

Description of Methods Used in Natural Experiment Studies

Description of Methods Used in Experimental Studies

Description of Methods Used in Other Study Approaches

Other titles in these collections

Recent Activity

Non-experimental research designs:: Investigating the spatial distribution and social ecology of male homicide

COMMENTS