what type of research method

Techniques or tools used for gathering research data include:

SAGE research methods

• SAGE research methods online This link opens in a new window Research methods tool to help researchers gather full-text resources, design research projects, understand a particular method and write up their research. Includes access to collections of video, business cases and eBooks,

Help and Information

• Next: Finding qualitative research >>
• Last Updated: Aug 19, 2024 3:39 PM
• URL: https://libguides.newcastle.edu.au/researchmethods

How To Choose Your Research Methodology

Qualitative vs quantitative vs mixed methods.

By: Derek Jansen (MBA). Expert Reviewed By: Dr Eunice Rautenbach | June 2021

Without a doubt, one of the most common questions we receive at Grad Coach is “ How do I choose the right methodology for my research? ”. It’s easy to see why – with so many options on the research design table, it’s easy to get intimidated, especially with all the complex lingo!

In this post, we’ll explain the three overarching types of research – qualitative, quantitative and mixed methods – and how you can go about choosing the best methodological approach for your research.

Overview: Choosing Your Methodology

Understanding the options – Qualitative research – Quantitative research – Mixed methods-based research

Choosing a research methodology – Nature of the research – Research area norms – Practicalities

1. Understanding the options

Before we jump into the question of how to choose a research methodology, it’s useful to take a step back to understand the three overarching types of research – qualitative , quantitative and mixed methods -based research. Each of these options takes a different methodological approach.

Qualitative research utilises data that is not numbers-based. In other words, qualitative research focuses on words , descriptions , concepts or ideas – while quantitative research makes use of numbers and statistics. Qualitative research investigates the “softer side” of things to explore and describe, while quantitative research focuses on the “hard numbers”, to measure differences between variables and the relationships between them.

Importantly, qualitative research methods are typically used to explore and gain a deeper understanding of the complexity of a situation – to draw a rich picture . In contrast to this, quantitative methods are usually used to confirm or test hypotheses . In other words, they have distinctly different purposes. The table below highlights a few of the key differences between qualitative and quantitative research – you can learn more about the differences here.

• Uses an inductive approach
• Is used to build theories
• Takes a subjective approach
• Adopts an open and flexible approach
• The researcher is close to the respondents
• Interviews and focus groups are oftentimes used to collect word-based data.
• Generally, draws on small sample sizes
• Uses qualitative data analysis techniques (e.g. content analysis , thematic analysis , etc)
• Uses a deductive approach
• Is used to test theories
• Takes an objective approach
• Adopts a closed, highly planned approach
• The research is disconnected from respondents
• Surveys or laboratory equipment are often used to collect number-based data.
• Generally, requires large sample sizes
• Uses statistical analysis techniques to make sense of the data

Mixed methods -based research, as you’d expect, attempts to bring these two types of research together, drawing on both qualitative and quantitative data. Quite often, mixed methods-based studies will use qualitative research to explore a situation and develop a potential model of understanding (this is called a conceptual framework), and then go on to use quantitative methods to test that model empirically.

In other words, while qualitative and quantitative methods (and the philosophies that underpin them) are completely different, they are not at odds with each other. It’s not a competition of qualitative vs quantitative. On the contrary, they can be used together to develop a high-quality piece of research. Of course, this is easier said than done, so we usually recommend that first-time researchers stick to a single approach , unless the nature of their study truly warrants a mixed-methods approach.

The key takeaway here, and the reason we started by looking at the three options, is that it’s important to understand that each methodological approach has a different purpose – for example, to explore and understand situations (qualitative), to test and measure (quantitative) or to do both. They’re not simply alternative tools for the same job. 

Right – now that we’ve got that out of the way, let’s look at how you can go about choosing the right methodology for your research.

2. How to choose a research methodology

To choose the right research methodology for your dissertation or thesis, you need to consider three important factors . Based on these three factors, you can decide on your overarching approach – qualitative, quantitative or mixed methods. Once you’ve made that decision, you can flesh out the finer details of your methodology, such as the sampling , data collection methods and analysis techniques (we discuss these separately in other posts ).

The three factors you need to consider are:

• The nature of your research aims, objectives and research questions
• The methodological approaches taken in the existing literature
• Practicalities and constraints

Let’s take a look at each of these.

Factor #1: The nature of your research

As I mentioned earlier, each type of research (and therefore, research methodology), whether qualitative, quantitative or mixed, has a different purpose and helps solve a different type of question. So, it’s logical that the key deciding factor in terms of which research methodology you adopt is the nature of your research aims, objectives and research questions .

But, what types of research exist?

Broadly speaking, research can fall into one of three categories:

• Exploratory – getting a better understanding of an issue and potentially developing a theory regarding it
• Confirmatory – confirming a potential theory or hypothesis by testing it empirically
• A mix of both – building a potential theory or hypothesis and then testing it

As a rule of thumb, exploratory research tends to adopt a qualitative approach , whereas confirmatory research tends to use quantitative methods . This isn’t set in stone, but it’s a very useful heuristic. Naturally then, research that combines a mix of both, or is seeking to develop a theory from the ground up and then test that theory, would utilize a mixed-methods approach.

Let’s look at an example in action.

If your research aims were to understand the perspectives of war veterans regarding certain political matters, you’d likely adopt a qualitative methodology, making use of interviews to collect data and one or more qualitative data analysis methods to make sense of the data.

If, on the other hand, your research aims involved testing a set of hypotheses regarding the link between political leaning and income levels, you’d likely adopt a quantitative methodology, using numbers-based data from a survey to measure the links between variables and/or constructs .

So, the first (and most important thing) thing you need to consider when deciding which methodological approach to use for your research project is the nature of your research aims , objectives and research questions. Specifically, you need to assess whether your research leans in an exploratory or confirmatory direction or involves a mix of both.

The importance of achieving solid alignment between these three factors and your methodology can’t be overstated. If they’re misaligned, you’re going to be forcing a square peg into a round hole. In other words, you’ll be using the wrong tool for the job, and your research will become a disjointed mess.

If your research is a mix of both exploratory and confirmatory, but you have a tight word count limit, you may need to consider trimming down the scope a little and focusing on one or the other. One methodology executed well has a far better chance of earning marks than a poorly executed mixed methods approach. So, don’t try to be a hero, unless there is a very strong underpinning logic.

Need a helping hand?

Factor #2: The disciplinary norms

Choosing the right methodology for your research also involves looking at the approaches used by other researchers in the field, and studies with similar research aims and objectives to yours. Oftentimes, within a discipline, there is a common methodological approach (or set of approaches) used in studies. While this doesn’t mean you should follow the herd “just because”, you should at least consider these approaches and evaluate their merit within your context.

A major benefit of reviewing the research methodologies used by similar studies in your field is that you can often piggyback on the data collection techniques that other (more experienced) researchers have developed. For example, if you’re undertaking a quantitative study, you can often find tried and tested survey scales with high Cronbach’s alphas. These are usually included in the appendices of journal articles, so you don’t even have to contact the original authors. By using these, you’ll save a lot of time and ensure that your study stands on the proverbial “shoulders of giants” by using high-quality measurement instruments .

Of course, when reviewing existing literature, keep point #1 front of mind. In other words, your methodology needs to align with your research aims, objectives and questions. Don’t fall into the trap of adopting the methodological “norm” of other studies just because it’s popular. Only adopt that which is relevant to your research.

Factor #3: Practicalities

When choosing a research methodology, there will always be a tension between doing what’s theoretically best (i.e., the most scientifically rigorous research design ) and doing what’s practical , given your constraints . This is the nature of doing research and there are always trade-offs, as with anything else.

But what constraints, you ask?

When you’re evaluating your methodological options, you need to consider the following constraints:

• Data access
• Equipment and software
• Your knowledge and skills

Let’s look at each of these.

Constraint #1: Data access

The first practical constraint you need to consider is your access to data . If you’re going to be undertaking primary research , you need to think critically about the sample of respondents you realistically have access to. For example, if you plan to use in-person interviews , you need to ask yourself how many people you’ll need to interview, whether they’ll be agreeable to being interviewed, where they’re located, and so on.

If you’re wanting to undertake a quantitative approach using surveys to collect data, you’ll need to consider how many responses you’ll require to achieve statistically significant results. For many statistical tests, a sample of a few hundred respondents is typically needed to develop convincing conclusions.

So, think carefully about what data you’ll need access to, how much data you’ll need and how you’ll collect it. The last thing you want is to spend a huge amount of time on your research only to find that you can’t get access to the required data.

Constraint #2: Time

The next constraint is time. If you’re undertaking research as part of a PhD, you may have a fairly open-ended time limit, but this is unlikely to be the case for undergrad and Masters-level projects. So, pay attention to your timeline, as the data collection and analysis components of different methodologies have a major impact on time requirements . Also, keep in mind that these stages of the research often take a lot longer than originally anticipated.

Another practical implication of time limits is that it will directly impact which time horizon you can use – i.e. longitudinal vs cross-sectional . For example, if you’ve got a 6-month limit for your entire research project, it’s quite unlikely that you’ll be able to adopt a longitudinal time horizon. 

Constraint #3: Money

As with so many things, money is another important constraint you’ll need to consider when deciding on your research methodology. While some research designs will cost near zero to execute, others may require a substantial budget .

Some of the costs that may arise include:

• Software costs – e.g. survey hosting services, analysis software, etc.
• Promotion costs – e.g. advertising a survey to attract respondents
• Incentive costs – e.g. providing a prize or cash payment incentive to attract respondents
• Equipment rental costs – e.g. recording equipment, lab equipment, etc.
• Travel costs
• Food & beverages

These are just a handful of costs that can creep into your research budget. Like most projects, the actual costs tend to be higher than the estimates, so be sure to err on the conservative side and expect the unexpected. It’s critically important that you’re honest with yourself about these costs, or you could end up getting stuck midway through your project because you’ve run out of money.

Constraint #4: Equipment & software

Another practical consideration is the hardware and/or software you’ll need in order to undertake your research. Of course, this variable will depend on the type of data you’re collecting and analysing. For example, you may need lab equipment to analyse substances, or you may need specific analysis software to analyse statistical data. So, be sure to think about what hardware and/or software you’ll need for each potential methodological approach, and whether you have access to these.

Constraint #5: Your knowledge and skillset

The final practical constraint is a big one. Naturally, the research process involves a lot of learning and development along the way, so you will accrue knowledge and skills as you progress. However, when considering your methodological options, you should still consider your current position on the ladder.

Some of the questions you should ask yourself are:

• Am I more of a “numbers person” or a “words person”?
• How much do I know about the analysis methods I’ll potentially use (e.g. statistical analysis)?
• How much do I know about the software and/or hardware that I’ll potentially use?
• How excited am I to learn new research skills and gain new knowledge?
• How much time do I have to learn the things I need to learn?

Answering these questions honestly will provide you with another set of criteria against which you can evaluate the research methodology options you’ve shortlisted.

So, as you can see, there is a wide range of practicalities and constraints that you need to take into account when you’re deciding on a research methodology. These practicalities create a tension between the “ideal” methodology and the methodology that you can realistically pull off. This is perfectly normal, and it’s your job to find the option that presents the best set of trade-offs.

Recap: Choosing a methodology

In this post, we’ve discussed how to go about choosing a research methodology. The three major deciding factors we looked at were:

• Exploratory
• Confirmatory
• Combination
• Research area norms
• Hardware and software
• Your knowledge and skillset

If you have any questions, feel free to leave a comment below. If you’d like a helping hand with your research methodology, check out our 1-on-1 research coaching service , or book a free consultation with a friendly Grad Coach.

Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

Very useful and informative especially for beginners

Nice article! I’m a beginner in the field of cybersecurity research. I am a Telecom and Network Engineer and Also aiming for PhD scholarship.

I find the article very informative especially for my decitation it has been helpful and an eye opener.

Hi I am Anna ,

I am a PHD candidate in the area of cyber security, maybe we can link up

The Examples shows by you, for sure they are really direct me and others to knows and practices the Research Design and prepration.

I found the post very informative and practical.

I struggle so much with designs of the research for sure!

I’m the process of constructing my research design and I want to know if the data analysis I plan to present in my thesis defense proposal possibly change especially after I gathered the data already.

Thank you so much this site is such a life saver. How I wish 1-1 coaching is available in our country but sadly it’s not.

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Ultimate Guide to the 7 Types of Research: Definitions, Examples, Advantages and Limitations!

About the article : different types of research.

Research is a fundamental aspect of any field of study, providing a systematic approach to gather and analyze information. It plays a crucial role in expanding knowledge, solving problems, and making informed decisions. Understanding the different types of research is essential for researchers to choose the most appropriate method for their study.

In this article, we will explore the various types of research methods commonly used in academic and professional settings. Each type of research has its own unique characteristics, strengths, and limitations. By gaining a comprehensive understanding of these research types, researchers can effectively design and conduct their studies to achieve their objectives.

Exploratory Research

Exploratory research is a type of research that is used to investigate a problem that is not clearly defined and gain a better understanding of the existing problem. It is often conducted when a researcher has just begun an investigation and wishes to understand the topic generally.

There are two main methods of conducting exploratory research: primary research and secondary research. Primary research involves collecting new data directly from the source, while secondary research involves analyzing existing data that has already been collected by others.

Under these two broad types, various methods can be employed to gather information. These methods include surveys, interviews , focus groups, observations, and case studies. Each method has its own advantages and disadvantages, and the choice of method depends on the nature of the research question and the available resources.

Exploratory research is valuable because it helps researchers gain insights and generate hypotheses for further investigation. It allows them to explore new areas of study and discover potential relationships between variables. However, it is important to note that exploratory research does not provide definitive answers or conclusive results. Instead, it lays the foundation for more in-depth research and helps researchers refine their research questions and methodologies.

Descriptive Research

Descriptive research is a methodological approach that seeks to depict the characteristics of a phenomenon or subject under investigation. It involves observing and measuring without manipulating variables, allowing researchers to identify characteristics, trends, and correlations. The main goal of descriptive research is to provide a detailed description of the population or phenomenon being studied. This type of research focuses on answering questions such as how, what, when, and where.

There are three basic approaches for gathering data in descriptive research: observational, case study, and survey. Observational research involves observing and recording behavior in its natural setting. Case study research involves in-depth analysis of a single individual, group, or situation. Survey research involves collecting data from a sample of individuals through questionnaires or interviews.

Descriptive research is particularly useful when researchers want to describe specific behaviors, characteristics, or trends as they occur in the environment. It provides a foundation for further research and can help generate hypotheses for future studies.

However, one limitation of descriptive research is that it does not establish causal relationships between variables. It can only provide a snapshot of the current state of the population or phenomenon being studied. Despite this limitation, descriptive research plays a crucial role in understanding and describing various aspects of the world around us.

Experimental Research

Experimental research is a quantitative research method with a scientific approach. It is the most familiar type of research design for individuals in the physical sciences and a host of other fields. This type of research design is popular in scientific experiments, social sciences, medical science, etc. Experimental research involves manipulating one or more variables to observe the effect on another variable. It aims to establish cause-and-effect relationships between variables. The researcher carefully controls and manipulates the independent variable(s) while measuring the dependent variable(s).

There are two broad categories of experimental research designs: true experimental designs and quasi-experimental designs. True experimental designs involve random assignment of participants to different groups and manipulation of the independent variable. Quasi-experimental designs lack random assignment but still involve manipulation of the independent variable.

One advantage of experimental research is its ability to establish causal relationships. By manipulating variables and controlling extraneous factors, researchers can determine whether changes in the independent variable(s) cause changes in the dependent variable(s). This allows for a more confident understanding of cause and effect.

Another advantage of experimental research is its versatility. It can be used in various fields and disciplines, allowing researchers to investigate a wide range of phenomena. Whether it’s testing the effectiveness of a new drug, studying the impact of different teaching methods, or exploring the relationship between variables, experimental research provides a powerful tool for scientific inquiry.

However, experimental research also has some limitations. One limitation is the potential for artificiality. In a controlled laboratory setting, variables may be manipulated in a way that does not fully reflect real-world conditions. This can limit the generalizability of the findings to real-life situations. Additionally, experimental research may face ethical considerations. Manipulating variables and potentially exposing participants to certain conditions can raise ethical concerns. Researchers must ensure that the benefits of the study outweigh any potential risks or harm to participants.

Correlational Research

Correlational research is a type of non-experimental research that focuses on observing and measuring the relationship between two or more variables. Unlike experimental research, the researcher does not control or manipulate the variables in correlational research. The main purpose of correlational research is to determine if there is a statistical relationship between the variables being studied. It involves comparing two variables and data sources, assessing the relationship between them, and identifying any trends or patterns.

There are several types of correlational studies that can be conducted. One type is positive correlation, which occurs when an increase in one variable is associated with an increase in another variable. For example, there may be a positive correlation between income and education level, meaning that as income increases, education level also tends to increase.

On the other hand, negative correlation refers to a relationship where an increase in one variable is associated with a decrease in another variable. An example of negative correlation could be the relationship between hours spent studying and test scores. As the number of hours spent studying increases, test scores tend to decrease. Lastly, zero correlation indicates that there is no relationship between the variables being studied. This means that changes in one variable do not affect the other variable. For instance, there may be zero correlation between shoe size and intelligence.

Correlational research is commonly used in various fields, including psychology, sociology, and marketing. It provides valuable insights into the relationships between variables and helps researchers understand the patterns and trends in data. However, correlational research has its limitations. Since it does not involve manipulation of variables, it cannot establish causation. It can only identify associations between variables. Additionally, correlational research relies on the accuracy and reliability of the data collected, which can be influenced by various factors.

Causal-Comparative Research

Causal-comparative research is a methodology used to identify cause-effect relationships between independent and dependent variables. It is a type of research method where the researcher tries to find out if there is a causal effect relationship between two or more groups or variables.

The main objective of causal-comparative research is to determine the cause or reason for pre-existing differences in groups of individuals. This research design involves comparing groups that have already been formed based on a specific characteristic or condition. The researcher then analyzes the differences between these groups to identify any causal relationships.

There are two types of causal-comparative research designs: retrospective and prospective. Retrospective causal-comparative research looks at past events or conditions to determine the cause-effect relationship. On the other hand, prospective causal-comparative research looks at current or future events or conditions to identify the causal relationship.

One example of causal-comparative research is a study comparing the critical thinking skills of students who were taught using the inquiry method versus those who were taught using the lecture method. The researcher would compare the two groups of students and analyze the differences in their critical thinking abilities to determine if the teaching method had a causal effect on their skills.

Causal-comparative research has its advantages and disadvantages. One advantage is that it allows researchers to study cause-effect relationships in situations where it is not possible or ethical to manipulate variables. It also provides valuable insights into the factors that contribute to differences between groups.

However, a limitation of causal-comparative research is that it cannot establish a cause-effect relationship with certainty, as there may be other variables or factors that influence the observed differences between groups.

Qualitative Research

Qualitative research is a type of research that explores and provides deeper insights into real-world problems. Instead of collecting numerical data, qualitative research deals with data types such as text, audio, images, and video, focusing on the variety of human experiences and perspectives.

There are different types of qualitative research methods that researchers can use depending on their study requirements. Some common qualitative research methods include in-depth interviews, focus groups, ethnographic research, content analysis, and case study. In-depth interviews involve conducting one-on-one interviews with participants to gather detailed information about their experiences, opinions, and perspectives. This method allows researchers to delve deep into the thoughts and feelings of individuals and gain a comprehensive understanding of their experiences.

Focus groups involve bringing together a small group of participants to discuss a specific topic or issue. The group dynamic allows for the exploration of different perspectives and the generation of rich and diverse insights. Focus groups are particularly useful for understanding social interactions and group dynamics. Ethnographic research involves immersing the researcher in the natural environment of the participants to observe and understand their behaviors, beliefs, and cultural practices. This method allows for a holistic understanding of the social and cultural context in which individuals operate.

Content analysis involves systematically analyzing textual, audio, or visual data to identify patterns, themes, and meanings. This method is often used to analyze documents, media content, or online discussions to gain insights into societal trends, attitudes, or representations. Case study research involves in-depth investigation of a specific individual, group, or organization. Researchers collect and analyze multiple sources of data to gain a comprehensive understanding of the case under study. Case studies are particularly useful for exploring complex phenomena or unique situations.

Qualitative research provides several advantages. It allows researchers to explore complex and nuanced phenomena in depth, providing rich and detailed insights. It also allows for flexibility and adaptability in the research process, as researchers can adjust their approach based on emerging findings. Additionally, qualitative research is often used to generate hypotheses or theories that can be further tested using quantitative research methods.

However, qualitative research also has some limitations. The findings are often context-specific and may not be generalizable to a larger population. The subjective nature of qualitative data collection and analysis can introduce bias and interpretation challenges. Qualitative research also requires significant time and resources, as data collection and analysis can be time-consuming and labor-intensive.

Quantitative Research

Quantitative research is a type of research that involves collecting and analyzing numerical data to describe characteristics, find correlations, or test hypotheses. It is characterized by structured tools like surveys, substantial sample sizes, closed-ended questions, and reliance on prior studies.

There are two main types of quantitative research: primary and secondary. Primary quantitative research involves collecting data directly from the source, such as through surveys or experiments. Secondary quantitative research, on the other hand, involves analyzing existing data that has been collected by someone else.

Quantitative research methods can be used to quantify opinions, behaviors, attitudes, and other definitive variables related to the market, customers, competitors, and more. It provides a systematic and objective approach to studying phenomena and allows for statistical analysis to draw conclusions.

There are several types of quantitative research designs that can be used, depending on the research objectives . These include descriptive research, correlational research, causal-comparative research, and experimental research as per explained above.

In conclusion, understanding the different types of research is essential for conducting effective and meaningful studies. Each type of research has its own strengths and limitations, and researchers must carefully consider which approach is the most appropriate for their specific research question and objectives. It is important to recognize that research is an iterative process, and different types of research may be used at different stages of a study.

In summary, the various types of research offer different perspectives and methodologies for investigating and understanding the world around us. By utilizing a combination of these approaches, researchers can gain a comprehensive understanding of complex phenomena and make meaningful contributions to their fields of study.

Leave a Comment Cancel reply

Save my name, email, and website in this browser for the next time I comment.

Related articles

How to Formulate Research Questions in a Research Proposal? Discover The No. 1 Easiest Template Here!

7 Easy Step-By-Step Guide of Using ChatGPT: The Best Literature Review Generator for Time-Saving Academic Research

Writing Engaging Introduction in Research Papers : 7 Tips and Tricks!

Understanding Comparative Frameworks: Their Importance, Components, Examples and 8 Best Practices

Revolutionizing Effective Thesis Writing for PhD Students Using Artificial Intelligence!

3 Types of Interviews in Qualitative Research: An Essential Research Instrument and Handy Tips to Conduct Them

Highlight Abstracts: An Ultimate Guide For Researchers!

Crafting Critical Abstracts: 11 Expert Strategies for Summarizing Research

Home > Blog > Tips for Online Students > A Comprehensive Guide to Different Types of Research

Higher Education News , Tips for Online Students , Tips for Students

A Comprehensive Guide to Different Types of Research

Updated: June 19, 2024

Published: June 15, 2024

When embarking on a research project, selecting the right methodology can be the difference between success and failure. With various methods available, each suited to different types of research, it’s essential you make an informed choice. This blog post will provide tips on how to choose a research methodology that best fits your research goals .

We’ll start with definitions: Research is the systematic process of exploring, investigating, and discovering new information or validating existing knowledge. It involves defining questions, collecting data, analyzing results, and drawing conclusions.

Meanwhile, a research methodology is a structured plan that outlines how your research is to be conducted. A complete methodology should detail the strategies, processes, and techniques you plan to use for your data collection and analysis.

Research Methods

The first step of a research methodology is to identify a focused research topic, which is the question you seek to answer. By setting clear boundaries on the scope of your research, you can concentrate on specific aspects of a problem without being overwhelmed by information. This will produce more accurate findings. 

Along with clarifying your research topic, your methodology should also address your research methods. Let’s look at the four main types of research: descriptive, correlational, experimental, and diagnostic.

Descriptive Research

Descriptive research is an approach designed to describe the characteristics of a population systematically and accurately. This method focuses on answering “what” questions by providing detailed observations about the subject. Descriptive research employs surveys, observational studies , and case studies to gather qualitative or quantitative data. 

A real-world example of descriptive research is a survey investigating consumer behavior toward a competitor’s product. By analyzing the survey results, the company can gather detailed insights into how consumers perceive a competitor’s product, which can inform their marketing strategies and product development.

Correlational Research

Correlational research examines the statistical relationship between two or more variables to determine whether a relationship exists. Correlational research is particularly useful when ethical or practical constraints prevent experimental manipulation. It is often employed in fields such as psychology, education, and health sciences to provide insights into complex real-world interactions, helping to develop theories and inform further experimental research.

An example of correlational research is the study of the relationship between smoking and lung cancer. Researchers observe and collect data on individuals’ smoking habits and the incidence of lung cancer to determine if there is a correlation between the two variables. This type of research helps identify patterns and relationships, indicating whether increased smoking is associated with higher rates of lung cancer.

Experimental Research

Experimental research is a scientific approach where researchers manipulate one or more independent variables to observe their effect on a dependent variable. This method is designed to establish cause-and-effect relationships. Fields like psychology , medicine, and social sciences frequently employ experimental research to test hypotheses and theories under controlled conditions. 

A real-world example of experimental research is Pavlov’s Dog experiment. In this experiment, Ivan Pavlov demonstrated classical conditioning by ringing a bell each time he fed his dogs. After repeating this process multiple times, the dogs began to salivate just by hearing the bell, even when no food was presented. This experiment helped to illustrate how certain stimuli can elicit specific responses through associative learning.

Diagnostic Research

Diagnostic research tries to accurately diagnose a problem by identifying its underlying causes. This type of research is crucial for understanding complex situations where a precise diagnosis is necessary for formulating effective solutions. It involves methods such as case studies and data analysis and often integrates both qualitative and quantitative data to provide a comprehensive view of the issue at hand. 

An example of diagnostic research is studying the causes of a specific illness outbreak. During an outbreak of a respiratory virus, researchers might conduct diagnostic research to determine the factors contributing to the spread of the virus. This could involve analyzing patient data, testing environmental samples, and evaluating potential sources of infection. The goal is to identify the root causes and contributing factors to develop effective containment and prevention strategies.

Using an established research method is imperative, no matter if you are researching for marketing , technology , healthcare , engineering, or social science. A methodology lends legitimacy to your research by ensuring your data is both consistent and credible. A well-defined methodology also enhances the reliability and validity of the research findings, which is crucial for drawing accurate and meaningful conclusions. 

Additionally, methodologies help researchers stay focused and on track, limiting the scope of the study to relevant questions and objectives. This not only improves the quality of the research but also ensures that the study can be replicated and verified by other researchers, further solidifying its scientific value.

How to Choose a Research Methodology

Choosing the best research methodology for your project involves several key steps to ensure that your approach aligns with your research goals and questions. Here’s a simplified guide to help you make the best choice.

Understand Your Goals

Clearly define the objectives of your research. What do you aim to discover, prove, or understand? Understanding your goals helps in selecting a methodology that aligns with your research purpose.

Consider the Nature of Your Data

Determine whether your research will involve numerical data, textual data, or both. Quantitative methods are best for numerical data, while qualitative methods are suitable for textual or thematic data.

Understand the Purpose of Each Methodology

Becoming familiar with the four types of research – descriptive, correlational, experimental, and diagnostic – will enable you to select the most appropriate method for your research. Many times, you will want to use a combination of methods to gather meaningful data. 

Evaluate Resources and Constraints

Consider the resources available to you, including time, budget, and access to data. Some methodologies may require more resources or longer timeframes to implement effectively.

Review Similar Studies

Look at previous research in your field to see which methodologies were successful. This can provide insights and help you choose a proven approach.

By following these steps, you can select a research methodology that best fits your project’s requirements and ensures robust, credible results.

Completing Your Research Project

Upon completing your research, the next critical step is to analyze and interpret the data you’ve collected. This involves summarizing the key findings, identifying patterns, and determining how these results address your initial research questions. By thoroughly examining the data, you can draw meaningful conclusions that contribute to the body of knowledge in your field. 

It’s essential that you present these findings clearly and concisely, using charts, graphs, and tables to enhance comprehension. Furthermore, discuss the implications of your results, any limitations encountered during the study, and how your findings align with or challenge existing theories.

Your research project should conclude with a strong statement that encapsulates the essence of your research and its broader impact. This final section should leave readers with a clear understanding of the value of your work and inspire continued exploration and discussion in the field.

Now that you know how to perform quality research , it’s time to get started! Applying the right research methodologies can make a significant difference in the accuracy and reliability of your findings. Remember, the key to successful research is not just in collecting data, but in analyzing it thoughtfully and systematically to draw meaningful conclusions. So, dive in, explore, and contribute to the ever-growing body of knowledge with confidence. Happy researching!

In this article

At UoPeople, our blog writers are thinkers, researchers, and experts dedicated to curating articles relevant to our mission: making higher education accessible to everyone. Read More

Home » Types of Research Methods

Types of Research Methods

When we talk about Types of Research Methods , we’re diving into the different ways researchers investigate and find answers to their questions. Think of research methods as tools in a toolbox, with each tool designed for a specific task.

Table of Contents

1. Qualitative Research

Imagine you’re an explorer trying to understand the culture of a remote village. You wouldn’t just count things, like how many houses there are; instead, you’d want to know the stories, beliefs, and feelings of the people living there. That’s what Qualitative Research is like. It’s all about diving deep into experiences, emotions, and narratives to understand the ‘why’ and ‘how’ of human behavior. Researchers might conduct interviews, observe behaviors, or analyze texts and artworks to gather rich, detailed insights.

2. Quantitative Research

Now, imagine you’re a scientist wanting to know if a new medicine works better than an old one. You’d probably give some people the new medicine and others the old one, then count how many in each group get better. This is Quantitative Research . It deals with numbers and statistics to answer questions like ‘how many’ or ‘how much’. By collecting numerical data and analyzing it, researchers can identify patterns and relationships, often leading to generalizable findings.

3. Mixed Methods

Sometimes, just counting things or just listening to stories isn’t enough. You might need both to get the full picture. That’s where Mixed Methods come in. It’s like using both a magnifying glass and a telescope; you get to see the intricate details up close and the big picture from afar. Researchers use a combination of qualitative and quantitative methods to benefit from the strengths of both, providing a more comprehensive understanding of the research problem.

4. Descriptive Research

Let’s say you’re curious about the daily routines of high school students. In Descriptive Research , you’d observe and describe what you see without trying to change anything. This method involves detailed observations to accurately depict situations or phenomena. It’s like painting a detailed landscape where every element is noted, but you’re not trying to interpret the scene or change the landscape.

5. Experimental Research

Imagine you’re curious about whether listening to classical music while studying improves test scores. In Experimental Research , you’d create a controlled environment, divide your subjects into groups, and introduce a variable (like classical music) to one group but not the other. Then, you’d compare the outcomes. This method allows researchers to determine cause-and-effect relationships by manipulating variables and controlling for outside influences.

6. Correlational Research

Suppose you wonder if there’s a relationship between the time students spend on social media and their grades. In Correlational Research , you’d collect data on both aspects and analyze it to see if there’s a link. However, it’s crucial to remember that correlation doesn’t imply causation; just because two things are related doesn’t mean one causes the other.

7. Action Research

Imagine you’re a teacher who wants to improve student engagement in your classroom. Action Research involves identifying a problem, implementing a strategy to address it, observing the results, and adjusting your approach based on the findings. It’s a cyclical process that combines research with action, often used in education, healthcare, and organizational development to bring about change.

8. Longitudinal and Cross-Sectional Research

Think of Longitudinal Research like taking a series of photographs of the same group of people every year to see how they change over time. It’s about observing the same subjects repeatedly over an extended period. On the other hand, Cross-Sectional Research is like taking a single snapshot of a diverse group at one moment in time to understand the current state of affairs. Each method has its strengths and is chosen based on the research question and available resources.

In summary, research methods are diverse, each suited to answering different types of questions. By selecting the appropriate method, researchers can gather the insights needed to advance our understanding of the world. Whether we’re exploring human behavior, testing new theories, or seeking to improve practices in various fields, these methods provide the foundation for building reliable and valuable knowledge.

You may also like

Types of Mixed Research Methods

What is Mixed Research Methods

Types of Quantitative Research Methods

Quantitative Research Methods

Types of Qualitative Research Methods

Qualitative Research Methods

• University Libraries
• Research Guides
• Topic Guides
• Research Methods Guide
• Research Design & Method

Research Methods Guide: Research Design & Method

• Introduction
• Survey Research
• Interview Research
• Data Analysis
• Resources & Consultation

Tutorial Videos: Research Design & Method

Research Methods (sociology-focused)

Qualitative vs. Quantitative Methods (intro)

Qualitative vs. Quantitative Methods (advanced)

FAQ: Research Design & Method

What is the difference between Research Design and Research Method?

Research design is a plan to answer your research question.  A research method is a strategy used to implement that plan.  Research design and methods are different but closely related, because good research design ensures that the data you obtain will help you answer your research question more effectively.

Which research method should I choose ?

It depends on your research goal.  It depends on what subjects (and who) you want to study.  Let's say you are interested in studying what makes people happy, or why some students are more conscious about recycling on campus.  To answer these questions, you need to make a decision about how to collect your data.  Most frequently used methods include:

• Observation / Participant Observation
• Focus Groups
• Experiments
• Secondary Data Analysis / Archival Study
• Mixed Methods (combination of some of the above)

One particular method could be better suited to your research goal than others, because the data you collect from different methods will be different in quality and quantity.   For instance, surveys are usually designed to produce relatively short answers, rather than the extensive responses expected in qualitative interviews.

What other factors should I consider when choosing one method over another?

Time for data collection and analysis is something you want to consider.  An observation or interview method, so-called qualitative approach, helps you collect richer information, but it takes time.  Using a survey helps you collect more data quickly, yet it may lack details.  So, you will need to consider the time you have for research and the balance between strengths and weaknesses associated with each method (e.g., qualitative vs. quantitative).

• << Previous: Introduction
• Next: Survey Research >>
• Last Updated: Aug 21, 2023 10:42 AM

• Tips & Tricks
• Website & Apps
• ChatGPT Blogs
• ChatGPT News
• ChatGPT Tutorial

Types of Research Methods Explained with Examples

Research methods are the various strategies, techniques, and tools that researchers use to collect and analyze data . These methods help researchers find answers to their questions and gain a better understanding of different topics. Whether conducting experiments, surveys, or interviews, choosing the right research method is crucial for obtaining accurate and reliable results.

In the ever-evolving world of academia and professional inquiry, understanding the various research methods is crucial for anyone looking to delve into a new study or project. Research is a systematic investigation aimed at discovering and interpreting facts , plays a pivotal role in expanding our knowledge across various fields.

Table of Content

What is Research?

Types of research methods, types of research methodology, difference between qualitative and quantitative research.

This article will explore the different types of research methods , how they are used, and their importance in the world of research.

Research is the process of studying a subject in detail to discover new information or understand it better. This can be anything from studying plants or animals, to learning how people think and behave, to finding new ways to cure diseases. People do research by asking questions, collecting information, and then looking at that information to find answers or learn new things.

This table provides a quick reference to understand the key aspects of each research type.

1. Qualitative Research

Qualitative research method is a methodological approach primarily used in fields like social sciences, anthropology, and psychology . It’s aimed at understanding human behavior and the motivations behind it. Qualitative research delves into the nature of phenomena through detailed, in-depth exploration.

Definition and Approach: Qualitative research focuses on understanding human behavior and the reasons that govern such behavior. It involves in-depth analysis of non-numerical data like texts, videos, or audio recordings.

Key Features:

• Emphasis on exploring complex phenomena
• Involves interviews, focus groups , and observations
• Generates rich, detailed data that are often subjective

Applications: Widely used in social sciences, marketing, and user experience research.

2. Quantitative Research

Quantitative research method is a systematic approach used in various scientific fields to quantify data and generalize findings from a sample to a larger population.

Definition and Approach: Quantitative research is centered around quantifying data and generalizing results from a sample to the population of interest. It involves statistical analysis and numerical data .

• Relies on structured data collection instruments
• Large sample sizes for generalizability
• Statistical methods to establish relationships between variables

Applications: Common in natural sciences, economics, and market research.

3. Descriptive Research

Descriptive research is a type of research method that is used to describe characteristics of a population or phenomenon being studied . It does not answer questions about how or why things are the way they are. Instead, it focuses on providing a snapshot of current conditions or describing what exists.

Definition and Approach: This Types of Research method aims to accurately describe characteristics of a particular phenomenon or population.

• Provides detailed insights without explaining why or how something happens
• Involves surveys and observations
• Often used as a preliminary research method

Applications: Used in demographic studies, census, and organizational reporting.

4. Analytical Research

Analytical research is a type of research that s eeks to understand the underlying factors or causes behind phenomena or relationships . It goes beyond descriptive research by attempting to explain why things happen and how they happen.

Definition and Approach: Analytical research method goes beyond description to understand the underlying reasons or causes.

• Involves comparing data and facts to make evaluations
• Critical thinking is a key component
• Often hypothesis-driven

Applications: Useful in scientific research, policy analysis, and business strategy.

5. Applied Research

Applied research is a type of scientific research method that aims to solve specific practical problems or address practical questions . Unlike fundamental research, which seeks to expand knowledge for knowledge’s sake, applied research is directed towards solving real-world issues .

Definition and Approach: Applied research focuses on finding solutions to practical problems.

• Direct practical application
• Often collaborative , involving stakeholders
• Results are immediately applicable

Applications: Used in healthcare, engineering, and technology development.

6. Fundamental Research

Fundamental research, also known as basic research or pure research, is a type of scientific research method that aims to expand the existing knowledge base. It is driven by curiosity, interest in a particular subject, or the pursuit of knowledge for knowledge’s sake , rather than with a specific practical application in mind.

Definition and Approach: Also known as basic or pure research, it aims to expand knowledge without a direct application in mind.

• Theoretical framework
• Focus on understanding fundamental principles
• Long-term in nature

Applications: Foundational in fields like physics, mathematics, and social sciences.

7. Exploratory Research

Exploratory research is a type of research method conducted for a problem that has not been clearly defined. Its primary goal is to gain insights and familiarity with the problem or to gain more information about a topic. Exploratory research is often conducted when a researcher or investigator does not know much about the issue and is looking to gather more information.

Definition and Approach: This type of research is conducted for a problem that has not been clearly defined.

• Flexible and unstructured
• Used to identify potential hypotheses
• Relies on secondary research like reviewing available literature

Applications: Often the first step in social science research and product development.

8. Conclusive Research

Conclusive research, also known as confirmatory research, is a type of research method that aims to confirm or deny a hypotheses or provide answers to specific research questions. It is used to make conclusive decisions or draw conclusions about the relationships among variables.

Definition and Approach: Conclusive research is designed to provide information that is useful in decision-making.

• Structured and methodical
• Aims to test hypotheses
• Involves experiments, surveys, and testing

Applications: Used in market research, clinical trials, and policy evaluations.

Here is detailed difference between the qualitative and quantitative research –

Understanding the different types of research methods is crucial for anyone embarking on a research project. Each type has its unique approach, methodology, and application area, making it essential to choose the right type for your specific research question or problem. This guide serves as a starting point for researchers to explore and select the most suitable research method for their needs, ensuring effective and reliable outcomes.

Types of Research Methods – FAQs

What are the 4 main types of research methods.

There are four main types of Quantitative research:  Descriptive, Correlational, Causal-Comparative/Quasi-Experimental, and Experimental Research . attempts to establish cause- effect relationships among the variables. These types of design are very similar to true experiments, but with some key differences.

What are the 5 main purpose of research?

The primary purposes of basic research (as opposed to applied research) are  documentation, discovery, interpretation, and the research and development (R&D) of methods and systems for the advancement of human knowledge .

What are 7 C’s of research?

The 7 C’s define the principles that are essential for conducting rigorous and credible research. They are Curiosity, Clarity, Conciseness, Correctness, Completeness, Coherence, Credibility.

• General Knowledge
• SSC/Banking
• tech-updates

Please Login to comment...

Types of irony explained (with examples), why can't a country print more currency - explained.

When is Easter 2024? - Date, Traditions, and Symbols Explained

What is youtube kids an app made just for kids, explained.

The Ballad of Songbirds and Snakes' Ending Explained: The Director and Producer Break It Down (Exclusive)

Project 2025 Explained: Impact on Public Health, Trump's Agenda, and Biden-Harris Opposition - Latest News

List of research institutes in india, studentship program for ayurveda research ken (spark), innovation in science pursuit for inspired research (inspire) programme, cognitive science research initiative (csri) - full form, history and objectives.

• Best External Hard Drives for Mac in 2024: Top Picks for MacBook Pro, MacBook Air & More
• How to Watch NFL Games Live Streams Free
• OpenAI o1 AI Model Launched: Explore o1-Preview, o1-Mini, Pricing & Comparison
• How to Merge Cells in Google Sheets: Step by Step Guide
• #geekstreak2024 – 21 Days POTD Challenge Powered By Deutsche Bank

Improve your Coding Skills with Practice

What kind of Experience do you want to share?

Research Methodologies

• What are research designs?
• What are research methodologies?

What are research methods?

Quantitative research methods, qualitative research methods, mixed method approach, selecting the best research method.

• Additional Sources

Research methods are different from research methodologies because they are the ways in which you will collect the data for your research project.  The best method for your project largely depends on your topic, the type of data you will need, and the people or items from which you will be collecting data.  The following boxes below contain a list of quantitative, qualitative, and mixed research methods.

• Closed-ended questionnaires/survey: These types of questionnaires or surveys are like "multiple choice" tests, where participants must select from a list of premade answers.  According to the content of the question, they must select the one that they agree with the most.  This approach is the simplest form of quantitative research because the data is easy to combine and quantify.
• Structured interviews: These are a common research method in market research because the data can be quantified.  They are strictly designed for little "wiggle room" in the interview process so that the data will not be skewed.  You can conduct structured interviews in-person, online, or over the phone (Dawson, 2019).

Constructing Questionnaires

When constructing your questions for a survey or questionnaire, there are things you can do to ensure that your questions are accurate and easy to understand (Dawson, 2019):

• Keep the questions brief and simple.
• Eliminate any potential bias from your questions.  Make sure that they do not word things in a way that favor one perspective over another.
• If your topic is very sensitive, you may want to ask indirect questions rather than direct ones.  This prevents participants from being intimidated and becoming unwilling to share their true responses.
• If you are using a closed-ended question, try to offer every possible answer that a participant could give to that question.
• Do not ask questions that assume something of the participant.  The question "How often do you exercise?" assumes that the participant exercises (when they may not), so you would want to include a question that asks if they exercise at all before asking them how often.
• Try and keep the questionnaire as short as possible.  The longer a questionnaire takes, the more likely the participant will not complete it or get too tired to put truthful answers.
• Promise confidentiality to your participants at the beginning of the questionnaire.

Quantitative Research Measures

When you are considering a quantitative approach to your research, you need to identify why types of measures you will use in your study.  This will determine what type of numbers you will be using to collect your data.  There are four levels of measurement:

• Nominal: These are numbers where the order of the numbers do not matter.  They aim to identify separate information.  One example is collecting zip codes from research participants.  The order of the numbers does not matter, but the series of numbers in each zip code indicate different information (Adamson and Prion, 2013).
• Ordinal: Also known as rankings because the order of these numbers matter.  This is when items are given a specific rank according to specific criteria.  A common example of ordinal measurements include ranking-based questionnaires, where participants are asked to rank items from least favorite to most favorite.  Another common example is a pain scale, where a patient is asked to rank their pain on a scale from 1 to 10 (Adamson and Prion, 2013).
• Interval: This is when the data are ordered and the distance between the numbers matters to the researcher (Adamson and Prion, 2013).  The distance between each number is the same.  An example of interval data is test grades.
• Ratio: This is when the data are ordered and have a consistent distance between numbers, but has a "zero point."  This means that there could be a measurement of zero of whatever you are measuring in your study (Adamson and Prion, 2013).  An example of ratio data is measuring the height of something because the "zero point" remains constant in all measurements.  The height of something could also be zero.

Focus Groups

This is when a select group of people gather to talk about a particular topic.  They can also be called discussion groups or group interviews (Dawson, 2019).  They are usually lead by a moderator  to help guide the discussion and ask certain questions.  It is critical that a moderator allows everyone in the group to get a chance to speak so that no one dominates the discussion.  The data that are gathered from focus groups tend to be thoughts, opinions, and perspectives about an issue.

Advantages of Focus Groups

• Only requires one meeting to get different types of responses.
• Less researcher bias due to participants being able to speak openly.
• Helps participants overcome insecurities or fears about a topic.
• The researcher can also consider the impact of participant interaction.

Disadvantages of Focus Groups

• Participants may feel uncomfortable to speak in front of an audience, especially if the topic is sensitive or controversial.
• Since participation is voluntary, not every participant may contribute equally to the discussion.
• Participants may impact what others say or think.
• A researcher may feel intimidated by running a focus group on their own.
• A researcher may need extra funds/resources to provide a safe space to host the focus group.
• Because the data is collective, it may be difficult to determine a participant's individual thoughts about the research topic.

Observation

There are two ways to conduct research observations:

• Direct Observation: The researcher observes a participant in an environment.  The researcher often takes notes or uses technology to gather data, such as a voice recorder or video camera.  The researcher does not interact or interfere with the participants.  This approach is often used in psychology and health studies (Dawson, 2019).
• Participant Observation:  The researcher interacts directly with the participants to get a better understanding of the research topic.  This is a common research method when trying to understand another culture or community.  It is important to decide if you will conduct a covert (participants do not know they are part of the research) or overt (participants know the researcher is observing them) observation because it can be unethical in some situations (Dawson, 2019).

Open-Ended Questionnaires

These types of questionnaires are the opposite of "multiple choice" questionnaires because the answer boxes are left open for the participant to complete.  This means that participants can write short or extended answers to the questions.  Upon gathering the responses, researchers will often "quantify" the data by organizing the responses into different categories.  This can be time consuming because the researcher needs to read all responses carefully.

Semi-structured Interviews

This is the most common type of interview where researchers aim to get specific information so they can compare it to other interview data.  This requires asking the same questions for each interview, but keeping their responses flexible.  This means including follow-up questions if a subject answers a certain way.  Interview schedules are commonly used to aid the interviewers, which list topics or questions that will be discussed at each interview (Dawson, 2019).

Theoretical Analysis

Often used for nonhuman research, theoretical analysis is a qualitative approach where the researcher applies a theoretical framework to analyze something about their topic.  A theoretical framework gives the researcher a specific "lens" to view the topic and think about it critically. it also serves as context to guide the entire study.  This is a popular research method for analyzing works of literature, films, and other forms of media.  You can implement more than one theoretical framework with this method, as many theories complement one another.

Common theoretical frameworks for qualitative research are (Grant and Osanloo, 2014):

• Behavioral theory
• Change theory
• Cognitive theory
• Content analysis
• Cross-sectional analysis
• Developmental theory
• Feminist theory
• Gender theory
• Marxist theory
• Queer theory
• Systems theory
• Transformational theory

Unstructured Interviews

These are in-depth interviews where the researcher tries to understand an interviewee's perspective on a situation or issue.  They are sometimes called life history interviews.  It is important not to bombard the interviewee with too many questions so they can freely disclose their thoughts (Dawson, 2019).

• Open-ended and closed-ended questionnaires: This approach means implementing elements of both questionnaire types into your data collection.  Participants may answer some questions with premade answers and write their own answers to other questions.  The advantage to this method is that you benefit from both types of data collection to get a broader understanding of you participants.  However, you must think carefully about how you will analyze this data to arrive at a conclusion.

Other mixed method approaches that incorporate quantitative and qualitative research methods depend heavily on the research topic.  It is strongly recommended that you collaborate with your academic advisor before finalizing a mixed method approach.

How do you determine which research method would be best for your proposal?  This heavily depends on your research objective.  According to Dawson (2019), there are several questions to ask yourself when determining the best research method for your project:

• Are you good with numbers and mathematics?
• Would you be interested in conducting interviews with human subjects?
• Would you enjoy creating a questionnaire for participants to complete?
• Do you prefer written communication or face-to-face interaction?
• What skills or experiences do you have that might help you with your research?  Do you have any experiences from past research projects that can help with this one?
• How much time do you have to complete the research?  Some methods take longer to collect data than others.
• What is your budget?  Do you have adequate funding to conduct the research in the method you  want?
• How much data do you need?  Some research topics need only a small amount of data while others may need significantly larger amounts.
• What is the purpose of your research? This can provide a good indicator as to what research method will be most appropriate.
• << Previous: What are research methodologies?
• Next: Additional Sources >>
• Last Updated: Aug 2, 2022 2:36 PM
• URL: https://library.tiffin.edu/researchmethodologies

• Translators
• Graphic Designers

Please enter the email address you used for your account. Your sign in information will be sent to your email address after it has been verified.

Understanding the Different Types of Research Methods

Research is at the heart of how we learn and grow as a society. It's all about digging into various topics, sparking discoveries, and shaping change. It doesn't matter whether we're talking about school, work, or personal projects—research helps us solve problems and build up our knowledge. It's not just about finding new things; it also helps us take a fresh look at what we already know.

But let's be clear, research isn't just jumping headfirst into the unknown. It requires careful planning, doing the work, and understanding the different research methods that guide us in collecting, examining, and presenting our findings.

Research methods are the techniques we use to gather and look at data while making sure we're thorough, fair, and trustworthy. These methods aren't meant to be one-size-fits-all; they offer different routes to learning, each one suited to specific research questions , goals, and resources.

In this post, we'll dive into different research methods, share examples, and give you some top tips for using them. Whether you're a seasoned researcher or just starting out, getting the hang of these methods will help you tackle your research projects with confidence and intention.

Understanding research methods

As we jump into our discussion on research methods, let's first get a handle on what they are. Simply put, research methods are the tools and tricks that researchers use to collect, analyze, and make sense of data in a clear, fair, and organized way. They're like the instruction manual for the research process, guiding us from coming up with a research question to sharing our findings.

Research methods come in all shapes and sizes, and they're as varied as the topics and fields they're used to study. They cover a wide range, each one suited to different kinds of data, research questions , and goals. The trick is to pick the right method that fits with your research aims, the resources you have, and the type of data you want to explore.

Generally, research methods come in two main flavors: qualitative and quantitative . Qualitative methods are all about digging into the deeper meanings and understandings of what we're studying. On the other side, quantitative methods are focused on counting and measuring things, giving us data we can run through statistical analysis.

There's also a third approach that's becoming more popular—mixed methods research. This approach blends elements of both qualitative and quantitative methods, taking advantage of the best of both worlds. It gives us a complete, rounded way to tackle complex research questions.

By understanding these methods, their pros and cons, and how to use them, we can make smart choices about our research design , making sure it's the best fit for answering our specific research questions effectively and reliably. As we dig further into these research methods, examples of them, and tips for using them, we're setting ourselves up to handle the research journey with skill, precision, and a clear roadmap.

Qualitative research methods

Qualitative research is all about getting a deep understanding of things in their natural settings. It helps researchers uncover meanings and interpretations that give us insight into people's experiences, behaviors, and social environments. These methods aren't about the big picture; instead, they offer a rich, detailed look at the subjects we're studying. They let us see things from the participants' point of view, reflecting their experiences and interpretations.

These methods are particularly handy when we want to get into the nitty-gritty of complex situations, get a handle on human behavior, or really get to know a specific context. They're often the starting point for studies that aim to guide the development of hypotheses for further research.

Interviews are a go-to method in qualitative research. They allow for direct conversation between the researcher and the participant. Interviews can range from structured, with set questions, to unstructured, allowing for open-ended exploration. They're highly adaptable and great for collecting detailed personal stories, understanding viewpoints, and finding out the how and why behind decision-making. However, they require some interviewing skills and can be time-consuming.

Focus groups

A focus group is a moderated discussion with a small group of participants. They're great for exploring group norms, generating ideas, and understanding shared experiences. They give researchers a chance to see how participants interact, giving insight into attitudes and decision-making processes within a social setting. They can be tough to manage, though, and need a skilled facilitator to balance the conversation.

Observation

Observational research means the researcher gets involved in a setting and takes notes on behaviors, interactions, and environmental details. The researcher can be a passive observer or an active participant. Observations can give authentic, first-hand data, revealing details that participants might not be aware of or able to express. However, they can take a lot of time, and there's a chance that the observer's presence might change the participants' behaviors.

By delving into qualitative research, we can discover the complex realities of people's experiences, behaviors, and social environments. Each method has its pros and cons, and the key is to choose the one that fits best with your research goals, questions, and resources. By understanding these qualitative research methods, we can really get into the heart of our research subjects, discovering truths that lie beneath the surface and painting a full, detailed picture of the situations we're studying.

Quantitative research methods

In contrast to qualitative research, quantitative research is all about measuring and counting. It dives into the world of numbers, providing data that can be analyzed using statistics. This approach allows researchers to test hypotheses, measure relationships, spot patterns, make predictions, and generalize findings to larger groups. Quantitative research strives for objectivity, consistency, and solid explanations.

Quantitative research methods aim to collect and analyze numerical data systematically, hoping to produce reliable and generalizable results. They work well when the goal is to measure a phenomenon, spot variations, test theories, or establish cause-and-effect relationships.

Surveys are a common way of collecting data quickly from many participants. They usually use structured questionnaires with closed-ended questions to gather specific information or measure certain variables. They can be done in person, over the phone, or online. Surveys can provide a broad picture of a population's attitudes, behaviors, or characteristics, but they can lack depth and depend heavily on the design of the questionnaire.

Experiments

Experimental research involves changing one variable to see if it causes a change in another. It allows researchers to establish cause-and-effect relationships and test theories under controlled conditions. Experiments are often used in fields like psychology, medicine, and the natural sciences. But they can be complex to design and conduct, and they require careful consideration of controls, randomization, and ethics.

Longitudinal studies

These studies collect data from the same subjects over a long period of time. They're useful for studying development, change, and trends over time. However, they can be costly, time-consuming, and subject to issues like participants dropping out.

In the precise world of quantitative research, we can measure phenomena, spot patterns, and draw conclusions that apply to larger groups. Each method has its pros and cons, and the choice depends on your research question, goals, and resources. By understanding these quantitative research methods, we can conduct solid, reliable research that contributes valuable knowledge to our field.

Mixed methods research

Mixed methods research is like getting the best of both worlds. It blends qualitative and quantitative research, giving a fuller picture of what's being studied. This approach recognizes that both types of data—the deep, rich descriptions from qualitative research and the precise, numerical data from quantitative research—offer valuable insights. By bringing both together, we get a more complete understanding of our topic.

Mixed methods research leverages the strengths of both qualitative and quantitative approaches, and at the same time, helps balance out their limitations. This approach encourages us to look beyond the "either/or" mindset, recognizing that complex research questions often need a varied approach for a full understanding.

There are multiple types of mixed methods research, each one serving different research goals and offering unique ways to bring together qualitative and quantitative data. Here are a few of the most common:

Sequential explanatory design

This design has two clear phases. First, the researcher collects and analyzes quantitative data. Then, they collect and analyze qualitative data. The qualitative phase is designed to help explain or expand upon the initial quantitative results. This design is particularly useful when unexpected results pop up from the quantitative phase that need further exploration.

Sequential exploratory design

This design flips the script, starting with qualitative data collection and analysis, followed by a quantitative phase. This design is usually used when a researcher wants to explore a phenomenon, create an instrument, or test elements that aren't well-understood or defined.

Concurrent design

In concurrent design, qualitative and quantitative data are collected at the same time but analyzed separately. The researcher then compares and contrasts the results to provide a comprehensive understanding of the research problem. This approach saves time and gives well-rounded findings.

Mixed methods research needs careful planning to effectively integrate qualitative and quantitative data and to make the most of each type's strengths. It requires flexibility and creativity from the researcher. When done well, it can give rich, detailed, and all-encompassing insights into the research problem. Despite the challenges, the value of mixed methods research lies in its ability to provide a more detailed and complete picture, leading to a deeper understanding of complex phenomena.

Tips for choosing the perfect research method

Choosing the right research method can sometimes feel a bit overwhelming. You need to choose the one that will best match your research goals, the questions you want to answer, and the resources you have available. Here are some tips to help you decide which method to pursue:

Match with your research question

The kind of question you're asking should guide your choice of method. If you're interested in understanding people's feelings, experiences, or how they make sense of things, qualitative methods like interviews or focus groups might be the best fit. If you're aiming to measure something, see if a pattern exists, or test an idea, quantitative methods like surveys or experiments might be more suitable. If your question needs a comprehensive answer that combines numbers with depth, mixed methods could be the way to go.

Check your resources

The resources you have at your disposal can really shape your choice of method. For instance, experiments might need special equipment or software, while interviews and focus groups need time and skill. Always choose a method that is doable with your resources but doesn't compromise the quality of your research.

Think about ethics

Every research method comes with ethical considerations . Make sure the rights, privacy, and well-being of your participants are at the forefront of your decision-making.

Look at past research

Review research that has been done on your topic to see what methods have been used before and how well they worked. Past studies can give you valuable insights into which methods might be suitable for your topic.

Ask for advice

If you're unsure, don't hesitate to ask for guidance from mentors, colleagues, or other researchers in your field. They can offer valuable insights and share their own experiences to help you make a well-informed decision.

Choosing the right research method is a crucial step in your research journey. By thinking through these factors, you can select the method that best fits your project. This will make sure your research is well-designed, ethical, and capable of answering your research questions effectively and reliably. Think of this process as laying a solid foundation for your research project, setting you up for discoveries and insights in your field of study.

Wrapping up our deep dive into research methods, it's clear how vital they are in shaping our research efforts and what we get out of them. Whether we're talking about qualitative, quantitative, or a mix of the two, each approach offers a unique way to get to grips with and dig into our research topics . They help guide us to findings that are both solid and have a real impact.

Choosing the right research method can be a bit tricky, kind of like picking the right tool from a toolbox. There's a lot that goes into this decision, from what kind of question we're trying to answer, to what resources we have at hand, the ethical considerations, and how comfortable we are with the method itself.

The process of doing research is just as important as the end results. The method we choose shapes our entire research journey and opens the door to meaningful discoveries that can make a difference in our fields. By putting time and thought into this process, we set ourselves up for success.

Header image by Tadamichi .

• Privacy Policy

Home » Research Design – Types, Methods and Examples

Research Design – Types, Methods and Examples

Table of Contents

Research Design

Definition:

Research design refers to the overall strategy or plan for conducting a research study. It outlines the methods and procedures that will be used to collect and analyze data, as well as the goals and objectives of the study. Research design is important because it guides the entire research process and ensures that the study is conducted in a systematic and rigorous manner.

Types of Research Design

Types of Research Design are as follows:

Descriptive Research Design

This type of research design is used to describe a phenomenon or situation. It involves collecting data through surveys, questionnaires, interviews, and observations. The aim of descriptive research is to provide an accurate and detailed portrayal of a particular group, event, or situation. It can be useful in identifying patterns, trends, and relationships in the data.

Correlational Research Design

Correlational research design is used to determine if there is a relationship between two or more variables. This type of research design involves collecting data from participants and analyzing the relationship between the variables using statistical methods. The aim of correlational research is to identify the strength and direction of the relationship between the variables.

Experimental Research Design

Experimental research design is used to investigate cause-and-effect relationships between variables. This type of research design involves manipulating one variable and measuring the effect on another variable. It usually involves randomly assigning participants to groups and manipulating an independent variable to determine its effect on a dependent variable. The aim of experimental research is to establish causality.

Quasi-experimental Research Design

Quasi-experimental research design is similar to experimental research design, but it lacks one or more of the features of a true experiment. For example, there may not be random assignment to groups or a control group. This type of research design is used when it is not feasible or ethical to conduct a true experiment.

Case Study Research Design

Case study research design is used to investigate a single case or a small number of cases in depth. It involves collecting data through various methods, such as interviews, observations, and document analysis. The aim of case study research is to provide an in-depth understanding of a particular case or situation.

Longitudinal Research Design

Longitudinal research design is used to study changes in a particular phenomenon over time. It involves collecting data at multiple time points and analyzing the changes that occur. The aim of longitudinal research is to provide insights into the development, growth, or decline of a particular phenomenon over time.

Structure of Research Design

The format of a research design typically includes the following sections:

• Introduction : This section provides an overview of the research problem, the research questions, and the importance of the study. It also includes a brief literature review that summarizes previous research on the topic and identifies gaps in the existing knowledge.
• Research Questions or Hypotheses: This section identifies the specific research questions or hypotheses that the study will address. These questions should be clear, specific, and testable.
• Research Methods : This section describes the methods that will be used to collect and analyze data. It includes details about the study design, the sampling strategy, the data collection instruments, and the data analysis techniques.
• Data Collection: This section describes how the data will be collected, including the sample size, data collection procedures, and any ethical considerations.
• Data Analysis: This section describes how the data will be analyzed, including the statistical techniques that will be used to test the research questions or hypotheses.
• Results : This section presents the findings of the study, including descriptive statistics and statistical tests.
• Discussion and Conclusion : This section summarizes the key findings of the study, interprets the results, and discusses the implications of the findings. It also includes recommendations for future research.
• References : This section lists the sources cited in the research design.

Example of Research Design

An Example of Research Design could be:

Research question: Does the use of social media affect the academic performance of high school students?

Research design:

• Research approach : The research approach will be quantitative as it involves collecting numerical data to test the hypothesis.
• Research design : The research design will be a quasi-experimental design, with a pretest-posttest control group design.
• Sample : The sample will be 200 high school students from two schools, with 100 students in the experimental group and 100 students in the control group.
• Data collection : The data will be collected through surveys administered to the students at the beginning and end of the academic year. The surveys will include questions about their social media usage and academic performance.
• Data analysis : The data collected will be analyzed using statistical software. The mean scores of the experimental and control groups will be compared to determine whether there is a significant difference in academic performance between the two groups.
• Limitations : The limitations of the study will be acknowledged, including the fact that social media usage can vary greatly among individuals, and the study only focuses on two schools, which may not be representative of the entire population.
• Ethical considerations: Ethical considerations will be taken into account, such as obtaining informed consent from the participants and ensuring their anonymity and confidentiality.

How to Write Research Design

Writing a research design involves planning and outlining the methodology and approach that will be used to answer a research question or hypothesis. Here are some steps to help you write a research design:

• Define the research question or hypothesis : Before beginning your research design, you should clearly define your research question or hypothesis. This will guide your research design and help you select appropriate methods.
• Select a research design: There are many different research designs to choose from, including experimental, survey, case study, and qualitative designs. Choose a design that best fits your research question and objectives.
• Develop a sampling plan : If your research involves collecting data from a sample, you will need to develop a sampling plan. This should outline how you will select participants and how many participants you will include.
• Define variables: Clearly define the variables you will be measuring or manipulating in your study. This will help ensure that your results are meaningful and relevant to your research question.
• Choose data collection methods : Decide on the data collection methods you will use to gather information. This may include surveys, interviews, observations, experiments, or secondary data sources.
• Create a data analysis plan: Develop a plan for analyzing your data, including the statistical or qualitative techniques you will use.
• Consider ethical concerns : Finally, be sure to consider any ethical concerns related to your research, such as participant confidentiality or potential harm.

When to Write Research Design

Research design should be written before conducting any research study. It is an important planning phase that outlines the research methodology, data collection methods, and data analysis techniques that will be used to investigate a research question or problem. The research design helps to ensure that the research is conducted in a systematic and logical manner, and that the data collected is relevant and reliable.

Ideally, the research design should be developed as early as possible in the research process, before any data is collected. This allows the researcher to carefully consider the research question, identify the most appropriate research methodology, and plan the data collection and analysis procedures in advance. By doing so, the research can be conducted in a more efficient and effective manner, and the results are more likely to be valid and reliable.

Purpose of Research Design

The purpose of research design is to plan and structure a research study in a way that enables the researcher to achieve the desired research goals with accuracy, validity, and reliability. Research design is the blueprint or the framework for conducting a study that outlines the methods, procedures, techniques, and tools for data collection and analysis.

Some of the key purposes of research design include:

• Providing a clear and concise plan of action for the research study.
• Ensuring that the research is conducted ethically and with rigor.
• Maximizing the accuracy and reliability of the research findings.
• Minimizing the possibility of errors, biases, or confounding variables.
• Ensuring that the research is feasible, practical, and cost-effective.
• Determining the appropriate research methodology to answer the research question(s).
• Identifying the sample size, sampling method, and data collection techniques.
• Determining the data analysis method and statistical tests to be used.
• Facilitating the replication of the study by other researchers.
• Enhancing the validity and generalizability of the research findings.

Applications of Research Design

There are numerous applications of research design in various fields, some of which are:

• Social sciences: In fields such as psychology, sociology, and anthropology, research design is used to investigate human behavior and social phenomena. Researchers use various research designs, such as experimental, quasi-experimental, and correlational designs, to study different aspects of social behavior.
• Education : Research design is essential in the field of education to investigate the effectiveness of different teaching methods and learning strategies. Researchers use various designs such as experimental, quasi-experimental, and case study designs to understand how students learn and how to improve teaching practices.
• Health sciences : In the health sciences, research design is used to investigate the causes, prevention, and treatment of diseases. Researchers use various designs, such as randomized controlled trials, cohort studies, and case-control studies, to study different aspects of health and healthcare.
• Business : Research design is used in the field of business to investigate consumer behavior, marketing strategies, and the impact of different business practices. Researchers use various designs, such as survey research, experimental research, and case studies, to study different aspects of the business world.
• Engineering : In the field of engineering, research design is used to investigate the development and implementation of new technologies. Researchers use various designs, such as experimental research and case studies, to study the effectiveness of new technologies and to identify areas for improvement.

Advantages of Research Design

Here are some advantages of research design:

• Systematic and organized approach : A well-designed research plan ensures that the research is conducted in a systematic and organized manner, which makes it easier to manage and analyze the data.
• Clear objectives: The research design helps to clarify the objectives of the study, which makes it easier to identify the variables that need to be measured, and the methods that need to be used to collect and analyze data.
• Minimizes bias: A well-designed research plan minimizes the chances of bias, by ensuring that the data is collected and analyzed objectively, and that the results are not influenced by the researcher’s personal biases or preferences.
• Efficient use of resources: A well-designed research plan helps to ensure that the resources (time, money, and personnel) are used efficiently and effectively, by focusing on the most important variables and methods.
• Replicability: A well-designed research plan makes it easier for other researchers to replicate the study, which enhances the credibility and reliability of the findings.
• Validity: A well-designed research plan helps to ensure that the findings are valid, by ensuring that the methods used to collect and analyze data are appropriate for the research question.
• Generalizability : A well-designed research plan helps to ensure that the findings can be generalized to other populations, settings, or situations, which increases the external validity of the study.

Research Design Vs Research Methodology

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Implications in Research – Types, Examples and...

Appendix in Research Paper – Examples and...

Ethical Considerations – Types, Examples and...

Assignment – Types, Examples and Writing Guide

Thesis Outline – Example, Template and Writing...

Research Paper Title Page – Example and Making...

Leave a comment x.

Save my name, email, and website in this browser for the next time I comment.

Home » Blog » Comprehensive Guide to Research Methodology – Design | Methods | Best Practices

Comprehensive Guide to Research Methodology – Design | Methods | Best Practices

• Other Laws | Blog |
• 19 Min Read
• Last Updated on 16 September, 2024

Recent Posts

Other Laws, Blog

Responsible Business Conduct and ESG in India – Best Practices | Emerging Regulation

News, Blog, Account & Audit

[Opinion] Internal Financial Control—Need to Focus?

Latest from taxmann.

Table of Contents

• Introduction
• Steps in Research Process
• Classification of Research Design

1. Introduction

This article describes the research process and different research designs in detail. Management and social science research, like other forms of scientific inquiry, require a structured sequence of highly interrelated steps (Zigmund et al., 2010). The research process involves a series of steps or actions essential for the smooth conduct of any research. The figure below illustrates the sequence of the research process. It is to be noted that these steps are not a road map to all kinds of research. Basically, it is applicable for deductive or functionalist research, and it can or needs to be revised to suit the requirements of a specific project. The research process doesn’t need to be followed successively; rather, the steps overlap frequently and are interrelated. The research process offers a comprehensive guideline that can be referred to for any management and social science research. It may happen that later stages can be accomplished before the earlier stages.

The steps involved in the research process are neither mutually exclusive nor separate and distinct. The selection of a research topic at the outset, defining the research problem and objectives, influences the selection of a sample and data collection. The sample selection may affect the design of questionnaire items. For example, suppose an organization wants to know the cause of attrition among lower-category employees with low educational qualifications. In that case, the wording for the questionnaire will be easier than for people in top management positions with professional educational qualifications. The steps may differ based on the objectives of the research. However, research based on deductive logic should follow the steps outlined below:

2. Steps in Research Process

• Problem Identification
• Literature Review
• Formulating Research Questions
• Research Design
• Data Collection
• Data Analysis
• Conclusions and Report Writing.

The quest for research must always be triggered by the longing to explore and gain more knowledge and understanding. The management dilemma encourages the need for a decision. The need may arise owing to the cause that the researchers want to discover or reestablish certain relationships. The orientation might be to solve immediate management issues, discover something new, or have purely academic intentions. For instance, in an organization, the manager may want to know the reason for high attrition and lack of job satisfaction, or a retail store may survey the post-purchase satisfaction among the customers.

2.1 Research Problem Identification

Defining the research problem is the first step in the research process. The researchers get the proper direction to conduct their research by first understanding the research problems. Hence, a well-defined research problem is crucial. When the problem is discovered, researchers and management can take further steps to define the problem clearly and precisely. A problem defined with accuracy and conscience helps the researchers utilize the available resources effectively. It is imperative for researchers to explore what exactly is the problem and what are the objectives of the research. The rule generally followed to define the research problem is that the definition should permit the researchers to acquire all details required to address the managerial issues and show guidelines for finding a solution. The researcher should be careful not to define the problem too broadly or narrowly. Examples of broad managerial problems are defining a strategy for enhancing organizational performance and a strategy to elevate the organization’s brand equity. An example of a narrow definition of a problem is how to match competitors’ recruitment strategies. To overcome the possibility of both errors while defining the research problem, the researchers must define the problem with broad, popular terms and devise its components. The broad general statement helps the researchers get a sound perspective on the research problem and avoid the error of defining the problem narrowly. On the other side, the specific component helps to identify the key aspects of the research problem, extend a transparent guideline to proceed further and avoid the error of defining the problem too broadly. In management and social science research, broad management problems need to be converted to information-oriented research problems that focus more on the cause than the symptoms. Some examples of managerial problems converted to research problems are presented in Table below. The conversion of management dilemma to managerial questions and further to research questions can be carried out through exploratory research. Such research incorporates an examination of past research studies, a review of extant literature and organizational records and interviewing experts (Cooper et al., 2016).

2.2 Literature Review

Exploring the existing literature is critical in the research process. Researchers must explore and investigate extant literature to observe whether other researchers have already addressed the identified research problem. A literature review is a systematic search of published work, including periodicals, books, journal papers (conceptual and empirical), and reports, representing theory and empirical work about the research problem and topic at hand. A survey of existing literature is customary in applied research and is an elementary requirement of a basic research report. The internet, electronic databases, websites, and e-library help the researcher to carry out literature surveys systematically and easily.

The literature review aims to study the existing state of knowledge in the domain of interest, to picture key authors, theories, methods, topics, and findings in that domain, and to explore the gaps in knowledge in that domain. A literature review conducted systematically reveals whether initial research questions have already gained substantial attention in the extant literature, whether more interesting newer research questions are available, whether past studies have consistent findings or contradictions exist, flaws in the body of research that the researchers can address, and whether the initial research questions need to be revised as per the findings of the literature review. Furthermore, the review can answer the proposed research questions and help identify theories used in previous studies to address similar research questions. For example, for an organization interested in determining the true cause of turnover, the researcher will study extensively the existing literature on attrition and its causes. By studying relevant journal articles, books, and book chapters, the researcher will discover the causes of attrition in general, find out the existing gaps, and suggest the management carry forward the research to find causes specific to the organization.

As deductive research primarily involves theory testing, the researchers must identify one or more theories that can illuminate the proposed research questions. Through an extensive literature review, researchers may uncover various concepts and constructs related to the phenomenon of interest. A theory will extend support to constructs/variables that are logically relevant to the chosen phenomenon. In the deductive approach, researchers use theory/theories as the logical basis for hypothesis testing. However, researchers must carefully select the theories appropriate for the identified problem to be studied. The hypotheses need to be logically formulated and connected to the research objectives.

2.3 Formulating Research Questions

After problem identification and clarification, with or without an exploratory research approach, the researchers should derive the research objectives. Cautious attention to problem definition helps the researchers devise proper research objectives. Research objectives are the goal to be achieved through research. The research objective drives the research process further. A well-devised research objective enhances the possibility of gathering, relevant information and avoiding unwanted information. The research objectives can be properly developed with the consensus of the researchers and management on the actual managerial and business problems. The researcher should ensure that the research objectives are clearly stated, appropriate, and will yield germane information. The research objective may involve exploring the likelihood of venturing into a new market or may necessitate examining the effect of a new organizational policy on employee performance. The nature and types of objectives lead to choosing an appropriate research design.

Research Objectives:  Research objectives represent the goal of the research the researchers want to accomplish.

2.3.1 Suitable Research Questions

Research questions are important to conduct effective research. Without a clear research question, the researcher may face the risk of unfocused research and will not be sure of what the research is about. Research questions are refined descriptions of the components of the research problem. These are questions related to behavior, events or phenomena of interest that the researchers search for answers in their research. Examples include what factors motivate the employees in an organization to apply the gained knowledge back to their jobs or what needs to be done to enhance the creativity of school-going students. Research questions can best state the objectives of the research. Each component of the research problem needs to be broken down into sub-parts or research questions. Research questions inquire about the information essential concerning the problem components. Properly answered research questions will lead to effective decision-making. While formulating research questions, researchers should be guided by the problem statement, theoretical background, and analytical framework.

Sources of Research Questions

• Extant Literature
• Personal experience
• Societal issues
• Managerial problems
• New theories
• Technological advancement
• Empirical cases
• Contradictory finding

2.3.1.1 Significance of Research Questions

Research questions are critical because they guide scientific and systematic literature search, the decision about appropriate research design, the decision about data collection and target audience, data analysis, selection of right tools and techniques and overall to move in the right direction.

The researcher can utilize different sources for formulating research questions, such as extant literature, personal experience, societal issues, managerial problems, new theories, technological advancement, and contradictory findings. The research question must portray certain attributes. Research questions in quantitative research are more specific compared to qualitative research. Sometimes, some qualitative research follows an open approach without any research questions. The main steps involved in formulating research questions are illustrated in Figure below.

Criteria of Effective Research Questions

• Rateability
• Systematic and logical
• Significant
• Fascinating
• Logical association among variables

The sequence in selecting research questions suggests that the researchers are engrossed in a process of progressive focusing down when developing the research questions. It helps them to slide down from the general research area to research questions. While formulating the research questions, the researchers should understand that ending a research question with a question mark is essential. Without a question mark, a statement cannot be considered as a research question. It is quite possible that the researchers may not get answers to all research questions. The research questions need to be related to each other.

2.4 Planning the Research Design

After formulating research problems and literature surveys, the next stage in the research process is to develop the research design. Research design is the blueprint of research activities to answer research questions. It is a master plan that includes research methods and procedures for gathering and analyzing the relevant information with minimum cost, time, and effort. A research design extends a plan for carrying out the research. The researchers need to decide the source to collect information, the techniques of research design (survey or experiment), sampling techniques, and the cost and schedule of the research. The success of these objectives depends on the purpose of the research. Usually, research purposes are segregated into four types: exploration, description, diagnosis, and experimentation.

There are varied designs, such as experimental or non-experimental hypotheses testing (details of different research designs are outlined in section 2.3 in this chapter). There are four primary research methods for descriptive and causal research: survey, experiments, secondary data, and observations. The selection of an appropriate research method relies on the research objectives, available data sources, the cost and effort of collecting data, and the importance of managerial decisions. If the research objective is exploration, a flexible research design can extend better opportunities to investigate different aspects of the research problem. On the other hand, if the intention is simply to describe any situation or phenomena of interest to examine the relationship between two or more variables, the appropriate design should prioritize minimizing bias and maximizing reliability in data collection and analysis. For example, suppose a researcher wants to conduct exploratory research to know the different types of arthritis common in India. In that case, it may require a flexible design relying on secondary data from hospital records or discussions with doctors or other experts to reach conclusions. However, to invent COVID-vaccination and medicine for the COVID-19 virus, the researchers conducted varied experiments to reach a conclusion.

2.4.1 Hypotheses Development

Exploratory research helps the researchers define the research questions, key variables, and theoretical underpinnings and formulate hypotheses if required in the research. The hypotheses must be logically derived based on the research questions and linked to research objectives. A hypothesis is a tentative proposition regarding a research phenomenon. It may be a tentative statement that indicates an association between two or more variables, guided by any supportive theory, theoretical framework, or analytical model. It is a viable answer to the research questions framed by the researchers. Hypotheses are statements of relationships or propositions that are declarative and can be tested with empirical data. Some examples are:

H 1 : Training influences organizational performance.

H 2 : Training enhances employee performance.

For two more research questions i.e., “to what extent does brand love determine purchase intention?” and “does age and family background moderate the relationship?”, the hypotheses are:

H 1 : Brand love is related to purchase intention.

H 2 : Age and Family status moderate the association between brand love and purchase intention. Figure below provides a pictorial representation of the hypotheses drawn.

However, it is not always feasible for researchers to formulate hypotheses in all situations. Sometimes, researchers may lack all relevant information, and theoretical support may not be available to formulate the hypotheses.

2.5 Sampling Design

This stage of the research process involves an investigation of the population under study. A complete investigation of the population under study is known as a census inquiry. Usually, in census investigation, all units or items of the population are studied with high accuracy and reliability. However, it is usually not practicable and feasible for the researchers to study the entire population. Researchers usually prefer to investigate small, representative subgroups from the population known as sample. The procedure to select the sub-groups/samples is called sampling design. Sampling entails the process of drawing conclusions based on a subgroup of the population. Hence, the sample is a subset of the population. The first question that needs to be addressed in sampling is “who is to be included in the sample?” and this requires the identification of the target population under study. It is difficult for the researcher to define the population and sampling unit. For example, if a researcher wants to investigate the financial savings and vehicle loan association survey. In that case, individuals with existing accounts will be taken, and this sample unit represents the existing customers and not the potential customers. Hence, it is critical in sampling design to determine the specific target population.

Secondly, the issue that concerns the researchers in sampling design is selecting an appropriate sample size, and the third concern is selecting the sampling units. Researchers need to address these concerns to justify the research. Samples can be selected either using probability sampling techniques or non-probability sampling techniques. There are four types of probability sampling such as simple random, systematic, stratified, and cluster sampling. Non-probability sampling includes convenience, judgmental, quota, and snowball sampling. Depending on the objective, researchers should select the appropriate sampling techniques for their study.

2.6 Fieldwork and Gathering Data

After the formalization of the sampling plan, the fieldwork and data-gathering stage begins. The researcher gathers data after finalizing what to research, among whom, and which method to use. Data gathering involves the process of information collection. Different data collection instruments are available for researchers to collect information or data. Broadly, there are two ways to collect data, such as primary and secondary data collection methods. Primary data include data collected firsthand and are original. Varied methods are available for primary data collection, such as structured and unstructured interviews, focused group discussion, observation, and survey using a structured questionnaire. The data can be collected offline or online. Secondary data included information collected from published or unpublished sources that were already available. Some secondary data collection sources are articles, magazines, company records, expert opinion survey data, feedback of customers, government data, and past research on the subject. For example, to conduct a survey of job satisfaction in an organization, the researcher may circulate a printed questionnaire offline or mail the questionnaire to the selected respondents following an appropriate sampling technique.

Another example could be a study that investigates the purchase preference for luxury cars, and the base model demands primary and empirical information. However, another study that intended to describe the financial investment behavior of existing customers will use secondary data. At this stage, the researchers need to ensure the reliability and validity of the data obtained for the study.

2.7 Data Processing and Analysis

After data gathering, the data needs to be converted or properly coded to answer the research question under study. The information gathered in the data collection phase should be mined from the primary raw data. Data processing starts with data editing, coding, and tabulation. First, it is vital for the researchers to check the data collection forms for missing data, clarity, and consistency in categorization. The editing process involves problems associated with data, such as respondents’ response errors. Editing improves the quality of the data and makes the data usable for tabulation, analysis, and interpretation. Tabulation is a technical process in which classified data are presented in tables. Researchers use computers to feed data to a computer spreadsheet for data analysis. The preparation of a spreadsheet also requires lots of expertise and experience.

After coding the data, the next step is to analyze the data. Data analysis is the utilization of reasoning to make sense of data gathered. Ample statistical techniques are available for the researchers to analyze the data. Based on the research questions, objectives, study types, sampling framework used, data types, and degree of accuracy involved in the research, one can choose from parametric or non-parametric techniques for data analysis. Researchers may adopt univariate, bi-variate or multi-variate methods for data analysis. The analysis may include simple frequency analysis, multiple regression, or structural equation modeling. Different techniques are available for qualitative data, presented in Part 3 of this book.

2.8 Drawing Conclusion and Preparing a Report

After data analysis, the final stage in the research process is the interpretation of the results. The researcher requires analytical skills to interpret the statistical results, link the output with the research objectives, and state the implications of the result.

Research Design:  Research design is the blueprint/systematic steps to carry out research smoothly

Finally, researchers must communicate the result in the form of a report. The preparation of the final report needs to be done with the utmost care. The final report should include the identified research questions, research approach, data collection method, data analysis techniques, study findings, and implications for theory and practice. The structure of the report will be discussed in the last section of this book. The report should be prepared comprehensively to be usable by management or organizations for decision-making.

3. Classification of Research Design

This section highlights the classification of research design. As mentioned in the previous section, research design is the framework for carrying out management and other research. After the identification of a problem, the researchers formulate the research design. A good research design ensures the effectiveness of the research work. The choice of selecting an appropriate design relies on the research objectives. The broad categorization of research design with sub-categorization is detailed in various sub-sections.

3.1 Exploratory Research Design

Methods to Conduct Exploratory Research

• Literature survey
• Secondary sources of data
• Experience survey
• Focused group discussions
• Observations
• Structured and unstructured interviews
• Pilot surveys
• Case Studies

Exploratory research design is the simplest form of research design. The researchers explore the true nature of the problem. When researchers aim to study a new area or examine a new interest, exploratory design is a good option. This research design is flexible and versatile in approach. The information required by the researchers is defined loosely and unstructured. Researchers carrying out qualitative research usually adopt exploratory research design. Exploratory research design serves three purposes (a) it helps the researchers to address their inquisitiveness and quest for better understanding (b) to assess the practicality of carrying out border research (c) and devise methods for further studies.

Methods to Conduct Descriptive Research

• Self-administered survey
• Phone survey
• Mail survey/online survey
• Observation
• Personal interview
• Telephone interview

Exploratory research design has paramount significance in management and social science research. They are crucial for researchers who want to study something new. To cite an example, during the COVID-19 pandemic, physical health, mental health, and safety of school and college-going children were a concern for most people. The online education system was the new normal at that time. Research studying the impact of digitalization, long time spent in online studies on students’ health and mental well-being during the COVID-19 pandemic, is of an exploratory kind. One of the disadvantages of exploratory research design is that researchers rarely get specific answers to the research questions.

3.2 Descriptive Research Design

The prime objective of descriptive research design is to describe certain situations or events. This type of design provides an extensive explanation of the research phenomena under study. In descriptive research, the researchers possess prior knowledge about the problem situations. The information is defined with clarity. This kind of research is preplanned and more structured than exploratory research. Researchers must formulate research questions properly and have clarity regarding the types of data needed and the procedure to be followed to achieve the research objectives. Researchers have the luxury of covering a large representative sample. Researchers must answer five Ws and one H – what, who, when, where, why, and how of research issues. What kind of information is required for the research, who are the target respondents, when the information will be collected, where to interact with the respondents, why information is collected from the respondents and how to collect data from the respondents. Descriptive research studies can be cross-sectional or longitudinal. The major objectives for the following descriptive research are given below.

• To explain the characteristics of certain groups such as the Indian population, employees, students, marketing personnel, organizations, sales persons. For example, a university to design a customized online higher studies course for working professionals needs a holistic profile of the interested population.
• To evaluate the portion of individuals in a specific population portraying a typical behavior. For instance, when a researcher is inclined to know the percentage of employees not interested in an online platform introduced for them in their organization.
• To predict for future. For instance, to know the future of physical retail stores due to the widespread expansion of online stores.
• To examine the extent to which management research variables relate to each other. For example, to what extent does work-life balance, salary, and conducive work environment enhance employee job satisfaction?

3.3 Causal Research Design

Usually, causal research design is adopted by researchers to explain causal relationships among phenomena under study. Causal research examines cause-and-effect relationships among variables. Causal research has certain criteria, as already discussed in Chapter 1. Causal research follows a planned and structured design like descriptive research. Though the magnitude of the relationship among variables is examined in descriptive research, the causal association cannot be explained through such research. Experimentation is one of the methods for carrying out causal research.

In causal research, the researchers usually examine the impact of one variable on another. The researchers try to explore the cause-and-effect relationship (nomothetic explanation). How can the researcher know whether cause and effect are associated? There are three criteria for a nomothetic causal relationship when (1) two or more variables are correlated, (2) the cause precedes the effect and (3) the absence of a plausible alternative explanation for the effect other than the proposed cause (Babbie, 2020). First, without establishing a correlation among two or more variables, causation cannot exist. Second, the cause should happen before the effect in time. For instance, it is more sensible to say that children’s religious affiliation is caused by their parents than to reflect that parents’ religious affiliation is due to children; even in some cases, it is plausible that children may convert to other religions later with their parent’s permission. The third significant condition for a causal relationship is that the effect cannot be attributed to any external third variable for establishing causation.

To cite one classic example, there is a causal association between sales of ice cream and death owing to drowning. Intake of more ice creams in summer does lead to a higher death rate due to drowning. The third intervening variable that causes higher death is season or temperature. In summer, higher deaths occur due to swimming and not because of taking ice-creams. The intervening variable season or temperature causes a higher death rate.

To establish a reliable causal relationship among two or more variables, other influencing variables must be controlled to neutralize their impact on the studied variables. For example, to study the effect of factors influencing training transfer in soft skill training, the other intervening variables such as age, gender, and educational qualification need to be controlled. This kind of research sometimes demands experimentation to establish causality. In most cases, causal research is quantitative and needs statistical hypothesis testing.

3.4 Experimental Research Design

Experimental research aims to examine the cause-effect relationship in a controlled setting by isolating the cause from the effect in time. The three criteria suggested by John Stuart Mill mirror in experimental research. In experimental research, the cause is administered to one group of subjects, known as the treatment group and not to the control group, and the researchers observe the difference in mean effect among the subjects of both groups. Whether variation in the cause is connected to variation in effect is observed. To be more specific, the researcher manipulates the independent variable and examines the change in the dependent variable, keeping other variables constant. Researchers used varied methods during the experiments to reduce the plausible effect of other explanations for the effect, along with ancillary methods to investigate the plausibility of those that cannot be ruled out. It is vital in experimental studies to control the extraneous and confounding variables while carrying out the experiments. Ignorance of such variables may lead to spurious relationships among studied variables. However, bringing many of the variables under experimental control is impossible. For example, personal characteristics of the subject like age, sex, intelligence, beliefs and persona. In such cases, the researchers must observe natural variations in the variables of concern. Then, statistical procedures are used to rule out the plausible impact of uncontrolled factors.

Experimental Research Design:  An experiment is a method of collecting evidence to indicate the effect of one variable on another.

Experimental research design can be conducted in a laboratory setting (laboratory experiment) or in a field setting (field experiments) where the phenomena of research interest happen. As an example, one of the most talked about and controversial experiments conducted on understanding human behavior has been the Stanford Prison Experiments, which took place at Stanford University in 1971. The experiments were funded by the US Office of Naval Research, and the principal investigator for the same was Prof Phillip Zimbardo. The major purpose of these experiments was to understand how norms develop and social expectations about roles shape group behavior. Experimental studies are segregated into four categories such as pre-experimental, true-experimental, quasi-experimental and statistical design.

3.4.1 Correlation, Causation and Cofounds

Correlation cannot be treated as causation, and correlation does not always prove causation. In correlation, it is unclear which variable comes first or whether any alternative explanation exists for the assumed effect. Two variables may be correlated due to chance. Correlation is symmetric, while causation is asymmetric. Two variables may be co-related, but their relationship may be affected by a third variable called cofounds. For example, let’s say that high salary and high educational qualifications are correlated. It is difficult to say with confirmation which comes first. Whether a high educational qualification leads to a high salary, or a high salary leads to a high educational qualification. Both possibilities can hold true and necessitate further investigation. Until researchers conclude through their investigation, a mere correlation among these two variables will not give a clear picture of their causal relationship. There is also the possibility of an alternative explanation for the relationship between high salary and high educational qualifications. The link may be due to a third variable called intellect, which results in high salary and high educational qualifications.

In management research, social science, and natural science, three significant pairs of components are required for experimentation: Experimental and control group, independent and dependent variable, and pre-test and post-test.

3.4.1.1 Experimental and Control Group

The group in which an experimental treatment is administered is known as the experimental or treatment group. In contrast, the group in which no experiment is administered is known as the control group. Using control groups enables the researchers to assess the experiment’s effects. For example, suppose a researcher wants to study the impact of rewards on employee productivity in an organization. In that case, the researcher can experiment with two groups of employees. One group will be given external rewards, known as the experimental group, and the other group (control group) will provide no external rewards. Then, the researcher can investigate the causal association between rewards on employees’ productivity through this experiment. The use of a control group is quite common in medical science research. In social science and management research, the use of control groups and experimental studies became popular with several experiments conducted in the late 1920s and early 1930s by F. J. Roethlisberger and W. J. Dickson (1939) to discover the changes required in working conditions to enhance employee satisfaction and productivity. Their series of experiments resulted in the Hawthorne effect.

3.4.1.2 Independent and Dependent Variables

In experimental research, the researchers study the impact of an independent variable on the dependent variable. Usually, experimental stimuli, whether present or absent, are considered independent variables. Independent variables are manipulated in the study, and their effects are assessed and compared. The researchers compare outcomes when the stimulus is present and not present. Hence, the independent variable is the cause, and the dependent variable is the presumed effect. It is to be noted that the independent variable in one study may serve as a dependent variable in another study. For example, an experiment intends to explore the causality between high salary and job satisfaction, job satisfaction is the dependent variable. However, in another experiment designed to explore the causality between job satisfaction and employee productivity, job satisfaction is the independent variable.

3.4.1.3 Pre- and Post-test

In an experiment, the experimenters measure the variable before conducting the experiment on the group known as the pre-test and measure the variable after conducting the experiments is called as post-test. Hence, subjects are exposed to a stimulus called a dependent variable (pre-testing), then exposed to a stimulus, i.e., independent variable, and again assessed with a dependent variable (post-testing). Any discrepancies between the two measurements of dependent variables are ascribed to the independent variable.

Disclaimer: The content/information published on the website is only for general information of the user and shall not be construed as legal advice. While the Taxmann has exercised reasonable efforts to ensure the veracity of information/content published, Taxmann shall be under no liability in any manner whatsoever for incorrect information, if any.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

PREVIOUS POST

To subscribe to our weekly newsletter please log in/register on Taxmann.com

Latest books.

R.K. Jain's Customs Tariff of India | Set of 2 Volumes

R.K. Jain's Customs Law Manual | 2023-24 | Set of 2 Volumes

R.K. Jain's GST Law Manual | 2023-24

R.K. Jain's GST Tariff of India | 2023-24

Everything on Tax and Corporate Laws of India

Everything you need on Tax & Corporate Laws. Authentic Databases, Books, Journals, Practice Modules, Exam Platforms, and More.

• Express Delivery | Secured Payment
• Free Shipping in India on order(s) above ₹500
• +91 8688939939
• About Company
• Media Coverage
• Union Budget 2024-25

• Business & Support
• Sell with Taxmann
• Locate Dealers
• Locate Representatives
• CD Key Activation
• Privacy Policy
• Return Policy
• Payment Terms

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Qualitative vs. Quantitative Research | Differences, Examples & Methods

Qualitative vs. Quantitative Research | Differences, Examples & Methods

Published on April 12, 2019 by Raimo Streefkerk . Revised on June 22, 2023.

When collecting and analyzing data, quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings. Both are important for gaining different kinds of knowledge.

Common quantitative methods include experiments, observations recorded as numbers, and surveys with closed-ended questions.

Quantitative research is at risk for research biases including information bias , omitted variable bias , sampling bias , or selection bias . Qualitative research Qualitative research is expressed in words . It is used to understand concepts, thoughts or experiences. This type of research enables you to gather in-depth insights on topics that are not well understood.

Common qualitative methods include interviews with open-ended questions, observations described in words, and literature reviews that explore concepts and theories.

Table of contents

The differences between quantitative and qualitative research, data collection methods, when to use qualitative vs. quantitative research, how to analyze qualitative and quantitative data, other interesting articles, frequently asked questions about qualitative and quantitative research.

Quantitative and qualitative research use different research methods to collect and analyze data, and they allow you to answer different kinds of research questions.

Quantitative and qualitative data can be collected using various methods. It is important to use a data collection method that will help answer your research question(s).

Many data collection methods can be either qualitative or quantitative. For example, in surveys, observational studies or case studies , your data can be represented as numbers (e.g., using rating scales or counting frequencies) or as words (e.g., with open-ended questions or descriptions of what you observe).

However, some methods are more commonly used in one type or the other.

Quantitative data collection methods

• Surveys :  List of closed or multiple choice questions that is distributed to a sample (online, in person, or over the phone).
• Experiments : Situation in which different types of variables are controlled and manipulated to establish cause-and-effect relationships.
• Observations : Observing subjects in a natural environment where variables can’t be controlled.

Qualitative data collection methods

• Interviews : Asking open-ended questions verbally to respondents.
• Focus groups : Discussion among a group of people about a topic to gather opinions that can be used for further research.
• Ethnography : Participating in a community or organization for an extended period of time to closely observe culture and behavior.
• Literature review : Survey of published works by other authors.

A rule of thumb for deciding whether to use qualitative or quantitative data is:

• Use quantitative research if you want to confirm or test something (a theory or hypothesis )
• Use qualitative research if you want to understand something (concepts, thoughts, experiences)

For most research topics you can choose a qualitative, quantitative or mixed methods approach . Which type you choose depends on, among other things, whether you’re taking an inductive vs. deductive research approach ; your research question(s) ; whether you’re doing experimental , correlational , or descriptive research ; and practical considerations such as time, money, availability of data, and access to respondents.

Quantitative research approach

You survey 300 students at your university and ask them questions such as: “on a scale from 1-5, how satisfied are your with your professors?”

You can perform statistical analysis on the data and draw conclusions such as: “on average students rated their professors 4.4”.

Qualitative research approach

You conduct in-depth interviews with 15 students and ask them open-ended questions such as: “How satisfied are you with your studies?”, “What is the most positive aspect of your study program?” and “What can be done to improve the study program?”

Based on the answers you get you can ask follow-up questions to clarify things. You transcribe all interviews using transcription software and try to find commonalities and patterns.

Mixed methods approach

You conduct interviews to find out how satisfied students are with their studies. Through open-ended questions you learn things you never thought about before and gain new insights. Later, you use a survey to test these insights on a larger scale.

It’s also possible to start with a survey to find out the overall trends, followed by interviews to better understand the reasons behind the trends.

Qualitative or quantitative data by itself can’t prove or demonstrate anything, but has to be analyzed to show its meaning in relation to the research questions. The method of analysis differs for each type of data.

Analyzing quantitative data

Quantitative data is based on numbers. Simple math or more advanced statistical analysis is used to discover commonalities or patterns in the data. The results are often reported in graphs and tables.

Applications such as Excel, SPSS, or R can be used to calculate things like:

• Average scores ( means )
• The number of times a particular answer was given
• The correlation or causation between two or more variables
• The reliability and validity of the results

Analyzing qualitative data

Qualitative data is more difficult to analyze than quantitative data. It consists of text, images or videos instead of numbers.

Some common approaches to analyzing qualitative data include:

• Qualitative content analysis : Tracking the occurrence, position and meaning of words or phrases
• Thematic analysis : Closely examining the data to identify the main themes and patterns
• Discourse analysis : Studying how communication works in social contexts

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Chi square goodness of fit test
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Inclusion and exclusion criteria

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

The research methods you use depend on the type of data you need to answer your research question .

• If you want to measure something or test a hypothesis , use quantitative methods . If you want to explore ideas, thoughts and meanings, use qualitative methods .
• If you want to analyze a large amount of readily-available data, use secondary data. If you want data specific to your purposes with control over how it is generated, collect primary data.
• If you want to establish cause-and-effect relationships between variables , use experimental methods. If you want to understand the characteristics of a research subject, use descriptive methods.

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

There are various approaches to qualitative data analysis , but they all share five steps in common:

• Prepare and organize your data.
• Review and explore your data.
• Develop a data coding system.
• Assign codes to the data.
• Identify recurring themes.

The specifics of each step depend on the focus of the analysis. Some common approaches include textual analysis , thematic analysis , and discourse analysis .

A research project is an academic, scientific, or professional undertaking to answer a research question . Research projects can take many forms, such as qualitative or quantitative , descriptive , longitudinal , experimental , or correlational . What kind of research approach you choose will depend on your topic.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Streefkerk, R. (2023, June 22). Qualitative vs. Quantitative Research | Differences, Examples & Methods. Scribbr. Retrieved September 16, 2024, from https://www.scribbr.com/methodology/qualitative-quantitative-research/

Is this article helpful?

Raimo Streefkerk

Other students also liked, what is quantitative research | definition, uses & methods, what is qualitative research | methods & examples, mixed methods research | definition, guide & examples, what is your plagiarism score.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 17 September 2024

A systematic review and meta analysis on digital mental health interventions in inpatient settings

• Alexander Diel 1 , 2 ,
• Isabel Carolin Schröter 1 , 2 ,
• Anna-Lena Frewer 1 , 2 ,
• Christoph Jansen 1 , 2 ,
• Anita Robitzsch 1 , 2 ,
• Gertraud Gradl-Dietsch 3 ,
• Martin Teufel 1 , 2 &
• Alexander Bäuerle 1 , 2  

npj Digital Medicine volume  7 , Article number:  253 ( 2024 ) Cite this article

Metrics details

• Psychiatric disorders
• Randomized controlled trials

E-mental health (EMH) interventions gain increasing importance in the treatment of mental health disorders. Their outpatient efficacy is well-established. However, research on EMH in inpatient settings remains sparse and lacks a meta-analytic synthesis. This paper presents a meta-analysis on the efficacy of EMH in inpatient settings. Searching multiple databases (PubMed, ScienceGov, PsycInfo, CENTRAL, references), 26 randomized controlled trial (RCT) EMH inpatient studies ( n  = 6112) with low or medium assessed risk of bias were included. A small significant total effect of EMH treatment was found ( g  = 0.3). The effect was significant both for blended interventions ( g  = 0.42) and post-treatment EMH-based aftercare ( g  = 0.29). EMH treatment yielded significant effects across different patient groups and types of therapy, and the effects remained stable post-treatment. The results show the efficacy of EMH treatment in inpatient settings. The meta-analysis is limited by the small number of included studies.

Introduction

Mental health disorders represent a prevalent set of clinical conditions associated with substantial personal and economic burdens. However, despite their prevalence and impact, there exists a conspicuous deficit in the provision of effective treatment 1 , 2 , 3 , 4 . Across Europe, estimates suggest that 15–40% of the population experiences some form of mental disorder, yet fewer than one-third of these cases receive treatment that meets the established standards of adequacy 5 , 6 , 7 , 8 , 9 .

One reason for the lack of adequate treatment of mental disorders are structural supply issues, for example caused by a shortage of mental healthcare providers in more rural areas 10 . Furthermore, negative attitudes towards mental health treatments hinder seeking help especially in mild to moderate cases 11 . Finally, prompt access to mental health treatment is paramount for its efficacy, yet mental health facilities and specialists often impose prolonged waiting periods spanning several months 12 . These extended waiting intervals amplify the economic strain of mental disorders 13 , exacerbate clinical manifestations 14 , 15 , diminish treatment adherence, and elevate dropout rates 16 , 17 . In summary, providing adequate mental health treatment is complicated by a variety of structural issues leading to several other problems like economic and patients’ personal costs.

E-mental health (EMH) interventions aim to provide adequate treatment of mental health disorders through technological means and channels, such as app- or web-based systems, text messages, videos, or digital monitoring. Here, the term EMH is used to describe any digitally delivered interventions with the goal of improving mental health outcomes. Due to the easy accessibility of EMH products, such interventions have many advantages: They can 1) fill structural supply gaps for rural areas, 2) bridge long waiting times for in-person mental health treatment, and 3) provide additional anonymity for those concerned about stigmatization 18 , 19 . Thus, EMH tools have the potential to be a viable method to overcome the various issues hindering adequate mental health treatment.

In outpatient settings, EMH interventions are effective tools to treat mental disorders according to several meta-analyses, including the treatment of anxiety and depression 20 , 21 , 22 eating disorders 23 , posttraumatic stress 24 , or work-related stress 25 . Furthermore, EMH interventions find predominantly positive acceptance from both patients and mental health practitioners 26 , 27 , 28 . Thus, EMH interventions are viable and accepted tools in the treatment of mental disorders in outpatient settings.

Inpatient treatment signals an especially high need for timely and adequate intervention and is indicated for cases considered too severe for outpatient treatment 29 . Inpatient interventions can profit from supportive EMH procedures either to bridge waiting times, to blend with in-person interventions, or to ensure stabilization and relapse prevention in aftercare treatments. Especially the implementation of post-treatment aftercare improves the chances of a favorable and sustained development 30 , 31 , 32 . Thus, EMH treatment can be an important factor in the long-term success of inpatient treatments. Adequate aftercare enhances rehabilitation according to several reviews and meta-analyses 33 , 34 , 35 . Randomized controlled trials (RCT) exist on EMH treatment as add-ons to regular inpatient interventions 36 and aftercare 37 . Furthermore, a systematic review 38 found support for the efficacy of EMH aftercare treatments but was limited by the small number of studies. Yet as of now, there are to our knowledge no meta-analyses on the use of EMH in inpatient treatments, nor are there meta-analyses on EMH for inpatient aftercare. In addition, the systematic review on EMH inpatient care is several years old and does not incorporate the more recent research 38 .

The present study seeks to summarize the findings of previous RCTs on EMH treatments in inpatient settings in a meta-analysis. In addition, the risk of bias of the studies are assessed 39 . Specifically, this meta-analysis seeks to investigate 1) the total effect of EMH treatments on mental health outcomes in inpatient settings; 2) the effects of EMH treatments, divided into interventions and aftercare treatments; 3) the effect of EMH treatments in relationship depending on the mental health disorder; 4) the effect of type of therapy on EMH efficacy; 5) the effects of follow-up measures on EMH interventions are investigated to test long-term effects; and 6) the assessment of bias of the currently published RCT literature. Furthermore, post-hoc analyses were conducted to investigate 1) the role of EMH medium (e.g., app-based, web-based, SMS-based) on EMH treatment efficacy, and 2) the effect of the type of control group on EMH efficacy.

Selected literature

A total of 26 research studies containing 123 effects and a sample size of n  = 6112 ( intervention group  = 3041, control group  = 3071) were included. A summary of the included studies is shown in Table 1 . Five studies used blended treatment during inpatient stay while 21 studies conducted post-inpatient aftercare treatment. The most common patient groups (according to the number of studies) were eating disorders ( k  = 7) followed by mood disorders ( k  = 6), transdiagnostic ( k  = 4), psychotic disorders ( k  = 3), return to work treatments ( k  = 2), mental comorbidities with somatic disorders ( k  = 2), anxiety disorders ( k  = 1), and substance abuse ( k  = 1).

Thirteen out of 26 studies utilized a passive control group for which participants did not receive any type of active treatment (e.g., waiting list); eight studies used an active control group with an active treatment alternative to the EMH treatment (e.g., aftercare e-mail reminders for mental health tools, psychoeducation, rehabilitation activities, or psychosocial support such as counsling); five studies used an active control group to which the EMH treatment as added to in the intervention group; finally, one study used both active and passive control groups.

Three studies used SMS-based EMH interventions. Eighteen studies used web-based interventions such as SUMMIT 40 , IN@ 41 , HEINS 42 , Deprexis 37 , 43 , 44 , GSA Online 45 , and EDINA 46 . Five studies used app-based tools such as MCT & More 47 and Mindshift 36 . Each specified tool was used by only one study except for Deprexis, which was used in three studies.

Out of all included studies, 17 were conducted in Germany, two in Sweden and USA respectively, and one in Hungary, Iran, Finland, Canada, and Australia, respectively.

Study search and selection flow is depicted in Fig. 1 .

Flowchart depicting study selection. The first selection of 726 studies was found in five different databases. Following the evaluation by exclusion criteria, 30 studies were selected for risk of bias evaluation. After four studies were excluded for risk of bias, 26 studies were included in the meta-analysis. EMH e-mental health, RCT randomized controlled trial.

Risk of bias assessment

Four studies were rated as high risk of bias and excluded from the analysis. Out of the remaining studies, 19 were rated as medium risk of bias and seven as low risk of bias. Among the most common bias concerns were asymmetrical attrition rates in control and intervention groups, high attrition rates with unclear reasons, alternating allocations (rather than random allocation), and inadequate information on blinding procedures (e.g., no specifications for statements such as “the procedure was blinded”). All four high risk studies were excluded also due to unclear, high, or uneven attrition rates between groups.

The risk assessment is summarized in Table 1 .

Publication bias analyses

Preliminary analyses were conducted to test for publication bias using funnel plot and p -curve analyses.

Funnel plot analysis

Funnel plots with effect sizes plotted against standard errors are depicted in Fig. 2a .

Funnel plot across all effects ( a ) and after excluding studies with the largest standard errors ( b ). The funnel plots depict the effect sizes (Hedges’ g ) plotted against the studies’ (reversed) standard errors. Asymmetry analyses found a significant asymmetry ( a ), but not when excluding four effects with the largest standard errors ( b ). As the effect size remains unaltered, the results do not indicate publication bias.

Publication bias would express itself in a preference for publishing significant compared to non-significant results. Because smaller studies need a higher effect size to reach significant effects compared to larger studies, an asymmetrical distribution with more smaller studies with larger effect sizes compared to larger studies would indicate publication bias. A regression analysis using standard error as a predictor of effect sizes suggests significant asymmetry ( z  = 3.6, p  < 0.001, i  = 123 effects).

Publication bias can be controlled by excluding the smallest studies 48 . After excluding studies with the largest standard errors ( i  = 4 effects, 3% of the total effects), another regression test showed no indicators of funnel plot asymmetry (z = 1.89, p  = 0.058, i  = 119, Fig. 2b ). The total effect size remained unaltered ( g  = 0.33 [0.2, 0.46], p  < 0.001), showing that the publication bias correction did not impact the results. Thus, the results do not indicate publication bias.

P-curve analysis

P -curve analysis was used to investigate publication bias further. A right-skewed p -curve would indicate an existing effect while a left-skewed p -curve would indicate publication bias or p -hacking as the latter curve would result from a tendency to acquire significant p -values of just below .05 despite the absence of a true effect indicated by a higher rate of results with smaller p -values. The p -curve is depicted in Fig. 3 .

P -curve including the meta-analysis’ 109 significant effects, compared to a hypothetical null-effect curve and a hypothetical 33% power effect curve. Analysis shows a significant right skewedness, indicating the existence of a true effect.

Out of all effects, i  = 109 effects provided a significant effect size of p  < 0.05, out of which i  = 108 showed a p -value of p  < 0.025. The significant right-skewedness test ( p binominal  < 0.001, z Full  = -65.51, z Half  = -64.65, p Half  < 0.001) suggested the existence of a true effect. Furthermore, the non-significant flatness test ( p binominal  = 1, z Full  = 64.13, z Half  = 65.88, p Half  = 1) provided no indicators that a true effect is not present.

In total, both funnel plot and p -curve analysis show no indicators of publication bias or p -hacking, and that the observed effect is true.

Effect size analysis

A summary of all results is presented in Fig. 4 .

Effect sizes, confidence intervals, and number of effects across conditions, controlled for study. Note. Total = across all data; relevant effects = only effects of measures relevant to the mental condition are included; blended = treatment with EMH blended with inpatient care; aftercare = treatment after inpatient care. CBT cognitive-behavioural therapy, PD Psychodynamic therapy.

Total effect

Total effect size with study as random effect revealed a significant positive effect of EMH intervention ( g  = 0.3 [0.2, 0.39], p  < .001, k  = 118). When only including effects of measures relevant to the mental disorder symptoms (e.g., Beck depression scores for depressive disorder patients) and removing measures not directly related to the mental disorder’s symptoms or clinical outcomes (e.g., social support, self-esteem), effect size increased ( g  = 0.36 [0.22, 0.5], p  < 0.001, k  = 83).

As expected given the variety of study designs and conditions, significant heterogeneity was observed for both the total effect (Q(117) = 408.25, p  < .001) and when including only clinically relevant outcomes (Q(82) = 647.91, p  < 0.001).

Treatment type

By-treatment type analysis revealed that both blended interventions during inpatient stay ( g  = 0.42 [0.27, 0.58], p  < 0.001, k  = 19) and aftercare treatments following inpatient stay ( g  = 0.29 [0.24, 0.34], p  < 0.001, k  = 99) showed significant effects.

Mental condition

By-condition analysis revealed significant effects of EMH interventions for eating disorder ( g  = 0.19 [0.07, 0.32], p  = .003, k  = 17), mood disorder ( g  = 0.38 [0.28, 0.49], p  < 0.001, k  = 22), psychotic disorder ( g  = 0.43 [0.27, 0.58], p  < 0.001, k  = 10), return to work ( g  = 0.21 [0.12, 0.3], p  < 0.001, k  = 24), and transdiagnostic patients ( g  = 0.4 [0.31, 0.49], p  < 0.001, k  = 34). No significant effects were found for anxiety disorders ( g  = 0.35 [−0.22, 0.93], p  = 0.23, k  = 3), mental comorbidity with somatic disorders ( g  = 0.19 [−0.02, 0.39], p  = 0.072, k  = 6), and substance abuse ( g  < 0.01 [−0.27, 0.28], p  = 0.964, k  = 2).

Type of therapy

Analysis by type of therapy revealed significant effects for cognitive behavioural therapy (CBT)-based treatments ( g  = 0.26 [0.18, 0.34], p  < 0.001, k  = 43) and psychodynamic (PD) treatments ( g  = 0.35 [0.27, 0.43], p  < 0.001, k  = 39).

Follow-up stability

To investigate potential effects of measurement time (e.g., a decrease of intervention efficacy for longer intervals after treatment), a linear mixed model with measurement time as the fixed effect and study as the random effect for effect sizes was calculated. Results showed no significant effect of measurement time ( t (61) = −00.97, p  = 0.337), showing no indication that the strength of the treatment effect is influenced by the time passed between intervention and measurement.

Post-hoc analyses

Post-hoc analyses were conducted to investigate differences between EMH medium/channel and effects of type of control group. EMH medium analysis revealed significant effects for EMH tools implemented as web-based tools ( g  = 0.32, CI [0.25, 0.37], p  < 0.001) and multimedia interventions ( g  = 0.79, CI [0.29, 1.29], p  = 0.002). Effects for app-based and SMS-based EMH tools were not significant. However, multimedia was used by only one study 43 . The only specific EMH tool used by multiple studies was Deprexis, which showed a significant effect ( g  = 0.61, CI [0.46, 0.77], p  < 0.001).

Control group analysis revealed that EMH interventions significantly improve mental health outcomes compared to passive controls (no active treatment; g  = 0.29, CI [0.19, 0.39], p  < .001), active controls (active treatment alternative to EMH; g  = 0.32, CI [0.24, 0.4], p  < .001), and active controls to which the EMH intervention was added to in the intervention condition ( g  = 0.3, CI [0.22, 0.39], p  < 0.001). Thus, EMH interventions show efficacy compared to active treatments and usual and when used in addition to usual treatments.

Few studies focused on patients affected by anxiety disorders, complicating interpretations of the presented results. Meanwhile, studies with transdiagnostic patients often included patients with anxiety disorders and measured anxiety symptoms (e.g., GAD-7). To gain further insight into the effects of EMH treatment on anxiety disoders, an additional post-hoc analysis has been conducted measuring the efficacy of EMH treatment on anxiety symptoms specifically. The analysis showed a significant effect on anxiety symptoms ( g  = 0.39, CI [0.18, 0.59], p  < 0.001).

EMH procedures have shown to be a viable tool for the treatment of mental disorders, yet research on EMH in inpatient settings is relatively sparse. The current work presents, to our knowledge, the first meta-analysis providing evidence for the efficacy of EMH in inpatient treatment and aftercare. We found a significant small effect of EMH treatment ( g  = 0.3).When focusing on disorder symptoms and clinically relevant outcomes, the effect size is further increased (g = 0.36), signalling that EMH procedures are suitable as interventions tailored to mental disorders in inpatient settings. A preliminary analysis further found no indicators of publication bias or p -hacking within the literature.

The effect remained significant when dividing the studies into the common implementation types of EMH, first when blended with in-person inpatient treatment ( g  = 0.42) and second as an aftercare treatment following inpatient intervention ( g  = 0.29). The majority of studies (21 out of 26) used an aftercare setting with the goal to ensure stabilization and prevent relapse of inpatient cases. Inpatient cases tend to be more severe compared to outpatient cases, with worse post-treatment outcomes when not sufficiently supported by aftercare following discharge 30 , 31 , 32 . The present results suggest that EMH can provide such an effective tool, closing an important mental health supply gap.

By-disorder analysis found that EMH was especially effective for psychotic disorders ( g  = 0.42), transdiagnostic patient groups ( g  = 0.4), and mood disorders ( g  = 0.38). The results are comparable to meta-analyses finding small yet significant effects of EMH in outpatient settings for mood disorders 22 , providing evidence that the effects are comparable to inpatient settings.

The positive effect of EMH treatment for psychotic disorders is surprising given that EMH interventions may worsen psychotic patients’ concerns about technology and being recorded due to psychopathological paranoid tendencies 49 . Furthermore, the effect contrasts the negative outcomes reported in studies investigating psychotic patients 42 , 50 , 51 . While the results complement previous research on the effectiveness of EMH outpatient treatments for schizophrenia and psychosis 52 , the usage of EMH interventions for psychotic disorders remains not well developed, and their efficacy cannot be reliably estimated with the current research. For inpatient settings, SMS-based aftercare reminders for medication adherence did not improve patient outcomes 50 . The HEINS web-based aftercare program containing multiple modules (including psychoeducation, crisis plans, contacts to psychiatrists, and supportive monitoring) meanwhile showed positive user acceptance and adherence 42 , and Horyzons, an online social therapy aftercare program containing multiple features (including psychoeducation, skill development support, peer-to-peer conversations, and expert support), improved patient employment and reduced emergency room visits compared to usual care 51 . Given that both Horyzons and HEINS are interactive support units containing multiple modules, the results suggest that more extensive EMH treatment is needed to ensure aftercare of patients with psychosis. Patients with severe illnesses such as psychosis may not be able to effectively utilize digital health tools. EMH tools are to be used with caution when treating patients with psychosis and should be used in addition to in-person treatment instead of an alternative.

For outpatient treatment, the efficacy of EMH treatment for anorexia nervosa is not well researched, potentially due to the severity of the disorder and the presumed necessity for face-to-face treatment by clinicians 21 . Out of seven studies investigating eating disorder patients, four focused mainly on bulimia nervosa 41 , 46 , 53 , 54 . When excluding a follow-up study 55 and a pilot RCT 56 , only one proper RCT study focused on anorexia nervosa 57 . Although the initial results are promising, caution should be taken when transferring the results onto patients with anorexia nervosa given that the disorder leads to severe consequences including somatic complications that may be insufficiently tracked and treated through digital means.

Meanwhile, no significant effects for anxiety symptoms, comorbidity with somatic disorders, or substance abuse disorders were found. However, only one study investigated anxiety symptoms 36 . Meanwhile, multiple studies with transdiagnostic patient groups included patients with anxiety disorders 37 , 39 , 58 . A post-hoc analysis focusing on anxiety symptoms revealed a significant effect ( g  = 0.39). Inpatient treatment is typically not indicated for anxiety disorders, which may explain the low number of studies. Given that EMH interventions are effective in treating anxiety disorders in outpatient settings 22 , and that the post-hoc analysis revealed a significant improvement in anxiety symptoms, the current negative findings on EMH inpatient treatment for anxiety disorders are to be interpreted with caution. A similar caution can be expressed for the negative result on substance abuse patients, which has been investigated by only one study 59 . Furthermore, future research ought to differentiate effects of EMH for different anxiety diagnoses in inpatient care, as EMH outpatient treatment effectiveness has been found to differ across anxiety disorders 22 .

Analysis by type of therapy revealed the effectiveness of both CBT- ( g  = 0.26) and PD- ( g  = 0.35) based interventions, showing that EMH treatment is effective when based on either of these types of psychotherapy

Finally, the result that observation period did not affect outcomes indicates that EMH-based treatment effects do not deteriorate with time passed after treatment, indicating the long-term stability of the effects. However, the latest measurement used in this analysis was 24 months after treatment. Hence, results cannot be interpreted for longer periods.

In general, the meta-analysis shows the efficacy of EMH treatment across different mental health disorders and types of therapy. Hence, mental health treatment can profit from integrating EMH into the patient journey. Given that EMH add-on also significantly improves outcomes compared to a regular active control group ( g  = 0.3), adding EMH to regular practices can improve overall treatment outcomes. Since treatment as usual tends to be minimal for aftercare treatment, EMH can facilitate long-term improvements and remission prevention following inpatient treatment since other aftercare practices are lacking or minimal. Especially web-based EMH treatment has been shown to be effective throughout multiple studies ( g  = 0.32) compared to SMS- or app-based approaches. Hence, practitioners may use EMH tools both as additives and as alternatives to regular treatment, and especially for aftercare following inpatient treatment. Web-based EMH tools have shown efficacy in most studies.

The meta-analysis is limited by the small number of studies especially for subgroup analyses, as some subgroups (e.g., anxiety disorder or substance abuse patients, or whole health approaches) only include a single study each and can thus not be properly interpreted. Although a total effect was found with a sufficient number of trials, further RCT research is needed to conduct more conclusive meta-analyses for subgroup-related research areas.

The small number of studies precludes further interesting analyses relevant to the design and implementation of EMH methods. For example, a previous meta-analysis on outpatient settings found that specific EMH methods were more effective for certain disorders (e.g., chatbots for depression, mood monitoring features for anxiety). Such research questions may be tackled in future meta-analyses when an adequate number of RCTs have been conducted. Meta-analyses and reviews are generally limited by the terms used and search outputs when conducting literature searches. Even though two literature searches (February 2024 and July 2024) were done for this meta-analysis, it may still not include all relevant literature. Furthermore, this meta-analysis was not preregistered. However, all relevant documents are publicly available.

Specific neuropsychological and cognitive measures were excluded from this meta-analysis to focus the research on explicit mental health outcomes. However, mental health deficits often co-occur with cognitive deficits, for example in memory, concentration, or problem-solving tasks. Although disorder-specific questionnaire measures encompass the measurement of such deficits, future research can focus on the effect of EMH interventions for the improvement of cognitive skills in patients affected by mental health disorders specifically.

Out of all 26 included studies 20 were conducted in Western or Northern Europe (17 in Germany, two in Sweden, one in Finland), three were conducted in North America (two in the USA and one in Canada), one in Australia, one in Hungary, and one in Iran. Research from other regions, such as Africa or East Asia, was absent. This may be due to differences in healthcare systems in different regions, and treatments alternative to inpatient treatment for more severe health cases. Thus, the results of this meta-analysis are mainly derived from studies conducted in countries with populations majorly of European descent. In order to generalize the reported findings, future research may aim to investigate EMH tools in more diverse populations.

Engagement and adherence are major concerns when applying EMH tools 60 , 61 , 62 , 63 . Effects of EMH on attrition were mitigated in this analysis by including group attrition effects in the RoB assessment: in fact, all four high risk studies were excluded due to unclear or uneven attrition rates. Engagement can be defined as usage as intended, measured for example through use frequency or completion 60 . Various included studies excluded participants with low engagement despite completion 41 and hence controlled for low engagement. Included studies mostly did not report direct effects on engagement on outcomes. One study found no effect of EMH tool use (assessed via logs) on symptom severity 56 . Similarly, other studies did not find a correlation between EMH use frequency and symptom improvement 47 , completed models and symptom improvement 64 , or differences between high- and low-frequency users 59 . Meanwhile, the number of completed EMH courses did significantly improve symptoms in patients with anorexia nervosa 55 . Although there are only few studies and results are not consistent, the results nevertheless indicate that use frequency or intensity does generally not affect the treatment efficacy. Finally, some studies report improved engagement in the intervention compared to a control group 65 , 66 , indicating that EMH intervention may improve engagement behaviour. Future research may investigate effects of such engagement when implementing EMH tools.

Given that various measurement outcomes were used and summarized to generalize a wider range of findings, results do not consistently reflect the most clinically relevant outcomes (e.g., remission or relapse rates) which were only reported by six studies for varying mental disorders. Instead, the majority of research studies relied on symptom questionnaires. In total, a majority of the studies included were assessed with some concerns regarding risk of bias. Due to the low number of high-quality research with low bias and large sample sizes, results should be interpreted with some degree of caution. EMH implementations furthermore involve certain risks 67 such as a lack of quality standards 68 , data privacy issues 18 , or a lack of digital literacy by practitioners 19 . Despite promising results in this meta analysis, in the context of such risks, more high quality RCT research is necessary for a more rigorous assessment of EMH efficacy.

In conclusion, the results indicate that EMH procedures are an effective tool in the treatment and aftercare of inpatients, especially for psychotic, mood disorder, and eating disorder, and patient groups combining different diagnoses. EMH tools can be used both in addition to in-person treatment and when in-person treatment is not available, e.g., for aftercare. Future research should investigate effects of EMH tools for the inpatient treatment of specific disorders and the relevance of the specific tools used. Larger sample sizes and randomized trials are warranted to substantiate these effects.

This review was conducted in accordance to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines 69 and Cochrane Handbook guidelines for meta-analyses and systematic reviews 39 .

Literature search

The literature databases SciencGov, PsycInfo, PubMed, and CENTRAL were searched for published literature. In addition, the ProQuest Database was searched for dissertation theses, and ICTRP and ClinicalTrials were searched for trial result registers.

To aim for high sensitivity according to Cochrane guidelines 39 , we used multiple search terms in relation to the following topics: e-mental health (digital, online, e-mental health, technology-based, web-based, internet-based, mobile-based), treatment setting (psychotherapy, psychiatric, psychosomatic), inpatient setting (inpatient, ward patient, hospitalized), and experimental design (RCT, randomized controlled trial). The search term used corresponds to the following: (“digital” OR “online” OR “e-mental health” OR “technology-based” OR “web-based” OR “internet-based” OR “mobile-based”) AND (“psychotherapy” OR “psychiatric” OR “psychosomatic”) AND (“inpatient” OR “ward patient” OR “hospitalized”) AND (“RCT” OR “randomized controlled trial”). Two researchers conducted the literature search in February 2024. Literature search was performed in English and German.

A secondary search was conducted in July 2024 by extending the search to use the terms “e-health”, “mhealth”, and “telemedicine”, and using the mesh terms “digital health”, “telemedicine”, “psychotherapy”, “psychosomatic medicine”, and “inpatients” if applicable, for the databases CENTAL and PubMed. The secondary literature search did not yield any new viable studies.

Literature selection

We included research studies providing EMH interventions during inpatient treatment or aftercare following inpatient treatment, and studies investigating psychiatric symptoms co-occurring in patients hospitalized for physical conditions (e.g., stress or depression symptoms in cancer patients). Cluster and pilot RCTs were included as well.

Studies were excluded if they 1) did not investigate the effect of EMH intervention or aftercare methods, 2) did not investigate inpatients (either during or after inpatient intervention), 3) did not investigate mental health measures as treatment outcomes (e.g., only focusing on somatic symptoms or acceptability of the intervention; specific neuropsychological or cognitive outcomes like problem-solving skills were also excluded), 4) were not randomized controlled trials, 5) did not provide sufficient information to extract the relevant data (e.g., outcome measures or sample sizes), and 6) showed a high risk of bias assessed via the Risk of Bias tool (see next section) 39 . Neuropsychological or cognitive were excluded to focus the meta-analysis on mental health treatment effects. Although cognitive or neuropsychological deficits can be symptoms of mental health disorders, symptom-focused measures of mental health deficits (e.g., depressiveness questionnaires for clinical depression) provide a more discriminative estimation of mental health deficits.

Three independent raters took part in the literature selection. In case of disagreements, the raters discussed the study until agreement was found.

Risk of bias was assessed using the Cochrane risk-of-bias tool for randomized trials (RoB 2) 39 . RoB 2 is designed to assess the risk of an RCT’s bias by classifying the level of risk for the following domains: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, and selective reporting. Examples of risk of bias include non-random or semi-random participant grouping (incl. alternating allocation); high, uneven, or unexplained participant attrition between groups; lack of blinding; or unreported discrepancies between the study protocol and study. If no information on a domain was provided, the particular domain was assessed with medium risk.

Domains were rated on three levels: low, medium, or high risk of bias. Research studies with a high risk of bias were excluded from the analysis.

Measurement selection

For the total analyses, only measures related to clinical symptoms and psychosocial performance were included. These include: metric variables of disorder-related incidents (relapses, readmissions, abstinence, admissions); disorder-related symptom severity measurements; general psychopathology, well-being, or quality of life; employment-related measures (when relevant); and specific mental or psychosocial measures expected to correlate with symptom severity (e.g., self-esteem, positive and negative affect, stress). All relevant measures in a study were included in an analysis and controlled by treating study as a random effect.

Variable summarization

To investigate the relevant research questions, studies and measures were categorized by the following system.

EMH treatment type was categorized into either blended intervention (EMH was implemented into the inpatient setting) or aftercare treatment (EMH was provided after completing inpatient setting).

The variable Disorder type was classified into the following categories based on the patient group investigated in the study: anxiety disorders (ICD-10 diagnoses F40 and F41), eating disorders (ICD-10 diagnoses F50) mood disorders (ICD-10 diagnoses F3), psychotic disorders (ICD-10 diagnoses F2), substance abuse disorders (ICD-10 diagnoses F1x.2), or their DSM-5 diagnostic equivalents. A study treating patient groups from different categories was classified as transdiagnostic . A study was categorized as somatic comorbidity if the effects of EMH interventions on mental health outcomes in somatic inpatient groups were investigated (e.g., stress or anxiety symptoms in cancer patients). Finally, the category return to work was used for studies focussing on outcomes related to workplace reintegration following inpatient care.

The variable type of therapy was classified according to the type of therapy the EMH intervention was based on according to the authors. If no type of therapy was mentioned, the variable was valued as not available .

Data extraction

Data was summarized on multiple variables: author, title, year, country, type (aftercare, blended treatment), treated mental disorder, somatic illness (if present), digital method, type of therapy, type of control group (active, passive), outcome measure, follow-up, sample sizes, and outcome results (means, standard deviations, odds ratios, effect sizes). Data was extracted by one rater and verified by two other independent raters. Study characteristics were tabulated according to the planned subgroup analyses. Studies with insufficient data were excluded from the (sub-)analyses.

Data transformation

Hedges’ g was used to report effect sizes as it outperforms Cohen’s d for small sample sizes. Cohen’s d effect sizes and variances were transformed to Hedges’ g values and variances using the following formulas 48 :

When a study reported odds ratio (OR) values, values were first transformed into Cohen’s d using the following formula 48 :

Cohen’s d values were then transformed into Hedges’ g according to Formula 1.

Data analysis

Heterogeneity was tested and pre-assumed given the variety of setups in research studies and subgroup analyses were therefore decided a priori. Fixed-random effects models were used with study as a random factor. To assess the results’ robustness, the total effect is analysed two times, first using the whole range of data, and second using only outcomes that are clinically relevant (limited to symptom severity and clinical outcomes). Effects’ certainty and confidence were assessed through risk of bias assessment according to Cochrane guidelines and by investigating publication bias using funnel plot and p -curve analyses. The meta-analysis was not preregistered. No protocol is available for the meta-analysis. Confidence was assessed by calculating confidence intervals from standard errors.

Post-hoc analyses were decided after the data was analysed for the main hypotheses. Post-hoc analyses included the effect of EMH medium, the role of control group, and the effect of EMH on anxiety symptoms specifically.

Data availability

Data including the complete list of searched literature, the included studies, extracted data, and risk assessment are publicly available at https://osf.io/bc59e . Thus, all data is provided to replicate assessment of literature according to inclusion criteria and risk of bias, as well as all data necessary to replicate the analyses.

Code availability

The R code for the analysis is publicly available at https://osf.io/bc59e . Main and subgroup analyses and visualization of results were conducted via RStudio (ver. 2021.9.1.0, R version 4.1.2). The R packages dmetar and metafor were used for the analyses 70 , 71 .

Borges, G. et al. Twelve-month prevalence of and risk factors for suicide attempts in the World Health Organization World Mental Health Surveys. J. Clin. Psychiatry 71 , 1617–1628 (2010).

PubMed   PubMed Central   Google Scholar  

Doran, C. M. & Kinchin, I. A review of the economic impact of mental illness. Aust. Health Rev. 43 , 43 (2019).

PubMed   Google Scholar  

Vos, T. et al. Global, regional, and national incidence, prevalence, and years lived with disability for 328 diseases and injuries for 195 countries, 1990–2016: A systematic analysis for the global burden of disease study 2016. Lancet 390 , 1211–1259 (2017).

Google Scholar  

Santomauro, D. F. et al. Global prevalence and burden of depressive and anxiety disorders in 204 countries and territories in 2020 due to the COVID-19 pandemic. Lancet 398 , 1700–1712 (2021).

Ahmed, N. et al. Mental health in Europe during the COVID-19 pandemic: A systematic review. Lancet Psychiatry 10 , 537–556 (2023).

Alonso, J. et al. Prevalence of mental disorders in Europe: Results from the European study of the epidemiology of Mental Disorders (esemed) project. Acta Psychiatr. Scandinavica 109 , 21–27 (2004).

Sacco, R., Camilleri, N., Eberhardt, J., Umla-Runge, K. & Newbury-Birch, D. A systematic review and meta-analysis on the prevalence of mental disorders among children and adolescents in Europe. European Child & Adolescent Psychiatry https://doi.org/10.1007/s00787-022-02131-2 (2022).

Wittchen, H. U. et al. The size and burden of mental disorders and other disorders of the brain in Europe 2010. Eur. Neuropsychopharmacol. 21 , 655–679 (2011).

PubMed   CAS   Google Scholar  

Zuberi, A. et al. Prevalence of mental disorders in the WHO Eastern Mediterranean Region: A systematic review and meta-analysis. Front. Psychiatry 12 , (2021).

Jacobi, F. et al. Psychische Störungen in der Allgemeinbevölkerung. Der Nervenarzt 85 , 77–87 (2014).

Andrade, L. H. et al. Barriers to mental health treatment: Results from the WHO World Mental Health Surveys. Psychol. Med. 44 , 1303–1317 (2013).

EU-Compass for action on Mental Health and well-being. Public Health Available at: https://health.ec.europa.eu/non-communicable-diseases/mental-health/eu-compass-action-mental-health-and-well-being_en (Accessed: 21st May 2024).

Koopmanschap, M. A., Brouwer, W. B. F., Hakkaart-van Roijen, L. & van Exel, N. J. A. Influence of waiting time on cost-effectiveness. Soc. Sci. Med. 60 , 2501–2504 (2005).

Reichert, A. & Jacobs, R. The impact of waiting time on patient outcomes: Evidence from early intervention in psychosis services in England. Health Econ. 27 , 1772–1787 (2018).

van Dijk, D. A. et al. Worse off by waiting for treatment? the impact of waiting time on clinical course and treatment outcome for depression in routine care. J. Affect. Disord. 322 , 205–211 (2023).

Sherman, M. L., Barnum, D. D., Buhman-Wiggs, A. & Nyberg, E. Clinical intake of child and adolescent consumers in a rural community mental health center: Does wait-time predict attendance? Community Ment. Health J. 45 , 78–84 (2008).

Williams, M. E., Latta, J. & Conversano, P. Eliminating the wait for Mental Health Services. J. Behav. Health Serv. Res. 35 , 107–114 (2007).

Köhnen, M., Dirmaier, J. & Härter, M. Potenziale und Herausforderungen von E-mental-health-interventionen in der Versorgung Psychischer störungen. Fortschr. der Neurologie · Psychiatr. 87 , 160–164 (2019).

Weitzel, E. C. et al. E-mental health in Germany — what is the current use and what are experiences of different types of health care providers for patients with mental illnesses? Arch. Public Health 81 , 1–6 (2023).

Bolinski, F. et al. The effect of E-mental health interventions on academic performance in university and college students: A meta-analysis of randomized controlled trials. Internet Interventions 20 , 100321 (2020).

PubMed   PubMed Central   CAS   Google Scholar  

Firth, J. et al. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J. Affect. Disord. 218 , 15–22 (2017).

Linardon, J. et al. Current evidence on the efficacy of mental health smartphone apps for symptoms of depression and anxiety. A meta‐analysis of 176 randomized controlled trials. World Psychiatry 23 , 139–149 (2024).

Linardon, J., Shatte, A., Messer, M., Firth, J. & Fuller-Tyszkiewicz, M. E-mental health interventions for the treatment and prevention of eating disorders: An updated systematic review and meta-analysis. J. Consulting Clin. Psychol. 88 , 994–1007 (2020).

Simblett, S., Birch, J., Matcham, F., Yaguez, L. & Morris, R. A systematic review and meta-analysis of E-mental health interventions to treat symptoms of posttraumatic stress. JMIR Mental Health 4 , (2017).

Stratton, E. et al. Effectiveness of ehealth interventions for reducing mental health conditions in employees: A systematic review and meta-analysis. PLOS ONE 12 , (2017).

Löbner, M. et al. What comes after the trial? an observational study of the real-world uptake of an e-mental health intervention by general practitioners to reduce depressive symptoms in their patients. Int. J. Environ. Res. Public Health 19 , 6203 (2022).

Rost, T. et al. User acceptance of computerized cognitive behavioral therapy for depression: Systematic review. Journal of Medical Internet Research 19 , (2017).

Wangler, J. & Jansky, M. How can primary care benefit from Digital Health Applications? – a quantitative, explorative survey on attitudes and experiences of General Practitioners in Germany. BMC Digital Health 2 , 1–14 (2024).

Zipfel, S., Herzog, W., Kruse, J. & Henningsen, P. Psychosomatic medicine in Germany: More timely than ever. Psychother. Psychosom. 85 , 262–269 (2016).

Franz, M. et al. Stationäre Tiefenpsychologisch Orientierte psychotherapie bei Depressiven Störungen (stop-D) - erste befunde einer naturalistischen, multizentrischen Wirksamkeitsstudie. Z. f.ür. Psychosomatische Med. und Psychotherapie 61 , 19–35 (2015).

Gönner, S., Bischoff, C., Ehrhardt, M. & Limbacher, K. Effekte therapiezielorientierter Kognitiv-Verhaltenstherapeutischer Nachsorgemaßnahmen auf den therapietransfer im Anschluss an eine Stationäre psychosomatische Rehabilitationsbehandlung. Die Rehabilitation 45 , 369–376 (2006).

Zeeck, A., Wietersheim, Jvon, Weiss, H., Beutel, M. & Hartmann, A. The INDDEP Study: Inpatient and day hospital treatment for depression – symptom course and predictors of change. BMC Psychiatry 13 , 100 (2013).

Giel, K. E. et al. Efficacy of post-inpatient aftercare treatments for anorexia nervosa: A systematic review of randomized controlled trials. J. Eat. Disord. 9 , 129 (2021).

Hegedüs, A., Kozel, B., Richter, D. & Behrens, J. Effectiveness of transitional interventions in improving patient outcomes and service use after discharge from psychiatric inpatient care: A systematic review and meta-analysis. Front. Psychiatry 10 , 969 (2020).

Vittengl, J. R., Clark, L. A., Dunn, T. W. & Jarrett, R. B. Reducing relapse and recurrence in Unipolar Depression: A comparative meta-analysis of cognitive-behavioral therapy’s effects. J. Consulting Clin. Psychol. 75 , 475–488 (2007).

Sharma, G. et al. Brief app-based cognitive behavioral therapy for anxiety symptoms in psychiatric inpatients: Feasibility randomized controlled trial. JMIR Formative Res. 6 , e38460 (2022).

Zwerenz, R. et al. Transdiagnostic, psychodynamic web-based self-help intervention following inpatient psychotherapy: Results of a feasibility study and randomized controlled trial. JMIR Ment. Health 4 , e41 (2017).

Hennemann, S., Farnsteiner, S. & Sander, L. Internet- and Mobile-based aftercare and relapse prevention in mental disorders: A systematic review and recommendations for Future Research. Internet Interventions 14 , 1–17 (2018).

Cochrane Handbook for Systematic Reviews of interventions. (Cochrane, 2019).

Kordy, H. et al. Internet-delivered disease management for recurrent depression: A multicenter randomized controlled trial. Psychother. Psychosom. 85 , 91–98 (2016).

Jacobi, C. et al. Web-based aftercare for women with bulimia nervosa following inpatient treatment: Randomized controlled efficacy trial. J. Med. Internet Res. 19 , e321 (2017).

Gallinat, C. et al. Feasibility of an intervention delivered via mobile phone and internet to improve the continuity of care in schizophrenia: A randomized controlled pilot study. Int. J. Environ. Res. Public Health 18 , 12391 (2021).

Nolte, S. et al. Do sociodemographic variables moderate effects of an internet intervention for mild to moderate depressive symptoms? an exploratory analysis of a randomised controlled trial (evident) including 1013 participants. BMJ Open 11 , e041389 (2021).

Abadi, M. et al. Achieving whole health: A preliminary study of TCMLH, a group-based program promoting self-care and empowerment among veterans. Health Educ. Behav. 49 , 347–357 (2021).

Becker, J., Kreis, A., Beutel, M. E. & Zwerenz, R. Wirksamkeit der Internetbasierten, Berufsbezogenen Nachsorge GSA-online im Anschluss an die stationäre psychosomatische Rehabilitation: Ergebnisse einer randomisiert Kontrollierten Studie. Die Rehabilitation 61 , 276–286 (2022).

Gulec, H. et al. A randomized controlled trial of an internet-based posttreatment care for patients with eating disorders. Telemed. e-Health 20 , 916–922 (2014).

Bruhns, A., Lüdtke, T., Moritz, S. & Bücker, L. A mobile-based intervention to increase self-esteem in students with depressive symptoms: Randomized controlled trial. JMIR mHealth uHealth 9 , e26498 (2021).

Borenstein, M. Computing effect sizes for meta-analysis. (Wiley-Blackwell, 2011).

Philippe, T. J. et al. Digital Health Interventions for delivery of mental health care: Systematic and comprehensive meta-review. JMIR Ment. Health 9 , e35159 (2022).

Välimäki, M., Kannisto, K. A., Vahlberg, T., Hätönen, H. & Adams, C. E. Short text messages to encourage adherence to medication and follow-up for people with psychosis (mobile.net): Randomized controlled trial in Finland. J. Med. Internet Res. 19 , e245 (2017).

Alvarez‐Jimenez, M. et al. The Horyzons Project: A randomized controlled trial of a novel online social therapy to maintain treatment effects from specialist first‐episode psychosis services. World Psychiatry 20 , 233–243 (2021).

Clarke, S., Hanna, D., Mulholland, C., Shannon, C. & Urquhart, C. A systematic review and meta-analysis of digital health technologies effects on psychotic symptoms in adults with psychosis. Psychosis 11 , 362–373 (2019).

Bauer, S., Okon, E., Meermann, R. & Kordy, H. Technology-enhanced maintenance of treatment gains in eating disorders: Efficacy of an intervention delivered via text messaging. J. Consulting Clin. Psychol. 80 , 700–706 (2012).

Bauer, S., Okon, E., Meermann, R. & Kordy, H. SMS-Nachsorge: Sektorenübergreifende Versorgung für patientinnen mit bulimia nervosa. Verhaltenstherapie 23 , 204–209 (2013).

Fichter, M. M., Quadflieg, N. & Lindner, S. Internet-based relapse prevention for anorexia nervosa: Nine- month follow-up. J. Eat. Disord. 1 , 23 (2013).

Neumayr, C., Voderholzer, U., Tregarthen, J. & Schlegl, S. Improving aftercare with technology for anorexia nervosa after intensive inpatient treatment: A pilot randomized controlled trial with a therapist‐guided smartphone app. Int. J. Eat. Disord. 52 , 1191–1201 (2019).

Fichter, M. M. et al. Does internet-based prevention reduce the risk of relapse for anorexia nervosa? Behav. Res. Ther. 50 , 180–190 (2012).

Ebert, D., Tarnowski, T., Gollwitzer, M., Sieland, B. & Berking, M. A transdiagnostic internet-based maintenance treatment enhances the stability of outcome after inpatient cognitive behavioral therapy: A randomized controlled trial. Psychother. Psychosom. 82 , 246–256 (2013).

Harrington, K. F. et al. Web-based smoking cessation intervention that transitions from inpatient to outpatient: Study protocol for a randomized controlled trial. Trials 13 , 123 (2012).

Kernebeck, S., Busse, T. S., Ehlers, J. P. & Vollmar, H. C. Adhärenz digitaler Interventionen im Gesundheitswesen: Definitionen, Methoden und offene Fragen [Adherence to digital health interventions: definitions, methods, and open questions]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 64 , 1278–1284 (2021). OctGermanEpub 2021 Sep 24. PMID: 34559252; PMCID: PMC8492574.

Sieverink, F., Kelders, S. M. & van Gemert-Pijnen, J. E. Clarifying the Concept of Adherence to eHealth Technology: Systematic Review on When Usage Becomes Adherence. J. Med Internet Res 19 , e402 (2017). Dec 6PMID: 29212630; PMCID: PMC5738543.

Beatty, L. & Binnion, C. A Systematic Review of Predictors of, and Reasons for, Adherence to Online Psychological Interventions. Int J. Behav. Med 23 , 776–794 (2016). DecPMID: 26957109.

Eysenbach, G. The law of attrition. J. Med. Internet Res. 7 , e11 (2005). Mar 31PMID: 15829473; PMCID: PMC1550631.

Norlund, F. et al. Internet-based cognitive behavioral therapy for symptoms of depression and anxiety among patients with a recent myocardial infarction: The U-CARE heart randomized controlled trial. J. Med. Internet Res. 20 , e88 (2018).

Holländare, F. et al. Randomized trial of internet-based relapse prevention for partially remitted depression. Acta Psychiatr. Scandinavica 124 , 285–294 (2011).

Zwerenz, R. et al. Online self-help as an add-on to inpatient psychotherapy: Efficacy of a new blended treatment approach. Psychother. Psychosom. 86 , 341–350 (2017).

Ebert, D. D. et al. Internet- and mobile-based psychological interventions: Applications, efficacy, and potential for improving mental health. Eur. Psychologist 23 , 167–187 (2018).

Weitzel, E. C. et al. E-mental-health und Digitale Gesundheitsanwendungen in Deutschland. Der Nervenarzt 92 , 1121–1129 (2021).

Page, M. J. et al. The Prisma 2020 statement: An updated guideline for reporting systematic reviews. BMJ. https://doi.org/10.1136/bmj.n71 (2021).

Harrer, M., Cuijpers, P., Furukawa, T. A. & Ebert, D. D. Doing meta-analysis with R https://doi.org/10.1201/9781003107347 (2021).

Viechtbauer, W. Conducting meta-analyses inrwith themetaforpackage. J. Stat. Softw. 36 , (2010).

Bischoff, C. et al. Wirksamkeit von Handheld-Gestütztem Selbstmanagement (e-coaching) in der rehabilitationsnachsorge. Verhaltenstherapie 23 , 243–251 (2013).

Ebert, D. et al. Web-basierte rehabilitationsnachsorge nach stationärer Psychosomatischer Therapie (W-rena). Die Rehab. 52 , 164–172 (2013).

CAS   Google Scholar  

Schmädeke, S. & Bischoff, C. Wirkungen smartphonegestützter Psychosomatischer Rehabilitationsnachsorge (eATROS) Bei depressiven Patienten. Verhaltenstherapie 25 , 277–286 (2015).

Willems, R. A. et al. Short‐term effectiveness of a web‐based tailored intervention for cancer survivors on quality of life, anxiety, depression, and fatigue: Randomized controlled trial. Psycho-Oncol. 26 , 222–230 (2016).

Zwerenz, R. et al. Evaluation of a transdiagnostic psychodynamic online intervention to support return to work: A randomized controlled trial. PLOS ONE 12 , e0176513 (2017).

Schlicker, S., Ebert, D. D., Middendorf, T., Titzler, I. & Berking, M. Evaluation of a text-message-based maintenance intervention for major depressive disorder after inpatient cognitive behavioral therapy. J. Affect. Disord. 227 , 305–312 (2018).

Zwerenz, R. et al. Improving the Course of Depressive Symptoms After Inpatient Psychotherapy Using Adjunct Web-Based Self-Help: Follow-Up Results of a Randomized Controlled Trial. J. Med. Internet Res. 21 , e13655 (2019).

Shaygan, M., Yazdani, Z. & Valibeygi, A. The effect of online multimedia psychoeducational interventions on the resilience and perceived stress of hospitalized patients with COVID-19: A pilot cluster randomized parallel-controlled trial. BMC Psychiatry 21 , 93 (2021).

Levis, M. et al. An implementation and effectiveness study evaluating Conflict Analysis in VA residential substance abuse services: Whole health informed self-guided online care. EXPLORE 18 , 688–697 (2022).

Download references

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

Clinic for Psychosomatic Medicine and Psychotherapy, LVR-University Hospital Essen, University of Duisburg-Essen, Essen, Germany

Alexander Diel, Isabel Carolin Schröter, Anna-Lena Frewer, Christoph Jansen, Anita Robitzsch, Martin Teufel & Alexander Bäuerle

Center for Translational Neuro- and Behavioral Sciences, University of Duisburg-Essen, Essen, Germany

Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, LVR-University Hospital Essen, University of Duisburg-Essen, Essen, Germany

Gertraud Gradl-Dietsch

You can also search for this author in PubMed   Google Scholar

Contributions

A.D. drafted the manuscript, conducted study search and selection, analysed the data, and revised the manuscript. I.S. and A.L.F. revised the manuscript and validated study search and selection. C.J., A.B., M.T., A.R., and G.G.D. revised the manuscript. M.T. acquired resources and funding.

Corresponding author

Correspondence to Alexander Diel .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Diel, A., Schröter, I.C., Frewer, AL. et al. A systematic review and meta analysis on digital mental health interventions in inpatient settings. npj Digit. Med. 7 , 253 (2024). https://doi.org/10.1038/s41746-024-01252-z

Download citation

Received : 24 May 2024

Accepted : 03 September 2024

Published : 17 September 2024

DOI : https://doi.org/10.1038/s41746-024-01252-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

STEM Outside of School: a Meta-Analysis of the Effects of Informal Science Education on Students' Interests and Attitudes for STEM

• Open access
• Published: 17 September 2024

Cite this article

You have full access to this open access article

• Xin Xia   ORCID: orcid.org/0009-0009-1717-8511 1 ,
• Lillian R. Bentley 2 ,
• Xitao Fan 3 &
• Robert H. Tai 4  

This meta-analysis explores the impact of informal science education experiences (such as after-school programs, enrichment activities, etc.) on students' attitudes towards, and interest in, STEM disciplines (Science, Technology, Engineering, and Mathematics). The research addresses two primary questions: (1) What is the overall effect size of informal science learning experiences on students' attitudes towards and interest in STEM? (2) How do various moderating factors (e.g., types of informal learning experience, student grade level, academic subjects, etc.) impact student attitudes and interests in STEM? The studies included in this analysis were conducted within the United States in K-12 educational settings, over a span of thirty years (1992–2022). The findings indicate a positive association between informal science education programs and student interest in STEM. Moreover, the variability in these effects is contingent upon several moderating factors, including the nature of the informal science program, student grade level, STEM subjects, publication type, and publication year. Summarized effects of informal science education on STEM interest are delineated, and the implications for research, pedagogy, and practice are discussed.

Avoid common mistakes on your manuscript.

STEM careers and opportunities, encompassing Science, Technology, Engineering, and Mathematics, are fundamental drivers of economic stability within contemporary society (Xie & Killewald, 2012 ; Xie et al., 2015 ). The United States has demonstrated a strategic commitment to fostering successive generations of STEM-focused individuals to maintain global competitiveness within the modern market (National Research Council [NRC], 2007 ). An educational priority has thus been the cultivation of student interest in STEM disciplines. It is widely acknowledged that students exhibiting such interest possess heightened prospects for navigating the STEM pipeline toward attaining a career in the field (Beier & Rittmayer, 2008 ; Business-Higher Education Forum, 2011 ; Tai et al., 2006 ). Attainment of a STEM career not only bolsters individual economic stability but also amplifies the potential for national STEM-driven innovation (Xie et al., 2015 ).

Students gain STEM experiences through interacting in formal (such as that occurring in schools, colleges, and universities) and informal learning environments (that occur everywhere else; Krishnamurthi & Rennie, 2013 ). On average, children spend approximately 20% of their time learning in formal educational environments (Eshach, 2007 ; Falk & Dierking, 2010 ; Sacco et al., 2014 ). This suggests that informal learning experiences could contain enormous potential to strengthen and enrich school STEM experiences (Bevan et al., 2010 ; NRC, 2009 ; Phillipset al., 2007 ). The United States has a variety of informal learning institutions for people to engage in STEM learning opportunities, including public libraries, zoos, aquariums, and museums (Falk & Dierking, 2010 ). Nevertheless, there is limited understanding regarding the specific types of informal experiences that ignite and sustain children's STEM learning (National Academies of Sciences, Engineering, and Medicine [NASEM], 2016 ). Therefore, understanding how these opportunities support STEM interest is an important area of research.

Research into the effects of informal science programs highlights that situational factors can create conflicting results. On the one hand, informal science learning was shown to promote and strengthen student understanding of school science and interest in STEM (Bevan et al., 2010 ; Maiorca et al., 2021 ; Phillips et al., 2007 ). On the other hand, however, informal science experiences could have negative effects on students’ attitudes toward STEM (Migliarese, 2011 ; Shields, 2010 ). The inconsistency of such research findings suggest that research attention is needed to understand how informal science learning may affect students’ attitudes toward STEM.

The aim of this investigation was to conduct a current quantitative meta-analysis examining the impacts of informal science programs on students' attitudes and interests towards STEM disciplines. This meta-analysis offers fresh perspectives on potential determinants underlying the variation in effect sizes observed across diverse studies. These nuanced insights offer valuable guidance for educators, policymakers, and practitioners in their endeavor to create informal science learning initiatives aimed at cultivating and nurturing students' interest and attitudes towards STEM.

Literature Review

Formal and informal learning experiences.

Formal and informal learning environments can support STEM-focused learning experiences (Decoito, 2014 ; Frantz et al., 2011 ). Formal STEM learning is any experience or activity that happens within a normal class period and at a school. Informal science learning is any activity or experience that occurs outside of the regular school day, and it includes activities that are not developed as part of an ongoing school curriculum (Crane, 1994 ). Formal STEM learning experiences tend to be mandatory, whereas informal science learning experiences are mostly voluntary (Crane, 1994 ). For this meta-analysis, we include school-based field trips, student out-of-school projects (after-school camps), community-based science youth programs, visits to museums and zoos, after-school programs, summer camps, and weekend school camps as informal science activities (Hofstein & Rosenfeld, 1996 ; So et al., 2018 ).

Attitude and Interest Toward STEM

Various conceptualizations of attitudes and interests in STEM exist within scholarly discourse. Gauld and Hukins ( 1980 ) delineated attitudes towards scientists, scientific inquiry, science learning, science-related activities, science careers, and the embrace of scientific attitudes as affective behaviors indicative of interests in science. Alternatively, Osborne et al. ( 2003 ) posited that attitudes towards science encompass emotions, beliefs, and values pertaining to science and its societal impact. Additionally, Potvin and Hasni ( 2014 ) categorized attitudes into distinct dimensions such as motivation, enjoyment, interest, self-efficacy related to being able to complete STEM tasks and STEM career aspirations.

There are diverse definitions of attitude and interest in STEM. Gauld and Hukins ( 1980 ) proposed that attitudes toward scientists, scientific inquiry, science learning, science-related activities, science careers, and the adoption of scientific attitudes were all affective behaviors representing interests in science. Osborne et al. ( 2003 ) proposed that attitudes toward science are composed of feelings, beliefs, and values held about science, and the impact of science on society. Attitudes can also include the ideas of enjoyment, motivation, interest, self-efficacy, and career aspirations (Potvin & Hasni, 2014 ).

Drawing from a comprehensive examination of pertinent scholarly literature concerning attitudes toward science, and extending this framework to encompass STEM disciplines, we operationalized attitude toward STEM across four distinct dimensions: interest, self-efficacy, overall attitude toward STEM, and career interests (Wiebe et al., 2018 ). Interest in STEM pertains to the affective domain, encapsulating individuals' emotions and sentiments regarding the learning of scientific subjects (e.g., Zhang & Tang, 2017 ). Self-efficacy refers to students' perceptions of their own capabilities to excel in STEM domains, as articulated by Bandura ( 1995 ). Attitude toward STEM denotes individuals' perspectives on the value, utility, and societal ramifications of science (e.g., Dowey, 2013 ). Lastly, STEM career interest is delineated as the extent to which students can furnish meaningful insights into their aspirations for future careers within STEM disciplines (Maltese & Tai, 2011 ).

This meta-analysis extends the scope of prior research conducted by Young et al. ( 2017 ). Young et al. completed a meta-analysis spanning 2009–2015 on the impact of out-of-school time learning on students' interests in STEM. The authors operationally defined out-of-school time as encompassing summer enrichment programs and after-school activities. Their findings indicated a favorable influence of out-of-school time on students' inclination towards STEM subjects. Furthermore, the variability in these effects was found to be moderated by factors such as the methodological rigor of the research design, the thematic focus of the programs, and the grade level of the participants.

Our analysis built on this foundation in several keyways. First, we extended the scope of time by including 30 years of empirical studies (1992–2022), compared to six from Young et al. ( 2017 ). We also broaden the types of programs evaluated by looking at all informal programs including field trips, outreach programs, and mobile labs. Additionally, we looked at the moderating effects of STEM as an entire discipline, then Science, Technology, Engineering, and Mathematics separately. We included the additional moderators of publication type, and year of study in the overall analysis. Finally, we extended the outcome variable by examining student interest, attitude, and self-efficacy toward STEM. The additional moderators and scope of the study will considerably increase the breadth and depth of the empirical knowledge base about the effects of informal science education experiences on students' interests and attitudes toward STEM.

Potential Moderators

The following literature review aims to elucidate salient findings concerning the different moderators used in this analysis. Specifically, the review focuses on the scholarship concerning different distinct categories of informal science programs, the influence of grade-level variations on students' STEM interests, the diverse typologies of STEM activities, as well as the impact of publication type and publication year on the overall effect of informal programing on students interests and attitudes toward STEM. Additionally, an examination of pertinent empirical conclusions concerning student interest, attitude, and self-efficacy regarding informal science programs will be summarized.

Type of Program

There are many different types of informal science programs. In this analysis, we consider after-school programs, summer camps, outreach programs, weekend school camps, field trips, mobile labs, and a mixture of these different out-of-school experiences to be included in informal education. Young et al. ( 2017 ) concluded that out-of-school programming (after school or summer school) did not produce significant effects on students' attitudes toward STEM. We are interested in re-examining the effects of after-school programs on students’ attitudes towards STEM and will also include other types of informal science experiences in our analysis. Variability in the outcomes of studies examining the association between informal STEM programs and students' interest levels in STEM may be attributable to the diversity of program structures investigated. Consequently, there exists potential for enriching our comprehension of informal science programming through the exploration of alternative forms of informal science experiences.

After-School Programs

After-school programs are any type of informal program that happens after-school and within or outside of school property. The programs can be administered by the school or by community-based organizations (Dryfoos, 1999 ). In this analysis, we define after-school programs as programs administered by the school.

Summer Enrichment Programs and Weekend School Camps

Summer camps and weekend school camps represent extracurricular educational endeavors that occur beyond the confines of the regular academic calendar, particularly during weekends and summer (Young et al., 2017 ). In the context of our investigation, we defined summer enrichment programs and weekend school camps as initiatives with a distinct focus on Science, Technology, Engineering, and Mathematics (STEM). Summer enrichment programs have been associated with accelerated learning trajectories including early graduation (Berliner, 2009 ), while concurrently catering to the educational needs of culturally and linguistically diverse students, as well as those hailing from economically disadvantaged backgrounds (Keiler, 2011 ; Matthews & Mellom, 2012 ).

However, discrepant to these positive associations, Young et al. concluded a lack of significant moderation effect exerted by summer camps on student interest in STEM disciplines. Concurrently, empirical inquiry into the impact of weekend school camps remains notably scarce, thus creating a paucity in the research space to investigate how these activities support interest in and attitudes toward STEM.

Outreach Programs

We define outreach programs as any STEM activity that is organized by an outside institution. There can be various formats to the program, from lecture-style outreach where a visiting professor will talk to a group of students about STEM, to a field trip to a university where STEM career pathways are highlighted (Tillinghast et al., 2020 ). Limited studies have measured the effects of these programs on students' interests and attitudes toward STEM, highlighting a need for research expansion, especially using meta-analysis techniques.

Field Trips and Virtual Museums

Alternative forms of informal science program engagements include field trips and virtual museum experiences. We delineate field trips and virtual museums as occasions wherein students embark on guided tours of STEM facilities, either physically or virtually, during school hours but outside the structured curriculum timeframe. These experiences have the potential to influence students' academic achievement and foster interest in STEM careers. For instance, secondary school students who interacted with scientists during field trips demonstrated heightened awareness regarding STEM career pathways (Jensen & Sjaastad, 2013 ). Furthermore, research indicates that students exhibited enhanced performance in mathematics subsequent to participating in a field trip to the New York Hall of Science (Alliance, 2011 ). Notably, despite these findings, there remains a paucity of systematic investigations utilizing meta-analytical approaches to scrutinize the collective impact of such field trip experiences.

Mobile Labs

The last type of informal science experience that we analyzed was the use of mobile labs. Mobile labs are mobile vehicles and buses that transport STEM labs for students to experience hands-on science at their schools. They first became popular in the late-1990s and currently have 29 member programs in 17 different states (Jones & Stapleton, 2017 ) .

Mixed Studies

Some programs are a combination of informal experiences. When we could not categorize an informal experience as one separate group, we analyzed the effect of the program in a mixed category.

Recent empirical scholarship focused on the association of students’ grade level and their interest and attitude towards STEM has produced conflicting conclusions. Scholars indicated a notable trend whereby students' attitudes and perceptions towards science and STEM disciplines diminish with increasing age as they progress through their educational schooling (George, 2000 ; Morrell & Lederman, 1998 ; Murphy & Beggs, 2003 ; Osbourne et al., 2003 ; Silver & Rushton, 2008 ). Additionally, these scholars underscored a discernible decline in student interest and enjoyment of science from intermediate to high school (George, 2000 ; Morrell & Lederman, 1998 ). Conversely, research conducted by Maltese and Tai ( 2011 ) revealed a positive association between eighth-grade students' perceptions of science's utility for their future and their likelihood of pursuing STEM degrees. Additionally, Sadler et al. ( 2012 ) established that students' career aspirations in STEM fields upon entering high school emerged as robust predictors of their vocational interests upon completing high school.

In summary, student attitudes towards STEM and their STEM career inclinations undergo a dynamic evolution throughout their elementary and secondary educational experiences. Once students reach early high school, research indicates that students’ interests and attitudes towards STEM solidify and can become predictors of their career choices later in college. Leveraging meta-analytic methodologies, our study aimed to augment the existing body of research by exploring the differential impacts of elementary and secondary education levels on students' overarching attitudes and interests in STEM disciplines as a result of their informal STEM experiences.

Scholarship investigating how students’ attitudes and interests might change when they study Science, Technology, Engineering, and Mathematics (STEM) as individual subjects and in conjunction, represents a research space that has limited conclusions. Looking at STEM experiences in conjunction, Wiebe et al. ( 2018 ) concluded that there was a reciprocal relationship between students' STEM experiences and the development of specific STEM career aspirations. Notably, success in mathematics (independent of STEM) was positively associated with the likelihood of students pursuing advanced education in STEM fields (Wang, 2012 ). Informal STEM learning activities have been shown to affect students’ self-efficacy related to mathematics (Jiang et al., 2024 ). Likewise, heightened self-efficacy related to science among students augments the propensity for embarking upon a career trajectory within the STEM domain. Additionally, throughout high school, an elevated interest in STEM is associated with a student's sustained commitment to undertaking advanced coursework in both mathematics and science (Simpkins et al., 2006 ). Overall, when students have positive experiences in STEM both interdisciplinary and individually, these experiences have a positive association with their STEM career interests and higher education aspirations.

Most STEM research focuses on science and mathematics, neglecting technology and engineering's impact on student attitudes and interest. This opens up a paucity in this research space to investigate the moderating effect of STEM as an entire discipline, and science, technology, engineering, and mathematics as separate subjects as they are associated with students' interests and attitudes towards informal science experiences.

Learning Outcomes

As delineated earlier, we operationalized attitude in four distinct facets: interest in STEM, self-efficacy related to STEM, attitude towards STEM, and STEM career interests. Interest in STEM pertains to the emotions and sentiments pertaining to the process of learning STEM subjects (Zhang & Tang, 2017 ). Self-efficacy, on the other hand, revolves around students' convictions regarding their competencies to excel in STEM endeavors, as well as their perseverance in persisting within STEM domains (Bandura, 1995 ). Attitude towards STEM signifies individuals' perceptions regarding the inherent value and significance of STEM disciplines (Dowey, 2013 ). Lastly, STEM career interest denotes students' proclivity towards prospective careers within STEM domains (Maltese & Tai, 2011 ).

Publication Type

Within the realm of meta-analysis, publication type stands out as a prominent moderator variable. Our study notably incorporates both peer-reviewed journal articles and dissertations. It is well-established that studies reporting statistically significant findings are more likely to be published compared to those reporting non-significant results, a phenomenon commonly termed publication bias (Rosenthal, 1979 ). The presence of heterogeneous findings across studies may arise from differences in publication types rather than alternative moderator variables. Consequently, we systematically examined publication type as a potential moderator variable within this meta-analytic inquiry, classifying it into two categories: "journal article" and "thesis_dissertation," with the latter encompassing both theses and dissertations.

Publication Year

Given the dynamic nature of education policies spanning from 1992 to 2022, exemplified by the publication of the Next Generation Science Standards (NGSS), the formats and instructional methodologies employed in science education have undergone significant transformations over time (NRC, 2011 ). Consequently, these alterations can change long-term attitudes towards and interest in STEM. Considering the possible changes in STEM education over the past 30 years, we included publication year as a potential moderator.

Aim of the Meta-analysis

Informal science learning plays an important role in science learning and contains various formats. Nonetheless, the effect of informal science learning in general, and the possible difference among different informal science learning settings in particular, have not been fully examined. The purpose of this meta-analysis is to explore whether informal science education is effective in increasing students’ learning interests and attitudes toward STEM. The following research questions were explored:

What is the overall effect size of informal science learning experiences on students' attitudes towards and interest in STEM?

How do various moderating factors, including the type of informal learning experience (such as museum visits, out-of-school programs, after-school activities), student grade level, academic subjects (science, technology, engineering, mathematics, or the broader STEM domain), type of publication (dissertation or peer-reviewed journal article), and publication year, impact student attitudes and interests in STEM?

Literature Search for Primary Studies

In this study, we sought primary studies from Proquest, EBSCO, Web of Science, and ScienceDirect. We concentrated on empirical studies exploring the effect between informal science education and K-12 students' interest in science. This encompassed peer-reviewed journal articles as well as unpublished dissertations or conference papers. Keywords used were as follows: (“Interest” OR “attitude”) AND (“out-of-school” OR “informal” OR “after school”) AND (“science education”). In addition, we reviewed articles that were cited in a previous meta-analysis (Young et al., 2017 ) and examined each article for potential addition to our analysis using Google Scholar (scholar.google.com). Articles were filtered by timeframe (1992–2022), language (English), and location (United States). In this study, we were interested in the effects of informal science programs within the United States. We conducted our search using the keywords individually or in various combinations and did not restrict the publication status, hence we can have both gray literature and journal articles.

Inclusion and Exclusion Criteria

For inclusion in this meta-analysis, we listed the criteria that studies must satisfy below:

A study necessitates the exploration of an informal science learning setting wherein explicit documentation of informal activities is provided. The study should align with the established criteria for informal learning activities as mentioned previously. The criteria were based on those used by Hofstein and Rosenfeld ( 1996 ).

A study was a quasi-experimental study, with an experimental group or groups (e.g., groups of students involved in an informal science learning program or activities) and a control or comparison group (business as usual or not involved in an informal science program or activities), respectively. Studies that lacked a control/comparison group, wherein participants did not engage in informal science learning programs, were systematically excluded from the analysis. For instance, Knapp and Barrie ( 2001 ) compared two field trips to a science center, and Wilson et al. ( 2012 ) tested the effectiveness of two versions of a film for part of the science center planetarium demographic to compare children's learning and attitude changes in response to films. These studies lacked a control group, and were not included in this study.

A pretest–posttest design was included in the study, typically involving collecting data from participants at two distinct time points: before the implementation of an informal science learning intervention or treatment (pretest) and after the intervention or treatment had been administered (posttest). The studies involve the same group of participants being measured on the same variables at both the pretest and posttest stages. Studies that did not collect the pretest data were excluded from this study.

A study had to include students’ interests, attitudes, and/or self-efficacy as outcomes. In our meta-analysis, we define attitude towards STEM in four ways (as mentioned above): interest, self-efficacy, attitude toward STEM, and career interests, as mentioned above. Studies that investigated the impacts of informal science learning on achievement were excluded from our study.

To be eligible for inclusion, a study had to provide sufficiently detailed quantitative data that facilitated the calculation and extraction of the relevant relationship as an effect size. This criterion ensured that the selected studies provide the necessary information required for effect size estimation, thereby enabling a rigorous analysis of the relationship under investigation. If a study only contained a qualitative interview for analysis, we did not include this study in our analysis because we could not calculate effect sizes.

The present study exclusively focused on samples comprising students from grades K-12, excluding college students or other participant groups.

A study must be published or available in English.

Study Selection

As illustrated in a PRISMA flowchart (Fig.  1 ), our initial search yielded 1042 potentially relevant studies as previously used by Maltese and Tai ( 2011 ). We included studies from published journal articles, and gray literature which included conference papers, and theses/dissertations. On the first round of screening, we removed 291 duplicate articles, hence 751 studies remained. Secondary round screening was to review the title and abstract of each article and determine if the study focused on the impact of informal science education on students' interests and attitudes. At this stage, 192 studies were reminded for further full-text examination.

Flowchart of the inclusion and exclusion in the meta-analysis

During the third round of screening, a comprehensive review of all articles was conducted independently by both the first and second authors to assess the eligibility of the 192 studies against the pre-established inclusion and exclusion criteria. Although an initial discrepancy emerged between the two reviewers, resolution was achieved through deliberation among the entire research team to determine the final inclusion status of each study.

As is common in many meta-analytic reviews, among the included primary studies, some, or even many, contain multiple effect sizes due to the following reasons: a) testing students’ attitudes and interests in different subjects (e.g. science, mathematics, technology, or engineering, etc.); b) measuring students’ attitudes in various dimensions (e.g. attitudes towards STEM content learning, attitudes towards STEM career). Subsequently, a total of 19 studies, comprising 68 effect sizes, were deemed to meet the predetermined inclusion criteria and were consequently incorporated into the meta-analysis. The cumulative sample size across these 19 studies amounted to 6160 participants.

Coding Process

Two authors of the article finished coding. Initially, the two coders collaborated in coding 25% of the primary studies together, which was the “trial coding” phase. The discordances encountered between the two coders were effectively addressed through a process of deliberation undertaken within the research team, whereby the identified issues observed during the coding procedure were thoroughly examined and subject to comprehensive discussion. Following the initial trial coding phase, the research team proceeded to establish a definitive coding table. Subsequently, two coders independently conducted coding on the remaining primary studies. Upon completion of the final coding phase, no significant disparities were observed, although minor variances were addressed through collaborative discussion.

Coding of Study Characteristics as Variables

Information from each study was coded about informal science learning program characteristics, student sample, publication year, and publication type as detailed below.

Studies were conducted on informal science learning in different formats. We divided the type of programs into eight categories: after-school, summer camp, outreach program, weekend school camp, field trip or a virtual museum, mobile lab, and mixed program. While studies contained more than one category of informal activities, we coded them as mixed programs.

Four categories of grades were coded: elementary (grade K-5), middle (grade 6–8), high (grade 9–12), and mixed grade in which studies included participants across school levels.

Research studies focusing on the examination of students' learning interests or attitudes in individual subjects, including science, mathematics, engineering and technology, and in STEM as a whole, were specifically considered in this analysis. Only those studies that encompassed all four domains of science, mathematics, engineering, and technology were coded and categorized as STEM. In this study, primary studies examined engineering and technology together, separate from science and mathematics. Therefore, we included an engineering and technology category.

Studies were categorized into four outcomes: self-efficacy, attitude, interest (i.e., in subject content), and career interest (i.e., in STEM careers).

Another frequently utilized moderator variable in meta-analysis is Publication type (Cai et al., 2017 ). In this study, we coded studies into two categories: journal articles or thesis/dissertations.

The last moderator in this study is the publication year that recorded the year of studies that were published.

Procedures of Meta-analytic

Calculation of effect size.

As described previously, the studies included in this meta-analytic review were based on the inclusion criteria mentioned above, and these studies had pre- and post-measures of interest/attitude of STEM subjects, thus providing interest/attitude change scores between pre- and post-measures from both experimental and control groups. Such a design was described as pretest–posttest-control (PPC) in the literature (Morris, 2008 ), which could be either experimental design with randomized assignment, or quasi-experimental design without randomized assignment (Morris, 2008 ). Becker ( 1988 ) presented an effect size measure for such PPC design, which was essentially the difference of standardized mean change scores between the two groups (treatment vs. control). Built upon Becker’s work on the effect size measure for the PPC design, Morris ( 2008 ) provided an in-depth discussion and empirical assessment about several alternative forms of effect size measures in such a pretest–posttest-control (PPC) design. The empirical findings in Morris ( 2008 ) showed that one effect size measure (labeled d ppc2 in Morris, 2008 , p. 369), has better and more robust performance than other alternatives. Guided by the empirical findings and conclusions of Morris ( 2008 ), in the current meta-analytical review, we used d ppc2 as the effect size measure between the students in an informal science learning program vs. those with no informal science learning. d ppc2 is based on the difference of standardized change scores (i.e., posttest score – pretest score) of the two groups, and it is conceptually equivalent to Hedges’ g (Hedges & Olkin, 1985 ). The technical details of d ppc2 and other alternative forms are available in Morris ( 2008 ). For the sake of simplicity for our readers, in the following, we will use the well-known g , instead of d ppc2 , in our presentation and discussion.

If a study did not directly provide the components needed for calculating effect size as described above, but provided sufficient other statistics (e.g., t-statistic, F-statistic, odds ratio, etc.) that allowed us to obtain the effect size measure based on available conversion formula in the literature (e.g., Hedges & Olkin, 1985 ), the effect size was obtained. In this context, a positive value of g was interpreted as an indication of improved performance by the treatment group of students who participated in an informal science learning program over the control group of students.

Random Effect Meta-Analysis Model

Some primary studies included in this meta-analysis presented multiple effect sizes within one study, thus, we used a random-effects model for analyzing the effect sizes. A random-effects model assumes that the effects of the variables are random and can vary across studies, depending on different study conditions. It is suitable when there is unobserved heterogeneity between the units or individuals. The weighting scheme in the random-effects model incorporates within-study variance alongside a constant value ( T 2 ), representing between-study variance, reducing the relative differences among the weights. Consequently, a random-effects model promotes a more balanced distribution of relative weights across studies compared to a fixed-effect model (Borenstein et al., 2010 ). This study reported and interpreted the weighted average effect sizes, confidence intervals (lower and upper limits), and z-test results. We used the R statistical platform (R Core Team, 2019 ) with Viechtbauer’s ( 2010 ) metafor package for analysis. The random effect coefficients were estimated using the maximum likelihood estimation method. Q statistics addressed the homogeneity of effect sizes.

Moderator Analysis

When Q statistics showed statistical heterogeneity across the studies, moderator analyses were conducted to test the potential study features that could have contributed to the inconsistency among the study's effect sizes. This study extracted four categorical variables (i.e., type of program, grade, disciplines, and type of publication) and one continuous variable (i.e., publication year). Significant moderation effects were assessed through the utilization of two distinct approaches. One is the omnibus test for categorical moderators which was employed to determine the presence of statistically significant moderation effects when dealing with categorical moderators, while the other is the slope analysis for the continuous moderator, which was utilized to identify significant moderation effects (Cai et al., 2022 ).

Publication Bias

Publication bias poses a significant challenge to the validity of meta-analytic findings. Research demonstrating large effect sizes or statistically significant findings may have a higher likelihood of being published and consequently included in a meta-analysis compared to studies with small effect sizes or statistically nonsignificant results (Mao et al., 2021 ; Rosenthal, 1979 ). As publication bias has the capacity to distort estimations of the true effect being investigated, it poses a pervasive challenge when conducting a meta-analysis (Thornton & Lee, 2000 ). To address the publication bias, this study used the funnel plot and Egger’s regression to assess potential publication bias. As Rothstein et al. ( 2005 ) discussed, the absence of bias can be inferred when the funnel plot demonstrates a symmetrical distribution. Egger et al. ( 1997 ) proposed a linear regression method for evaluating publication bias, which a p-value greater than 0.05 in this test suggests the absence of publication bias.

Viechtbauer and Cheung ( 2010 ) demonstrated that meta-analyses need to include influential case diagnostics to identify outliers or extreme effect sizes and separate them from the rest of the data. We used Cook's distances (Fig.  2 ), DFBETAS (Fig.  3 ), and standardized deleted residuals (Fig.  4 ) to detect the potential outliers. As Figs.  2 , 3 , and 4 showed below, two effect sizes (i.e., the 30th and 58th effect sizes) were identified as influential outliers. After excluding the influential outlier, the revised pooled effect size was determined to be 0.21 (95% CI [0.12, 0.30], p  < 0.001), demonstrating close similarity to the previous pooled effect size of 0.21 (95% CI [0.10, 0.32], p  < 0.001). Sensitivity analysis showed that the overall effect size remained virtually unchanged even after removing influential outliers. Therefore, we can conclude that including influential outliers did not change the main results of our meta-analysis.

Cook’s distances of the effect sizes

EFBETA values of effect sizes

Standentized deleted residuals of effect sizes

We include independent studies in the analysis that spanned from 1997 to 2022. A total of 19 studies were incorporated, yielding a comprehensive set of 68 effect sizes. Data sheet can be provided by contacting the corresponding author. These effect sizes exhibited a diverse range, extending from -1.15 to 1.59, with a median value of 0.17. The magnitude of the effect size and the accuracy of its estimation differ. The majority of effect sizes were positive (81%), while 10 effect sizes were negative and three were close to zero.

The Q test was statistically significant (As shown in Table  1 ), indicating significant heterogeneity across the effect sizes ( Q ( df  = 67) = 1663.24, p  < 0.001), with a total heterogeneity ( I 2 ) of 91.63%. This means that the variation in effect sizes across the studies cannot be explained solely by sampling error (Borenstein et al., 2017 ). We can reject the null hypothesis that the true effect sizes are homogeneous. The true effectiveness of informal science programs appears to differ across the studies.

As shown in Table  1 , the estimated effect size of 0.21 with a 95% confidence interval ranging from 0.10 to 0.32 suggests a statistically significant effect. Kraft ( 2020 ) put forth a comprehensive framework aimed at elucidating effect sizes within the context of educational interventions targeting the academic achievement of pre-K-12 students. Specifically, the author delineated the categorization of effect sizes based on their magnitude and argued that effect sizes below 0.05 donate a small effect size, while effect sizes ranging from 0.05 to 0.2 represent a medium effect size, and effect sizes surpassing the threshold of 0.2 were identified as indicative of a large effect size. Therefore, interpreted within this framework, 0.21 in our study was characterized as a statistically significant medium to large effect size in terms of the overall effect of informal science learning programs on students' interests or attitudes in STEM.

“Forest plot” is a graphic tool that presents the effect sizes of each study and the overall effect size derived from random-effect modeling in one graph. Figure  5 presents all 68 effect sizes. The position of the square dot along the horizontal axis represents the estimated effect size for a given study and the size of the dot indicates the weight or precision of the study’s estimate. Confidence intervals (CI) were presented by the horizontal bars extending from the square dot with two ends where the bar's length shows the confidence interval's width. In addition, the overall effect size is located below all individual effect sizes in a diamond shape, and its width represents the 95% confidence interval around the summary estimate. The vertical dot line is the “null” effect, meaning that there is zero effect by informal science learning activities on students' learning interests and attitudes. Since the “null” effect line did not cross the diamond shape, we concluded that the overall “effect” was statistically significant.

Forest plot

Specifically, in Fig.  5 , the predominant trend manifests as positive outcomes, with a notable presence of 55 positive effect sizes positioned to the right of the "null" effect line. Conversely, 10 effect sizes exhibited negative trends, while three were proximate to zero. It is noteworthy that certain effect sizes demonstrated exceptionally broad confidence intervals, as evidenced by Garvin ( 2015 ) and Parker and Gerber ( 2000 ), while others exhibited narrower confidence intervals, exemplified by Crawford and Huscroft-D’Angelo ( 2015 ) and Roberson ( 2010 ). The sample size associated with an effect size affected the confidence interval width, with smaller sample sizes associated with wider confidence intervals. Consequently, in order to mitigate against the potential influence stemming from variations in confidence intervals on the overall outcomes of the study, it was necessary to assign appropriate weights to the effect size within a meta-analysis. The accumulated overall effect size is based on weighted individual effect sizes across the studies, with effect sizes from larger samples weighted more than the effect sizes from smaller samples.

Moderators Analysis

We conducted random effects analysis to explore how moderators influence the effects of informal science learning programs. Table 2 summarizes the results, and the following discussion will address each moderator separately below.

The effect of informal science learning was found to be moderated by the type of program. As the omnibus test show, different types of the program explained the effect-size heterogeneity was statistically significant ( Q moderators ( df  = 7) = 31.80, p  < 0.0001), indicating that the effect sizes from the studies based on different programs could differ statistically. Under this moderator of Type of Program , Outreach Program stood out as showing the largest average effect size of 0.72, which is also statistically significant (Mean g  = 0.72; 95% CI [0.43, 1.02]; p  = 0.0043). Studies under other types of programs showed smaller effect sizes in general, although all positive.

Similar to the results for program type above, the grade was also a statistically significant moderator ( Q moderators ( df  = 4) = 30.48, p  < 0.0001). The studies involving middle school students (Mean g  = 0.43; 95% CI [0.12, 0.59]; p  = 0.0033) and high school students (Mean g  = 0.42; 95% CI [0.15, 0.69]; p  = 0.0024) showed larger effect sizes than those involving elementary school students (Mean g  =   −  0.24; 95% CI [ −  0.77, 0.29]; p  = 0.3657) and mixed grade students (Mean g  = 0.08; 95% CI [ −  0.06, 0.21]; p  = 0.2490).

We found a statistically significant moderating effect of subjects ( Q moderators ( df  = 4) = 19.53, p  = 0.0006). This suggests that the studies that examined interests or attitudes toward science (Mean g  = 0.14; 95% CI [0.01, 0.26]; p  = 0.0318), mathematics (Mean g  = 0.44; 95% CI [0.04, 0.84]; p  = 0.03), technology and engineering (Mean g  = 0.48; 95% CI [0.06, 0.90]; p  = 0.03), or full STEM (Mean g  = 0.36; 95% CI [0.05, 0.68]; p  = 0.02), although all having statistically significant effect sizes positively related to informal science activities, had statistical variations in their respective effect sizes related to the subject areas, with those studies focusing on interests or attitudes toward science showing smaller effect sizes.

Similarly, we found a significant moderating effect of outcome types ( Q moderators ( df  = 4) = 22.08, p  = 0.0002). More specifically, the studies that examined self-efficacy and attitude showed larger effect sizes than those that examined interest or career interest. However, in regard to self-efficacy, there was only one article with two effect sizes that addressed this outcome. For this reason, the reliability of the finding is questionable because of the small sample size, and caution is warranted for the interpretation of this sub-group finding. In general, the finding suggests that the informal science learning activities showed a significant positive effect on students’ attitudes toward STEM areas.

The dataset used in this meta-analysis consisted of 38 effect sizes from journal articles, and 30 effect sizes from dissertations or theses. The results revealed a statistically significant effect of the publication type moderator ( Q moderators ( df  = 2) = 22.50, p  < 0.0001), indicating publication type explains heterogeneity in the observed effect sizes. Specifically, the effect sizes derived from journal articles exhibited a larger magnitude (Mean g  = 0.34; 95% CI [0.205, 0.48]; p  < 0.0001) compared to those obtained from dissertations or theses (Mean g  = 0.05; 95% CI [ −  0.10, 0.21]; p  = 0.4913).

No statistically significant difference was revealed in terms of the effect size heterogeneity of publication year ( Q moderators ( df  = 1) = 2.80, p  = 0.09). The slope of publication year was statistically non-significant ( \(\beta\) =  −  0.01; 95% CI [ −  0.03, 0.00]; p  = 0.0941), suggesting that the variable of publication year did not have a significant impact on the effect sizes.

To assess the potential publication bias, we used both the funnel plot and Egger’s regression method (Egger et al., 1997 ) in this study. As Fig.  6 shows, the funnel plot had an approximately symmetrical distribution, which indicates a general absence of publication bias. In addition, Egger’s regression test (p = 0.28) is not statistically significant, indicating a lack of evidence for publication bias.

Funnel plot for the effect sizes

This meta-analysis served as a comprehensive summary of quantitative studies carried out within the context of informal science learning in the United States. Building upon previous meta-analytic research, we aimed to conduct a systematic meta-analysis to examine the impact of informal science learning by focusing on changes in interests or attitudes toward STEM areas before and after students participated in informal science activities. More specifically, we compared the interest or attitude change scores across treatment groups (i.e., with informal science learning activities) and control groups (i.e., without informal science learning activities). By addressing these aspects, our meta-analysis provides valuable insights into the role of informal science learning and its effects on students' interests and attitudes toward STEM areas.

Specifically, the random effect modeling revealed the statistically significant overall mean effect size (Mean g  = 0.21), indicating that informal science learning opportunities had a positive effect on students' STEM interests, and this positive effect, as discussed in Kraft ( 2020 ), could be characterized as a medium to large effect. The result was consistent with previous funding by Young et al. ( 2017 ) and consistent with empirical findings on informal science learning toward STEM interests (e.g., Crawford & Huscroft-D’Angelo, 2015 ; Havasy, 1997 ; Yang & Chittoori, 2022 ).

The statistically significant outcomes of the heterogeneity test conducted among the effect sizes revealed notable statistical diversity in the impacts of informal science learning opportunities on students' attitudes and interests in STEM across the encompassed studies. Consequently, an examination of potential moderators was initiated to address the second research question. The significant moderators include the type of informal science program, grade level, publication type, and sample size. However, due to our small sample size in this meta-analysis, we highlight that the overall effect of informal STEM programming was large (0.21) and had a significant impact on students' interests and attitudes toward STEM, rather than focusing on the differences between moderators. The heterogeneity could be due to the program’s focus. Laurer et al. ( 2006 ) concluded that informal program focus was a significant moderator of mathematics achievement among at-risk youth. Since STEM interest has been associated with achievement (Maltese & Tai, 2011 ), it is important to determine how program focus affects student interest in STEM (Young et al., 2017 ). Shaby et al. ( 2024 ) pointed out the importance of pedagogical design of Laboratory Group Activity in a Science Museum for students’ interaction and learning outcomes. The various pedagogical designs of the same type of program might result in different effects. Future research in program focus might uncover why there is a difference between different types of informal science experiences.

Grade level emerged as a significant moderator on student interest in and attitudes toward STEM. However, the available data are insufficient to draw a statistical conclusion regarding the disparity between grade levels. Specifically, only a very limited number of studies conducted at the elementary school level were available for this meta-analytic review, with only 2 out of 19 studies involving elementary students. Nevertheless, there is accumulating evidence suggesting that early engagement in STEM learning, particularly during elementary school, may yield long-term benefits for sustained interest in STEM (Curran & Kitchin, 2019 ; Morgan et al., 2016 ). The scant representation of elementary-level studies in the research literature underscores a notable gap in the research landscape, emphasizing the necessity for further investigation to elucidate the impact of informal STEM experiences on elementary school students.

Although there was a lack of evidence for publication bias for studies included in this meta-analysis, the type of publications (i.e., journal articles vs. thesis/dissertations) showed different magnitude of effect sizes. Publication bias is influenced by various factors such as language bias, time lag bias, and selective reporting. The publication bias test as implemented in this study may not be sufficiently sensitive to capture all aspects of bias. Therefore, it is possible for the publication bias test to indicate no bias while the moderator test reveals an impact of publication type.

Conclusions and Future Directions

Students acquire STEM-related knowledge through formal and informal education experiences (Goldstein, 2015 ). Formal STEM experiences are part of the national curriculum provided to the students in their schools. Researchers can gauge the effectiveness of formal STEM programs by tracking students’ progress in classrooms and with national testing data. However, it is more difficult to assess the impact of informal science learning experiences, since they happen outside of school. Based on this meta-analysis, informal STEM experiences have a positive effect on students' interest in and attitudes toward STEM, and should be incorporated into students’ educational experiences.

Despite our exhaustive efforts in locating relevant studies conducted in a span of thirty years (1992–2022), we were only able to find a very limited number of studies that quantitatively assessed the effects of informal science learning on students’ interest and attitude for science and STEM subjects. The majority of studies in this area were not appropriate for our quantitative synthesis because of different reasons: lack of sufficient information for effect size calculations for meta-analysis; studies of qualitative approaches that did not have quantifiable data on STEM interest changes; quantitative studies with only one-time data collection, but could not be used to assess the effect of informal science learning on students’ interests/attitude for science/STEM. Future research may consider these and other similar issues for designing methodologically rigorous quantitative studies that examine how informal science learning experiences contribute to students’ interest/attitude for STEM.

Limitations

Several limitations were identified during the evaluation and analysis process of this meta-analysis. During the data collection process, several articles were not available for the researchers to review, due to the internal restrictions of our database access and limitations from our university library. Another limitation is the conflicting results of our publication type analysis and publication bias assessment. Publication type was used as a moderator in our analysis, and we concluded that journal articles had a statistically significant effect size when compared to dissertation articles. This contradicted our publication bias assessment results, which suggested a lack of sufficient statistical evidence for publication bias, and this could be a limitation of this analysis. But the overall very limited number of studies ( N  = 19) available for this meta-analysis made it difficult to explore this issue in a more meaningful way. Because of these considerations, caution is warranted in the interpretation of the findings related to this issue.

Small sample size (in terms of both the number of studies and number of effect sizes) is a general limitation in this meta-analysis, especially for some moderator analysis. Obviously, this meta-analysis, like other synthesis studies, is at the mercy of what is available in the research literature. Because of the small sample size issue, we need to exercise caution when interpreting the findings, especially the findings in the moderator analysis as shown in Table  2 . As discussed above, future research may further examine some of the issues revealed in the study.

We did not include some other possible variables, such as dosage, duration, gender, race, school type, or sampling method. Although these could be potential moderators, the lack of relevant information in the primary studies made it impossible for us to consider these in the meta-analysis. Furthermore, the interaction effects of moderator variables on the effect of learning interest and attitudes were not addressed in this study, because the limited sample size made it statistically impractical to conduct interaction analysis.

In addition, we were not able to explore what components of different types of programs actually had an effect on students’ interests and attitudes. Previous studies revealed that hands-on activities and challenge assessments could enhance students' interest and motivation (Hamari et al., 2016 ; Parsons & Taylor, 2011 ; Poudel et al., 2005 ). It is possible that the outreach program included in this meta-analytic study contained more challenging activities, which might explain that such studies' larger effect size than some other studies. But lack of relevant information in the primary studies included in our meta-analysis made it impossible for us to explore such potentially relevant issues.

Data Availability

The datasets used in this meta-analysis are derived from publicly available sources (Proquest, EBSCO, Web of Science, and ScienceDirect) and previously published studies. The references and citations for all included studies are provided in the reference list with the asterisk accompanying this publication.

*: A primary study with effect size(s) included in this meta-analysis.

Alliance, A. (2011). STEM learning in afterschool: An analysis of impact and outcomes . Afterschool Alliance .

Bandura, A. (1995). Exercise of personal and collective efficacy in changing societies (A. Bandura, Ed.). Cambridge University.

Becker, B. J. (1988). Synthesizing standardized mean-change measures. British Journal of Mathematical and Statistical Psychology, 41 , 257–278.

Article   Google Scholar  

Beier, M., & Rittmayer, A. (2008). Literature overview: Motivational factors in STEM: Interest and self-concept . Assessing Women and Men in Engineering.

Berliner, D. C. (2009). Poverty and potential: Out-of-school-factors and school success . Education and the Public Interest Center, University of Colorado/Education Policy Research Unit, Arizona State University. Retrieved October 26, 2015, from http://epicpolicy.org/publication/poverty-and-potential

Bevan, B., Dillon, J., Hein, G. E., Macdonald, M., Michalchik, V., Miller, D., Root, D., Rudder-Kilkenny, L., Xanthoudaki, M., & Yoon, S. (2010). Making science matter: Collaborations between informal science education organizations and schools . Center for Advancement of Informal Science Education.

Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010). A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods, 1 (2), 97–111. https://doi.org/10.1002/jrsm.12

Borenstein, M., Higgins, J. P., Hedges, L. V., & Rothstein, H. R. (2017). Basics of meta-analysis: I^2 is not an absolute measure of heterogeneity. Research Synthesis Methods, 8 (1), 5–18.

Business-Higher Education Forum (2011). Creating the workforce of the future: The STEM interest and proficiency challenge . BHEF Research Brief. Business-Higher Education Forum.

Cai, Z., Fan, X., & Du, J. (2017). Gender and attitudes toward technology use: A meta-analysis. Computers & Education, 105 , 1–13.

Cai, Z., Mao, P., Wang, D., He, J., Chen, X., & Fan, X. (2022). Effects of scaffolding in digital game-based learning on student’s achievement: A three-level meta-analysis. Educational Psychology Review, 34 (2), 537–574.

Crane, V. (1994). Informal science learning: What the research says about television, science museums, and community-based projects . Research Communications, Limited.

*Crawford, L., & Huscroft-D’Angelo, J. (2015). Mission to space: Evaluating one type of informal science education. The Electronic Journal for Research in Science & Mathematics Education , 19 (1). https://ejrsme.icrsme.com/article/view/13753

Curran, F. C., & Kitchin, J. (2019). Early elementary science instruction: Does more time on science or science topics/skills predict science achievement in the early grades? AERA Open, 5 , 1–18. https://doi.org/10.1177/2332858419861081

DeCoito, I. (2014). Focusing on science, technology, engineering, and mathematics (STEM) in the 21st century. Ontario Professional Surveyor, 57 (1), 34–36.

Google Scholar  

Dowey, A. L. (2013). Attitudes, Interests, and Perceived Self-Efficacy toward Science of Middle School Minority Female Students: Considerations for Their Low Achievement and Participation in STEM Disciplines (Unpublished doctoral dissertation). University of California.

Dryfoos, J. G. (1999). The role of the school in children’s out-of-school time. The Future of Children, 9 (2), 117–134.

Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315 (7109), 629–634. https://doi.org/10.1136/bmj.315.7109.629

Eshach, H. (2007). Bridging in-school and out-of-school learning: Formal, non-formal, and informal education. Journal of Science Education and Technology, 16 , 171–190.

Falk, J. H., & Dierking, L. D. (2010). The 95 percent solution. American Scientist, 98 (6), 486–493.

Frantz, T. D., Miranda, M., & Siller, T. (2011). Knowing what engineering and technology teachers need to know: An analysis of pre-service teachers’ engineering design problems. International Journal of Technology Design Education, 21 , 307–320.

*Garvin, B. A. (2015). An investigation of a culturally responsive approach to science education in a summer program for marginalized youth (Doctoral dissertation). University of South Carolina. Retrieved from ProQuest Dissertation Theses Global.

Gauld, C. F., & Hukins, A. A. (1980). Scientific attitudes: A review. Studies in Science Education, 7 (1), 129–161.

George, R. (2000). Measuring change in students’ attitudes toward science over time: An application of latent variable growth modeling. Journal of Science Education and Technology, 9 (3), 213–225.

Goldstein, D. (2015). The teacher wars: A history of America’s most embattled profession . Anchor.

Hamari, J., Shernoff, D. J., Rowe, E., Coller, B., Asbell-Clarke, J., & Edwards, T. (2016). Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning. Computers in Human Behavior, 54 , 170–179.

*Havasy, R. A. D. P. (1997). The effect of informal science experiences on science achievement and attitude of high school biology students (Ed.D.). Teachers College, Columbia University. Retrieved from ProQuest Dissertation Theses Global.

Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis . Academic press.

Hofstein, A., & Rosenfeld, S. (1996). Bridging the gap between formal and informal science learning. Studies in Science Education, 28 , 87–112.

Jensen, F., & Sjaastad, J. (2013). A Norwegian out-of-school mathematics project’s influence on secondary students’ STEM motivation. International Journal of Science and Mathematics Education, 11 (6), 1437–1461. https://doi.org/10.1007/s10763-013-9401-4

Jiang, H., Chugh, R., Turnbull, D., Wang, X., & Chen, S. (2024). Exploring the effects of technology-related informal mathematics learning activities: A structural equation modeling analysis. International Journal of Science and Mathematics Education , 1–21. https://doi.org/10.1007/s10763-024-10456-4

Jones, A. L., & Stapleton, M. K. (2017). 1.2 million kids and counting—Mobile science laboratories drive student interest in STEM. PLoS biology, 15 (5), e2001692.

Keiler, L. S. (2011). An effective urban summer school: Students’ perspectives on their success. The Urban Review, 43 (3), 358–378.

Knapp, D., & Barrie, E. (2001). Content evaluation of an environmental science field trip. Journal of Science Education and Technology, 10 (4), 351–357. https://doi.org/10.1023/A:1012247203157

Kraft, M. A. (2020). Interpreting effect sizes of education interventions. Educational Researcher, 49 (4), 241–253. https://doi.org/10.3102/0013189X20912798

Krishnamurthi, A., & Rennie, L. J. (2013). Informal science learning and education: Definition and goals. Afterschool Alliance.

Laurer, P. A., Akiba, M., Wilkerson, S. B., Apthorp, H. S., Snow, D., & Martin-Glen, M. L. (2006). Out-of-school-time programs: A meta-analysis of effects for at-risk students. Review of Educational Research, 76 (2), 275–313.

Maiorca, C., Roberts, T., Jackson, C., Bush, S., Delaney, A., Mohr-Schroeder, M. J., & Soledad, S. Y. (2021). Informal learning environments and impact on interest in STEM careers. International Journal of Science and Mathematics Education, 19 , 45–64.

Maltese, A. V., & Tai, R. H. (2011). Pipeline persistence: Examining the association of educational experiences with earned degrees in STEM among U.S. students. Science Education, 95 (5), 877–907.

Mao, P., Cai, Z., He, J., Chen, X., & Fan, X. (2021). The relationship between attitude toward science and academic achievement in science: A three-level meta-analysis.  Frontiers in psychology ,  12 , 784068.

Matthews, P. H., & Mellom, P. J. (2012). Shaping aspirations, awareness, academics, and action outcomes of summer enrichment programs for English-learning secondary students. Journal of Advanced Academics, 23 (2), 105–124.

*Migliarese, N. L. (2011). "That's my kind of animal!" Designing and assessing an outdoor science education program with children's megafaunaphilia in mind. (Doctoral dissertation). University of California, Berkeley. Retrieved from ProQuest Dissertation Theses Global.

Morgan, P. L., Farkas, G., Hillemeier, M. M., & Maczuga, S. (2016). Science achievement gaps begin very early, persist, and are largely explained by modifiable factors. Educational Researcher, 45 (1), 18–35.

Morrell, P. D., & Lederman, N. G. (1998). Student’s attitudes toward school and classroom science: Are they independent phenomena? School Science and Mathematics, 98 (2), 76–83.

Morris, S. B. (2008). Estimating effect sizes from pretest-posttest-control group designs. Organizational Research Methods, 11 (2), 364–386.

Murphy, C., & Beggs, J. (2003). Children’s perceptions of school science. School Science Review, 84 , 109–116.

National Research Council. (2007). Taking science to school: Learning and teaching science in grades K-8. National Academies Press.

National Research Council. (2009). Learning science in informal environments: People, places, and pursuits. National Academies Press.

National Research Council, Division of Behavioral, Board on Testing, Assessment, Board on Science Education, & Committee on Highly Successful Schools or Programs for K-12 STEM Education. (2011). Successful K-12 STEM education: Identifying effective approaches in science, technology, engineering, and mathematics . National Academies Press.

National Academies of Sciences, Engineering, and Medicine. (2016). Parenting matters: Supporting parents of children ages 0–8. The National Academies Press.

Osborne, J., Simon, S., & Collins, S. (2003). Attitudes towards science: A review of the literature and its implications. International Journal of Science Education, 25 (9), 1049–1079.

*Parker, V., & Gerber, B. (2000). Effects of a science intervention program on middle‐grade student achievement and attitudes.  School Science and Mathematics, 100 (5), 236-242.

Parsons, J., & Taylor, L. (2011). Improving student engagement. Current Issues in Education, 14 (1).

Phillips, M., Finkelstein, D., & Wever-Frerichs, S. (2007). School site to museum floor: How informal science institutions work with schools. International Journal of Science Education, 29 (12), 1489–1507.

Potvin, P., & Hasni, A. (2014). Interest, motivation and attitude towards science and technology at K-12 levels: A systematic review of 12 years of educational research. Studies in Science Education, 50 (1), 85–129.

Poudel, D. D., Vincent, L. M., Anzalone, C., Huner, J., Wollard, D., Clement, T., DeRamus, A., & Blakewood, G. (2005). Hands-on activities and challenge tests in agricultural and environmental education. The Journal of Environmental Education, 36 (4), 10–22.

R Core Team. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved March 10, 2020 from  https://www.R-project.org/

*Roberson, S. V. (2010). Science skills on wheels: The exploration of a mobile science lab's influence on teacher and student attitudes and beliefs about science. (Ed.D dissertation). University of Pennsylvania. Retrieved from ProQuest Dissertation Theses Global.

Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86 , 638–641. https://doi.org/10.1037/0033-2909.86.3.638

Rothstein, H., Sutton, A. J., & Borenstein, M. (Eds.). (2005). Publication bias in meta-analysis: Prevention, assessment and adjustments. Wiley.

Sadler, P. M., Sonnert, G., Hazari, Z., & Tai, R. (2012). Stability and volatility of STEM career interest in high school: A gender study. Science Education, 96 (3), 411–427

Sacco, K., Falk, J. H., & Bell, J. (2014). Informal science education: Lifelong, life-wide, life-deep. PLoS Biology, 12 (11), e1001986.

Shaby, N., Assaraf, O. B. Z., & Koch, N. P. (2024). Students’ interactions during laboratory group activity in a science museum. International Journal of Science and Mathematics Education, 22 (4), 703–720.

*Shields, N. C. (2010). Elementary students' knowledge and interests related to active learning in a summer camp at a zoo (Doctoral dissertation). Purdue University. Retrieved from ProQuest Dissertation Theses Global.

Silver, A., & Rushton, B. (2008). Primary-school children’s attitudes towards science, engineering and technology and their images of scientists and engineers. Education 3–13, 36 (1), 51–67.

Simpkins, S. D., Davis-Kean, P. E., & Eccles, J. S. (2006). Math and science motivation: A longitudinal examination of the links between choices and beliefs. Developmental Psychology, 42 (1), 70–83.

So, W. W. M., Zhan, Y., Chow, S. C. F., & Leung, C. F. (2018). Analysis of STEM activities in primary students’ science projects in an informal learning environment. International Journal of Science and Mathematics Education, 16 , 1003–1023.

Tai, R. H., Liu, C. Q., Maltese, A. V., & Fan, X. (2006). Career choice: Planning early for careers in science. Science, 312 (5777), 1143–1144.

Thornton, A., & Lee, P. (2000). Publication bias in meta-analysis: Its causes and consequences. Journal of Clinical Epidemiology, 53 (2), 207–216. https://doi.org/10.1016/S0895-4356(99)00161-4

Tillinghast, R. C., Appel, D. C., Winsor, C., & Mansouri, M. (2020, August). STEM outreach: A literature review and definition. Paper presented at 2020 IEEE Integrated STEM Education Conference (ISEC).

Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36 (3), 1–48. https://doi.org/10.18637/jss.v036.i03

Viechtbauer, W., & Cheung, M. W. L. (2010). Outlier and influence diagnostics for meta-analysis. Research Synthesis Methods, 1 (2), 112–125. https://doi.org/10.1002/jrsm.11

Wang, M.-T. (2012). Educational and career interests in math: A longitudinal examination of the links between classroom environment, motivational beliefs, and interests. Developmental Psychology, 48 (6), 1643–1657.

Wiebe, E., Unfried, A., & Faber, M. (2018). The relationship of STEM attitudes and career interest. EURASIA Journal of Mathematics, Science and Technology Education, 14 (10), 1–17.

Wilson, A. C., Gonzalez, L. L., & Pollock, J. A. (2012). Evaluating learning and attitudes on tissue engineering: A study of children viewing animated digital dome shows detailing the biomedicine of tissue engineering. Tissue Engineering Part A, 18 (5–6), 576–586. https://doi.org/10.1089/ten.tea.2011.0242

Xie, Y., Fang, M., & Shauman, K. (2015). STEM education. Annual Review of Sociology, 41 , 331–357.

Xie, Y., & Killewald, A. A. (2012). Is American science in decline? Harvard University Press.

Book   Google Scholar  

*Yang, D., & Chittoori, B. (2022). Investigating Title I school student STEM attitudes and experience in an after-school problem-based bridge building project. Journal of STEM Education: Innovations and Research, 23 (1), 17–24.

Young, J., Ortiz, N., & Young, J. (2017). STEMulating interest: A meta-analysis of the effects of out-of-school time on student STEM interest. International Journal of Education in Mathematics, Science and Technology, 5 (1), 62–74.

Zhang, D., & Tang, X. (2017). The influence of extracurricular activities on middle school students’ science learning in China. International Journal of Science Education, 39 , 1381–1402. https://doi.org/10.1080/09500693.2017.1332797

Download references

Parts of this work have been supported by the US National Science Foundation (NSF DRL 1811265). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the US National Science Foundation.

Author information

Authors and affiliations.

University of Virginia, 405 Emmet St S, Charlottesville, VA, USA

Georgia State University, 33 Gilmer Street SE, Atlanta, GA, 30303, USA

Lillian R. Bentley

The Chinese University of Hong Kong, Hong Kong SAR, China

Australian Catholic University, 25A Barker Road, Strathfield, NSW, 2135, Australia

Robert H. Tai

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection were performed by Xin Xia, Lillian Bentley; and analysis by Xin Xia. The first draft of the manuscript was written by Xin Xia and Lillian Bentley, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Xin Xia .

Ethics declarations

Ethical approval.

Not applicable.

Competing Interests

We declare that there is no competing interests relevant to this meta-analysis. No financial or personal relationships with individuals or organizations have influenced the study design, data collection, analysis, interpretation, or the decision to publish the findings. The research was conducted with impartiality and objectivity to ensure the integrity of the meta-analysis process and the credibility of its outcomes.

Conflict of Interest

The authors have no conflicts of interest to declare. All co-authors have read and agree with the contents of the manuscript. All co-authors have no financial interest to report.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Xia, X., Bentley, L.R., Fan, X. et al. STEM Outside of School: a Meta-Analysis of the Effects of Informal Science Education on Students' Interests and Attitudes for STEM. Int J of Sci and Math Educ (2024). https://doi.org/10.1007/s10763-024-10504-z

Download citation

Received : 12 February 2024

Accepted : 06 September 2024

Published : 17 September 2024

DOI : https://doi.org/10.1007/s10763-024-10504-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Informal science education
• Meta-analysis
• Student interests and attitudes
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 17 September 2024

Lessons from the COVID-19 pandemic to strengthen NCD care and policy in humanitarian settings: a mixed methods study exploring humanitarian actors’ experiences

• Éimhín Ansbro 1 , 2 ,
• Olivia Heller 3 ,
• Lavanya Vijayasingham 1 , 2 ,
• Caroline Favas 1 , 2 ,
• Jacqueline Rintjema 2 , 4 ,
• Alyssa Chase-Vilchez 5 ,
• Claire Stein 6 , 7 ,
• Rita Issa 8 , 9 ,
• Leah Sanga 1 , 2 ,
• Adrianna Murphy 2 , 10 &
• Pablo Perel 1 , 2  

BMC Health Services Research volume  24 , Article number:  1081 ( 2024 ) Cite this article

Metrics details

The COVID-19 pandemic and response severely impacted people living with non-communicable diseases (PLWNCDs) globally. It exacerbated pre-existing health inequalities, severely disrupted access to care, and worsened clinical outcomes for PLWNCDs, who were at higher risk of morbidity and mortality from the virus. The pandemic’s effects were likely magnified in humanitarian settings, where there were pre-existing gaps in continuity of care for non-communicable diseases (NCDs). We sought to explore factors affecting implementation of NCD care in crisis settings during the COVID-19 pandemic and the adaptations made to support implementation.

Guided by the Consolidated Framework for Implementation Research, we undertook an online survey of 98 humanitarian actors from multiple regions and organization types (March-July 2021), followed by in-depth interviews with 13 purposively selected survey respondents (October-December 2021) . Survey data were analysed using descriptive statistics, while interview data were analysed thematically, using both deductive and inductive approaches.

Initially, humanitarian actors faced challenges influenced by external actors’ priorities, such as de-prioritisation of NCD care by governments, travel restrictions and supply chain interruptions. With each infection wave and lockdown, humanitarian actors were better able to adapt and maintain NCD services. The availability of COVID-19 vaccines was a positive turning point, especially for the risk management of people with NCDs and protection of health workers. Key findings include that, despite pre-existing challenges, humanitarian actors largely continued NCD services during the crisis. Enabling factors that supported continuity of NCD services included the ability to quickly pivot to remote means of communication with PLWNCDs, flexibility in medicine dispensing, and successful advocacy to prioritize NCD management within health systems. Key lessons learned included the importance of partnerships and cooperation with other health actors, and the mobilisation or repurposing of community health workers/volunteer networks.

Conclusions

The COVID-19 experience should prompt national and global health stakeholders to strengthen inclusion of NCDs in emergency preparedness, response, and resilience planning. Key lessons were learned around remote care provision, including adapting to NCD severity, integrating community health workers, providing context-adapted patient information, combating misinformation, and strengthening cross-sectoral partnerships.

Peer Review reports

The SARS-2 Coronavirus (COVID-19) pandemic caused unprecedented challenges worldwide, testing healthcare systems across continents, affecting populations’ health and wellbeing, and highlighting global and national inequities [ 1 , 2 ]. COVID-19 was more likely to cause severe infection and death in people who were older (75 years and above), immunocompromised or living with non-communicable disease (NCD) [ 1 , 3 ]. As early as May 2020, NCDs and COVID-19 were cast as twin epidemics and later as a “syndemic.” They acted synergistically on morbidity and mortality, and shared a common set of underlying risk factors, including socio-economic deprivation, obesity, older age, and ethnicity [ 4 ]. As COVID-19 deaths reached one million worldwide, and the key roles of social inequity and failed political leadership were recognised, there was growing acknowledgement that tackling NCDs would be a “prerequisite for successful containment” of COVID-19 [ 5 ]. This required a broader syndemic approach, encompassing housing, education, employment, health, and the environmental sectors.

For decades before the pandemic, NCDs, notably cardiovascular diseases, cancers, diabetes, and chronic respiratory diseases, were the leading causes of mortality globally. They are responsible for 41 million deaths each year, equating to 75% of total global deaths [ 6 ]. People living in low and middle-income countries (LMICs), where the majority (70%) of global NCD deaths occur, are disproportionally affected by premature NCD mortality (i.e., deaths occurring before the age of 70) [ 6 ]. For best outcomes, people living with NCDs (PLWNCDs) require functioning health systems to deliver a continuum of care. This includes early detection through screening and diagnosis; accessible and continuous care and medications; and supported self-care and education, as well as context-adapted healthy eating and exercise opportunities [ 7 , 8 ].

In parallel, more people than ever are affected by humanitarian crises, which have become more complex and prolonged [ 7 , 8 , 9 ]. Conflict, violence, and socio-economic inequity drive most of these crises, and many are now compounded by climate change. In 2021, COVID-19 overlaid other pre-existing and emerging crisis risks, as humanitarian needs remained at historically high levels. An estimated 306 million people were in need in 2021, 90.4 million more than in 2019, before the COVID-19 pandemic hit [ 10 , 11 ].

Humanitarian emergencies disrupt care for NCDs, through destruction of health infrastructure and supply chains, and by reducing access to care. The continuum from diagnosis and screening services, to medical consultation, provision of regular medicines and equipment, and referral pathways may all be affected. Limited evidence also shows that the rates of acute exacerbations, including heart attacks, strokes, asthma attacks, and amputations are increased by stress, and are higher both during emergencies and in their immediate aftermath [ 7 , 8 ]. Recent World Health Assembly resolutions and the World Health Organization (WHO) NCD Global Action Plan 2013–2030 underlined the importance of ensuring that refugees and internally displaced people can access care for NCDs [ 12 ]. However, until recently, NCDs have not been afforded the same priority as other important health concerns during acute crises, and have often been insufficiently integrated into emergency preparedness and response [ 13 ].

Refugees and other displaced people and those with limited health care access—as well as PLWNCDs—were considered “high burden” populations affected by the pandemic and its response [ 14 ]. Many national response policies to manage COVID-19 infections directly caused disruptions of NCD services along the continuum of care [ 15 , 16 ]. A WHO survey conducted from May to July 2020 indicated that about 75% of global NCD services were disrupted in the early days of the pandemic, with low (65%) and lower- middle income (49%) countries most affected [ 17 ]. In the initial months of the pandemic in 2020, NCD care was commonly disrupted because of the urgent diversion of health care resources towards the COVID-19 response, government-imposed travel restrictions, advice to high-risk people to isolate, and people’s understandable fear of attending health facilities [ 17 , 18 , 19 , 20 ]. Data from high- and middle-income countries demonstrate the consequences of foregone or delayed NCD-related healthcare seeking. These include poorer rates of diabetes diagnosis, control and up-titration of medications, and poorer CVD outcomes due to decreased access to care [ 21 , 22 , 23 , 24 ]. Reduced facility attendance or admission for acute NCD complications, such as heart attacks, often increase out-of-hospital deaths, and worsen long-term complications, including functional impairments and disability [ 20 ].

Some humanitarian actors have signalled their ability to continue NCD services with minimal disruptions during the peak of the COVID-19 pandemic [ 25 ], an ability that was not demonstrated even in stable high- and middle-income settings in the early phases of the response [ 18 ]. However, we know little about how COVID-19 disrupted NCD services in crisis settings more broadly, how actors adapted, and what factors enabled or hindered them to do so.

Though the peak of the COVID-19 pandemic is behind us, it is important that we learn lessons from this experience that may shape future NCD services and policies. Given the likelihood of another pandemic, and the fact that the climate crisis will cause more extreme weather events and compound the vulnerabilities that lead to conflict, WHO and other actors are placing greater emphasis on health system preparedness, response, and resilience. Therefore, factors affecting continuity of care for NCDs and successful adaptations to care delivery in the context of COVID-19 are important for preparing for future health service disruptions, for ongoing crises, and where marginalised or vulnerable communities have limited access to care [ 26 ]. Accordingly, we sought to explore factors affecting implementation of NCD care in crisis settings during the COVID-19 pandemic in LMICs, and the adaptations made to support implementation.

Study team and setting

The Centre for Global Chronic Conditions, in collaboration with the Health in Humanitarian Crises Centre, from the London School of Hygiene and Tropical Medicine (LSHTM), led the study in partnership with the Global Alliance for Chronic Disease (GACD) Humanitarian Crises Working Group. The research design was guided by an advisory committee of experts from key humanitarian organisations and agencies [WHO, United Nations High Commission for Refugees (UNHCR), International Committee of the Red Cross, Médecins sans Frontières, and International Rescue Committee] who work on global policies and programmes delivering NCD care in humanitarian settings. This was a global study, targeting humanitarian actors in all geographical settings, who were involved in direct delivery of NCD care during the COVID-19 pandemic.

Study design

The study used a newly developed online survey in English (Additional file 1 ) targeting humanitarian actors, followed by individual interviews (Additional file 2 ) with selected participants. We focussed on the delivery of care for hypertension, type-1 and type-2 diabetes (“DM/HTN”, implying care for either or all conditions) as these are the most common NCD types currently addressed by humanitarian organisations [ 13 , 27 ]. These conditions are also established tracer conditions, used in the healthcare quality assessment literature to assess health system or service performance [ 28 , 29 , 30 ]. These example conditions tend to be well defined, prevalent, relatively easy to diagnose, and have effective, available treatments.

Conceptual framework and definitions

We used an implementation science framework, the Consolidated Framework for Implementation Research (CFIR – Fig.  1 ) to inform the design and analysis of the survey and interviews [ 31 , 32 ]. CFIR is a practical framework, which provides a list of constructs, organised within domains, that are believed to influence implementation, either positively or negatively. It is intended to help guide the systematic assessment of potential barriers and facilitators and, thus, tailor implementation strategies and adaptations, and/or to explain outcomes. The five major domains of the framework – 1) intervention characteristics, 2) outer setting, 3) inner setting, 4) characteristics of individuals, and 5) process – provided a means to synthesise diverse interventions or adaptations in various contexts in response to a global pandemic.

The Consolidated Framework for Implementation Research framework (2009), Source: [ 31 , 32 ]

For this study, we conceptualised the “intervention” as maintaining access to NCD care while responding to the health risks of the COVID-19 pandemic. “Maintaining access to care” was defined as the continued provision of care to the target population at a minimum acceptable level, compared to the baseline (e.g., before the pandemic), so that the services were available (i.e., with adequate human resources, equipment – including drugs – to safely deliver quality services), physically accessible and affordable, and utilised by the target population. NCD care refers to primary health care level activities for people with hypertension and/or diabetes that we propose are essential to maintain during the COVID-19 pandemic.

Data collection

The online survey (Additional file 1 ) was designed by the LSHTM team, guided by the CFIR framework constructs, reviewed by the advisory committee, and piloted. Questions focussed on the delivery of a specific programme/project, focussing on the characteristics of pre-pandemic NCD services, adaptations made in response to the pandemic, individual and inner and outer setting challenges or facilitators, and decision making. We defined the components of NCD services as: medical consultation, disease monitoring, PLWNCDs’ education and support services, and primary prevention and community screening. The survey was hosted on the BOS Online Survey tool ©. A survey link was shared with all participants via email, and the survey included screening questions to restrict participation to people with relevant profiles. It was launched in March 2021 and closed in June 2021.

For the in-depth interviews, a structured topic guide (Additional file 2 ) was used to direct the flow of conversation, and ensure coherence of discussions with the study’s aims and survey. To facilitate rapid data collection, a team of four female interviewers with a public health background (CS, AC, JS, RI) was trained by EA. Each interviewer invited two to four participants and undertook between one and three interviews. From October to December 2021, thirty participants were contacted by email, of whom 13 took part in an interview. Interviewers probed the participants with follow-up questions based on their unique responses, and at the interviewer’s discretion. Interviews took place from November to 2021 to January 2022, and lasted between 45–60 min. They were conducted online, over the phone, or via Skype or Zoom audio-conferencing platforms. Interviews were conducted in English and were digitally audio-recorded, and transcribed for analysis using MS Word and Excel. Written, informed consent, was transmitted via e-mail. Weekly meetings were held with the study team to debrief on interviews, discuss initial findings and iteratively adapt the topic guide.

Participant sampling

Project managers or medical staff directly involved in NCD care delivery at project/programme level in humanitarian settings during the COVID-19 pandemic were eligible for the online survey. Programming professionals are directly involved in the implementation of NCD programmes and service delivery, and their tacit working knowledge and experience provide invaluable insights into how the COVID-19 pandemic and policies affected NCD programmes, as well as how adaptations were formulated, coordinated, and implemented during this crisis. Using our existing GACD, LSHTM, and advisory committee networks, our partners emailed a convenience sample of their contacts who fit the sampling criteria, sharing information on the study, and inviting them to fill in the online survey. Snowball sampling of the respondents’ contacts was used to extend the sampling frame.

A sub-set of survey participants was invited to participate in in-depth interviews, six months after the survey was administered. The interview cohort was purposively selected to represent voices of participants in a range of roles in NCD programmes, from different organisation types that employed different types of adaptations, across different global regions. With input from the advisory committee, the study team defined the following selection criteria to identify follow-up interview participants: 1) geographical spread, 2) range of adaptations/ adjustments, 3) range of organizations, and 4) range of positions/ roles in NCD care delivery.

Data analysis

Descriptive tabulation of quantitative survey responses was undertaken using the Stata statistical software package [ 33 ]. The survey was conducted as a rapid response to the initial phase of the pandemic, and early findings were shared with the advisory committee.

Qualitative data from a) survey free text responses, and from b) interview transcripts, were analysed jointly, using a combination of Framework Analysis (deductive coding) and inductive open coding approaches [ 34 ]. The Framework Method provides clear steps to follow and produces highly structured outputs of summarised data. It is therefore useful where multiple researchers are working on a project, particularly in multi-disciplinary research teams where not all members have experience of qualitative data analysis. First, an a priori coding template using MS Excel was developed by EA based on the CFIR framework (Fig. 1 ) to guide the deductive coding process (performed by OH, AC, CS). A separate data-driven inductive coding exercise was conducted by EA and LV. Repeated review and the complimentary coding approaches enriched the research team’s interpretive and analytic understanding of the data. The qualitative data is presented as reconstructed narratives using both a descriptive and interpretive stance, by themes, and with direct quotes from the participants.

The survey received 98 responses, from 38 different organisations, operating in 21 different countries. Most survey respondents were working in South-East Asia, Africa, and the Eastern Mediterranean (34%, 33% and 28% respectively), and their programmes were based in protracted conflict areas (32%), and targeted refugees (83%), although 60% targeted mixed populations [i.e., a mix of refugees, internally displaced populations (IDPs), and/or host populations]. Most programmes were in camp settings (70%), and provided DM/HTN care integrated within general primary health care (63%) or with other NCDs (including cardiovascular disease and mental health care) (26%). Table 1 outlines the characteristics of the survey respondents and the NCD programmes they were involved in.

Interviews were conducted with 13 of these survey respondents. Table 2 outlines the interview participants’ characteristics.

Findings from both the survey and interviews are reported below, following the CFIR implementation framework constructs ( intervention characteristics, process, outer setting, inner setting, and characteristics of individuals ) and subconstructs, which are highlighted in italics. As mentioned, we defined the “intervention” as maintaining continuity of NCD services, while mitigating the threat of COVID-19.

Intervention characteristics

Before the pandemic, medical consultation was provided by generalist doctors in 90% of respondents’ NCD programmes; specialist doctors, nurses, and lay- or community-based health workers/volunteers were involved in 27%, 41% and 43% of respondent’s programmes, respectively. Consultations were done individually and face-to-face in most (98%) cases. Groups were utilised for consultation and monitoring, but mainly for education and prevention/screening activities. Most medical consultations were delivered in a primary care centre or health posts (89%), fewer in secondary or tertiary level hospitals (36%), and services included home visits in 25% and mobile clinics in 15% of cases.

During the pandemic response, more than half of the NCD service components provided before the pandemic were partially or fully maintained, including medical consultation (94%), disease monitoring (90%), PLWNCDs’ education and support (88%) and primary prevention and community screening services (61%). As might be expected, face-to-face individual services declined, with more than 50% of these services reduced during the pandemic, and medical consultation via home visits were cut by half. More detail on the characteristics of NCD service components before and during the pandemic are available in Additional file 3 .

Organisations’ implementation processes varied as they experienced different organisational ( inner setting ) and contextual ( outer setting ) barriers and facilitators. Services were adapted iteratively as the pandemic progressed. For example, survey respondents reported outer setting factors that hampered continuity of service delivery, including poor mobile phone coverage (28%), smartphone availability (35%) and internet connectivity (35%). PLWNCDs faced challenges in managing their disease, especially financially (49%) and mentally (42%).

The key CFIR intervention constructs that were generated from interview and survey free text data were source, evidence strength and quality, adaptability, and cost. At the onset of the pandemic, national policies immediately targeted infection prevention and control (IPC) to limit the pandemic’s spread, introducing movement restrictions, and diverting health system policy and resources to the pandemic response. In the early days, interviewees reported initial uncertainty in how to respond to these policies.

The decision to prioritise PLWNCDs and the specific adaptations made to service delivery were perceived as coming strongly from within individual organisations, with recommendations coming from WHO/UNHCR, rather than from national governments. The latter were largely perceived as having “ forgotten ” PLWNCDs in their initial pandemic response plans. The source of IPC guidance, training and equipment was perceived to be national governments, Ministries of Health, and international actors, such as the WHO and UNHCR. The UN sources were considered trustworthy and of good quality, filling essential gaps when information or action was lagging from national resources. The cost of maintaining NCD care was mainly spoken of in terms of the cost and diversion of funds into IPC measures, and the fact that pandemic-related inflation increased costs for governments, organisations, and PLWNCDs, for example, significantly increasing transportation costs. The CFIR constructs complexity, trialability and relative advantage versus other interventions did not feature strongly in the data. There were many unknowns at the beginning of the pandemic response, and there was acknowledgement that organisations did not have time to trial interventions but, instead, needed to act quickly.

In most settings, the process of maintaining NCD care could be summarised as involving the following key components: a) the introduction or enhancement of IPC measures; b) prioritisation of PLWNCDs and maintenance of clinical contact, including through remote means; c) maintenance of medication and equipment supplies; d) maintenance or adaptation of the health workforce; e) information sharing between organisations and with PLWNCDs, and countering misinformation; and iteratively adapting these approaches as the pandemic evolved:

“Adaptations done in NCD service delivery were aimed to address the safety of NCD patients from COVID-19, considering their susceptibility to mortality due to COVID-19, also safety of health care staff, from community level to health facility level” [ID01]

The CFIR constructs planning, engaging, executing, and reviewing were discussed in interviews and survey free text responses. Evaluating was less prominent in the data, given that data were collected relatively early in the pandemic response, and programmes did not have time to formally evaluate their response strategies. However, respondents reported anecdotally that their interventions were successful.

The WHO Health Sector Cluster System or UNHCR-coordination systems, which are used to coordinate multiple agencies during emergency responses, were instrumental in planning and executing the pandemic response in places where it was already established. For example, in these settings, collaboration and information sharing occurred early in the pandemic. Decisions on how to respond were generally made by the organisation’s management, although one interviewee described close engagement of clinical staff in an iterative decision-making process:

“…clinic staff, budget staff and … coordination, all three … were working together to come up with these recommendations of how to overcome the challenges at the clinic level. So, I think the recommendations came mostly from the clinic staff …but it was a collective decision. [ID31]

Infection prevention and control

Interview participants described rapidly introducing COVID-19 risk mitigation measures, including IPC protocols, such as the use of personal protective equipment (PPE), hand hygiene, and social distancing, and training on the clinical management of COVID-19. A number of participants noted there were supply delays in some circumstances. Where organisations initially suspended DM/HTN services, shortages in PPE (14%) was the most commonly reported reason. Masks and PPE were introduced as soon as supplies were available and were often provided by international non-governmental organisations (NGOs) and United Nations (UN) organisations, who stepped in when national supply chains were inadequate or too slow.

Prioritisation of people with NCDs

Respondents consistently reported that their organisations, unlike many national governments, recognised the increased risk PLWNCDS faced, and the need to prioritise their continuity of care. Organisations took varying approaches to social distancing to protect and prioritise PLWNCD and staff. For example, in some contexts, outdoor waiting areas were created, and temperature checks and triage of PLWNCDs were introduced. PLWNCDs were often separated from other primary care patients. In many, although not all, cases, only PLWNCDs with severe or uncontrolled conditions continued to be seen at facilities, by appointment only, while those with stable conditions were advised to remain at home. In a minority of cases, facility-based consultations were maintained for all PLWNCDs, while group-based activities were adapted (Additional file 3 ).

Maintaining NCD consultations

Table 3 outlines the survey response on the change or termination of NCD programmes implemented by the respondents’ organisations. Medical consultations were largely maintained or immediately adapted – only 12% of respondents reported initially suspending and then resuming them in an adapted format. The major reasons reported for suspending consultations were government-mandated movement restrictions (33%) and PLWNCDs’ fear of face-to-face attendance (24%). These factors also reduced the numbers of consultations in the initial months.

Other NCD programme components were also adapted, either immediately or after a period of brief suspension. In most cases, disease monitoring continued unchanged (46%), and the remainder of programmes simplified or reduced monitoring frequency. The few service components that were completely stopped without resumption tended to be at the community level (2% of education and support services, and 6% of primary prevention and community screening services) or involving group-based activities or mobile units (Table  3 and Additional file 3 ).

Reducing facility-based contact

Adaptations were introduced to maintain contact when PLWNCDs could not attend facilities. Face-to-face consultations were either dropped entirely (reducing from 93 to 39%) or decreased in frequency (73%). The principle means used to maintain contact with PLWNCDs remotely were via community health workers or volunteers (CHW), and via use of telemedicine.

CHWs were involved in some aspect of NCD service provision, mainly in education and support and/or NCD prevention and screening activities (Additional file 3 ). They played a role in medical and in disease monitoring in about one fifth and one third of cases, respectively. In response to the pandemic, one fifth of respondents (21%) reported additional task sharing to community-based staff. Their role was expanded to include education around COVID-19, IPC, and vaccination, active follow up of PLWNCDs, home-based clinical and adherence monitoring, and liaison with clinicians, supporting remote management of PLWNCDs. Interview participants from diverse settings highlighted the key role that CHWs played in reaching the community and gaining real-time insights on community needs, disseminating information, and gaining community trust.

In parallel, however, participants emphasised the need for adequate and regularly updated training, communication pathways, and support for CHWs:

“We ensured CHWs (were) kept on their toes in terms of trainings and refresher, information on COVID and NCD and management of NCD within the COVID-19 pandemic. Two, we ensured that CHWs also (were) giving (clinical) information back …It’s also very important to have (a) communication system where CHWs can … share information directly to you and … tell you the situation in the community…. [ID26]

Prior to the pandemic, the survey findings suggest that telemedicine via mobile or landline telephone, WhatsApp, or video consultation, was utilized by a very small proportion of our study respondents’ organisations (Additional file 3 ). The survey results also indicate a higher use of telephones during the pandemic to provide medical consultations, disease monitoring, education and support services, and primary prevention and screening. For example, 2% of respondents reported their organisations using telephone consultations pre-pandemic, which increased to 23% during the pandemic (Fig.  2 ).

Use of technology to support medical consultations before and during the pandemic

Access to and use of blood pressure and blood sugar monitors was variable. Similarly, access to digital devices with internet connectivity such as telephone, smartphones, and tablets, to communicate remotely with health facilities varied significantly. Where there was phone and internet connectivity and access to use of smart devices, programme staff were able to engage with, and monitor PLWNCDs through online platforms. Stable PLWNCDs with controlled disease were supported to self-manage at home via phone consultations or CHW visits, and this was facilitated by PLWNCDs having home monitoring devices (blood pressure machines and glucometers). This was more common in the Middle East and North African region than in Sub Saharan Africa. Lack of available self-care resources in other settings meant that PLWNCDs were not able to monitor and manage their health within their homes. In one setting, PLWNCDs were taught to self-inject insulin rather than having to attend the facility for health workers do it.

In some instances, this change in remote consultation approach was met with initial resistance. As the approach was normalised, PLWNCDs reportedly began to prefer these modes of communication.

Communication via these platforms spanned from health education and awareness, to targeted counselling and psycho-social support, where its wide reach was deemed beneficial in reducing stigma. For example, one programme provided nurse-led psychosocial support via WhatsApp groups. Uptake was increased through the delivery of “ ice breaking ” messages and the service was offered to all PLWNCDs, and therefore engagement with the service was not associated with having a mental illness.

Several examples of the CFIR constructs reviewing and evaluating were offered by interviewees. For example, several organisations realised that their initial attempts to use internet or smart phone-based technology were hampered by PLWNCDs’ lack of or uneven access to digital infrastructure, and they reverted to using telephones or community health workers to maintain contact. One interview respondent also described realising, after a period of implementing phone consultations, that doctors required specific guidance and tools to undertake these safely and consistently.

Maintaining supply of medication and equipment

At the beginning of the pandemic, most interview participants described issues with procurement of medication and IPC equipment, and national level supply chains being diverted to the pandemic response. Supply issues were reported as the main reason some programmes initially stopped or suspended DM/HTN service. In addition, almost half (45%) reported internal supply issues within their organisation which hampered continuity of care, and one third (32%) reported introducing adaptations to medication procurement or supply in response to the pandemic.

Key adaptations to medication supply included increasing the dispensing interval to three months (32%) (following WHO guidance), allowing family and friends to pick up medications from facilities (48%), and in one case, having community health workers deliver medication to people’s homes. The reduced frequency of medication pick-ups was seen as a useful to mitigate exposure to the virus in high-risk populations and to reduce crowding, caseload, and the number of people in health facilities.

Interviewees indicated that supply chain challenges lasted up to about four months and were resolved through national and international interagency collaboration.

Maintaining the health workforce

Survey participants cited staff absence due to COVID-19-related illness or quarantine (60%), and staff burnout (49%) as key internal organisational challenges to maintaining continuity of NCD care during the pandemic. Many health care workers were diverted from their usual roles to the pandemic response, their movements were physically restricted during the “lockdowns”, and interviewees recounted their initial “ panic” and high stress levels.

Strong interorganisational collaboration, particularly within camp settings, allowed organisations to pool their human resources and “cross-cover”, for example, taking on another organisation’s PLWNCDs when they had a COVID-19 outbreak among staff. One organisation reported creating two teams of staff who worked in separate shifts, to minimise burn-out and infection risk. To alleviate these workforce challenges, several reported task-sharing within the facility (25%) and/ or to community-based staff (21%) (Additional file 4 ). Interviewees cited improved supply of PPE and the introduction of COVID-19 vaccines as pivotal changes that protected staff and reduced their fear.

Sharing information and countering misinformation

Themes around use of existing data and data sharing between organisations were generated inductively from the interviews. The importance of patient registries was clearly highlighted, since they allowed staff to track NCD patients, which enabled continuity of care, and information sharing with patients. Where the WHO Health Cluster and UNHCR coordination mechanisms were strong, particularly in camp-based settings, agencies pooled their NCD patient lists and supply data, allowing agencies to share resources and collectively respond.

Communication strategies were key throughout the pandemic response. During the initial phase of the pandemic, programmes focussed on urgently communicating the infection risks and prevention strategies, through public and programme-based communication. Additional messaging on the importance of follow-up care for NCDs was then necessary, to counter people’s fear of attending facilities. Once vaccines were introduced, a new wave of messaging was required and implemented in many of the programmes—this time on the merits and safety of COVID-19 vaccines, and to counter misinformation and vaccine myths.

“At the beginning it was very difficult. You know, the misinformation “oh the COVID-19 vaccine it makes you die.” …we worked in coordination with other health services with the refugee camp and community health volunteers conducting home visits to ensure all NCD patients (got) the vaccine… [ID09]

Community health workers, where they were active, played an important role in delivering these messages, and interviewees also reported using social media, such as Facebook and WhatsApp, SMS messages in some settings, and more traditional loudhailers to spread educational messages, where settings were conducive to this e.g. in camps.

Inner setting

Structural characteristics of the surveyed organisations – most of which were humanitarian actors used to working in volatile settings, assessing acute needs, and rapidly intervening – and their internal networks and communications were important elements in quickly responding, and iteratively adapting to the pandemic. Narratives from the interviews, which were conducted about six months after the survey took place, highlighted that after the initial uncertainty, programme staff felt better equipped to manage the evolving circumstances. Interviewees highlighted their organisations’ resilience, inherent agility, and ability to adapt, and several expressed pride in their organisation’s success in coping, and maintaining continuity of care for PLWNCDs. Furthermore, teamwork and coordination were often strengthened by the pandemic response and several respondents proposed retaining these adaptations after the pandemic.

The physical infrastructure and camp versus urban setting characteristics were highly influential. Movement within camps was less challenging than moving in and out of camps, or within urban areas, and, where host populations used health services within camps, their access was jeopardised.

Strong baseline data collection systems and processes within an organisation enabled assessment of the situation, follow-up of individual patients and data sharing with other organisations:

“In our facility, we have one dedicated register for non-communicable diseases patient… so our dedicated team, continuously (kept) tracking these patients…and we (kept) connection with our community health workers ...” [ID02]

However, other organisations felt hindered by the lack of available data and data infrastructure in planning and rolling out their response.

Generally, interviewees were receptive to the changes that had to be made in response to COVID-19, the idea of protecting PLWNCDs, while maintaining continuity of care fit with individual and organisational norms and values. Interviewees generally felt they had support and feedback from managers. However, many described undertaking additional tasks with a reduced workforce and staff burnout as a prominent theme in both survey and interviews. Some participants also described a lack of “back-up” emergency plans, including alternative workflow plans when staffing was short.

Views on PPE training were mixed; some described it as delayed or improperly carried out. There were also contrasting accounts of CHW training, which was poor in some settings and highly successful with bespoke CHW training packages being developed in other settings. Overall, quick development and dissemination of training programmes, including for non-medical and CHWs, often through online/remote modules from various international and local health actors were recognised as an important enabling factor in continuing NCD care in a safe manner:

“All health workers had training about the IPC measures during COVID-19, and how to deal with patients. This was online training… done at the beginning of the crisis, through the WHO…on their website….” [ID09]

Outer setting

Participants were asked about their awareness of PLWNCDs’ needs and resources and their attempts to prioritise them. Survey respondents cited physical restrictions (88%), social restrictions (60%), fear of attending health services (54%), financial hardship (49%), and poor mental health (42%) as the key challenges faced by PLWNCDs during the pandemic (Additional file 4 ). They attempted to overcome them by introducing remote modalities for consultations and monitoring, and strong, agile messaging campaigns.

As anticipated, respondents highlighted established structural and infrastructural challenges in providing NCD care that existed before the pandemic, including a lack of NCD policy and funding and national economic pressures. More general challenges faced by humanitarians operating within an emergency response, such as fragmented health systems, with pluralistic actors, sometimes operating in vertical programmes with limited integration, were also noted.

The degree to which an organisation was networked with other external organisations ( cosmopolitanism within CFIR) proved a crucial enabler in rapidly adapting and maintaining care for PLWNCDs during the pandemic, and a key theme that was identified from surveys and interview data. Interviewees described utilising pre-existing networks of health actors and WHO-led health cluster meetings, especially in camp-based settings, with a significant strengthening of these relationships, and day-to-day collaboration increasing far beyond pre-pandemic levels. Examples of this included creating a master list of NCD patients within camps, cross covering each other’s operations and borrowing each other’s resources, including health workers, medical supplies, and community volunteer networks. These networks offered key support and a degree of peer pressure or competitive pressure to implement interventions.

One example of a new cross-sectoral collaboration was offered, whereby a health organisation repurposed a CHW network, which was usually involved in protection activities, to engage in active follow up of PLWNCDs. Government stewardship and leadership were also highlighted as key enablers to rapid response and adaptation.

External policies and incentives played a key role as either barriers or enablers. The lack of national-level emergency preparedness plans and mechanisms for coordination between health actors were highlighted by many respondents. Narratives around the early instructions from various Ministries of Health suggest a strong initial focus on infection control, and de-prioritisation of other services, including those for chronic disease:

"COVID took all the, let's say the light and only cases with COVID were prioritized. So no, I think NCDs were pulled back during the pandemic." [ID09]

A lack of pre-existing national-level policies and funding for NCDs, followed by the diversion of funding and staff time in public facilities to infection control measures and COVID-19 treatment hampered the continuity of NCD services and referrals. External policies by partner hospitals or health facilities also influenced the continuity of some NCD programme components. For example, non-emergency referrals to secondary and tertiary care hospitals were often postponed.

Other potential adaptations to reduce facility-based contact for PLWNCDs were hindered by the lack of enabling policies and national infrastructure. For example, policy barriers prevented longer-term dispensing of medicines in some contexts, and the lack of legal mechanisms to enable task sharing or telehealth consultations limited adaptions of service delivery in others. The baseline utility and availability of technology in the local context was a clear influence on the remote care modalities that could be introduced. Respondents reported a lack of national infrastructure to facilitate virtual or remote health activities prior to the pandemic, including for consultations, prescriptions, and medication delivery. Thus, while organisations were initially advised to use social media, smartphones etc., many found that this was unrealistic in their settings.

Persistent advocacy and engagement with Ministries of Health was successful in changing the policy approach towards NCD services and dispensing of medicines. Respondents suggested further advocacy was needed with governments to include NCDs as priority conditions in future emergency response, to allow for longer dispensing intervals to reduce the burden of facility attendance, and to build on technology and infrastructure to allow for remote consultation and dispensing. In Table 4 , below, we summarise our findings around the contextual factors, intervention characteristics and other barriers and enablers that influenced the continuity of NCD care in humanitarian settings during the COVID-19 pandemic. We also note our study participants' recommendations for action to maintain NCD care continuity during future crises (Table 4 ).

To our knowledge, this is one of the first studies to document factors affecting the implementation of NCD care in LMIC humanitarian settings during the COVID-19 pandemic [ 25 ]. A key finding was that NCD services were largely maintained throughout the pandemic response. Respondents’ organisations minimised interruptions to NCD care, while mitigating the risks of COVID-19, by adapting to enable remote care and reduce facility-based contact. Our study respondents highlighted how the pandemic response exacerbated the pre-existing challenges they faced in delivering NCD care in crisis-affected countries. Most humanitarian actors operate in fragile LMIC settings, where health systems are often under-resourced and fragmented, and where national-level emergency preparedness and response mechanisms may be limited. Reflecting the experience in other parts of the world, our data highlighted that initial COVID-19 responses seemed to de-prioritise PLWNCDs, health system resources were diverted away from NCD care and, especially in many LMIC settings, access to pandemic mitigation strategies, PPE and vaccines was frequently delayed [ 11 ]. Maintaining NCD care during the pandemic was also hampered by the lack of pre-existing policy or infrastructure to support remote care modalities, the fear and misinformation around COVID-19, and the initial resistance to remote care expressed by PLWNCDs.

Despite the challenges, humanitarian actors were adept at implementing context-adapted changes to support continuity of NCD services, which is consistent with findings from a similar study [ 25 ]. The humanitarian system’s in-built flexibility and agility, existing humanitarian coordination mechanisms, and strong experience communicating with PLWNCDs and advocating with authorities were all supportive factors. The UN agency coordination mechanisms, including the WHO health cluster approach and UNCHR working groups enabled quick coordination and sharing or repurposing of partner resources. When it was available, strong data collection on NCDs, such as patient registries and supply monitoring, underpinned this effective interagency coordination. Humanitarians’ experience with previous outbreaks, such as cholera and Ebola, while different, may have allowed them to react in a more agile manner than national health systems could. In keeping with this, LMIC countries that were most successful in their pandemic response built on prior outbreak experience and on existing community resources, including community health workers [ 14 ].

The key role of community health workers and volunteers in facilitating continuity of NCD care, sharing key information, and building trust among communities stood out in our data. This is consistent with other studies, which found that, with adequate and timely resources, including adapted protocols, training, and PPE, pre-existing CHW programmes were able to continue with minimal disruption during the pandemic [ 15 , 33 ] . The key part CHWs played in many of the pandemic responses recounted here reflects their pre-existing role in refugee camp settings and within Sub-Saharan African and in Southeast Asian health systems. By contrast, the role is not often utilised in the Middle East and North Africa, and it has been highlighted as a potential area for development [ 35 ]. There is growing evidence for the positive impact of CHWs on NCD management both in stable LMIC settings, and in maintaining services during periods of disruption [ 36 , 37 , 38 , 39 , 40 , 41 ]. However, in expanding this role in future NCD programmes, lessons must be learned around the need to adequately support CHWs with resources, supervision and training [ 42 ].

Telehealth, defined as “the combined use of the internet and information technology for clinical and organisational purposes, both locally and remotely”, has been touted as one innovative approach to maintaining continuity of care for PLWNCDs that should be retained and built upon post-pandemic [ 43 , 44 ]. According to the WHO, telemedicine and patient triage were the most common mitigation strategies used to reduce NCD service disruption in the early days of the pandemic [ 17 ]. However, our study reflects the literature around the introduction of telehealth – its success is highly contingent on national infrastructure, smartphone ownership rates, and internal organisational factors. Moreover, clear guidance, training and culturally-congruent communication all support its successful implementation [ 45 ]. Our data also highlight the need for guidance for clinicians in the use of telemedicine, in keeping with previous calls for specific WHO guidance on the development and use of digital health solutions for NCD care [ 20 ]. Narratives from this study suggest that the wider use of self-care, via home-based monitoring equipment, coupled with tele-health or CHW networks may be beneficial. These modalities may increase access to care, especially in crisis settings, where populations may be cut off from facilities, or where populations are marginalised or hard to reach. However, their cost effectiveness, acceptability and feasibility in different contexts must be tested with robust implementation research [ 46 , 47 ].

Introducing telemedicine may increase health inequalities [ 42 ]. Throughout the pandemic, the use of digital health for NCDs has not been equitable across world regions, disease types, or populations [ 43 ]. Indeed, the COVID-19 pandemic has highlighted and entrenched existing global inequalities - essential health workers, migrants, refugees and other displaced or marginalised populations, and those living with NCDs were among the groups most burdened by its effects [ 14 ]. It shone a spotlight on the global NCD epidemic and the enormous negative health, social and financial effects NCDs bring, the magnitude of which far outweighs that of the pandemic [ 48 ].

Implications and recommendations for practice and policy

Humanitarian actors and health systems continue to learn lessons from the COVID-19 response that may enhance models of NCD care. Our data support calls for more person-centred, community-based care that limits facility-based contact. Developing such models would be useful beyond the pandemic, as they bring care closer to people’s homes and communities and improve access by decreasing transport and time cost burden on vulnerable, resource-limited, and marginalised patients. They also decrease the risk of nosocomial infections, and potentially decrease the burden on health facilities and staff, allowing more time to be spent on quality care. The means of achieving this must be adapted to the context, but may include increased use of community health workers, telephone consultations, home-based disease monitoring and adapted dispensing practices. The potential for social media and CHW networks to spread reliable health messaging was also highlighted in our study. We recommend that new or adapted models of care should be co-developed with PLWNCDs, and evaluated for cost-effectiveness, using implementation research approaches. Training on NCDs and adequate supervision and funding is needed for health care providers – including CHWs – to build and retain their role in supporting communities. Increased funding and advocacy for the inclusion of NCDs in emergency preparedness and response is essential. Finally, we recommend further implementation research to evaluate some of the adaptations described here, for example, CHW- an/or tele-health supported self-care.

The COVID-19 pandemic exposed how underprepared the health systems of many countries were to respond to the global NCD epidemic. For example, only 42% of low-income countries included the continuity of NCD services in their national COVID-19 plan [ 20 ]. WHO has highlighted steps to “build back better” NCD services post-pandemic, such as including NCDs in national emergency response and preparedness plans, and strengthening baseline NCD data collection and NCD supply management systems [ 49 ]. In keeping with the “health for all” paradigm, NCDs should be integrated into strengthened primary health care within a universal health care approach, and access must be extended to people who are forcibly displaced by humanitarian crises.

Strengths and limitations

This study was designed in the early days of the pandemic to gain insights that could be useful to humanitarians as they rolled out their responses. Engagement with an expert advisory committee, and pre-existing relationships with global humanitarian actors, provided access to respondents from multiple global regions. The survey and interviews took place at different time points in the pandemic, enabling the generation of insights relating to different response phases. Analysis was guided by an implementation study framework, which helped synthesise findings from diverse contexts.

However, the survey was not designed to identify the number of unique programmes, nor was it designed to detect differences in service delivery approaches before and during pandemic with statistical power. We cannot comment on the actual level of service use, on how it may have changed, nor on what impact any of the documented adaptations may have had on clinical outcomes, including complication rates and mortality.

We note that our survey’s initial convenience sampling approach, via study partners and existing networks, facilitated reaching major international humanitarian actors, such as UNHCR, but resulted in few local NGOs being included. This sampling frame meant that most survey participants worked in camp settings, despite most refugees and other forcibly displaced populations now living in urban, integrated settings [ 50 ]. The findings around enhanced communication and collaboration may, therefore, be less generalisable to non-camp-based settings. Despite producing a version in Spanish to encourage responses from South America, we had few responses from the Americas and from the Western Pacific. This was presumably because the major relevant NGOs had limited operations in these regions. Offering the survey in French and Arabic, for example, may have increased responses from other regions. Fewer than half of the invited interviewees accepted to participate, possibly because they were still actively involved in the pandemic response. We also acknowledge that PLWNCDs themselves were not included as participants in this study and recommend further research to learn from and respond to their experiences of the pandemic.

The lessons around factors affecting continuity of care for NCDs and successful adaptations to care delivery in the context of COVID-19 are important for preparing for future health service disruptions, including in contexts experiencing ongoing crises or where marginalised or vulnerable communities have limited access to care. Our study findings reenforce global calls for more investment, strengthened partnerships and greater integration of NCDs into emergency preparedness, and building of resilient health systems.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Data availability

The data that support the findings of this study are not openly available due to reasons of sensitivity and are available from the corresponding author upon reasonable request. Data are located in controlled access data storage at the London School of Hygiene & Tropical Medicine.

Abbreviations

Community Health Worker or Volunteer

Consolidated Framework for Implementation Research

SARS CoV-2 Coronavirus

Diabetes Mellitus

Global Alliance for Chronic Diseases

• Hypertension

Infection Prevention and Control

Low- and Middle-Income Country

London School of Hygiene & Tropical Medicine

Non-communicable Diseases

Non-governmental Organisation

People Living with Non-communicable Disease

Personal Protective Equipment

United Nations

United Nations High Commissioner for Refugees

World Health Organization

Al-Oraibi A, Nellums LB, Chattopadhyay K. COVID-19, conflict, and non-communicable diseases among refugees. eClinicalMedicine. 2021;34. Available from: https://www.thelancet.com/journals/eclinm/article/PIIS2589-5370(21)00093-6/fulltext . Cited 14 Feb 2023.

Booth A, Reed AB, Ponzo S, Yassaee A, Aral M, Plans D, et al. Population risk factors for severe disease and mortality in COVID-19: A global systematic review and meta-analysis. PLoS ONE. 2021;16:e0247461.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Kluge HHP, Wickramasinghe K, Rippin HL, Mendes R, Peters DH, Kontsevaya A, et al. Prevention and control of non-communicable diseases in the COVID-19 response. The Lancet. 2020;395:1678–80.

Article   CAS   Google Scholar  

Sheldon TA, Wright J. Twin epidemics of covid-19 and non-communicable disease. BMJ. 2020;369: m2618.

Article   PubMed   Google Scholar  

Horton R. Offline: COVID-19 is not a pandemic. The Lancet. 2020;396:874.

World Health Organization. Non communicable diseases: Key Facts. 2022. Available from: https://www.who.int/news-room/fact-sheets/detail/noncommunicable-diseases . Cited 31 Oct 2022.

Jawad M, Vamos EP, Najim M, Roberts B, Millett C. Impact of armed conflict on cardiovascular disease risk: a systematic review. Heart Br Card Soc. 2019;105:1388–94.

Google Scholar  

Ngaruiya C, Bernstein R, Leff R, Wallace L, Agrawal P, Selvam A, et al. Systematic review on chronic non-communicable disease in disaster settings. BMC Public Health. 2022;22:1234.

Article   PubMed   PubMed Central   Google Scholar  

UNHCR. UNHCR Global Trends 2023. UNHCR. 2023. Available from: https://www.unhcr.org/ie/global-trends . Cited 17 Feb 2024.

OCHA. OCHA’s Strategic Plan 2023–2026: Transforming Humanitarian Coordination | OCHA. 2023. Available from: https://www.unocha.org/publications/report/world/ochas-strategic-plan-2023-2026-transforming-humanitarian-coordination . Cited 13 Dec 2023.

UNHCR. Global Trends Report 2022. UNHCR. 2022 [cited 2023 Dec 13]. Available from: https://www.unhcr.org/global-trends-report-2022 . Cited 13 Dec 2023.

World Health Organisation. WHO | Global Action Plan for the Prevention and Control of NCDs 2013–2020. WHO. 2015. Available from: https://www.who.int/nmh/events/ncd_action_plan/en/ . Cited 17 Feb 2015.

Ansbro É, Issa R, Willis R, Blanchet K, Perel P, Roberts B. Chronic NCD care in crises: A qualitative study of global experts’ perspectives on models of care for hypertension and diabetes in humanitarian settings. J Migr Health. 2022;5:100094.

Sachs JD, Karim SSA, Aknin L, Allen J, Brosbøl K, Colombo F, et al. The Lancet Commission on lessons for the future from the COVID-19 pandemic. The Lancet. 2022;400:1224–80.

Article   Google Scholar  

Luciani S, Caixeta R, Chavez C, Ondarsuhu D, Hennis A. What is the NCD service capacity and disruptions due to COVID-19? Results from the WHO non-communicable disease country capacity survey in the Americas region. BMJ Open. 2023;13: e070085.

Yaacoub S, Zmeter C, Abbas LA, Leresche E, Kdouh O, Hammoud R, et al. Has the COVID-19 pandemic changed the utilization and provision of essential health care services from 2019 to 2020 in the primary health care network in Lebanon? Results from a nationwide representative cross-sectional survey. PLoS ONE. 2023;18: e0288387.

World Health Organization. Pulse survey on continuity of essential health services during the COVID-19 pandemic: interim report, 27 August 2020. 2020. Available from: https://www.who.int/publications-detail-redirect/WHO-2019-nCoV-EHS_continuity-survey-2020.1 . Cited 31 Oct 2022.

World Health Organization. Non-communicable Diseases Progress Monitor 2022. 2022. Available from: https://www.who.int/publications-detail-redirect/9789240047761 . Cited 12 Nov 2022.

Nikoloski Z, Alqunaibet AM, Alfawaz RA, Almudarra SS, Herbst CH, El-Saharty S, et al. Covid-19 and non-communicable diseases: evidence from a systematic literature review. BMC Public Health. 2021;21:1068.

The World Health Organization. The impact of the COVID-19 pandemic on noncommunicable disease resources and services: results of a rapid assessment. 2020 Sep. Available from: https://www.who.int/publications-detail-redirect/9789240010291

Holland D, Heald AH, Stedman M, Hanna F, Wu P, Duff C, et al. Assessment of the effect of the COVID-19 pandemic on UK HbA1c testing: implications for diabetes management and diagnosis. J Clin Pathol. 2021; Available from: https://jcp.bmj.com/content/early/2021/10/12/jclinpath-2021-207776 . Cited 15 Feb 2023.

Carr MJ, Wright AK, Leelarathna L, Thabit H, Milne N, Kanumilli N, et al. Impact of COVID-19 on diagnoses, monitoring, and mortality in people with type 2 diabetes in the UK. Lancet Diabetes Endocrinol. 2021;9:413.

Del Pinto R, Ferri C, Mammarella L, Abballe S, Dell’Anna S, Cicogna S, et al. Increased cardiovascular death rates in a COVID-19 low prevalence area. J Clin Hypertens. 2020;22:1932–5.

Wright FL, Cheema K, Goldacre R, Hall N, Herz N, Islam N, et al. Effects of the COVID-19 pandemic on secondary care for cardiovascular disease in the UK: an electronic health record analysis across three countries. Eur Heart J - Qual Care Clin Outcomes. 2023;9:377–88.

PubMed   Google Scholar  

Miller L, Alani AH, Avril N, Jingree ML, Atwiine AB, Amire KA, et al. Adaptation of care for non-communicable diseases during the COVID-19 pandemic: a global case study. BMJ Glob Health. 2022;7: e006620.

World Health Organization. Strengthening health emergency prevention, preparedness, response and resilience. 2023. Available from: https://cdn.who.int/media/docs/default-source/emergency-preparedness/who_hepr_wha2023-21051248b.pdf?sfvrsn=a82abdf4_3&download=true . Cited 17 Feb 2024.

Jaung MS, Willis R, Sharma P, Aebischer Perone S, Frederiksen S, Truppa C, et al. Models of care for patients with hypertension and diabetes in humanitarian crises: a systematic review. Health Policy Plan. 2021;36:509–32.

Kessner DM, Kalk CE, Singer J. Assessing Health Quality — The Case for Tracers. N Engl J Med. 1973;288:189–94.

Article   PubMed   CAS   Google Scholar  

Lindström K, Berg L, Rylander B, Hagman A, Olsson L, Bengtsson C. A model for quality assessment in primary health care using the tracer condition technique with insulin treated diabetes as one of the tracers. Scand J Prim Health Care. 1997;15:92–6.

Nolte E, Bain C, McKee M. Diabetes as a Tracer Condition in International Benchmarking of Health Systems. Diabetes Care. 2006;29:1007–11.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.

Research Team-Center for Clinical Management Research. The Consolidated Framework for Implementation Research – Technical Assistance for users of the CFIR framework. 2022. Available from: https://cfirguide.org/ . Cited 11 Nov 2022.

StataCorp. Stata Statistical Software: Release 18. College Station, TX: StataCorp LLC.; 2023. https://www.stata.com/support/faqs/resources/citing-softwaredocumentation-faqs/ .

Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13:117.

Cragg, L, Davies, M, Macdowall W. Health Promotion Theory. 2013. Available from: https://www.mheducation.co.uk/health-promotion-theory-9780335263202-emea-group . Cited 17 Feb 2024.

Washington CH, Tyler FJ, Davis J, Shapiro DR, Richards A, Richard M, et al. Trauma training course: innovative teaching models and methods for training health workers in active conflict zones of Eastern Myanmar. Int J Emerg Med. 2014;7:46.

Perri HB, Ed. National Community Health Worker Programs from Afghanistan to Zimbabwe. 2020. https://chwcentral.org/resources/health-for-the-people%E2%80%8B-national-community-health-worker-programs-from-afghanistan-to-zimbabwe . Cited 17 Feb 2024.

Farzadfar F, Murray CJ, Gakidou E, Bossert T, Namdaritabar H, Alikhani S, et al. Effectiveness of diabetes and hypertension management by rural primary health-care workers (Behvarz workers) in Iran: a nationally representative observational study. The Lancet. 2012;379:47–54.

Newman PM, Franke MF, Arrieta J, Carrasco H, Elliott P, Flores H, et al. Community health workers improve disease control and medication adherence among patients with diabetes and/or hypertension in Chiapas, Mexico: an observational stepped-wedge study. BMJ Glob Health. 2018;3: e000566.

Neupane B. Integrating nutrition in local governance structures: An example from suaahara program Nepal. FASEB J Conf Exp Biol. 2015;29.1. Supplement.741.3.  https://doi.org/10.1096/FASEBJ.29.1_SUPPLEMENT.741.3 .

National Health Mission India. Module for Multi-Purpose Workers (MPW) - Female/Male on Prevention, Screening and Control of Common Non-Communicable Diseases. Available from: https://main.mohfw.gov.in/sites/default/files/Module%20for%20Multi-Purpose%20Workers%20-%20Prevention%2C%20Screening%20and%20Control%20of%20Common%20NCDS_2.pdf . Cited 17 Feb 2024.

Salve S, Raven J, Das P, Srinivasan S, Khaled A, Hayee M, et al. Community health workers and Covid-19: Cross-country evidence on their roles, experiences, challenges and adaptive strategies. PLOS Glob Public Health. 2023;3: e0001447.

Bouabida K, Lebouché B, Pomey M-P. Telehealth and COVID-19 Pandemic: An Overview of the Telehealth Use, Advantages, Challenges, and Opportunities during COVID-19 Pandemic. Healthcare. 2022;10:2293.

Abd-Alrazaq A, Hassan A, Abuelezz I, Ahmed A, Alzubaidi MS, Shah U, et al. Overview of Technologies Implemented During the First Wave of the COVID-19 Pandemic: Scoping Review. J Med Internet Res. 2021;23: e29136.

Favas C, Ansbro É, Eweka E, Agarwal G, Lazo Porras M, Tsiligianni I, et al. Factors Influencing the Implementation of Remote Delivery Strategies for Non-Communicable Disease Care in Low- and Middle-Income Countries: A Narrative Review. Public Health Rev. 2022;0. Available from: https://www.ssph-journal.org/articles/ https://doi.org/10.3389/phrs.2022.1604583/full . Cited 17 Aug 2022.

Slama S, Kim H-J, Roglic G, Boulle P, Hering H, Varghese C, et al. Care of non-communicable diseases in emergencies. The Lancet. 2017;389:326–30.

Remme M, Narasimhan M, Wilson D, Ali M, Vijayasingham L, Ghani F, et al. Self care interventions for sexual and reproductive health and rights: costs, benefits, and financing. BMJ. 2019;365: l1228.

Pan X-F, Yang J, Wen Y, Li N, Chen S, Pan A. Non-Communicable Diseases During the COVID-19 Pandemic and Beyond. Eng Beijing China. 2021;7:899–902.

CAS   Google Scholar  

World Health Organization Regional Office for South-East Asia. Integration of NCD care in emergency response and preparedness. 2018. Available from: https://www.who.int/publications-detail-redirect/9789290226352 . Cited 17 Feb 2024.

Guterres A, Spiegel P. The state of the world’s refugees: adapting health responses to urban environments. JAMA. 2012;308:673–4.

Download references

Acknowledgements

We would like to acknowledge the Global Alliance for Chronic Disease Humanitarian Working Group, Dr James Smith, and our Advisory Board of humanitarian and United Nations actors: Dr Philippa Boulle, Dr Sigiriya Aebischer Perone, Dr Lilian Kiapi, Dr Mike Woodman, and Dr Slim Slama.

EA, CF, AM, PP received LSHTM salary support from Novo Nordisk AAS for this study. The co-authors received no specific funding for their role in the study. The funder had no role in the conceptualization, design, data collection, analysis, decision to publish, or preparation of the manuscript.

Author information

Authors and affiliations.

Department of Epidemiology of Noncommunicable Diseases, Epidemiology & Population Health, London School of Hygiene & Tropical Medicine, London, UK

Éimhín Ansbro, Lavanya Vijayasingham, Caroline Favas, Leah Sanga & Pablo Perel

Centre for Global Chronic Conditions, London, School of Hygiene & Tropical Medicine , London, UK

Éimhín Ansbro, Lavanya Vijayasingham, Caroline Favas, Jacqueline Rintjema, Leah Sanga, Adrianna Murphy & Pablo Perel

Service de Médecine Tropicale Et Humanitaire, Hôpitaux Universitaires de Genève, Geneva, Switzerland

Olivia Heller

Faculty of Law, University of Toronto, Toronto, Canada

Jacqueline Rintjema

Global Alliance for Chronic Diseases, Wellcome, London, UK

Alyssa Chase-Vilchez

Help Age International, Yangon, Myanmar

Claire Stein

Faculty of Economics and Business, University of Groningen, Groningen, The Netherlands

School of Global Development, University of East Anglia, Norwich, UK

T.H. Chan School of Public Health, FXB Center for Health and Human Rights, Harvard University, Boston, USA

Department of Health Services Research and Policy, Public Health & Policy, London School of Hygiene & Tropical Medicine, London, UK

Adrianna Murphy

You can also search for this author in PubMed   Google Scholar

Contributions

EA, CF, AM and PP conceived of and designed the study. EA, CS, ACV, RI collected data. CS, ACV, OH, LV, EA, LS analysed data. EA, OH, LV drafted the manuscript and all authors reviewed drafts.

Corresponding author

Correspondence to Éimhín Ansbro .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval to conduct this study was obtained from the LSHTM Ethical Review Committee (ID 22825). Details of the study focus were shared with participants prior to the survey and interviews. Their written informed consent was obtained before data collection commenced. Participants’ identifying information or the organisations they represent have not been included to ensure their anonymity.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., supplementary material 4., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Ansbro, É., Heller, O., Vijayasingham, L. et al. Lessons from the COVID-19 pandemic to strengthen NCD care and policy in humanitarian settings: a mixed methods study exploring humanitarian actors’ experiences. BMC Health Serv Res 24 , 1081 (2024). https://doi.org/10.1186/s12913-024-11458-2

Download citation

Received : 29 March 2024

Accepted : 19 August 2024

Published : 17 September 2024

DOI : https://doi.org/10.1186/s12913-024-11458-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Humanitarian
• Noncommunicable
• Implementation
• Service delivery

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

For Researchers

Celiac Disease Foundation

• Sign Up for Email Updates!

About Celiac Disease

About the disease.

• What is Celiac Disease?
• Symptoms of Celiac Disease
• Testing & Diagnosis
• Treatment & Follow-Up
• Celiac Disease in Children
• Related Conditions & Mental Health
• What You Need to Know Video Series
• Symptoms Assessment Tool
• Virtual Celiac and Nutrition Education Classes
• Find a Healthcare Practitioner

Gluten-Free Resources

• What is Gluten?
• Label Reading & the FDA
• Eat! Gluten-Free and Recipes
• Gluten-Free Candy List
• Patient Brochures
• Celiac Disease Centers
• Raising Celiac Podcast

Advancing Research

Research resources.

• Join the iCureCeliac Patient Registry
• Why Enroll in a Celiac Disease Trial
• Qualify for a Celiac Disease Trial
• Future Therapies for Celiac Disease
• GlutenDetect Gluten Exposure Test Kit
• Research News
• Join the iCureCeliac Registry
• Qualify for a Trial
• Apply for a Research Award
• Research Awards and Prizes
• NASPGHAN Diagnosis Tool
• Patient Recruitment Services
• Access iCureCeliac Registry Data
• NIH Funding for Celiac Disease Research
• Department of Defense Funding for Celiac Disease Research
• Join the CDF Research Working Group

Get Involved

• Become a Policy Advocate
• Public Policy Priorities
• Contact Your Members of Congress
• Federal Benefits
• Celiac Disease Management in Schools Training Program
• Advocacy News
• Register for the Turkey Trot

Join the Effort

• Ways To Give
• Join Team Gluten-Free
• Social Media Toolkit
• Get Phone Advice
• Sign Up For Email Updates

About the Foundation

• Mission & Purpose
• Leaders, Advisors & Staff
• Strategic Plan
• Financials & Annuals
• Foundation Press
• Contact Information

Marilyn's Message

• Make a Donation
• Become a Research Partner
• Become a Proud Sponsor

Support the Foundation

• Gifts of Stock and Securities
• Employer Matching
• Other Ways to Give
• Proud Sponsors and Partners
• Team Gluten-Free Shop
• Researchers identify how biomarkers may help predict celiac disease in patients with type 1 diabetes

Celiac disease prevalence in children with T1DM is approximately 5-8%, which is significantly higher than the general population. Because these conditions frequently co-occur, screening for celiac disease is recommended in children with T1DM, even if in the absence of symptoms. Current celiac disease screening methods include blood tests like TTG IgA. However, TTG IgA levels can sometimes be falsely elevated in children with T1DM, leading to unnecessary endoscopy procedures.

Celiac disease and type 1 diabetes are both autoimmune disorders, and they share a genetic predisposition. In individuals with type 1 diabetes, the immune system attacks insulin-producing cells in the pancreas, while in celiac disease, the immune system targets the small intestine in response to gluten. Both conditions are linked to specific genes, particularly HLA-DQ2 and HLA-DQ8, which increase susceptibility to these autoimmune diseases. This genetic overlap explains why children with type 1 diabetes are at higher risk for developing celiac disease. Managing both conditions requires careful attention to diet and regular screening to prevent complications, making early detection essential for optimal care.

In the new study, led by Dr. Danny Mallon, researchers aimed to identify laboratory markers to more accurately predict celiac disease in pediatric T1DM patients. They found that elevated TTG IgA levels (>10x the upper limit of normal) and positive endomysial antibody (EMA) tests were strong predictors of celiac disease in this pediatric cohort. They concluded that while symptoms and other factors are inconsistent predictors of celiac disease in children with T1DM, elevated TTG IgA and positive EMA results may be more reliable markers. Additional research in this area is needed to establish serological thresholds for celiac disease diagnosis in T1DM patients.

The findings highlight the importance of proactive celiac disease screening in children with T1DM. Elevated TTG IgA levels should prompt further testing for celiac disease to ensure early diagnosis and better management of both conditions.

Study Authors : Jessica Rutsky, Andrew Krueger, Qin Sun, Lin Fei, and Daniel Mallon of Cincinnati Children’s Hospital

Link to study

Search the blog

Recent posts.

• Download Your CEU Certificate: Supporting Families With Picky Eaters
• New Wheat Discovery Could Reduce the Impact of Celiac Disease
• Adriana Carrieri's Rising Villus Award for Celiac College Research

Search the archive

• Celiac Disease in the News
• Clinical Trials
• Eat! Gluten-Free
• iCureCeliac
• Marilyn’s Message
• Monthly eNewsletter
• Patient Profile Advocacy Series
• Research Press
• Trending Topics

Sign up for our mailing list