qualitative and quantitative research instruments

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is Quantitative Research? | Definition, Uses & Methods

What Is Quantitative Research? | Definition, Uses & Methods

Published on June 12, 2020 by Pritha Bhandari . Revised on June 22, 2023.

Quantitative research is the process of collecting and analyzing numerical data. It can be used to find patterns and averages, make predictions, test causal relationships, and generalize results to wider populations.

Quantitative research is the opposite of qualitative research , which involves collecting and analyzing non-numerical data (e.g., text, video, or audio).

Quantitative research is widely used in the natural and social sciences: biology, chemistry, psychology, economics, sociology, marketing, etc.

• What is the demographic makeup of Singapore in 2020?
• How has the average temperature changed globally over the last century?
• Does environmental pollution affect the prevalence of honey bees?
• Does working from home increase productivity for people with long commutes?

Table of contents

Quantitative research methods, quantitative data analysis, advantages of quantitative research, disadvantages of quantitative research, other interesting articles, frequently asked questions about quantitative research.

You can use quantitative research methods for descriptive, correlational or experimental research.

• In descriptive research , you simply seek an overall summary of your study variables.
• In correlational research , you investigate relationships between your study variables.
• In experimental research , you systematically examine whether there is a cause-and-effect relationship between variables.

Correlational and experimental research can both be used to formally test hypotheses , or predictions, using statistics. The results may be generalized to broader populations based on the sampling method used.

To collect quantitative data, you will often need to use operational definitions that translate abstract concepts (e.g., mood) into observable and quantifiable measures (e.g., self-ratings of feelings and energy levels).

Note that quantitative research is at risk for certain research biases , including information bias , omitted variable bias , sampling bias , or selection bias . Be sure that you’re aware of potential biases as you collect and analyze your data to prevent them from impacting your work too much.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Once data is collected, you may need to process it before it can be analyzed. For example, survey and test data may need to be transformed from words to numbers. Then, you can use statistical analysis to answer your research questions .

Descriptive statistics will give you a summary of your data and include measures of averages and variability. You can also use graphs, scatter plots and frequency tables to visualize your data and check for any trends or outliers.

Using inferential statistics , you can make predictions or generalizations based on your data. You can test your hypothesis or use your sample data to estimate the population parameter .

First, you use descriptive statistics to get a summary of the data. You find the mean (average) and the mode (most frequent rating) of procrastination of the two groups, and plot the data to see if there are any outliers.

You can also assess the reliability and validity of your data collection methods to indicate how consistently and accurately your methods actually measured what you wanted them to.

Quantitative research is often used to standardize data collection and generalize findings . Strengths of this approach include:

• Replication

Repeating the study is possible because of standardized data collection protocols and tangible definitions of abstract concepts.

• Direct comparisons of results

The study can be reproduced in other cultural settings, times or with different groups of participants. Results can be compared statistically.

• Large samples

Data from large samples can be processed and analyzed using reliable and consistent procedures through quantitative data analysis.

• Hypothesis testing

Using formalized and established hypothesis testing procedures means that you have to carefully consider and report your research variables, predictions, data collection and testing methods before coming to a conclusion.

Despite the benefits of quantitative research, it is sometimes inadequate in explaining complex research topics. Its limitations include:

• Superficiality

Using precise and restrictive operational definitions may inadequately represent complex concepts. For example, the concept of mood may be represented with just a number in quantitative research, but explained with elaboration in qualitative research.

• Narrow focus

Predetermined variables and measurement procedures can mean that you ignore other relevant observations.

• Structural bias

Despite standardized procedures, structural biases can still affect quantitative research. Missing data , imprecise measurements or inappropriate sampling methods are biases that can lead to the wrong conclusions.

• Lack of context

Quantitative research often uses unnatural settings like laboratories or fails to consider historical and cultural contexts that may affect data collection and results.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Chi square goodness of fit test
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Inclusion and exclusion criteria

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

Operationalization means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioral avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalize the variables that you want to measure.

Reliability and validity are both about how well a method measures something:

• Reliability refers to the  consistency of a measure (whether the results can be reproduced under the same conditions).
• Validity   refers to the  accuracy of a measure (whether the results really do represent what they are supposed to measure).

If you are doing experimental research, you also have to consider the internal and external validity of your experiment.

Hypothesis testing is a formal procedure for investigating our ideas about the world using statistics. It is used by scientists to test specific predictions, called hypotheses , by calculating how likely it is that a pattern or relationship between variables could have arisen by chance.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 22). What Is Quantitative Research? | Definition, Uses & Methods. Scribbr. Retrieved June 18, 2024, from https://www.scribbr.com/methodology/quantitative-research/

Is this article helpful?

Pritha Bhandari

Other students also liked, descriptive statistics | definitions, types, examples, inferential statistics | an easy introduction & examples, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Best Family Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Guided Meditations
• Verywell Mind Insights
• 2024 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

Quantitative vs. Qualitative Research in Psychology

Anabelle Bernard Fournier is a researcher of sexual and reproductive health at the University of Victoria as well as a freelance writer on various health topics.

Emily is a board-certified science editor who has worked with top digital publishing brands like Voices for Biodiversity, Study.com, GoodTherapy, Vox, and Verywell.

• Key Differences

Quantitative Research Methods

Qualitative research methods.

• How They Relate

In psychology and other social sciences, researchers are faced with an unresolved question: Can we measure concepts like love or racism the same way we can measure temperature or the weight of a star? Social phenomena⁠—things that happen because of and through human behavior⁠—are especially difficult to grasp with typical scientific models.

At a Glance

Psychologists rely on quantitative and quantitative research to better understand human thought and behavior.

• Qualitative research involves collecting and evaluating non-numerical data in order to understand concepts or subjective opinions.
• Quantitative research involves collecting and evaluating numerical data. 

This article discusses what qualitative and quantitative research are, how they are different, and how they are used in psychology research.

Qualitative Research vs. Quantitative Research

In order to understand qualitative and quantitative psychology research, it can be helpful to look at the methods that are used and when each type is most appropriate.

Psychologists rely on a few methods to measure behavior, attitudes, and feelings. These include:

• Self-reports , like surveys or questionnaires
• Observation (often used in experiments or fieldwork)
• Implicit attitude tests that measure timing in responding to prompts

Most of these are quantitative methods. The result is a number that can be used to assess differences between groups.

However, most of these methods are static, inflexible (you can't change a question because a participant doesn't understand it), and provide a "what" answer rather than a "why" answer.

Sometimes, researchers are more interested in the "why" and the "how." That's where qualitative methods come in.

Qualitative research is about speaking to people directly and hearing their words. It is grounded in the philosophy that the social world is ultimately unmeasurable, that no measure is truly ever "objective," and that how humans make meaning is just as important as how much they score on a standardized test.

Used to develop theories

Takes a broad, complex approach

Answers "why" and "how" questions

Explores patterns and themes

Used to test theories

Takes a narrow, specific approach

Answers "what" questions

Explores statistical relationships

Quantitative methods have existed ever since people have been able to count things. But it is only with the positivist philosophy of Auguste Comte (which maintains that factual knowledge obtained by observation is trustworthy) that it became a "scientific method."

The scientific method follows this general process. A researcher must:

• Generate a theory or hypothesis (i.e., predict what might happen in an experiment) and determine the variables needed to answer their question
• Develop instruments to measure the phenomenon (such as a survey, a thermometer, etc.)
• Develop experiments to manipulate the variables
• Collect empirical (measured) data
• Analyze data

Quantitative methods are about measuring phenomena, not explaining them.

Quantitative research compares two groups of people. There are all sorts of variables you could measure, and many kinds of experiments to run using quantitative methods.

These comparisons are generally explained using graphs, pie charts, and other visual representations that give the researcher a sense of how the various data points relate to one another.

Basic Assumptions

Quantitative methods assume:

• That the world is measurable
• That humans can observe objectively
• That we can know things for certain about the world from observation

In some fields, these assumptions hold true. Whether you measure the size of the sun 2000 years ago or now, it will always be the same. But when it comes to human behavior, it is not so simple.

As decades of cultural and social research have shown, people behave differently (and even think differently) based on historical context, cultural context, social context, and even identity-based contexts like gender , social class, or sexual orientation .

Therefore, quantitative methods applied to human behavior (as used in psychology and some areas of sociology) should always be rooted in their particular context. In other words: there are no, or very few, human universals.

Statistical information is the primary form of quantitative data used in human and social quantitative research. Statistics provide lots of information about tendencies across large groups of people, but they can never describe every case or every experience. In other words, there are always outliers.

Correlation and Causation

A basic principle of statistics is that correlation is not causation. Researchers can only claim a cause-and-effect relationship under certain conditions:

• The study was a true experiment.
• The independent variable can be manipulated (for example, researchers cannot manipulate gender, but they can change the primer a study subject sees, such as a picture of nature or of a building).
• The dependent variable can be measured through a ratio or a scale.

So when you read a report that "gender was linked to" something (like a behavior or an attitude), remember that gender is NOT a cause of the behavior or attitude. There is an apparent relationship, but the true cause of the difference is hidden.

Pitfalls of Quantitative Research

Quantitative methods are one way to approach the measurement and understanding of human and social phenomena. But what's missing from this picture?

As noted above, statistics do not tell us about personal, individual experiences and meanings. While surveys can give a general idea, respondents have to choose between only a few responses. This can make it difficult to understand the subtleties of different experiences.

Quantitative methods can be helpful when making objective comparisons between groups or when looking for relationships between variables. They can be analyzed statistically, which can be helpful when looking for patterns and relationships.

Qualitative data are not made out of numbers but rather of descriptions, metaphors, symbols, quotes, analysis, concepts, and characteristics. This approach uses interviews, written texts, art, photos, and other materials to make sense of human experiences and to understand what these experiences mean to people.

While quantitative methods ask "what" and "how much," qualitative methods ask "why" and "how."

Qualitative methods are about describing and analyzing phenomena from a human perspective. There are many different philosophical views on qualitative methods, but in general, they agree that some questions are too complex or impossible to answer with standardized instruments.

These methods also accept that it is impossible to be completely objective in observing phenomena. Researchers have their own thoughts, attitudes, experiences, and beliefs, and these always color how people interpret results.

Qualitative Approaches

There are many different approaches to qualitative research, with their own philosophical bases. Different approaches are best for different kinds of projects. For example:

• Case studies and narrative studies are best for single individuals. These involve studying every aspect of a person's life in great depth.
• Phenomenology aims to explain experiences. This type of work aims to describe and explore different events as they are consciously and subjectively experienced.
• Grounded theory develops models and describes processes. This approach allows researchers to construct a theory based on data that is collected, analyzed, and compared to reach new discoveries.
• Ethnography describes cultural groups. In this approach, researchers immerse themselves in a community or group in order to observe behavior.

Qualitative researchers must be aware of several different methods and know each thoroughly enough to produce valuable research.

Some researchers specialize in a single method, but others specialize in a topic or content area and use many different methods to explore the topic, providing different information and a variety of points of view.

There is not a single model or method that can be used for every qualitative project. Depending on the research question, the people participating, and the kind of information they want to produce, researchers will choose the appropriate approach.

Interpretation

Qualitative research does not look into causal relationships between variables, but rather into themes, values, interpretations, and meanings. As a rule, then, qualitative research is not generalizable (cannot be applied to people outside the research participants).

The insights gained from qualitative research can extend to other groups with proper attention to specific historical and social contexts.

Relationship Between Qualitative and Quantitative Research

It might sound like quantitative and qualitative research do not play well together. They have different philosophies, different data, and different outputs. However, this could not be further from the truth.

These two general methods complement each other. By using both, researchers can gain a fuller, more comprehensive understanding of a phenomenon.

For example, a psychologist wanting to develop a new survey instrument about sexuality might and ask a few dozen people questions about their sexual experiences (this is qualitative research). This gives the researcher some information to begin developing questions for their survey (which is a quantitative method).

After the survey, the same or other researchers might want to dig deeper into issues brought up by its data. Follow-up questions like "how does it feel when...?" or "what does this mean to you?" or "how did you experience this?" can only be answered by qualitative research.

By using both quantitative and qualitative data, researchers have a more holistic, well-rounded understanding of a particular topic or phenomenon.

Qualitative and quantitative methods both play an important role in psychology. Where quantitative methods can help answer questions about what is happening in a group and to what degree, qualitative methods can dig deeper into the reasons behind why it is happening. By using both strategies, psychology researchers can learn more about human thought and behavior.

Gough B, Madill A. Subjectivity in psychological science: From problem to prospect . Psychol Methods . 2012;17(3):374-384. doi:10.1037/a0029313

Pearce T. “Science organized”: Positivism and the metaphysical club, 1865–1875 . J Hist Ideas . 2015;76(3):441-465.

Adams G. Context in person, person in context: A cultural psychology approach to social-personality psychology . In: Deaux K, Snyder M, eds. The Oxford Handbook of Personality and Social Psychology . Oxford University Press; 2012:182-208.

Brady HE. Causation and explanation in social science . In: Goodin RE, ed. The Oxford Handbook of Political Science. Oxford University Press; 2011. doi:10.1093/oxfordhb/9780199604456.013.0049

Chun Tie Y, Birks M, Francis K. Grounded theory research: A design framework for novice researchers .  SAGE Open Med . 2019;7:2050312118822927. doi:10.1177/2050312118822927

Reeves S, Peller J, Goldman J, Kitto S. Ethnography in qualitative educational research: AMEE Guide No. 80 . Medical Teacher . 2013;35(8):e1365-e1379. doi:10.3109/0142159X.2013.804977

Salkind NJ, ed. Encyclopedia of Research Design . Sage Publishing.

Shaughnessy JJ, Zechmeister EB, Zechmeister JS.  Research Methods in Psychology . McGraw Hill Education.

By Anabelle Bernard Fournier Anabelle Bernard Fournier is a researcher of sexual and reproductive health at the University of Victoria as well as a freelance writer on various health topics.

Qualitative vs. quantitative data in research: what's the difference?

If you're reading this, you likely already know the importance of data analysis. And you already know it can be incredibly complex.

At its simplest, research and it's data can be broken down into two different categories: quantitative and qualitative. But what's the difference between each? And when should you use them? And how can you use them together?

Understanding the differences between qualitative and quantitative data is key to any research project. Knowing both approaches can help you in understanding your data better—and ultimately understand your customers better. Quick takeaways:

Quantitative research uses objective, numerical data to answer questions like "what" and "how often." Conversely, qualitative research seeks to answer questions like "why" and "how," focusing on subjective experiences to understand motivations and reasons.

Quantitative data is collected through methods like surveys and experiments and analyzed statistically to identify patterns. Qualitative data is gathered through interviews or observations and analyzed by categorizing information to understand themes and insights.

Effective data analysis combines quantitative data for measurable insights with qualitative data for contextual depth.

What is quantitative data?

Qualitative and quantitative data differ in their approach and the type of data they collect.

Quantitative data refers to any information that can be quantified — that is, numbers. If it can be counted or measured, and given a numerical value, it's quantitative in nature. Think of it as a measuring stick.

Quantitative variables can tell you "how many," "how much," or "how often."

Some examples of quantitative data :  

How many people attended last week's webinar? 

How much revenue did our company make last year? 

How often does a customer rage click on this app?

To analyze these research questions and make sense of this quantitative data, you’d normally use a form of statistical analysis —collecting, evaluating, and presenting large amounts of data to discover patterns and trends. Quantitative data is conducive to this type of analysis because it’s numeric and easier to analyze mathematically.

Computers now rule statistical analytics, even though traditional methods have been used for years. But today’s data volumes make statistics more valuable and useful than ever. When you think of statistical analysis now, you think of powerful computers and algorithms that fuel many of the software tools you use today.

Popular quantitative data collection methods are surveys, experiments, polls, and more.

Quantitative Data 101: What is quantitative data?

Take a deeper dive into what quantitative data is, how it works, how to analyze it, collect it, use it, and more.

Learn more about quantitative data →

What is qualitative data?

Unlike quantitative data, qualitative data is descriptive, expressed in terms of language rather than numerical values.

Qualitative data analysis describes information and cannot be measured or counted. It refers to the words or labels used to describe certain characteristics or traits.

You would turn to qualitative data to answer the "why?" or "how?" questions. It is often used to investigate open-ended studies, allowing participants (or customers) to show their true feelings and actions without guidance.

Some examples of qualitative data:

Why do people prefer using one product over another?

How do customers feel about their customer service experience?

What do people think about a new feature in the app?

Think of qualitative data as the type of data you'd get if you were to ask someone why they did something. Popular data collection methods are in-depth interviews, focus groups, or observation.

Start growing with data and Fullstory.

Request your personalized demo of the Fullstory behavioral data platform.

What are the differences between qualitative vs. quantitative data?

When it comes to conducting data research, you’ll need different collection, hypotheses and analysis methods, so it’s important to understand the key differences between quantitative and qualitative data:

Quantitative data is numbers-based, countable, or measurable. Qualitative data is interpretation-based, descriptive, and relating to language.

Quantitative data tells us how many, how much, or how often in calculations. Qualitative data can help us to understand why, how, or what happened behind certain behaviors .

Quantitative data is fixed and universal. Qualitative data is subjective and unique.

Quantitative research methods are measuring and counting. Qualitative research methods are interviewing and observing.

Quantitative data is analyzed using statistical analysis. Qualitative data is analyzed by grouping the data into categories and themes.

As you can see, both provide immense value for any data collection and are key to truly finding answers and patterns. 

More examples of quantitative and qualitative data

You’ve most likely run into quantitative and qualitative data today, alone. For the visual learner, here are some examples of both quantitative and qualitative data: 

Quantitative data example

The customer has clicked on the button 13 times. 

The engineer has resolved 34 support tickets today. 

The team has completed 7 upgrades this month. 

14 cartons of eggs were purchased this month.

Qualitative data example

My manager has curly brown hair and blue eyes.

My coworker is funny, loud, and a good listener. 

The customer has a very friendly face and a contagious laugh.

The eggs were delicious.

The fundamental difference is that one type of data answers primal basics and one answers descriptively. 

What does this mean for data quality and analysis? If you just analyzed quantitative data, you’d be missing core reasons behind what makes a data collection meaningful. You need both in order to truly learn from data—and truly learn from your customers. 

What are the advantages and disadvantages of each?

Both types of data has their own pros and cons. 

Advantages of quantitative data

It’s relatively quick and easy to collect and it’s easier to draw conclusions from. 

When you collect quantitative data, the type of results will tell you which statistical tests are appropriate to use. 

As a result, interpreting your data and presenting those findings is straightforward and less open to error and subjectivity.

Another advantage is that you can replicate it. Replicating a study is possible because your data collection is measurable and tangible for further applications.

Disadvantages of quantitative data

Quantitative data doesn’t always tell you the full story (no matter what the perspective). 

With choppy information, it can be inconclusive.

Quantitative research can be limited, which can lead to overlooking broader themes and relationships.

By focusing solely on numbers, there is a risk of missing larger focus information that can be beneficial.

Advantages of qualitative data

Qualitative data offers rich, in-depth insights and allows you to explore context.

It’s great for exploratory purposes.

Qualitative research delivers a predictive element for continuous data.

Disadvantages of qualitative data

It’s not a statistically representative form of data collection because it relies upon the experience of the host (who can lose data).

It can also require multiple data sessions, which can lead to misleading conclusions.

The takeaway is that it’s tough to conduct a successful data analysis without both. They both have their advantages and disadvantages and, in a way, they complement each other. 

Now, of course, in order to analyze both types of data, information has to be collected first.

Let's get into the research.

Quantitative and qualitative research

The core difference between qualitative and quantitative research lies in their focus and methods of data collection and analysis. This distinction guides researchers in choosing an appropriate approach based on their specific research needs.

Using mixed methods of both can also help provide insights form combined qualitative and quantitative data.

Best practices of each help to look at the information under a broader lens to get a unique perspective. Using both methods is helpful because they collect rich and reliable data, which can be further tested and replicated.

What is quantitative research?

Quantitative research is based on the collection and interpretation of numeric data. It's all about the numbers and focuses on measuring (using inferential statistics ) and generalizing results. Quantitative research seeks to collect numerical data that can be transformed into usable statistics.

It relies on measurable data to formulate facts and uncover patterns in research. By employing statistical methods to analyze the data, it provides a broad overview that can be generalized to larger populations.

In terms of digital experience data, it puts everything in terms of numbers (or discrete data )—like the number of users clicking a button, bounce rates , time on site, and more. 

Some examples of quantitative research: 

What is the amount of money invested into this service?

What is the average number of times a button was dead clicked ?

How many customers are actually clicking this button?

Essentially, quantitative research is an easy way to see what’s going on at a 20,000-foot view. 

Each data set (or customer action, if we’re still talking digital experience) has a numerical value associated with it and is quantifiable information that can be used for calculating statistical analysis so that decisions can be made. 

You can use statistical operations to discover feedback patterns (with any representative sample size) in the data under examination. The results can be used to make predictions , find averages, test causes and effects, and generalize results to larger measurable data pools. 

Unlike qualitative methodology, quantitative research offers more objective findings as they are based on more reliable numeric data.

Quantitative data collection methods

A survey is one of the most common research methods with quantitative data that involves questioning a large group of people. Questions are usually closed-ended and are the same for all participants. An unclear questionnaire can lead to distorted research outcomes.

Similar to surveys, polls yield quantitative data. That is, you poll a number of people and apply a numeric value to how many people responded with each answer.

Experiments

An experiment is another common method that usually involves a control group and an experimental group . The experiment is controlled and the conditions can be manipulated accordingly. You can examine any type of records involved if they pertain to the experiment, so the data is extensive. 

What is qualitative research?

Qualitative research does not simply help to collect data. It gives a chance to understand the trends and meanings of natural actions. It’s flexible and iterative.

Qualitative research focuses on the qualities of users—the actions that drive the numbers. It's descriptive research. The qualitative approach is subjective, too. 

It focuses on describing an action, rather than measuring it.

Some examples of qualitative research: 

The sunflowers had a fresh smell that filled the office.

All the bagels with bites taken out of them had cream cheese.

The man had blonde hair with a blue hat.

Qualitative research utilizes interviews, focus groups, and observations to gather in-depth insights.

This approach shines when the research objective calls for exploring ideas or uncovering deep insights rather than quantifying elements.

Qualitative data collection methods

An interview is the most common qualitative research method. This method involves personal interaction (either in real life or virtually) with a participant. It’s mostly used for exploring attitudes and opinions regarding certain issues.

Interviews are very popular methods for collecting data in product design .

Focus groups

Data analysis by focus group is another method where participants are guided by a host to collect data. Within a group (either in person or online), each member shares their opinion and experiences on a specific topic, allowing researchers to gather perspectives and deepen their understanding of the subject matter.

Digital Leadership Webinar: Accelerating Growth with Quantitative Data and Analytics

Learn how the best-of-the-best are connecting quantitative data and experience to accelerate growth.

So which type of data is better for data analysis?

So how do you determine which type is better for data analysis ?

Quantitative data is structured and accountable. This type of data is formatted in a way so it can be organized, arranged, and searchable. Think about this data as numbers and values found in spreadsheets—after all, you would trust an Excel formula.

Qualitative data is considered unstructured. This type of data is formatted (and known for) being subjective, individualized, and personalized. Anything goes. Because of this, qualitative data is inferior if it’s the only data in the study. However, it’s still valuable. 

Because quantitative data is more concrete, it’s generally preferred for data analysis. Numbers don’t lie. But for complete statistical analysis, using both qualitative and quantitative yields the best results. 

At Fullstory, we understand the importance of data, which is why we created a behavioral data platform that analyzes customer data for better insights. Our platform delivers a complete, retroactive view of how people interact with your site or app—and analyzes every point of user interaction so you can scale.

Unlock business-critical data with Fullstory

A perfect digital customer experience is often the difference between company growth and failure. And the first step toward building that experience is quantifying who your customers are, what they want, and how to provide them what they need.

Access to product analytics is the most efficient and reliable way to collect valuable quantitative data about funnel analysis, customer journey maps , user segments, and more.

But creating a perfect digital experience means you need organized and digestible quantitative data—but also access to qualitative data. Understanding the why is just as important as the what itself.

Fullstory's DXI platform combines the quantitative insights of product analytics with picture-perfect session replay for complete context that helps you answer questions, understand issues, and uncover customer opportunities.

Start a free 14-day trial to see how Fullstory can help you combine your most invaluable quantitative and qualitative insights and eliminate blind spots.

About the author

Our team of experts is committed to introducing people to important topics surrounding analytics, digital experience intelligence, product development, and more.

Related posts

Quantitative data is used for calculations or obtaining numerical results. Learn about the different types of quantitative data uses cases and more.

Discover how data discovery transforms raw data into actionable insights for informed decisions, improved strategies, and better customer experiences.

Learn the 3 key benefits democratized data can achieve, and 3 of the most pertinent dangers of keeping data (and teams) siloed.

Learn the essentials of behavioral data and its transformative impact on customer experience. Our comprehensive guide provides the tools and knowledge to harness this power effectively.

Discover how Fullstory leverages AI to turn raw data into actionable insights, transforming user experiences and driving business growth.

Discover how just-in-time data, explained by Lane Greer, enhances customer insights and decision-making beyond real-time analytics.

Quantitative and Qualitative Research

• I NEED TO . . .

What is Quantitative Research?

• What is Qualitative Research?
• Quantitative vs Qualitative
• Step 1: Accessing CINAHL
• Step 2: Create a Keyword Search
• Step 3: Create a Subject Heading Search
• Step 4: Repeat Steps 1-3 for Second Concept
• Step 5: Repeat Steps 1-3 for Quantitative Terms
• Step 6: Combining All Searches
• Step 7: Adding Limiters
• Step 8: Save Your Search!
• What Kind of Article is This?
• More Research Help This link opens in a new window

Quantitative methodology is the dominant research framework in the social sciences. It refers to a set of strategies, techniques and assumptions used to study psychological, social and economic processes through the exploration of numeric patterns . Quantitative research gathers a range of numeric data. Some of the numeric data is intrinsically quantitative (e.g. personal income), while in other cases the numeric structure is  imposed (e.g. ‘On a scale from 1 to 10, how depressed did you feel last week?’). The collection of quantitative information allows researchers to conduct simple to extremely sophisticated statistical analyses that aggregate the data (e.g. averages, percentages), show relationships among the data (e.g. ‘Students with lower grade point averages tend to score lower on a depression scale’) or compare across aggregated data (e.g. the USA has a higher gross domestic product than Spain). Quantitative research includes methodologies such as questionnaires, structured observations or experiments and stands in contrast to qualitative research. Qualitative research involves the collection and analysis of narratives and/or open-ended observations through methodologies such as interviews, focus groups or ethnographies.

Coghlan, D., Brydon-Miller, M. (2014).  The SAGE encyclopedia of action research  (Vols. 1-2). London, : SAGE Publications Ltd doi: 10.4135/9781446294406

What is the purpose of quantitative research?

The purpose of quantitative research is to generate knowledge and create understanding about the social world. Quantitative research is used by social scientists, including communication researchers, to observe phenomena or occurrences affecting individuals. Social scientists are concerned with the study of people. Quantitative research is a way to learn about a particular group of people, known as a sample population. Using scientific inquiry, quantitative research relies on data that are observed or measured to examine questions about the sample population.

Allen, M. (2017).  The SAGE encyclopedia of communication research methods  (Vols. 1-4). Thousand Oaks, CA: SAGE Publications, Inc doi: 10.4135/9781483381411

How do I know if the study is a quantitative design?  What type of quantitative study is it?

Quantitative Research Designs: Descriptive non-experimental, Quasi-experimental or Experimental?

Studies do not always explicitly state what kind of research design is being used.  You will need to know how to decipher which design type is used.  The following video will help you determine the quantitative design type.

• << Previous: I NEED TO . . .
• Next: What is Qualitative Research? >>
• Last Updated: Jun 12, 2024 8:56 AM
• URL: https://libguides.uta.edu/quantitative_and_qualitative_research

University of Texas Arlington Libraries 702 Planetarium Place · Arlington, TX 76019 · 817-272-3000

• Internet Privacy
• Accessibility
• Problems with a guide? Contact Us.

Root out friction in every digital experience, super-charge conversion rates, and optimise digital self-service

Uncover insights from any interaction, deliver AI-powered agent coaching, and reduce cost to serve

Increase revenue and loyalty with real-time insights and recommendations delivered straight to teams on the ground

Know how your people feel and empower managers to improve employee engagement, productivity, and retention

Take action in the moments that matter most along the employee journey and drive bottom line growth

Whatever they’re are saying, wherever they’re saying it, know exactly what’s going on with your people

Get faster, richer insights with qual and quant tools that make powerful market research available to everyone

Run concept tests, pricing studies, prototyping + more with fast, powerful studies designed by UX research experts

Track your brand performance 24/7 and act quickly to respond to opportunities and challenges in your market

Meet the operating system for experience management

• Free Account
• Product Demos
• For Digital
• For Customer Care
• For Human Resources
• For Researchers
• Financial Services
• All Industries

Popular Use Cases

• Customer Experience
• Employee Experience
• Employee Exit Interviews
• Net Promoter Score
• Voice of Customer
• Customer Success Hub
• Product Documentation
• Training & Certification
• XM Institute
• Popular Resources
• Customer Stories
• Artificial Intelligence

Market Research

• Partnerships
• Marketplace

The annual gathering of the experience leaders at the world’s iconic brands building breakthrough business results.

• English/AU & NZ
• Español/Europa
• Español/América Latina
• Português Brasileiro
• REQUEST DEMO
• Experience Management
• Ultimate Guide to Market Research
• Quantitative Research

Try Qualtrics for free

Your ultimate guide to quantitative research.

10 min read You may be already using quantitative research and want to check your understanding, or you may be starting from the beginning. Here’s an exploration of this research method and how you can best use it for maximum effect for your business.

You may be already using quantitative research and want to check your understanding, or you may be starting from the beginning. Here’s an exploration of this research method and how you can best use it for maximum effect for your business.

What is quantitative research?

Quantitative is the research method of collecting quantitative data – this is data that can be converted into numbers or numerical data, which can be easily quantified, compared, and analysed.

Quantitative research deals with primary and secondary sources where data is represented in numerical form. This can include closed-question poll results, statistics, and census information or  demographic data .

Quantitative data tends to be used when researchers are interested in understanding a particular moment in time and examining data sets over time to find trends and patterns.

To collect numerical data, surveys are often employed as one of the main research methods to source first-hand information in  primary research . Qualitative research can also  come from third-party research studies .

Quantitative research is widely used in the realms of social sciences, such as psychology, economics, sociology, and marketing.

Research teams collect data that is significant to proving or disproving a hypothesis research question – known as the research objective. When they collect quantitative data, researchers will  aim to use a sample size that is representative  of the total population of the target market they’re interested in.

Then the data collected will be manually or automatically stored and compared for insights.

Learn how Qualtrics can enhance & simplify the quantitative research process

Qualitative vs quantitative research

While the quantitative research definition focuses on numerical data, qualitative research is defined as data that supplies non-numerical information.

Qualitative research focuses on the thoughts, feelings, and values of a participant, to understand why people act in the way they do. They result in data types like quotes, symbols, images, and written testimonials.

These data types tell researchers subjective information, which can help us assign people into categories, such as a participant’s religion, gender, social class, political alignment, likely favoured products to buy, or their preferred training learning style.

For this reason, qualitative research is often used in social research, as this gives a window into the behaviour and actions of people.

In general, if you’re interested in measuring something or testing a hypothesis, use quantitative methods. If you want to explore ideas, thoughts, and meanings, use qualitative methods.

However, quantitative and qualitative research methods are both recommended when you’re looking to understand a point in time, while also finding out the reason behind the facts.

Quantitative research data collection methods

Quantitative research methods can use structured research instruments like:

A survey is a simple-to-create and easy-to-distribute research method, which helps gather information from large groups of participants quickly. Traditionally, paper-based surveys can now be made online, so costs can stay quite low.

Quantitative questions tend to be closed questions that ask for a numerical result, based on a range of options, or a yes/no answer that can be tallied quickly.

Face-to-face or phone interviews

Interviews are a great way to connect with participants , though they require time from the research team to set up and conduct.

Researchers may also have issues connecting with participants in different geographical regions. The researcher uses a set of predefined close-ended questions, which ask for yes/no or numerical values.

Polls can be a shorter version of surveys, used to get a ‘flavour’ of what the current situation is with participants. Online polls can be shared easily, though polls are best used with simple questions that request a range or a yes/no answer.

Quantitative data is the opposite of qualitative research, another dominant framework for research in the social sciences, explored further below.

Quantitative data types

Quantitative research methods often deliver the following data types:

• Test Scores
• Per cent of training course completed
• Performance score out of 100
• Number of support calls active
• Customer Net Promoter Score (NPS)

When gathering numerical data, the emphasis is on how specific the data is, and whether they can provide an indication of what ‘is’ at the time of collection. Pre-existing statistical data can tell us what ‘was’ for the date and time range that it represented.

Quantitative research design methods (with examples)

Quantitative research has a number of quantitative research designs you can choose from:

Descriptive

This design type describes the state of a data type is telling researchers, in its native environment. There won’t normally be a clearly defined research question to start with. Instead,  data analysis will suggest a conclusion, which can become the hypothesis to investigate further.

Examples of descriptive quantitative design include:

• A description of child’s Christmas gifts they received that year
• A description of what businesses sell the most of during Black Friday
• A description of a product issue being experienced by a customer

Correlational

This design type looks at two or more data types, the relationship between them, and the extent that they differ or align. This does not look at the causal links deeper – instead statistical analysis looks at the variables in a natural environment.

Examples of correlational quantitative design include:

• The relationship between a child’s Christmas gifts and their perceived happiness level
• The relationship between a business’ sales during Black Friday and the total revenue generated over the year
• The relationship between a customer’s product issue and the reputation of the product

Causal-Comparative/Quasi-Experimental

This design type looks at two or more data types and tries to explain any relationship and differences between them, using a cause-effect analysis. The research is carried out in a near-natural environment, where information is gathered from two groups – a naturally occurring group that matches the original natural environment, and one that is not naturally present.

This allows for causal links to be made, though they might not be correct, as other variables may have an impact on results.

Examples of causal-comparative/quasi-experimental quantitative design include:

• The effect of children’s Christmas gifts on happiness
• The effect of Black Friday sales figures on the productivity of company yearly sales
• The effect of product issues on the public perception of a product

Experimental Research

This design type looks to make a controlled environment in which two or more variables are observed to understand the exact cause and effect they have. This becomes a quantitative research study, where data types are manipulated to assess the effect they have. The participants are not naturally occurring groups, as the setting is no longer natural. A quantitative research study can help pinpoint the exact conditions in which variables impact one another.

Examples of experimental quantitative design include:

• The effect of children’s Christmas gifts on a child’s dopamine (happiness) levels
• The effect of Black Friday sales on the success of the company
• The effect of product issues on the perceived reliability of the product

Quantitative research methods need to be carefully considered, as your data collection of a data type can be used to different effects. For example, statistics can be descriptive or correlational (or inferential). Descriptive statistics help us to summarise our data, while inferential statistics help infer conclusions about significant differences.

Advantages of quantitative research

• Easy to do : Doing quantitative research is more straightforward, as the results come in numerical format, which can be more easily interpreted.
• Less interpretation : Due to the factual nature of the results, you will be able to accept or reject your hypothesis based on the numerical data collected.
• Less bias : There are higher levels of control that can be applied to the research, so  bias can be reduced , making your data more reliable and precise.

Disadvantages of quantitative research

• Can’t understand reasons:  Quantitative research doesn’t always tell you the full story, meaning you won’t understand the context – or the why, of the data you see, why do you see the results you have uncovered?
• Useful for simpler situations:  Quantitative research on its own is not great when dealing with complex issues. In these cases, quantitative research may not be enough.

How to use quantitative research to your business’s advantage

Quantitative research methods may help in areas such as:

• Identifying which advert or landing page performs better
• Identifying  how satisfied your customers are
• How many customers are likely to recommend you
• Tracking how your brand ranks in awareness  and customer purchase intent
• Learn what consumers are likely to buy from your brand.

6 steps to conducting good quantitative research

Businesses can benefit from quantitative research by using it to evaluate the impact of data types. There are several steps to this:

• Define your problem or interest area : What do you observe is happening and is it frequent? Identify the data type/s you’re observing.
• Create a hypothesis : Ask yourself what could be the causes for the situation with those data types.
• Plan your quantitative research : Use structured research instruments like surveys or polls to ask questions that test your hypothesis.
• Data Collection : Collect quantitative data and understand what your data types are telling you. Using data collected on different types over long time periods can give you information on patterns.
• Data analysis : Does your information support your hypothesis? (You may need to redo the research with other variables to see if the results improve)
• Effectively present data : Communicate the results in a clear and concise way to help other people understand the findings.

Related resources

Market intelligence 9 min read, qualitative research questions 11 min read, ethnographic research 11 min read, business research methods 12 min read, qualitative research design 12 min read, business research 10 min read, qualitative research interviews 11 min read, request demo.

Ready to learn more about Qualtrics?

• Open access
• Published: 17 June 2024

Qualitative and quantitative assessment of non-clear cell renal cell carcinoma using contrast-enhanced ultrasound

• WeiPing Zhang 1 ,
• JingLing Wang 1 ,
• Li Chen 1 &
• Jiayu Shi 1 , 2  

BMC Urology volume  24 , Article number:  129 ( 2024 ) Cite this article

Metrics details

Non-clear cell renal cell carcinoma (nccRCC) represents a rare form of renal cell carcinoma (RCC) in the clinic. It is now understood that contrast-enhanced ultrasound (CEUS) exhibits diverse manifestations and can be prone to misdiagnosis. Therefore, summarizing the distinctive features of contrast-enhanced ultrasonography is essential for differentiation from ccRCC.

This study aims to evaluate the diagnostic efficacy of qualitative and quantitative CEUS in diagnosing nccRCC to enhance our understanding of this condition.

We conducted a retrospective analysis of 21 patients with confirmed nccRCC following surgery and assessed the characteristic conventional ultrasound and CEUS imaging features. The paired Wilcoxon signed-rank sum test was employed to compare differences in CEUS time-intensity curve (TIC) parameters between the lesions and the normal renal cortex.

Routine ultrasound revealed the following primary characteristics in the 21 nccRCC cases: hypoechoic appearance (10/21, 47.6%), absence of liquefaction (18/21, 66.7%), regular shape (19/21, 90.5%), clear boundaries (21/21, 100%), and absence of calcification (17/21, 81%). Color Doppler flow imaging (CDFI) indicated a low blood flow signal (only 1 case of grade III). Qualitative CEUS analysis demonstrated that nccRCC predominantly exhibited slow progression (76.1%), fast washout (57%), uniformity (61.9%), low enhancement (71.5%), and ring enhancement (61.9%). Quantitative CEUS analysis revealed that parameters such as PE, WiAUC, mTTI, WiR, WiPI, WoAUC, WiWoAUC, and WOR in the lesions were significantly lower than those in the normal renal cortex ( Z =-3.980, -3.563, -2.427, -3.389, -3.980, -3.493, -3.528, -2.763, P  < 0.001, < 0.001, = 0.015, = 0.001, < 0.001, < 0.001, < 0.001, = 0.006). However, there were no significant differences in RT, TTP, FT, or QOF (all P  > 0.05).

nccRCC exhibits distinctive CEUS characteristics, including slow progression, fast washout, low homogeneity enhancement, and ring enhancement, which can aid in distinguishing nccRCC from ccRCC.

Peer Review reports

Introduction

Renal cell carcinoma (RCC) stands as one of the most prevalent malignancies in the field of urology. Approximately 70% of RCC cases manifest as clear cell renal cell carcinoma (ccRCC). In addition to ccRCC, there exist several rarer variants of non-clear cell renal cell carcinoma (nccRCC), including papillary renal cell carcinoma (pRCC), chromophobe renal cell carcinoma (ChRCC), translocation cancer, and collecting duct cancer [ 1 , 2 ].

The majority of RCC cases remain asymptomatic, frequently detected incidentally through imaging procedures [ 3 ]. While enhanced computed tomography (CT) has traditionally served as the gold standard for diagnosing renal cell carcinoma, it carries inherent drawbacks, such as ionizing radiation exposure and the potential for hypersensitivity reactions to iodine-based contrast agents, thereby limiting its application [ 4 ]. In contrast, ultrasound, a non-invasive examination, can provide an initial assessment of tumor location, size, shape, and vascularization, making it the preferred method for initial diagnosis [ 5 ]. However, conventional ultrasound often exhibits limitations stemming from restricted two-dimensional resolution and discrepancies in tumor images, significantly impeding its effectiveness in diagnosis, differentiation, prognosis prediction, and evaluating the therapeutic outcomes of RCC.

Contrast-enhanced ultrasound (CEUS) has emerged as a more sensitive alternative to conventional ultrasound. It offers clear visualization of microvascular changes within and around tumors, coupled with quantitative analysis software that generates time-intensity curves for qualitative and quantitative assessments of blood perfusion within the lesion. CEUS brings several advantages, including convenience, affordability, broad applicability, absence of ionizing radiation, negligible risk of iodine contrast agent allergies, and the ability to conduct repeated examinations in quick succession. Notably, CEUS demonstrates diagnostic sensitivity and specificity for RCC akin to that of CT [ 6 , 7 ]. Hence, it has attracted significant interest from clinicians in recent years. ccRCC, characterized by a rich blood supply and susceptibility to hemorrhage, cystic degeneration, and necrosis, typically exhibits a CEUS pattern of “rapid enhancement followed by rapid washout” [ 8 ]. Conversely, the histological characteristics of nccRCC are complex and changeable, and the manifestations of different types of tumors may be different in CEUS. It leads to nccRCC presents diverse ultrasound and CEUS manifestations, which often pose a diagnostic challenge prior to surgical intervention [ 9 ]. Herein, we retrospectively analyzed the CEUS characteristics of 21 surgically and pathologically confirmed nccRCC cases. Importantly, we sought to enhance diagnostic ability for this tumor subtype and offer valuable guidance for clinical decision-making.

Materials and methods

Study patients.

From January 2020 to May 2023, 21 patients with nccRCC confirmed by pathology underwent CEUS in the First Affiliated Hospital of Nanchang University. Patients were included in the present study based on the following criteria: (1) Evidence of renal masses on gray-scale ultrasound with complete conventional ultrasound and CEUS imaging data; (2) No history of renal invasive procedures or other treatments before the ultrasound examination; (3) all patients underwent surgical treatment, and pathologically confirmed as nccRCC; (4) age ≥ 18 years; (5) lesion diameter more than 1 cm; (6) Quality of fit (QOF) > 0.7. Exclusion criteria: (1) nccRCC without enhancement or nodular enhancement of the cyst wall; (2) excluding tumors with poor dynamic image storage and large respiratory range, which could not be quantitatively analyzed in the later stage; (3) CEUS images were uninterpretable due to factors such as the deep location of the tumor, patient obesity, or inadequate ultrasound penetration. The study was approved by the hospital’s ethics committee and the institutional review board (NO: IIT2023174), and informed consent was obtained from every participant.

Instruments

Routine ultrasound and CEUS examinations were conducted using either the Mindray Resona R9 ultrasonic diagnostic instrument (Shenzhen Mairui Biomedical Electronics Co., Ltd, probe model SC5-1U) or the Siemens ACUSON Sequoia ultrasonic diagnostic instrument (Siemens Medical System Co., Ltd, probe model C5-1). Conventional ultrasound was used for the observation of lesion size, location, shape, boundary, internal echo, and blood flow. Subsequently, the optimal section that offered clear visualization of the tumor and surrounding renal parenchyma was selected for CEUS, and the target focus was located in the middle of the image as far as possible. Patients were instructed to maintain calm and slow breathing during the procedure. CEUS was performed using a low mechanical index (MI 0.06–0.08) and SonoVue (SonoVue, 2.5 μm in diameter, Bracco, Italy) contrast medium. Prior to examination, 59 mg of SonoVue was mixed with 5 ml of 0.9% sodium chloride solution to form a suspension. During the imaging procedure, 1.0 ml of the SonoVue suspension was injected through the superficial vein of the elbow, followed by the injection of 5 ml of 0.9% sodium chloride solution to maintain consistency. Dynamic images were captured between 120 s and 180 s. Two experienced ultrasound physicians with over 5 years of expertise in CEUS, blinded to the pathological results, conducted qualitative and quantitative analyses of the images.

Qualitative analysis of CEUS images

The renal CEUS phase was divided into the perfusion phase (0–30 s) and the washout phase (> 30 s). During this analysis, attention was given to enhancement and washout times, peak intensity, enhancement uniformity, enhancement shape, and annular enhancement. Specific CEUS analysis components included: enhancement kinetics (categorized as fast wash-in [tumor enhancement faster than renal cortex], slow wash-in [tumor enhancement slower than cortex], or isoenhancing [tumor and cortex enhance simultaneously]); wash-out timing (fast wash-out [tumor washout faster than cortex], slow wash-out [tumor washout slower than cortex], or isoregressive [tumor and cortex washout concurrently]); peak intensity (high [tumor enhancement higher than cortex], isoenhancing, or low enhancement [tumor enhancement lower than cortex] relative to the renal cortex); post-enhancement morphology (regular or irregular); and enhancement homogeneity (uniform or heterogeneous). Pseudocapsule ring hyperenhancement indicated the presence of a circular hyperenhancement zone around the tumor, with significantly higher enhancement than the normal renal cortex inside and around the tumor.

Quantitative analysis of CEUS images

Dynamic storage images in DICOM format were obtained and analyzed using Vuebox software. A region of interest (ROI) was delineated, with ROI1 representing the total area for image analysis, ROI2 indicating the area of uniform enhancement (in cases of inhomogeneous enhancement, the region with the highest enhancement intensity was selected while avoiding annular enhancement), and ROI3 encompassing the normal renal cortex with uniform enhancement at the same depth. Various time-intensity curve (TIC) parameters were measured, including (1) peak enhancement (PE), representing the intensity of peak enhancement; (2) rise time (RT); (3) mean transit time local (mTTI); (4) time to peak (TTP), signifying the time when the contrast intensity within the mass reached its peak enhancement; (5) fall time (FT); (6) wash in area under the curve (WiAUC), indicating the area under the curve from arrival time to peak enhancement; (7) wash in rate (WiR), defined as the tangent of the rising part of the TIC curve; (8) wash in perfusion index (WiPI), calculated as the ratio of WiAUC to time; (9) wash out rate (WoR), denoting the tangent of the descending part of the TIC curve; (10) wash out area under the curve (WoAUC); (11) wash in and wash out area under the curve (WiWoAUC); and (12) QOF.

Statistical methods

Statistical analysis was performed using SPSS 23.0 statistical software. Age and tumor diameter were assessed for normal distribution and expressed as mean ± s.d. TIC parameters were expressed as median (interquartile range, IQR). Differences in TIC parameters between the lesions and normal renal cortex were compared using the paired Wilcoxon signed-rank sum test for paired samples. Statistical significance was determined by a p -value < 0.05.

Patients and US results

In terms of general patient characteristics and conventional ultrasonography findings, all 21 tumor cases were solitary, comprising lesions located in the left kidney ( n  = 10), the right kidney ( n  = 11), the superior pole ( n  = 6), the inferior pole ( n  = 5), and the middle pole ( n  = 10) of the kidney, and the mean lesion diameter measured (3.55 ± 1.43) cm. The included patients exhibited male predominance ( n  = 16, 76.2%) with a median age of 55 years. Conventional ultrasound observations revealed predominant features such as hypoechoic appearance (10/21, 47.6%), absence of liquefaction (18/21, 66.7%), regular morphology (19/21, 90.5%), clear boundaries (21/21, 100%), absence of calcification (17/21, 81%), and limited color Doppler flow (only 1 case with grade III blood flow) (Fig.  1 ; Table  1 ). All cases were later confirmed as nccRCC through surgical and pathological evaluation at our hospital. This included cases of pRCC ( n  = 8, comprising 4 type I and 4 type II), chRCC ( n  = 5), renal cell carcinoma associated with Xp11.2 translocation/TFE-3 gene fusion ( n  = 4), sarcomatoid carcinoma ( n  = 1), collecting duct carcinoma ( n  = 1), and eosinophilic papillary carcinoma ( n  = 2).

Representative CEUS images of nccRCC versus ccRCC. ( a-d ): A patient, female, 63 years old, with left chRCC. ( a ): Conventional ultrasound image displaying a hypoechoic mass in the left lower kidney, measuring 2.86 cm x 1.98 cm, with clear boundaries and a regular shape; ( b ): Contrast-enhanced ultrasound (CEUS) image during the perfusion phase at 9 s, demonstrating homogeneous low enhancement within the lesion and peripheral annular enhancement; ( c ): CEUS image during the washout phase at 52 s, depicting low enhancement within the lesion and peripheral ring enhancement; ( d ): Time-intensity curve (TIC) of nccRCC showing slow advance, fast retreat and low enhancement. ( e-h ): A patient, female, 72 years old, with left ccRCC. ( e ): Conventional ultrasound image displaying a hypoechoic mass in the left middle kidney, measuring 7.55 cm x 5.43 cm, with clear boundaries and a regular shape; ( f ): Contrast-enhanced ultrasound (CEUS) image during the perfusion phase at 9 s, demonstrating Heterogeneous high enhancement within the lesion and peripheral annular enhancement; ( g ): CEUS image during the washout phase at 52 s, depicting high enhancement within the lesion and peripheral ring enhancement; ( h ): Time-intensity curve (TIC) of nccRCC showing fast advance, slow retreat and high enhancement

Results of qualitative analysis of CUES images

Qualitative analysis of ultrasonography findings in the 21 nccRCC patients demonstrated that the enhancement patterns included slow enhancement (16/21, 76.1%), synchronous enhancement (4/21, 19.0%), and rapid enhancement (1/21, 4.9%). Peak intensity observations revealed low enhancement (15/21, 71.5%), moderate enhancement (4/21, 19.0%), and high enhancement (2/21, 9.5%). Enhancement uniformity analysis indicated homogeneous enhancement (13/21, 61.9%) and inhomogeneous enhancement (8/21, 38.1%). The rapid decline in enhancement was observed in 8 cases (8/21, 38.1%), with a synchronous decline in 1 case (4.9%). Enhancement direction analysis showed diffuse enhancement (14/21, 66.7%) and concentric enhancement (7/21, 33.3%). Pseudocapsule ring hyperenhancement was present in 13 cases (13/21, 61.9%). Post-enhancement, the boundary appeared clear in 19 cases (19/21, 90.5%) and unclear in 2 cases (2/21, 9.5%). The post-enhancement range correlated with the grayscale ultrasound findings in 19 cases (19/21, 90.5%) and was enlarged in 2 cases (2/21, 9.5%), with one case being Type II pRCC and the other case being sarcomatoid carcinoma. The main characteristics observed in nccRCC patients included slow enhancement, rapid wash-out, low uniformity enhancement, and pseudocapsule ring hyperenhancement (Fig.  1 ; Table  2 ).

Results of quantitative analysis of CUES images

In this study, a total of 21 nccRCC patients were included. Vuebox quantitative analysis software was employed to analyze the time-intensity curve of the lesions and normal renal parenchyma. TIC parameters were initially assessed using box plots, followed by the removal of outliers and supplementation through multiple random methods. Differences in TIC parameters between the lesions and normal renal cortex were evaluated using the Wilcoxon symbolic rank sum test for paired samples. The results indicated that PE, WiAUC, mTTI, WiR, WiPI, WoAUC, WiWoAUC, and WOR in the lesions were significantly lower than those in the normal renal cortex (Z = -3.980, -3.563, -2.427, -3.389, -3.980, -3.493, -3.528, -2.763, P  < 0.001, < 0.001, = 0.015, = 0.001, < 0.001, < 0.001, < 0.001, = 0.006). However, there were no significant differences in RT, TTP, FT, and QOF between the lesions and normal renal cortex (Table  3 ).

It is widely acknowledged that the histopathological variations within renal cell carcinoma significantly impact prognosis and tumor biology. The 5-year survival rate for ccRCC stands at only 55–60%, whereas pRCC and chRCC exhibit significantly higher rates of 80–90% [ 10 ]. Consequently, the preoperative classification of renal cell carcinoma holds substantial importance for clinical management. CEUS offers real-time and dynamic insight into microperfusion within lesions, aiding in the differentiation of malignant and benign renal lesions and enhancing the evaluation of complex renal cysts [ 11 ]. According to the literature, most ccRCC cases typically present with fast enhancement and high peak intensity, which correlates with invasiveness [ 12 ]. However, there is limited literature on the quantitative analysis of CEUS in nccRCC. This study sought to elucidate the CEUS characteristics of 21 nccRCC cases, conducting both qualitative and quantitative analyses to open new avenues for the diagnosis of this disease.

Prior studies [ 13 ] have explored CEUS characteristics in nccRCC, emphasizing that nccRCC predominantly exhibits low enhancement in peak intensity and a contrast pattern characterized by slow advancement, rapid washout, and low enhancement. In this study of 21 nccRCC patients, “slow advancement” accounted for 76.19%, low enhancement for 71.43%, and “rapid washout” for 57.14% of cases. Enhanced uniformity was observed in 61.90% of cases, with diffuse enhancement in 66.67%. This angiographic pattern, primarily displaying “slow advancement, rapid washout, and diffuse homogeneous low enhancement”, is consistent with previous research findings [ 14 ]. The primary reason behind this “slow advancement, low enhancement” pattern may be attributed to nccRCC’s typically hypovascular nature, with fewer vascular components and greater interstitial content, resulting in slow progression and limited enhancement. The “rapid washout” phenomenon may be linked to the presence of pRCC and chRCC within this nccRCC subgroup, accounting for 61.9%. These tumor types often possess an incomplete capillary network, arteriovenous fistulas, and direct arterial-to-venous blood flow, leading to faster contrast medium clearance than in the surrounding renal cortex. Additionally, nccRCC tumors tend to grow slowly, with rare occurrences of necrosis and cystic degeneration. Consequently, they appear more homogeneous when reaching their peak enhancement. Literature reports [ 15 ] suggest that the pseudocapsule results from the deposition, ischemia, or necrosis of fibrous tissue in the adjacent renal tissue during tumor growth, with circular enhancement being a common CEUS manifestation of the pseudocapsule. The contrast-enhanced pseudocapsule sign has demonstrated utility in differentiating benign and malignant renal tumors, with an AUC of 0.777 (95% confidence interval 0.701–0.853), sensitivity of 67.4%, and specificity of 88.0% [ 16 ]. However, multivariate Logistic regression analysis showed that the pseudocapsule sign is an independent predictor of RCC [ 16 ]. Zhu et al. [ 17 ] reported a significant difference in the incidence of CEUS pseudocapsule visualization among RCC patients across various tumor-size subgroups. Notably, the highest pseudocapsule visualization rate was observed in medium-sized tumors (with a diameter of 2–4 cm), reaching 79.3%. Furthermore, there was a statistical difference in the detection rate of CEUS pseudocapsules between ccRCC and nccRCC. Specifically, the detection rate of pseudocapsules in pRCC and chRCC was higher than that in ccRCC, possibly due to the distinct contrast enhancement patterns observed in different subtypes of renal cell carcinoma. pRCC and chRCC are categorized as hypovascular lesions, displaying slow progression and limited enhancement during perfusion in CEUS, followed by rapid fading. Consequently, high-echo rings could be distinctly visualized surrounding the tumor [ 18 , 19 ]. However, multivariate analysis indicated that the pseudocapsule ring enhancement index is not an independent predictor for distinguishing RCC subtypes [ 15 ]. The abovementioned CEUS characteristics primarily rely on subjective qualitative analysis, are susceptible to human factors, and are characterized by subjectivity and low reproducibility.

Vuebox introduces a novel approach for quantitatively evaluating CEUS characteristics. In our study, we employed the quality of fit to assess curve fitting reliability. There were no significant differences in QOF between lesions and the normal renal cortex, and all curves had a GOF > 0.7, indicating reliable and comparable results. Time-related parameters such as RT, TTP, and FT represent the speed and quantity of contrast agent in the flushing stage, reflecting neovascularization within the mass [ 20 ]. In our study, RT, TTP, and FT for lesions were lower than those for the normal cortex, although without significant differences ( P  < 0.05), which might be attributed to the kidney’s single blood supply, with the tumor relying on the renal artery or accessory renal artery, similar to the normal renal cortex, resulting in no significant difference in blood flow perfusion rate. However, the perfusion time parameter mTTI within the lesion was significantly lower than that within the normal renal cortex ( P  < 0.05), suggesting a shorter overall perfusion time within the lesion, possibly due to the diminished blood supply in nccRCC. PE, WiAUC, WoAUC, and WiWoAUC parameters reflect microcirculation within the tumor. WiAUC, WoAUC, and WiWoAUC represent the time-intensity integral during inflow, outflow, and the combined inflow and outflow phases, respectively, offering a comprehensive and intuitive view of microvessel density within renal tumors and serving as unique quantitative Vuebox indices [ 21 ]. Our study results revealed that PE, WiAUC, WoAUC, and WiWoAUC were significantly lower within the lesion than in the normal renal cortex, indicating reduced perfusion, washout, and global blood flow within the lesion. To mitigate potential interference from PE and WiAUC caused by technical or individual variability, we further evaluated relative indicators such as WiPI, WiR, and WoR, ensuring the independence of curve parameters. Our findings showed that WiPI, WiR, and WoR within the lesions were lower than those in the normal renal cortex, with statistically significant differences. This implies reduced speed and quantity of contrast media during perfusion and washout phases within the lesion, leading to low enhancement. These results align with the qualitative analysis findings discussed earlier.

This study qualitatively and quantitatively analyzed the CEUS characteristics of nccRCC, improved the understanding of the disease, and provided a basis for clinical differential diagnosis. Ping Zhao et al. [ 13 ] showed that CEUS and contrast-enhanced magnetic resonance imaging (MRI) showed good diagnostic performance in the differential diagnosis of ccRCC and non-ccRCC, with AUC of 0.834 and 0.803 respectively, and there was no significant difference between the two methods ( p  = 0.54). Rong-xi Liang et al. [ 9 ] retrospectively analyzed CEUS and contrast-enhanced CT images of 82 cases with ccRCC, 24 cases with pRCC and 19 cases with ChRCC. The results showed that the enhancement patterns of CEUS and contrast-enhanced CT in the three subtypes of RCC were similar, and all of them could accurately diagnose the lesions of ccRCC, pRCC and ChRCC. Microbubble ultrasound contrast agent is a real blood pool imaging agent, which will not spread to the intercellular space, which greatly improves the sensitivity of blood flow detection at low flow rate and accurately reflects renal tumor perfusion, especially for nccRCC with lack of blood supply. Therefore, CEUS can be used as an alternative for patients with renal insufficiency or hypersensitivity to iodine contrast media that are not suitable for contrast-enhanced MRI or contrast-enhanced CT.

Indeed, the limitations of this study should be acknowledged. Firstly, this is a single-center retrospective study. Secondly, the inclusion of subjects who were pathologically confirmed after surgery may introduce selection bias. Thirdly, the sample size is relatively small, and the study did not differentiate between different subtypes of nccRCC, focusing mainly on the two most common subtypes. Additionally, rare subtypes were underrepresented in the study.

In conclusion, when compared to ccRCC, nccRCC exhibits a higher 5-year survival rate. Thus, precise preoperative classification of RCC holds substantial clinical value in assessing prognosis. CEUS in nccRCC show cases distinctive characteristics, including slow advancement, rapid washout, low uniformity enhancement, and annular enhancement. These characteristic CEUS patterns are helpful in distinguishing nccRCC from ccRCC. CEUS can be utilized as an alternative examination for patients who are contraindicated for contrast-enhanced MRI or contrast-enhanced CT.

Data availability

Data is available upon reasonable request.

Bukavina L, Bensalah K, Bray F, et al. Epidemiology of renal cell carcinoma: 2022 update. Eur Urol. 2022;82(5):529–42.

Article   PubMed   Google Scholar  

Bahadoram S, Davoodi M, Hassanzadeh S, et al. Renal cell carcinoma: an overview of the epidemiology, diagnosis, and treatment. G Ital Nefrol. 2022;39(3):2022–vol3.

PubMed   Google Scholar  

Escudier B, Porta C, Schmidinger M, et al. Renal cell carcinoma: ESMO clinical practice guidelines for diagnosis, treatment and follow-up. Ann Oncol. 2016;27(suppl 5):v58–68.

Article   CAS   PubMed   Google Scholar  

Gray RE, Harris GT. Renal cell carcinoma: diagnosis and management. Am Fam Physician. 2019;99(3):179–184. Erratum in: Am Fam Physician. 2019;99(12):732.

King KG. Use of contrast ultrasound for renal mass evaluation. Radiol Clin North Am. 2020;58(5):935–49.

Barr RG. Use of lumason/sonovue in contrast-enhanced ultrasound of the kidney for characterization of renal masses-a meta-analysis. Abdom Radiol (NY). 2022;47(1):272–87.

Marschner CA, Ruebenthaler J, Schwarze V, et al. Comparison of computed tomography (CT), magnetic resonance imaging (MRI) and contrast-enhanced ultrasound (CEUS) in the evaluation of unclear renal lesions. Rofo. 2020;192(11):1053–9.

Tufano A, Leonardo C, Di Bella C, et al. Qualitative assessment of contrast-enhanced ultrasound in differentiating clear cell renal cell carcinoma and oncocytoma. J Clin Med. 2023;12(9):3070.

Article   PubMed   PubMed Central   Google Scholar  

Liang R-X, Wang H, Zhang H-P, et al. The value of real-time contrast-enhanced ultrasound combined with CT enhancement in the differentiation of subtypes of renal cell carcinoma. [J] Urol Oncol. 2021;39:837.e19–837.e28.

Huang X, Wang N, Liu L, et al. Pre-operative prediction of invasiveness in renal cell carcinoma: the role of conventional ultrasound and contrast-enhanced ultrasound. [J] Ultrasound Med Biol. 2023;49:2169–76.

Article   Google Scholar  

Calio Brian P, Lyshchik A, Li J, et al. Long term surveillance of renal cell carcinoma recurrence following ablation using 2D and 3D contrast-enhanced ultrasound. [J] Urology. 2018;121:189–196.

Meng X, Yang R, Zhao S, et al. Associations between tumor grade, contrast-enhanced ultrasound features, and microvascular density in patients with clear cell renal cell carcinoma: a retrospective study. [J] Quant Imaging Med Surg. 2022;12:1882–1892.

Zhao P, Zhu J, Wang L, Li N, et al. Comparative diagnostic performance of contrast-enhanced ultrasound and dynamic contrast-enhanced magnetic resonance imaging for differentiating clear cell and non-clear cell renal cell carcinoma. Eur Radiol. 2023;33:3766–74.

Xue LY, Lu Q, Huang BJ, et al. Differentiation of subtypes of renal cell carcinoma with contrast-enhanced ultrasonography. Clin Hemorheol Microcirc. 2016;63:361–71.

Wang L, Wu H, Li J, Wang Y, et al. Diagnostic value of pseudocapsule ring hyperenhancement in contrast-enhanced ultrasound in renal focal lesions. Ir J Med Sci. 2023;192:1631–6.

Dai W-B, Yu B, Diao X-H, et al. Renal masses: evaluation with contrast-enhanced ultrasound, with a special focus on the pseudocapsule sign. [J] Ultrasound Med Biol. 2019;45:1924–32.

Zhu J, Li N, Zhao P, et al. Contrast-enhanced ultrasound (CEUS) of benign and malignant renal tumors: distinguishing CEUS features differ with tumor size. Cancer Med. 2023;12:2551–9.

Liu H, Cao H, Chen L, et al. The quantitative evaluation of contrast-enhanced ultrasound in the differentiation of small renal cell carcinoma subtypes and angiomyolipoma. [J] Quant Imaging Med Surg. 2022;12:106–118.

Rübenthaler J, Reimann R, Hristova P, et al. Parametric imaging of clear cell and papillary renal cell carcinoma using contrast-enhanced ultrasound (CEUS). Clin Hemorheol Microcirc. 2015;16(2):89–97.

Sun D, Wei C, Li Y, et al. Contrast-enhanced ultrasonography with quantitative analysis allows differentiation of renal tumor histotypes. Sci Rep. 2016;11:6:35081.

He M, Gao Q, Xiang J, et al. Diagnostic value of qualitative and quantitative contrast-enhanced Ultrasound for pathological subtypes of small solid renal masses. J Ultrasound Med. 2023;42(8):1663–73.

Download references

Acknowledgements

Declared none.

This study was supported by the Health Commission Science and Technology Plan of Jiangxi Province (Grant Number 202210452), the Administration of Traditional Chinese Medicine Science and Technology Plan of Jiangxi Province (Grant Number 2021A060), and the provincial project of teaching reform in colleges and universities of Jiangxi Province (Grant Number JXJG-22-1-60).

Author information

Authors and affiliations.

Department of Ultrasound, The First Affiliated Hospital, Jiangxi Medical College, Nanchang University, Nanchang, China

WeiPing Zhang, JingLing Wang, Li Chen & Jiayu Shi

The First Clinical Medical College, Jiangxi Medical College, Nanchang University, Nanchang, China

You can also search for this author in PubMed   Google Scholar

Contributions

Weiping Zhang was responsible for the article writing and ultrasound image processing; Jingling Wang and Jiayu Shi was in charge of cases collection; Li Chen was responsible for the article design. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to WeiPing Zhang .

Ethics declarations

Ethics approval and consent to participate.

This research was approved by the Ethics Committee of first affiliated Hospital of Nanchang University(IIT2023174), and Informed consent was obtained from all participants of this study.

Human and animal rights

No animals were used in this research. All the human procedures used were in accordance with the ethical standards of the committee responsible for human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2013 ( http://ethics.iit.edu/ecodes/node/3931 ).

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Zhang, W., Wang, J., Chen, L. et al. Qualitative and quantitative assessment of non-clear cell renal cell carcinoma using contrast-enhanced ultrasound. BMC Urol 24 , 129 (2024). https://doi.org/10.1186/s12894-024-01514-8

Download citation

Received : 28 January 2024

Accepted : 10 June 2024

Published : 17 June 2024

DOI : https://doi.org/10.1186/s12894-024-01514-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Contrast-enhanced ultrasound
• Qualitative
• Non-clear cell renal cell carcinoma
• Quantitative

BMC Urology

ISSN: 1471-2490

• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• BMJ Glob Health
• v.4(Suppl 1); 2019

Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods

1 School of Social Sciences, Bangor University, Wales, UK

Andrew Booth

2 School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK

Graham Moore

3 School of Social Sciences, Cardiff University, Wales, UK

Kate Flemming

4 Department of Health Sciences, The University of York, York, UK

Özge Tunçalp

5 Department of Reproductive Health and Research including UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), World Health Organization, Geneva, Switzerland

Elham Shakibazadeh

6 Department of Health Education and Promotion, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran

Associated Data

bmjgh-2018-000893supp001.pdf

bmjgh-2018-000893supp002.pdf

bmjgh-2018-000893supp003.pdf

bmjgh-2018-000893supp005.pdf

bmjgh-2018-000893supp004.pdf

Guideline developers are increasingly dealing with more difficult decisions concerning whether to recommend complex interventions in complex and highly variable health systems. There is greater recognition that both quantitative and qualitative evidence can be combined in a mixed-method synthesis and that this can be helpful in understanding how complexity impacts on interventions in specific contexts. This paper aims to clarify the different purposes, review designs, questions, synthesis methods and opportunities to combine quantitative and qualitative evidence to explore the complexity of complex interventions and health systems. Three case studies of guidelines developed by WHO, which incorporated quantitative and qualitative evidence, are used to illustrate possible uses of mixed-method reviews and evidence. Additional examples of methods that can be used or may have potential for use in a guideline process are outlined. Consideration is given to the opportunities for potential integration of quantitative and qualitative evidence at different stages of the review and guideline process. Encouragement is given to guideline commissioners and developers and review authors to consider including quantitative and qualitative evidence. Recommendations are made concerning the future development of methods to better address questions in systematic reviews and guidelines that adopt a complexity perspective.

Summary box

• When combined in a mixed-method synthesis, quantitative and qualitative evidence can potentially contribute to understanding how complex interventions work and for whom, and how the complex health systems into which they are implemented respond and adapt.
• The different purposes and designs for combining quantitative and qualitative evidence in a mixed-method synthesis for a guideline process are described.
• Questions relevant to gaining an understanding of the complexity of complex interventions and the wider health systems within which they are implemented that can be addressed by mixed-method syntheses are presented.
• The practical methodological guidance in this paper is intended to help guideline producers and review authors commission and conduct mixed-method syntheses where appropriate.
• If more mixed-method syntheses are conducted, guideline developers will have greater opportunities to access this evidence to inform decision-making.

Introduction

Recognition has grown that while quantitative methods remain vital, they are usually insufficient to address complex health systems related research questions. 1 Quantitative methods rely on an ability to anticipate what must be measured in advance. Introducing change into a complex health system gives rise to emergent reactions, which cannot be fully predicted in advance. Emergent reactions can often only be understood through combining quantitative methods with a more flexible qualitative lens. 2 Adopting a more pluralist position enables a diverse range of research options to the researcher depending on the research question being investigated. 3–5 As a consequence, where a research study sits within the multitude of methods available is driven by the question being asked, rather than any particular methodological or philosophical stance. 6

Publication of guidance on designing complex intervention process evaluations and other works advocating mixed-methods approaches to intervention research have stimulated better quality evidence for synthesis. 1 7–13 Methods for synthesising qualitative 14 and mixed-method evidence have been developed or are in development. Mixed-method research and review definitions are outlined in box 1 .

Defining mixed-method research and reviews

Pluye and Hong 52 define mixed-methods research as “a research approach in which a researcher integrates (a) qualitative and quantitative research questions, (b) qualitative research methods* and quantitative research designs, (c) techniques for collecting and analyzing qualitative and quantitative evidence, and (d) qualitative findings and quantitative results”.A mixed-method synthesis can integrate quantitative, qualitative and mixed-method evidence or data from primary studies.† Mixed-method primary studies are usually disaggregated into quantitative and qualitative evidence and data for the purposes of synthesis. Thomas and Harden further define three ways in which reviews are mixed. 53

• The types of studies included and hence the type of findings to be synthesised (ie, qualitative/textual and quantitative/numerical).
• The types of synthesis method used (eg, statistical meta-analysis and qualitative synthesis).
• The mode of analysis: theory testing AND theory building.

*A qualitative study is one that uses qualitative methods of data collection and analysis to produce a narrative understanding of the phenomena of interest. Qualitative methods of data collection may include, for example, interviews, focus groups, observations and analysis of documents.

†The Cochrane Qualitative and Implementation Methods group coined the term ‘qualitative evidence synthesis’ to mean that the synthesis could also include qualitative data. For example, qualitative data from case studies, grey literature reports and open-ended questions from surveys. ‘Evidence’ and ‘data’ are used interchangeably in this paper.

This paper is one of a series that aims to explore the implications of complexity for systematic reviews and guideline development, commissioned by WHO. This paper is concerned with the methodological implications of including quantitative and qualitative evidence in mixed-method systematic reviews and guideline development for complex interventions. The guidance was developed through a process of bringing together experts in the field, literature searching and consensus building with end users (guideline developers, clinicians and reviewers). We clarify the different purposes, review designs, questions and synthesis methods that may be applicable to combine quantitative and qualitative evidence to explore the complexity of complex interventions and health systems. Three case studies of WHO guidelines that incorporated quantitative and qualitative evidence are used to illustrate possible uses of mixed-method reviews and mechanisms of integration ( table 1 , online supplementary files 1–3 ). Additional examples of methods that can be used or may have potential for use in a guideline process are outlined. Opportunities for potential integration of quantitative and qualitative evidence at different stages of the review and guideline process are presented. Specific considerations when using an evidence to decision framework such as the Developing and Evaluating Communication strategies to support Informed Decisions and practice based on Evidence (DECIDE) framework 15 or the new WHO-INTEGRATE evidence to decision framework 16 at the review design and evidence to decision stage are outlined. See online supplementary file 4 for an example of a health systems DECIDE framework and Rehfuess et al 16 for the new WHO-INTEGRATE framework. Encouragement is given to guideline commissioners and developers and review authors to consider including quantitative and qualitative evidence in guidelines of complex interventions that take a complexity perspective and health systems focus.

Designs and methods and their use or applicability in guidelines and systematic reviews taking a complexity perspective

Supplementary data

Taking a complexity perspective.

The first paper in this series 17 outlines aspects of complexity associated with complex interventions and health systems that can potentially be explored by different types of evidence, including synthesis of quantitative and qualitative evidence. Petticrew et al 17 distinguish between a complex interventions perspective and a complex systems perspective. A complex interventions perspective defines interventions as having “implicit conceptual boundaries, representing a flexible, but common set of practices, often linked by an explicit or implicit theory about how they work”. A complex systems perspective differs in that “ complexity arises from the relationships and interactions between a system’s agents (eg, people, or groups that interact with each other and their environment), and its context. A system perspective conceives the intervention as being part of the system, and emphasises changes and interconnections within the system itself”. Aspects of complexity associated with implementation of complex interventions in health systems that could potentially be addressed with a synthesis of quantitative and qualitative evidence are summarised in table 2 . Another paper in the series outlines criteria used in a new evidence to decision framework for making decisions about complex interventions implemented in complex systems, against which the need for quantitative and qualitative evidence can be mapped. 16 A further paper 18 that explores how context is dealt with in guidelines and reviews taking a complexity perspective also recommends using both quantitative and qualitative evidence to better understand context as a source of complexity. Mixed-method syntheses of quantitative and qualitative evidence can also help with understanding of whether there has been theory failure and or implementation failure. The Cochrane Qualitative and Implementation Methods Group provide additional guidance on exploring implementation and theory failure that can be adapted to address aspects of complexity of complex interventions when implemented in health systems. 19

Health-system complexity-related questions that a synthesis of quantitative and qualitative evidence could address (derived from Petticrew et al 17 )

It may not be apparent which aspects of complexity or which elements of the complex intervention or health system can be explored in a guideline process, or whether combining qualitative and quantitative evidence in a mixed-method synthesis will be useful, until the available evidence is scoped and mapped. 17 20 A more extensive lead in phase is typically required to scope the available evidence, engage with stakeholders and to refine the review parameters and questions that can then be mapped against potential review designs and methods of synthesis. 20 At the scoping stage, it is also common to decide on a theoretical perspective 21 or undertake further work to refine a theoretical perspective. 22 This is also the stage to begin articulating the programme theory of the complex intervention that may be further developed to refine an understanding of complexity and show how the intervention is implemented in and impacts on the wider health system. 17 23 24 In practice, this process can be lengthy, iterative and fluid with multiple revisions to the review scope, often developing and adapting a logic model 17 as the available evidence becomes known and the potential to incorporate different types of review designs and syntheses of quantitative and qualitative evidence becomes better understood. 25 Further questions, propositions or hypotheses may emerge as the reviews progress and therefore the protocols generally need to be developed iteratively over time rather than a priori.

Following a scoping exercise and definition of key questions, the next step in the guideline development process is to identify existing or commission new systematic reviews to locate and summarise the best available evidence in relation to each question. For example, case study 2, ‘Optimising health worker roles for maternal and newborn health through task shifting’, included quantitative reviews that did and did not take an additional complexity perspective, and qualitative evidence syntheses that were able to explain how specific elements of complexity impacted on intervention outcomes within the wider health system. Further understanding of health system complexity was facilitated through the conduct of additional country-level case studies that contributed to an overall understanding of what worked and what happened when lay health worker interventions were implemented. See table 1 online supplementary file 2 .

There are a few existing examples, which we draw on in this paper, but integrating quantitative and qualitative evidence in a mixed-method synthesis is relatively uncommon in a guideline process. Box 2 includes a set of key questions that guideline developers and review authors contemplating combining quantitative and qualitative evidence in mixed-methods design might ask. Subsequent sections provide more information and signposting to further reading to help address these key questions.

Key questions that guideline developers and review authors contemplating combining quantitative and qualitative evidence in a mixed-methods design might ask

Compound questions requiring both quantitative and qualitative evidence?

Questions requiring mixed-methods studies?

Separate quantitative and qualitative questions?

Separate quantitative and qualitative research studies?

Related quantitative and qualitative research studies?

Mixed-methods studies?

Quantitative unpublished data and/or qualitative unpublished data, eg, narrative survey data?

Throughout the review?

Following separate reviews?

At the question point?

At the synthesis point?

At the evidence to recommendations stage?

Or a combination?

Narrative synthesis or summary?

Quantitising approach, eg, frequency analysis?

Qualitising approach, eg, thematic synthesis?

Tabulation?

Logic model?

Conceptual model/framework?

Graphical approach?

• WHICH: Which mixed-method designs, methodologies and methods best fit into a guideline process to inform recommendations?

Complexity-related questions that a synthesis of quantitative and qualitative evidence can potentially address

Petticrew et al 17 define the different aspects of complexity and examples of complexity-related questions that can potentially be explored in guidelines and systematic reviews taking a complexity perspective. Relevant aspects of complexity outlined by Petticrew et al 17 are summarised in table 2 below, together with the corresponding questions that could be addressed in a synthesis combining qualitative and quantitative evidence. Importantly, the aspects of complexity and their associated concepts of interest have however yet to be translated fully in primary health research or systematic reviews. There are few known examples where selected complexity concepts have been used to analyse or reanalyse a primary intervention study. Most notable is Chandler et al 26 who specifically set out to identify and translate a set of relevant complexity theory concepts for application in health systems research. Chandler then reanalysed a trial process evaluation using selected complexity theory concepts to better understand the complex causal pathway in the health system that explains some aspects of complexity in table 2 .

Rehfeuss et al 16 also recommends upfront consideration of the WHO-INTEGRATE evidence to decision criteria when planning a guideline and formulating questions. The criteria reflect WHO norms and values and take account of a complexity perspective. The framework can be used by guideline development groups as a menu to decide which criteria to prioritise, and which study types and synthesis methods can be used to collect evidence for each criterion. Many of the criteria and their related questions can be addressed using a synthesis of quantitative and qualitative evidence: the balance of benefits and harms, human rights and sociocultural acceptability, health equity, societal implications and feasibility (see table 3 ). Similar aspects in the DECIDE framework 15 could also be addressed using synthesis of qualitative and quantitative evidence.

Integrate evidence to decision framework criteria, example questions and types of studies to potentially address these questions (derived from Rehfeuss et al 16 )

GIS, Geographical Information System; RCT, randomised controlled trial.

Questions as anchors or compasses

Questions can serve as an ‘anchor’ by articulating the specific aspects of complexity to be explored (eg, Is successful implementation of the intervention context dependent?). 27 Anchor questions such as “How does intervention x impact on socioeconomic inequalities in health behaviour/outcome x” are the kind of health system question that requires a synthesis of both quantitative and qualitative evidence and hence a mixed-method synthesis. Quantitative evidence can quantify the difference in effect, but does not answer the question of how . The ‘how’ question can be partly answered with quantitative and qualitative evidence. For example, quantitative evidence may reveal where socioeconomic status and inequality emerges in the health system (an emergent property) by exploring questions such as “ Does patterning emerge during uptake because fewer people from certain groups come into contact with an intervention in the first place? ” or “ are people from certain backgrounds more likely to drop out, or to maintain effects beyond an intervention differently? ” Qualitative evidence may help understand the reasons behind all of these mechanisms. Alternatively, questions can act as ‘compasses’ where a question sets out a starting point from which to explore further and to potentially ask further questions or develop propositions or hypotheses to explore through a complexity perspective (eg, What factors enhance or hinder implementation?). 27 Other papers in this series provide further guidance on developing questions for qualitative evidence syntheses and guidance on question formulation. 14 28

For anchor and compass questions, additional application of a theory (eg, complexity theory) can help focus evidence synthesis and presentation to explore and explain complexity issues. 17 21 Development of a review specific logic model(s) can help to further refine an initial understanding of any complexity-related issues of interest associated with a specific intervention, and if appropriate the health system or section of the health system within which to contextualise the review question and analyse data. 17 23–25 Specific tools are available to help clarify context and complex interventions. 17 18

If a complexity perspective, and certain criteria within evidence to decision frameworks, is deemed relevant and desirable by guideline developers, it is only possible to pursue a complexity perspective if the evidence is available. Careful scoping using knowledge maps or scoping reviews will help inform development of questions that are answerable with available evidence. 20 If evidence of effect is not available, then a different approach to develop questions leading to a more general narrative understanding of what happened when complex interventions were implemented in a health system will be required (such as in case study 3—risk communication guideline). This should not mean that the original questions developed for which no evidence was found when scoping the literature were not important. An important function of creating a knowledge map is also to identify gaps to inform a future research agenda.

Table 2 and online supplementary files 1–3 outline examples of questions in the three case studies, which were all ‘COMPASS’ questions for the qualitative evidence syntheses.

Types of integration and synthesis designs in mixed-method reviews

The shift towards integration of qualitative and quantitative evidence in primary research has, in recent years, begun to be mirrored within research synthesis. 29–31 The natural extension to undertaking quantitative or qualitative reviews has been the development of methods for integrating qualitative and quantitative evidence within reviews, and within the guideline process using evidence to decision-frameworks. Advocating the integration of quantitative and qualitative evidence assumes a complementarity between research methodologies, and a need for both types of evidence to inform policy and practice. Below, we briefly outline the current designs for integrating qualitative and quantitative evidence within a mixed-method review or synthesis.

One of the early approaches to integrating qualitative and quantitative evidence detailed by Sandelowski et al 32 advocated three basic review designs: segregated, integrated and contingent designs, which have been further developed by Heyvaert et al 33 ( box 3 ).

Segregated, integrated and contingent designs 32 33

Segregated design.

Conventional separate distinction between quantitative and qualitative approaches based on the assumption they are different entities and should be treated separately; can be distinguished from each other; their findings warrant separate analyses and syntheses. Ultimately, the separate synthesis results can themselves be synthesised.

Integrated design

The methodological differences between qualitative and quantitative studies are minimised as both are viewed as producing findings that can be readily synthesised into one another because they address the same research purposed and questions. Transformation involves either turning qualitative data into quantitative (quantitising) or quantitative findings are turned into qualitative (qualitising) to facilitate their integration.

Contingent design

Takes a cyclical approach to synthesis, with the findings from one synthesis informing the focus of the next synthesis, until all the research objectives have been addressed. Studies are not necessarily grouped and categorised as qualitative or quantitative.

A recent review of more than 400 systematic reviews 34 combining quantitative and qualitative evidence identified two main synthesis designs—convergent and sequential. In a convergent design, qualitative and quantitative evidence is collated and analysed in a parallel or complementary manner, whereas in a sequential synthesis, the collation and analysis of quantitative and qualitative evidence takes place in a sequence with one synthesis informing the other ( box 4 ). 6 These designs can be seen to build on the work of Sandelowski et al , 32 35 particularly in relation to the transformation of data from qualitative to quantitative (and vice versa) and the sequential synthesis design, with a cyclical approach to reviewing that evokes Sandelowski’s contingent design.

Convergent and sequential synthesis designs 34

Convergent synthesis design.

Qualitative and quantitative research is collected and analysed at the same time in a parallel or complementary manner. Integration can occur at three points:

a. Data-based convergent synthesis design

All included studies are analysed using the same methods and results presented together. As only one synthesis method is used, data transformation occurs (qualitised or quantised). Usually addressed one review question.

b. Results-based convergent synthesis design

Qualitative and quantitative data are analysed and presented separately but integrated using a further synthesis method; eg, narratively, tables, matrices or reanalysing evidence. The results of both syntheses are combined in a third synthesis. Usually addresses an overall review question with subquestions.

c. Parallel-results convergent synthesis design

Qualitative and quantitative data are analysed and presented separately with integration occurring in the interpretation of results in the discussion section. Usually addresses two or more complimentary review questions.

Sequential synthesis design

A two-phase approach, data collection and analysis of one type of evidence (eg, qualitative), occurs after and is informed by the collection and analysis of the other type (eg, quantitative). Usually addresses an overall question with subquestions with both syntheses complementing each other.

The three case studies ( table 1 , online supplementary files 1–3 ) illustrate the diverse combination of review designs and synthesis methods that were considered the most appropriate for specific guidelines.

Methods for conducting mixed-method reviews in the context of guidelines for complex interventions

In this section, we draw on examples where specific review designs and methods have been or can be used to explore selected aspects of complexity in guidelines or systematic reviews. We also identify other review methods that could potentially be used to explore aspects of complexity. Of particular note, we could not find any specific examples of systematic methods to synthesise highly diverse research designs as advocated by Petticrew et al 17 and summarised in tables 2 and 3 . For example, we could not find examples of methods to synthesise qualitative studies, case studies, quantitative longitudinal data, possibly historical data, effectiveness studies providing evidence of differential effects across different contexts, and system modelling studies (eg, agent-based modelling) to explore system adaptivity.

There are different ways that quantitative and qualitative evidence can be integrated into a review and then into a guideline development process. In practice, some methods enable integration of different types of evidence in a single synthesis, while in other methods, the single systematic review may include a series of stand-alone reviews or syntheses that are then combined in a cross-study synthesis. Table 1 provides an overview of the characteristics of different review designs and methods and guidance on their applicability for a guideline process. Designs and methods that have already been used in WHO guideline development are described in part A of the table. Part B outlines a design and method that can be used in a guideline process, and part C covers those that have the potential to integrate quantitative, qualitative and mixed-method evidence in a single review design (such as meta-narrative reviews and Bayesian syntheses), but their application in a guideline context has yet to be demonstrated.

Points of integration when integrating quantitative and qualitative evidence in guideline development

Depending on the review design (see boxes 3 and 4 ), integration can potentially take place at a review team and design level, and more commonly at several key points of the review or guideline process. The following sections outline potential points of integration and associated practical considerations when integrating quantitative and qualitative evidence in guideline development.

Review team level

In a guideline process, it is common for syntheses of quantitative and qualitative evidence to be done separately by different teams and then to integrate the evidence. A practical consideration relates to the organisation, composition and expertise of the review teams and ways of working. If the quantitative and qualitative reviews are being conducted separately and then brought together by the same team members, who are equally comfortable operating within both paradigms, then a consistent approach across both paradigms becomes possible. If, however, a team is being split between the quantitative and qualitative reviews, then the strengths of specialisation can be harnessed, for example, in quality assessment or synthesis. Optimally, at least one, if not more, of the team members should be involved in both quantitative and qualitative reviews to offer the possibility of making connexions throughout the review and not simply at re-agreed junctures. This mirrors O’Cathain’s conclusion that mixed-methods primary research tends to work only when there is a principal investigator who values and is able to oversee integration. 9 10 While the above decisions have been articulated in the context of two types of evidence, variously quantitative and qualitative, they equally apply when considering how to handle studies reporting a mixed-method study design, where data are usually disaggregated into quantitative and qualitative for the purposes of synthesis (see case study 3—risk communication in humanitarian disasters).

Question formulation

Clearly specified key question(s), derived from a scoping or consultation exercise, will make it clear if quantitative and qualitative evidence is required in a guideline development process and which aspects will be addressed by which types of evidence. For the remaining stages of the process, as documented below, a review team faces challenges as to whether to handle each type of evidence separately, regardless of whether sequentially or in parallel, with a view to joining the two products on completion or to attempt integration throughout the review process. In each case, the underlying choice is of efficiencies and potential comparability vs sensitivity to the underlying paradigm.

Once key questions are clearly defined, the guideline development group typically needs to consider whether to conduct a single sensitive search to address all potential subtopics (lumping) or whether to conduct specific searches for each subtopic (splitting). 36 A related consideration is whether to search separately for qualitative, quantitative and mixed-method evidence ‘streams’ or whether to conduct a single search and then identify specific study types at the subsequent sifting stage. These two considerations often mean a trade-off between a single search process involving very large numbers of records or a more protracted search process retrieving smaller numbers of records. Both approaches have advantages and choice may depend on the respective availability of resources for searching and sifting.

Screening and selecting studies

Closely related to decisions around searching are considerations relating to screening and selecting studies for inclusion in a systematic review. An important consideration here is whether the review team will screen records for all review types, regardless of their subsequent involvement (‘altruistic sifting’), or specialise in screening for the study type with which they are most familiar. The risk of missing relevant reports might be minimised by whole team screening for empirical reports in the first instance and then coding them for a specific quantitative, qualitative or mixed-methods report at a subsequent stage.

Assessment of methodological limitations in primary studies

Within a guideline process, review teams may be more limited in their choice of instruments to assess methodological limitations of primary studies as there are mandatory requirements to use the Cochrane risk of bias tool 37 to feed into Grading of Recommendations Assessment, Development and Evaluation (GRADE) 38 or to select from a small pool of qualitative appraisal instruments in order to apply GRADE; Confidence in the Evidence from Reviews of Qualitative Research (GRADE-CERQual) 39 to assess the overall certainty or confidence in findings. The Cochrane Qualitative and Implementation Methods Group has recently issued guidance on the selection of appraisal instruments and core assessment criteria. 40 The Mixed-Methods Appraisal Tool, which is currently undergoing further development, offers a single quality assessment instrument for quantitative, qualitative and mixed-methods studies. 41 Other options include using corresponding instruments from within the same ‘stable’, for example, using different Critical Appraisal Skills Programme instruments. 42 While using instruments developed by the same team or organisation may achieve a degree of epistemological consonance, benefits may come more from consistency of approach and reporting rather than from a shared view of quality. Alternatively, a more paradigm-sensitive approach would involve selecting the best instrument for each respective review while deferring challenges from later heterogeneity of reporting.

Data extraction

The way in which data and evidence are extracted from primary research studies for review will be influenced by the type of integrated synthesis being undertaken and the review purpose. Initially, decisions need to be made regarding the nature and type of data and evidence that are to be extracted from the included studies. Method-specific reporting guidelines 43 44 provide a good template as to what quantitative and qualitative data it is potentially possible to extract from different types of method-specific study reports, although in practice reporting quality varies. Online supplementary file 5 provides a hypothetical example of the different types of studies from which quantitative and qualitative evidence could potentially be extracted for synthesis.

The decisions around what data or evidence to extract will be guided by how ‘integrated’ the mixed-method review will be. For those reviews where the quantitative and qualitative findings of studies are synthesised separately and integrated at the point of findings (eg, segregated or contingent approaches or sequential synthesis design), separate data extraction approaches will likely be used.

Where integration occurs during the process of the review (eg, integrated approach or convergent synthesis design), an integrated approach to data extraction may be considered, depending on the purpose of the review. This may involve the use of a data extraction framework, the choice of which needs to be congruent with the approach to synthesis chosen for the review. 40 45 The integrative or theoretical framework may be decided on a priori if a pre-developed theoretical or conceptual framework is available in the literature. 27 The development of a framework may alternatively arise from the reading of the included studies, in relation to the purpose of the review, early in the process. The Cochrane Qualitative and Implementation Methods Group provide further guidance on extraction of qualitative data, including use of software. 40

Synthesis and integration

Relatively few synthesis methods start off being integrated from the beginning, and these methods have generally been subject to less testing and evaluation particularly in a guideline context (see table 1 ). A review design that started off being integrated from the beginning may be suitable for some guideline contexts (such as in case study 3—risk communication in humanitarian disasters—where there was little evidence of effect), but in general if there are sufficient trials then a separate systematic review and meta-analysis will be required for a guideline. Other papers in this series offer guidance on methods for synthesising quantitative 46 and qualitative evidence 14 in reviews that take a complexity perspective. Further guidance on integrating quantitative and qualitative evidence in a systematic review is provided by the Cochrane Qualitative and Implementation Methods Group. 19 27 29 40 47

Types of findings produced by specific methods

It is highly likely (unless there are well-designed process evaluations) that the primary studies may not themselves seek to address the complexity-related questions required for a guideline process. In which case, review authors will need to configure the available evidence and transform the evidence through the synthesis process to produce explanations, propositions and hypotheses (ie, findings) that were not obvious at primary study level. It is important that guideline commissioners, developers and review authors are aware that specific methods are intended to produce a type of finding with a specific purpose (such as developing new theory in the case of meta-ethnography). 48 Case study 1 (antenatal care guideline) provides an example of how a meta-ethnography was used to develop a new theory as an end product, 48 49 as well as framework synthesis which produced descriptive and explanatory findings that were more easily incorporated into the guideline process. 27 The definitions ( box 5 ) may be helpful when defining the different types of findings.

Different levels of findings

Descriptive findings —qualitative evidence-driven translated descriptive themes that do not move beyond the primary studies.

Explanatory findings —may either be at a descriptive or theoretical level. At the descriptive level, qualitative evidence is used to explain phenomena observed in quantitative results, such as why implementation failed in specific circumstances. At the theoretical level, the transformed and interpreted findings that go beyond the primary studies can be used to explain the descriptive findings. The latter description is generally the accepted definition in the wider qualitative community.

Hypothetical or theoretical finding —qualitative evidence-driven transformed themes (or lines of argument) that go beyond the primary studies. Although similar, Thomas and Harden 56 make a distinction in the purposes between two types of theoretical findings: analytical themes and the product of meta-ethnographies, third-order interpretations. 48

Analytical themes are a product of interrogating descriptive themes by placing the synthesis within an external theoretical framework (such as the review question and subquestions) and are considered more appropriate when a specific review question is being addressed (eg, in a guideline or to inform policy). 56

Third-order interpretations come from translating studies into one another while preserving the original context and are more appropriate when a body of literature is being explored in and of itself with broader or emergent review questions. 48

Bringing mixed-method evidence together in evidence to decision (EtD) frameworks

A critical element of guideline development is the formulation of recommendations by the Guideline Development Group, and EtD frameworks help to facilitate this process. 16 The EtD framework can also be used as a mechanism to integrate and display quantitative and qualitative evidence and findings mapped against the EtD framework domains with hyperlinks to more detailed evidence summaries from contributing reviews (see table 1 ). It is commonly the EtD framework that enables the findings of the separate quantitative and qualitative reviews to be brought together in a guideline process. Specific challenges when populating the DECIDE evidence to decision framework 15 were noted in case study 3 (risk communication in humanitarian disasters) as there was an absence of intervention effect data and the interventions to communicate public health risks were context specific and varied. These problems would not, however, have been addressed by substitution of the DECIDE framework with the new INTEGRATE 16 evidence to decision framework. A d ifferent type of EtD framework needs to be developed for reviews that do not include sufficient evidence of intervention effect.

Mixed-method review and synthesis methods are generally the least developed of all systematic review methods. It is acknowledged that methods for combining quantitative and qualitative evidence are generally poorly articulated. 29 50 There are however some fairly well-established methods for using qualitative evidence to explore aspects of complexity (such as contextual, implementation and outcome complexity), which can be combined with evidence of effect (see sections A and B of table 1 ). 14 There are good examples of systematic reviews that use these methods to combine quantitative and qualitative evidence, and examples of guideline recommendations that were informed by evidence from both quantitative and qualitative reviews (eg, case studies 1–3). With the exception of case study 3 (risk communication), the quantitative and qualitative reviews for these specific guidelines have been conducted separately, and the findings subsequently brought together in an EtD framework to inform recommendations.

Other mixed-method review designs have potential to contribute to understanding of complex interventions and to explore aspects of wider health systems complexity but have not been sufficiently developed and tested for this specific purpose, or used in a guideline process (section C of table 1 ). Some methods such as meta-narrative reviews also explore different questions to those usually asked in a guideline process. Methods for processing (eg, quality appraisal) and synthesising the highly diverse evidence suggested in tables 2 and 3 that are required to explore specific aspects of health systems complexity (such as system adaptivity) and to populate some sections of the INTEGRATE EtD framework remain underdeveloped or in need of development.

In addition to the required methodological development mentioned above, there is no GRADE approach 38 for assessing confidence in findings developed from combined quantitative and qualitative evidence. Another paper in this series outlines how to deal with complexity and grading different types of quantitative evidence, 51 and the GRADE CERQual approach for qualitative findings is described elsewhere, 39 but both these approaches are applied to method-specific and not mixed-method findings. An unofficial adaptation of GRADE was used in the risk communication guideline that reported mixed-method findings. Nor is there a reporting guideline for mixed-method reviews, 47 and for now reports will need to conform to the relevant reporting requirements of the respective method-specific guideline. There is a need to further adapt and test DECIDE, 15 WHO-INTEGRATE 16 and other types of evidence to decision frameworks to accommodate evidence from mixed-method syntheses which do not set out to determine the statistical effects of interventions and in circumstances where there are no trials.

When conducting quantitative and qualitative reviews that will subsequently be combined, there are specific considerations for managing and integrating the different types of evidence throughout the review process. We have summarised different options for combining qualitative and quantitative evidence in mixed-method syntheses that guideline developers and systematic reviewers can choose from, as well as outlining the opportunities to integrate evidence at different stages of the review and guideline development process.

Review commissioners, authors and guideline developers generally have less experience of combining qualitative and evidence in mixed-methods reviews. In particular, there is a relatively small group of reviewers who are skilled at undertaking fully integrated mixed-method reviews. Commissioning additional qualitative and mixed-method reviews creates an additional cost. Large complex mixed-method reviews generally take more time to complete. Careful consideration needs to be given as to which guidelines would benefit most from additional qualitative and mixed-method syntheses. More training is required to develop capacity and there is a need to develop processes for preparing the guideline panel to consider and use mixed-method evidence in their decision-making.

This paper has presented how qualitative and quantitative evidence, combined in mixed-method reviews, can help understand aspects of complex interventions and the systems within which they are implemented. There are further opportunities to use these methods, and to further develop the methods, to look more widely at additional aspects of complexity. There is a range of review designs and synthesis methods to choose from depending on the question being asked or the questions that may emerge during the conduct of the synthesis. Additional methods need to be developed (or existing methods further adapted) in order to synthesise the full range of diverse evidence that is desirable to explore the complexity-related questions when complex interventions are implemented into health systems. We encourage review commissioners and authors, and guideline developers to consider using mixed-methods reviews and synthesis in guidelines and to report on their usefulness in the guideline development process.

Handling editor: Soumyadeep Bhaumik

Contributors: JN, AB, GM, KF, ÖT and ES drafted the manuscript. All authors contributed to paper development and writing and agreed the final manuscript. Anayda Portela and Susan Norris from WHO managed the series. Helen Smith was series Editor. We thank all those who provided feedback on various iterations.

Funding: Funding provided by the World Health Organization Department of Maternal, Newborn, Child and Adolescent Health through grants received from the United States Agency for International Development and the Norwegian Agency for Development Cooperation.

Disclaimer: ÖT is a staff member of WHO. The author alone is responsible for the views expressed in this publication and they do not necessarily represent the decisions or policies of WHO.

Competing interests: No financial interests declared. JN, AB and ÖT have an intellectual interest in GRADE CERQual; and JN has an intellectual interest in the iCAT_SR tool.

Patient consent: Not required.

Provenance and peer review: Not commissioned; externally peer reviewed.

Data sharing statement: No additional data are available.

Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

QUALITATIVE AND QUANTITATIVE RESEARCH INSTRUMENTS Research tools

A brief explanation with examples about qualitative and quantitative research tools.

Related Papers

Social Indicators Research

Allison Williams

Amanda Bolderston

Interviews are a cornerstone of modern health care research and can be used by both experienced and novice researchers to gather data for projects. For qualitative research, the semistructured or unstructured interview is often used and this can be carried out in various ways. Methods discussed in this directed reading include the face-to-face interview, group interviews such as focus groups, and remote interview conducted by telephone or using the computer. These methods are discussed in detail including advantages and disadvantages of each as well as accompanying practical considerations. Regardless of the method used, there are several matters to be considered in the process of planning, conducting, and analyzing interviews. These consist of initially selecting appropriate participants for the study, preparing a research protocol, and writing useful interview questions designed to capture the information required. It is also important for the researcher to be able to develop and demonstrate rapport with the participant and use attentive listening. Ethical issues relating to the conduct of interviews are also considered, including consent, privacy, and confidentiality. Finally, there is a discussion on the method of analyzing qualitative interview data to prepare for its dissemination in the form of an article or presentation.

Gregor Torkar

Nature conservation research is increasingly concerned with the human component as with the ecosystem or species in focus. Natural scientists that are loyal to their education tend to favour quantitative methods. These methods are not necessarily the only and most suitable tools in human dimensions studies. The aim of this article is to describe the importance of quantitative and qualitative research methods in human dimensions studies pertaining to nature conservation, particularly interviewing. Differences between structured, semi-structured and unstructured interviews are explained and examples given. Additionally, some guidelines for conducting a qualitative interview are presented. Research scientists working with human dimension studies should be aware that nature conservation is a complex cultural problem and that complexity and creativity should therefore be recognized when addressing research methodologies. IZVLEČEK V naravovarstvenih raziskavah dobiva vse večjo težo človeš...

Health and Quality of Life Outcomes

Catherine Acquadro

Dr.PRASANTH VENPAKAL

ASSESSMENT Assessment is a systematic process of gathering information about what a student knows, is able to do, and is learning to do. Assessment information provides the foundation for decision-making and planning for instruction and learning. Assessment is an integral part of instruction that enhances, empowers, and celebrates student learning. Using a variety of assessment techniques, teachers gather information about what students know and are able to do, and provide positive, supportive feedback to students. They also use this information to diagnose individual needs and to improve their instructional programs, which in turn helps students learn more effectively. Assessment must be considered during the planning stage of instruction when learning outcomes and teaching methods are being targeted. It is a continuous activity, not something to be dealt with only at the end of a unit of study. Students should be made aware of the expected outcomes of the course and the procedures to be used in assessing performance relative to the learning outcomes. Students can gradually become more actively involved in the assessment process in order to develop lifelong learning skills. Evaluation refers to the decision making which follows assessment. Evaluation is a judgment regarding the quality, value, or worth of

Md Raihan Ubaidullah

Data collection is the process of gathering and measuring information on variables of interest, in an established systematic fashion that enables one to answer stated research questions, test hypotheses, and evaluate outcomes. The data collection component of research is common to all fields of study including physical and social sciences, humanities, business, etc. While methods vary by discipline, the emphasis on ensuring accurate and honest collection remains the same. The goal for all data collection is to capture quality evidence that then translates to rich data analysis and allows the building of a convincing and credible answer to questions that have been posed (www.en.wikipedia.org). It is an endeavor to discuss about some techniques of data collection.

Lowella Viray

This study attempts to explain and explore the research methods in Information Systems (IS), while on the other it seeks to provide starting point for their use. In order to support the purpose of this study, a guide was written to introduce a toolkit of methods, explaining how to use them, showing how to analyze the data you obtained, and listing techniques to help you further understand each of the research methods used. More importantly, it aims to help novice or expert researchers to determine the appropriate methods they can apply in their research since the guide provided by this study created a thorough distinction between each research methods. The study employs quantitative research that applies basic descriptive qualitative study. In pursuit of this study’s aim, an existing literature and studies review was done. The literature and studies cited in this study, focus mainly in information systems research and research methods. The literature section reviews IS discipline in literature, IS way back from when it started, IS research in literature, issues concerning IS research and its future. To acquire further understanding about quantitative, qualitative and mixed-method in IS, the researcher sites six related studies to review. All the literature and studies reviewed in this section had showed how the nature of IS are constantly changing and most of the study suggests further advancement in IS research. Consequently, research methods like qualitative, quantitative and mixed-methods can make an important contribution to IS research and development. Thus, the work that described in this study plays an important role to help novice and experienced researchers to learn and review as reference, the use of research methods in evaluating Information System (IS) research. Since the work in this study aims to produce an instructional guide based from the curriculum, a selection of the required textbook for the guide was done. The researcher adopts Seif and Champine Criteria for Selecting Era 3, 21st Century Outcomes Curriculum Materials. Other criteria are also added like book review, cost-efficiency, availability, most recent copy, and content to produce neutral results. The assessment for other criteria like book review, cost-efficiency and availability was done adopting Upstill, Craswell and Hawking observations on their case study in Search and Searchability. The assessment was conducted by the researcher with four books about research methods and design employed and after conducting the assessment one book was selected as the primary textbook while the other three books serves as supplementary textbooks. The guide contains three major sections that explores and explains quantitative, qualitative and mixed-method approaches. The distinctions are based on different fundamental questions about methods used in IS research. Such are: (1) What is the method, (2) 2.When it should be used, (3) What do I need to consider, (4) How it should be used, (5) What is the output, (6) How should it be analyzed, and (7) What are the advantage and disadvantages. Within each distinctions includes the different quantitative methods used in information systems like surveys, experimentation particularly quasi-experiment, and statistical analysis. After the discussion, an example was given and list of readings to further enhance the learning on quantitative methods. The next major section of the guide is the qualitative method where it outlines the core qualitative research methods used in information systems research namely, open-ended and survey questions, participant observation, interviews, and document analysis. Example and further readings is included too. The last part of the guide is the discussion on mixed-method approach. The format of the section when it comes to questions is a little bit different compared to quantitative and qualitative sections because the third section is like the integration part of the two first-mentioned methods. In conclusion, the value of this study resides in the learning and knowledge it can provide for the novice or experienced researchers who are planning to conduct research in information systems or any related area of discipline. This guide reflects the wider aim of this study to support the need of further enhancement in Information Systems (IS) research and development. In this, we hope this guide goes in some way to help make the application rate of Information Systems (IS) research high. The topics within this guide are not fully comprehensive that you should follow-up at least some of the references suggested in further readings since the examples and focus is pointing to Information Systems (IS) research. In addition, the information contained in this study will not always appear in order that suits your perspective and circumstances. Due to this reasons, suggestions for future study were offered.

Lisa Farndon

Data generated from quantitative studies is normally in the form of numbers, whereas a qualitative study will generate data that is in the form of words. These words will most commonly be from transcripts of interviews, written observations of situations or documents. A large amount of textual data is normally produced from a qualitative study, which can make analysis a

Jackie Campbell

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

• Systematic Review
• Open access
• Published: 13 June 2024

Infant feeding experiences among Indigenous communities in Canada, the United States, Australia, and Aotearoa: a scoping review of the qualitative literature

• Hiliary Monteith 1 ,
• Carly Checholik 2 ,
• Tracey Galloway 2 ,
• Hosna Sahak 1 ,
• Amy Shawanda 3 ,
• Christina Liu 1 &
• Anthony J. G. Hanley 1 , 4 , 5  

BMC Public Health volume  24 , Article number:  1583 ( 2024 ) Cite this article

152 Accesses

Metrics details

Although exclusive breastfeeding is recommended for the first six months of life, research suggests that breastfeeding initiation rates and duration among Indigenous communities differ from this recommendation. Qualitative studies point to a variety of factors influencing infant feeding decisions; however, there has been no collective review of this literature published to date. Therefore, the objective of this scoping review was to identify and summarize the qualitative literature regarding Indigenous infant feeding experiences within Canada, the United States, Australia, and Aotearoa.

Using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses- Scoping Reviews and the Joanna Briggs Institute Guidelines, in October 2020, Medline, Embase, CINAHL, PsycINFO, and Scopus were searched for relevant papers focusing on Indigenous infant feeding experiences. Screening and full-text review was completed by two independent reviewers. A grey literature search was also conducted using country-specific Google searches and targeted website searching. The protocol is registered with the Open Science Framework and published in BMJ Open.

Forty-six papers from the five databases and grey literature searches were included in the final review and extraction. There were 18 papers from Canada, 11 papers in the US, 9 studies in Australia and 8 studies conducted in Aotearoa. We identified the following themes describing infant feeding experiences through qualitative analysis: colonization, culture and traditionality, social perceptions, family, professional influences, environment, cultural safety, survivance, establishing breastfeeding, autonomy, infant feeding knowledge , and milk substitutes , with family and culture having the most influence on infant feeding experiences based on frequency of themes.

Conclusions

This review highlights key influencers of Indigenous caregivers’ infant feeding experiences, which are often situated within complex social and environmental contexts with the role of family and culture as essential in supporting caregivers. There is a need for long-term follow-up studies that partner with communities to support sustainable policy and program changes that support infant and maternal health.

Peer Review reports

Introduction

Nutritional status is a key aspect of infant health with recommendations for exclusive breastfeeding for the first six months of life, which can also influence and be influenced by maternal health and wellbeing [ 1 , 2 ]. Breastfeeding has several benefits for the health and development of infants, including a reduced risk of ear and respiratory infections, obesity, asthma, skin conditions, childhood leukemia, and gastroenteritis [ 3 , 4 , 5 ]. It also supports bonding between the child and parent with improved intimacy [ 3 ]. Additionally, breastfeeding has several maternal physical and mental health benefits, including a reduced risk of breast and ovarian cancer, depression, and type 2 diabetes due to immunoprotective antibodies in breastmilk [ 3 ]. The World Health Organization (WHO) recommends exclusive breastfeeding for the first 6 months of life and initiation within the first hour after birth; however, less than half of infants 0–6 months old are exclusively breastfed worldwide [ 6 ]. Many countries are not meeting the WHO recommendations, with notable differences between low, middle, and high-income countries [ 2 ]. Differences in breastfeeding initiation rates and duration have been observed between Indigenous and non-Indigenous groups, with 6–10% lower breastfeeding initiation rates and shorter duration for Indigenous peoples [ 7 , 8 , 9 ].

Despite the many benefits of breastfeeding, bottle feeding with milk substitutes is a common form of infant nutrition and its common usage is related to a multi-dimensional set of factors influencing infant feeding decision-making. Breastfeeding is considered a traditional practice within many Indigenous cultures; however, disruptions to traditional lifeways through colonization have influenced intergenerational knowledge sharing, particularly within high-income, settler states like Canada, the US, Australia, and Aotearoa (New Zealand) [ 10 ]. Rollins et al. [ 1 ] summarize factors that influence the global breastfeeding environment including the sociocultural and market contexts, the healthcare system and services, family and community settings, employment, and individual determinants like the mother and infant attributes. However, these core breastfeeding environments for general populations overlook key considerations for Indigenous communities given the unique historical, cultural, and socio-economic contexts specific to Indigenous groups [ 11 ].

Many studies to date have focused on quantitative infant feeding data, incorporating structured questionnaires that have provided some insight into breastfeeding barriers and enablers for Indigenous caregivers [ 7 , 12 , 13 , 14 ]. However, these studies are informed by specific research questions and do not capture important nuances that caregivers experience related to infant feeding. Qualitative research can enhance our understanding of phenomena by providing flexible means for participants to engage in the research topic of interest without the constraints of structured instruments, and can even transform the research by highlighting community needs [ 15 , 16 ]. Qualitative research can also have synergy with Indigenous methodologies, supporting the use of qualitative research with Indigenous communities [ 17 ]. Given the value of qualitative inquiry and breastfeeding as traditional practice for many Indigenous cultures, disrupted by colonial influences and the burden of conditions that breastfeeding has been shown to mitigate [ 3 , 5 , 10 , 11 , 16 , 17 ], it is imperative that we consider Indigenous caregiver infant feeding experiences and perspectives to understand what needs exist as defined by communities and caregivers. Therefore, the overall aim of this scoping review was to identify and summarize the qualitative literature on infant nutrition experiences to inform needs as expressed qualitatively by Indigenous caregivers in Canada, the US, Australia, and Aotearoa. These regions are included given the shared colonial influences on Indigenous peoples with overlapping outcomes on health [ 10 , 18 ]. This review will also assess the qualitative methodologies used to understand what can be learned to inform Indigenous infant feeding services, policies, and research gaps.

Protocol and registration

This scoping review adheres to guidelines from Tricco and colleagues’ [ 19 ] Preferred Reporting Items for Systematic Reviews and Meta-Analyses ( PRISMA) extension for scoping reviews , the Joanna Briggs Institute’s Reviewer’s Manual Chap. 11 [ 20 ], as well as Arksey & O’Malley’s [ 21 ] foundational article on scoping studies. The protocol for the review is registered with the Open Science Framework ( https://doi.org/10.17605/OSF.IO/J8ZW2 ) and published with BMJ Open [ 22 ].

Eligibility criteria

Works included in this review must have focused on Indigenous populations in Canada, the United States, Australia, and/or Aotearoa. These four countries share commonalities in that they are colonial countries in which Indigenous peoples face inequitable health outcomes [ 10 , 18 , 23 ]. The topic of interest for this review was caregivers’ experiences of infant feeding within one or more of these regions. “Caregivers” refer to individuals in the infants’ immediate familial and social circles who are directly responsible for the regular care of the infant. A broad definition of those involved in caregiving was used, recognizing that within many Indigenous communities, traditional adoption practices occur, or biological parents may not be the primary caregivers in part related to complex socio-ecological challenges. The experiences of healthcare professionals were not included as they were not considered “caregivers” by this definition. Works that discussed breastfeeding, as well as alternative forms of infant feeding, such as formula and cow’s milk, were included. Works that only focused on the introduction of solid foods were excluded. To capture caregivers’ experiences of infant feeding, qualitative and mixed-method studies that discussed experiences, perspectives, and/or practices as described by caregivers were included. Studies that used exclusively quantitative methods or that only described an outsider perspective (e.g. health professional) were excluded. Peer-reviewed journal articles and grey literature were included if they met the above criteria, were published in the English language, and were published after 1969 [ 22 ].

Various types of grey literature such as government documents, dissertations, and research reports by academic and non-academic institutions, including Indigenous organizations, were included. Media reports (including videos, news, and blogs) were excluded from the grey literature as they did not follow a research design with results that could be considered alongside the studies included in the review, hindering our ability to compare and critically analyze the results. Similarly, publications that consisted of only an abstract were excluded from both grey and database publications during full-text review as not enough information was present for analysis.

Information sources

The search strategy was created with guidance from a research librarian at the Gerstein Science Information Centre, University of Toronto. The complete search strategy can be found as supplementary material in our published protocol [ 22 ]. Search terms primarily included broad terminology for Indigenous peoples (e.g. Native American) rather than specific Nation names (e.g. Ojibwe) as this would have significantly extended the search term list while not resulting in additional sources given how sources are indexed within Library systems. A database and grey literature search were conducted for this scoping review, completed independently from one another until final data extraction when the data were combined for analysis. For both searches, the reviewers followed a step-by-step process of title and abstract screening, followed by full-text screening, and then data extraction.

The database search planning and calibration occurred in August and September of 2020, and all data were exported in English on October 20, 21, and 22 of 2020. Exportation occurred over three days given feasibility of exporting the high number of citations and time capacity of the reviewers. A total of 16734 relevant sources available in the following databases were included: Medline, Embase, CINAHL, PsycINFO, and Scopus. These databases were selected to ensure a broad range of research given the multidisciplinary nature of research on this topic. The grey literature search consisted of a targeted search of a variety of Indigenous focused websites specific to the four countries and a thorough Google search with each of the country-specific Google versions (Google.com.au, Google.co.nz, Google.ca, and Google.com) where the first 10 pages of results were reviewed (Supplementary File 1 ). Lastly, Indigenous Studies Portal (I-Portal) was searched as part of the grey literature as this database uses a different indexing system than other research databases. The Canadian Agency for Drugs and Technologies in Health (CADTH)’s “Grey Matters” checklist [ 24 ] was used in the planning and tracking of grey literature searches and findings.

The results of the database search including 16734 citations were uploaded to Covidence (Veritas Health Innovation Ltd., Melbourne, Australia), a data management platform for systematic and scoping reviews, where 3928 duplicates were automatically removed. The 284 results of the grey literature search were recorded on Google Sheets (Alphabet Inc. California, USA) and 146 duplicates were manually removed by the reviewers. Due to the large number of results retrieved in the database and grey literature search, a hand-search of reference lists was not conducted.

A list of key words developed by HM were searched on each site and can be found in Supplementary File 1 . The grey literature search was completed by HM, CC, and HS with all reviewers assigned to search a Country-specific Google database for one of the included countries. Using a template created by Stapleton [ 25 ] at the University of Waterloo based on methods described by Godin et al. [ 26 ], the reviewers kept track of which search terms were searched on the websites, the number of results retrieved, and the number of items screened and saved for further full-text analysis. If a website did not have a search bar, relevant tabs were examined for research, resources, and other publications. I-Portal was originally searched on August 15th, 2021 (yielding 10 results), however the search was revised to remove Indigenous search terms as the database was already Indigenous-specific. The search was repeated on August 18th, 2021, and yielded 77 additional results. The grey literature search was completed between May 25, 2021 – August 18, 2021. No search limitations or filters were used for the grey literature search or the database search.

The database abstract screening was initially completed by HM and CC starting in October 2020. They were then joined by HS and CL in February 2021. To ensure all reviewers had a shared understanding of the eligibility criteria, two search results were screened together and each reviewer discussed their reasoning for inclusion or exclusion. HM also hosted an introductory meeting to review the screening process using Covidence Software [ 27 ] in detail. All 12806 database results were saved in Covidence [ 27 ].

Abstract and full-text screening was completed in Covidence by two independent reviewers. Any conflicts at the screening stage were resolved by AH after all the results had been screened by two reviewers. Full-text screening was completed by HM, AH, and CC, and when conflicts arose, the reviewers met to discuss the difference in opinion until a consensus was reached. A third reviewer joined to offer impartial opinions for full-text conflicts.

Grey literature results were not imported to Covidence. Instead, the team used Google Sheets to organize the publications. Similar to the database review process, each study was screened by two independent reviewers and conflicts were resolved by a third party and discussed for consensus. Full-text review of the grey literature was completed by HM, AH, CC, and HS.

Data extraction and analysis

HM compiled a list of variables to extract (Supplementary File 2 ), and the data extraction was completed by HM, AH, and CC in Covidence for database results and Google Sheets for the grey literature. The extraction template was reviewed and tested by all three reviewers using the same two articles. Discussion about any areas of confusion followed by minor edits to the data extraction template were completed prior to extraction.

Only one reviewer extracted data from most publications, however in circumstances where an article was complex or data extraction was not clear given the format of the article, two reviewers extracted data from the publication. An additional subset of five publications were also randomly double-reviewed by HM to ensure consistency in data extraction. There were an additional two articles that were excluded at this step after review and discussion by AH and HM.

Review findings using the extraction template (supplementary file 2 ) were exported into Microsoft Excel (Microsoft Corporation, Washington, USA) and reviewed by HM. HM compiled all data and completed summary figures for variables of interest. The primary analysis consisted of a qualitative review of the included papers’ results and recommendations using a thematic synthesis informed by grounded theory and meta-ethnography, where the included papers are synthesized together, and interpreted using descriptive and analytical themes [ 28 ]. Similar to grounded theory, this process was inductive and identifies themes through comparisons. HM reviewed all extracted data from the excel files, coding for overlapping themes and taking notes throughout. The full-text of the extracted papers were then revisited to identify overall concepts, followed by descriptive themes. Categorization of descriptive themes was completed based on the results and interpretations of included papers. Descriptive themes were refined through additional comparisons between papers. The same analytical process was used for both database and grey literature results, and final analysis involved the integration of themes from the database and grey literature papers. Supplementary file 3 provides a summary table of the included papers in this scoping review.

Characteristics of included articles

Of the final sample of 46 articles from which data was extracted (Fig.  1 ), there were studies from each of the four countries, with the most studies (39%) published from Canada. In addition, this qualitative literature on infant feeding included several Indigenous groups within the four countries. The studies retained in this review included authors who identified as either Indigenous or non-Indigenous, and several did not mention positionality (Fig.  2 ). 13% more grey literature studies discussed positionality and had Indigenous sole authorship compared to the database papers. Regarding methodologies utilized, several described Indigenous methodologies and used thematic analysis as an analytic tool (Figs.  3 and 4 ). However, a third of the studies did not describe their theoretical foundations for the qualitative inquiry. Over 60% of the studies were published in the fields of public health and/or nursing as per the authors stated fields of study and/or the Journal’s field, and although there were studies published from 1984 to 2019, 50% of the retained papers were published after 2010.

PRISMA flow diagram for studies identified, screened, and included in this review from both database and grey literature searches. Note that records not retrived are those in which the full-text was not accessible. This diagram was created from the PRISMA 2020 statement [ 29 ]

Author positionality as described in the retained papers

Summary of analytic tools used in the retained studies

Summary of theoretical foundations informing the retained studies’ methodologies

Analysis revealed a variety of important themes that aligned with Indigenous and public health perspectives on health, including the socioecological model. There were twelve final overarching themes including colonization, social perceptions, family, professional influences, culture and traditionality , environment (i.e. built environment) , autonomy, survivance, infant feeding knowledge, cultural safety , milk substitutes , and establishing breastfeeding with evidence of connections among these themes. These themes are shown in Fig.  5 in a circular pattern where the themes intersect with the infant and caregiver represented at the centre. This model is conceptually aligned with that of Dodgson et al. [ 30 ], who considered the “contextual influences within the social structures of family and community, Ojibwe culture, and mainstream culture.”

Scoping review research model of themes

The twelve final themes are shown as the main influences on infant feeding experiences. The themes are arranged in a circular pattern with the infant and caregiver represented at the centre, emphasizing the connection between all of the themes

Theme one: colonization

There were 14 papers that discussed colonization of Indigenous peoples as a key factor influencing infant feeding decisions and experiences (Fig.  6 ) [ 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ]. Colonization has meant the dispossession of land and limited access to culturally safe healthcare, malnutrition, and loss of language through residential schools, loss of culture and traditional knowledge through assimilation and separation of families, disrupting breastfeeding practices and limiting income for infant formula. Eni et al. [ 36 ] described the policies leading to evacuation from communities to tertiary-care hospitals for birthing as the medicalization of birthing practices, which creates various challenges for First Nations women in Canada. One participant also shared about the impacts of intergenerational trauma related to colonization on breastfeeding, ‘‘You can’t teach about breastfeeding technique and think things will change. It’s the spirit that’s been affected, our experience with trauma. Our women need to relearn how to bond with their children.’’.

A qualitative study with Aboriginal Australian first-time mothers noted the disruptions to breastfeeding practices over time, providing a historical chart detailing how infant feeding practices changed as a result of colonial influences [ 38 ]. Brittany Luby [ 39 ] described how hydroelectric flooding from 1900 to 1975 in Northwestern Ontario reduced breastfeeding practices for Anishinabek mothers and their infants. Although not all studies specifically discussed history and colonization, those that considered the broader historical context highlighted how important this issue is in understanding the factors that lead to infant feeding decisions, particularly those that do not align with breastfeeding as a traditional feeding practice.

Frequency of identified themes in the database papers and the grey literature

Theme two: culture and traditionality

Culture , including traditionality, was the second most described theme throughout all papers, identified both directly and indirectly in 31 papers (Fig.  6 ) [ 30 , 31 , 32 , 34 , 35 , 37 , 38 , 39 , 40 , 41 , 42 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 ]. The Navajo Infant Feeding Project focused on cultural beliefs influencing infant feeding practices within three Navajo communities in the United States [ 48 ] and emphasized breastfeeding’s significance for nutritional, physical, and psychological health where mothers not only pass along physical health benefits, but also their wellbeing to their children. The Baby Teeth Talk Study in Cree communities in Northern Manitoba, Canada, has identified breastfeeding as a cultural intervention for the prevention of early childhood caries [ 52 ]. Several studies included a variety of generations in data collection, contributing to rich discussion of how breastfeeding rates and connection to traditionality has changed in some communities [ 48 , 57 , 64 , 65 ]. For example, grandmothers living on the Fort Peck Reservation in Montana, US, were interviewed about their perspectives on infant feeding [ 65 ]. In one of the ethnographic studies, there was a specific focus on the Ojibwe culture relating to infant feeding practices from the perspective of mothers, professionals who were also community members, and Elders [ 35 ]. This study emphasized the holistic and collective worldview of the community, influencing women’s roles within the family and how teachings were passed on from generation to generation [ 35 ]. This was considered to be important in influencing effective and culturally safe breastfeeding promotion. Within the Northwest Territories, Canada, Moffitt and Dickinson [ 53 ] supported breastfeeding knowledge translation tools for Tłı̨chǫ women with one of the themes focused on factors that “pull to breastfeeding,” including breastfeeding as a traditional feeding method. In general, Indigenous communities described breastfeeding as a cultural practice; however, how this is supported and the traditional knowledge surrounding this practice may differ from community to community. Therefore, health providers must be aware of community-specific protocols and support these within programs and recommendations.

Theme three: social perceptions

Societal influences are often considered alongside cultural perspectives of infant feeding; therefore, this theme was also commonly discussed in the papers retained in this scoping review (Fig.  6 ) [ 30 , 32 , 33 , 36 , 37 , 38 , 40 , 42 , 49 , 50 , 52 , 54 , 57 , 58 , 59 , 61 , 64 , 66 , 67 , 68 , 69 , 70 , 71 ]. In New South Wales, Australia, Aboriginal mothers and key informants noted the need for “a safe place to feed,” including concerns about the social acceptability to breastfeed in public [ 32 ]. Broader social “norms” are also discussed as influencing maternal behavior [ 68 ], and respondents in some studies expressed concern about judgements from others [ 32 , 36 ]. Tapera et al. [ 40 ] described concerns about social pressures and a lack of support with one grandparent sharing, “well here in New Zealand, I know we have a problem with this [breast-feeding], especially when mothers go out and they breast-feed their babies in public. There’s a lot of people that moan and groan about this.” Similarly, regarding social norms, a grandmother living in the US shared,

“a long time ago that, it [breastfeeding] was acceptable and nobody had any qualms about it but today, I mean you read continually about, people, mother’s tryin’ ta breastfeed and they’re being chased out a places or stores or people are rude about it […]. Society’s changed, you know, it’s […] society, has come to the point where it’s […] trying to tell us what’s the right way ta live what’s the right way ta raise our kids” [ 65 ].

Dodgson et al. [ 30 ] described how in an Ojibwe community in Minnesota, US, participants noted the dominant societal influences in contrast to community traditions, with women making an effort to engage in traditional practices. The sexualization of breasts in mainstream society sometimes influenced Indigenous mothers’ infant feeding experiences [ 36 ], although Ojibwe caregivers in Minnesota attributed shyness with breastfeeding to traditional value opposed to sexualization of breasts [ 30 ]. Eni et al. [ 36 ] included sexual objectification of the feminine body as a subtheme in their study, describing how this social perception damages maternal mental health, creating a barrier to breastfeeding. While shifting social norms is a significant challenge, breastfeeding supports can address concerns about the sexual objectification of breasts by creating safe spaces for parents to talk about the challenges and ensure that parents have access to mental health resources.

Theme four: family

Dodgson et al. [ 30 ] described family as a pattern that influences breastfeeding intersecting with the social structures of the community, culture, and the broader society. There were 33 other papers that described the influence of family on infant feeding practices making this the most discussed theme (Fig.  6 ) [ 30 , 31 , 32 , 33 , 36 , 38 , 39 , 40 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 53 , 54 , 55 , 57 , 58 , 59 , 60 , 61 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 ]. Native American mothers living in six communities highlighted the importance of family as a key theme [ 47 ]. One mother shared, “For me, it’s my mom definitely [whose advice is most important] because she has had three kids and I lived with her or near her for all of my kids. So I’ve always gone to her first for advice.” This was echoed by many other participants with a paraprofessional adding, “family [advice is most important], because they are around their family most. And they always hear from their aunties, or from grandma, baby’s fussing, baby must be hungry, baby needs this and baby needs that.” The Baby Basket Program in Cape York, Australia identified that empowering families was the foundation of the program to ensure that mothers and their partners were equipped for the arrival of their babies [ 50 ]. Family often plays an integral role in supporting mothers in infant feeding practices. Bauer and Wright [ 45 ] note that even when mothers don’t have other supports or conditions in place to support breastfeeding, they may still choose to breastfeed if their family is supportive. However, when this support is lacking, mothers find it challenging to breastfeed [ 31 , 36 ]. Some studies identified the significance of family in the study design, integrating family caregiver perspectives in data collection [ 64 , 65 ]. Therefore, health programs and research studies should consider the role and experience of non-primary caregivers within family networks for infant and maternal health and nutrition.

Theme five: professional influences

This theme represents the influence of formal systems including healthcare professionals, health and social programs, child services, and the legal system. In total, there were 26 papers that referenced professional influences on infant feeding experiences (Fig.  6 ) [ 30 , 31 , 33 , 38 , 41 , 42 , 43 , 45 , 47 , 48 , 50 , 51 , 52 , 54 , 58 , 59 , 61 , 62 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 ]. Some studies incorporate health workers as participants in data collection [ 47 , 50 , 65 ]. One health paraprofessional shares about some of the pressures experienced by mothers to formula feed, “sometimes hospitals and doctors want to push formula in bottles on moms [ 47 ].” One of the main themes in a study with Sioux and Assiniboine Nations in the US was the ‘ Overburdened Healthcare System’ , describing a lack of resources and infrastructure to support breastfeeding, including a subtheme of mistrust in the healthcare system due to previous negative experiences such as forced sterilization of Indigenous women [ 65 ]. However, some caregivers also expressed positive healthcare supports, “when I was at home, [clinic midwife] and [lactation consultant made home visits] … they encouraged me … And then it started getting a little bit better, but it was still a bit hard. Now he feeds pretty all right [ 73 ].” Professional influences on infant feeding are nuanced and may differ significantly within various contexts and individuals; therefore, tailored interventions are needed.

Theme six: environment

This theme represents the external variables within the built environment that influence decision making including work, school, remoteness, and cost of formula. Eighteen papers addressed this theme [ 30 , 31 , 44 , 45 , 46 , 47 , 48 , 49 , 51 , 53 , 58 , 59 , 66 , 67 , 68 , 70 , 71 , 72 ]. Wright et al. [ 74 ] specifically considered the challenge of breastfeeding with maternal employment among the Navajo population in the US. In Bauer and Wright’s [ 45 ] study that explored infant feeding decision models, they identified that work and school are part of the decision-making process on whether to breastfeeding or to use formula, but even when these environmental challenges are present they can be further influenced by other factors, like family . For example, a mother may choose to breastfeed and use a breast pump to navigate work/school schedules, but family members may recommend that they can incorporate formula; decision-making is not only about the main caregiver’s desires but can involve various decision-makers.

Theme seven: autonomy

This theme describes parents’ freedom to make infant feeding decisions that fit for them and their priorities. Maternal desire to breast- or bottle-feed was discussed in select papers in this review [ 45 , 51 ]. In addition, other papers describe parents’ freedom to do activities outside of infant feeding in the early months of baby’s life with discussion of time required to breastfeed or prepare bottles for feeding [ 31 , 58 , 72 , 74 ]. A key informant in a study with an Aboriginal community in Northern New South Wales, Australia, shares, “they want to breastfeed, but then it comes down to when they want to go out, or keep up with their man [ 32 ].” Some parents report that they experienced judgements from others or feel forced into making a specific decision on infant feeding method, highlighting a desire to have support and freedom to make their own decisions [ 36 , 56 ].

Theme eight: infant feeding knowledge

Several studies emphasize the importance of knowledge on infant feeding experiences, highlighting the value of infant feeding education, both within the overall healthcare system and from traditional teachings [ 30 , 32 , 35 , 40 , 42 , 43 , 47 , 52 , 57 , 58 , 62 , 64 , 66 , 67 , 68 , 69 , 70 , 71 , 72 ]. Within the theme of addressing feeding challenges in one study [ 66 ], a caregiver shared how knowledge helped her to work through a challenge,

“He did start fussing at about 6 weeks and that was kind of hard because I thought, ‘No, I have got this perfect now, and he has started to muck up’. But then I read, because I had those booklets and I read that sometimes they — at a certain point — they get a bit fussy and you just have to work through it. [Ml7]” [ 66 ].

Traditional breastfeeding knowledge is important for many communities; one Anishinaabe community knowledge keeper shared that “breast milk is a gift and a medicine a mother gives her child” [ 35 ]. This study also discusses feeding patterns as shared by Elders and traditional teachers. Traditional knowledge considers holistic perspectives of health where caregivers are also focused on the baby’s spiritual wellbeing [ 48 , 56 ].

Theme nine: milk substitutes

Bottle feeding (formula or canned milk) and solid foods are described in several papers as alternatives or complements to breastfeeding [ 31 , 33 , 34 , 37 , 39 , 47 , 48 , 49 , 51 , 52 , 53 , 58 , 66 , 67 , 74 , 75 ]. In Neander and Morse’s [ 37 ] study with a Cree community in Alberta, Canada, bottle feedings were offered particularly when mothers felt that they were not producing adequate milk supply to meet the baby’s nutritional needs. Insufficient milk supply is echoed as a concern in several other papers resulting in complementary bottle feeding or weaning [ 48 , 51 , 56 , 66 , 67 ]. A Māori father shares,

“about the second week, baby just wanted more food. She (partner) would end her day and baby was just hungry. We had to [give her] the bottle and then she would be finally satisfied. It wasn’t that she made a choice. Baby was actually demanding more and more and she couldn’t produce it. (First-time father, mid 20’s) [ 56 ].”

This theme particularly overlaps with autonomy as parents balance infant feeding decisions with breastmilk supply, work, school, and other personal commitments.

Theme ten: cultural safety

Indigenous caregivers interact with a variety of health services postnatally; however, there is a need to address cultural safety within the healthcare system. Twelve retained papers highlighted this theme either directly as one of their themes or as part of another theme (Fig.  6 ) [ 30 , 31 , 44 , 47 , 50 , 64 , 66 , 67 , 69 , 71 , 73 , 74 ]. One health worker in Victoria, Australia, shared,

“I can’t say often enough or long enough, loud enough the ideal for children 0–8 is to have access to maternal and child health. You might say ‘oh yes, they’ve got access to mainstream and they’re culturally going to put up a few Indigenous prints in their rooms’ It’s not the same. Our families are telling us with their feet it’s not the same.”

Mothers expressed a desire for more traditional infant feeding knowledge within services and culturally relevant supports [ 47 , 64 ]. A study that focused on a baby basket program to support families in a Murri (Local Australian Aboriginal Group) Way identified how important culturally safe language and relationships are for families,

“…the nurse is also learning what the best way is to approach a family and what the wording has to be, what the languaging is around things, what the traditional words are for Indigenous language and are appropriate for use in certain circumstances” [ 50 ].

Theme eleven: survivance

Indigenous caregivers experience a variety of hardships; however, through resistance and survival, they practice cultural revitalization [ 76 ]. This theme is discussed in 15 papers and is often described through a lens of maternal mental health (Fig.  6 ) [ 30 , 31 , 33 , 43 , 53 , 54 , 57 , 58 , 59 , 63 , 64 , 66 , 68 , 36 , 74 ]. Some parents express feelings of guilt for the challenges they encounter, which can further contribute to negative emotions [ 58 ]. Maternal mental and emotional health can impact infant feeding experiences,

“…sometimes people’s psychological health, mental health is more of a risk factor, you know if you’re not sleeping and you’re bordering on depression and you’re not coping well and you can’t get the baby to latch and you’re constantly feeling like a failure and you can’t get out of that rut, is it worth it?…People have to decide that for themselves. (Key Informant #5)” [ 33 ].

A grandmother in the Northwest Territories of Canada noted the disembodiment caused by residential schools as expressed as a disconnection between physical experiences and relationships,

“You know in those days, I mean residential school. In those days, they never did talk about their body parts because I think they were too ashamed [of your body] to say to your kids. I never did hear it [breastfeeding] from my sisters or nobody in the family. They were so private (L151-156)” [ 57 ].

Traumatic experiences, like residential schools, can have a lasting impact on how caregivers navigate motherhood and infant feeding, and the support they receive from family members.

Theme twelve: establishing breastfeeding

There are several practical challenges that mothers encounter while breastfeeding like pain, latching issues, and low milk supply, discussed in 11 of the studies (Fig.  6 ) [ 48 , 51 , 54 , 56 , 58 , 61 , 66 , 68 , 71 , 72 ]. A mother shared,

“He wouldn’t latch on all the time, like, the nurses and stuff tried to help me but then it would be all frustrating…. He didn’t really know what to do. He tried and then they gave him formula. He really loved it. [MI5]” [ 66 ].

Although these challenges are most discussed at the beginning of breastfeeding, sometimes concerns arise when babies are older.

“Yeah it was 8 or 9 months after she was born. After a while there was too much pressure on me. She was getting up all through the night and she would eat and eat and eat and not get full…” [ 33 ].

Overall, many caregivers reported that breastfeeding is difficult; therefore, supports that consider the variety of challenges that can arise are needed.

Study recommendations

The studies included in this review were published over three decades starting in 1984 until 2019 and were completed with various Indigenous communities in four countries. We anticipated that earlier work would demonstrate markedly different infant feeding recommendations than more recent research; however, this was not necessarily the case. For example, cultural safety is a more recent discussion within the health literature; however, although we see some discussion of this in more recent studies, studies in the 80’s and 90’s also highlight the importance of incorporating traditional teaching and consulting community members [ 37 , 48 ]. Therefore, supporting Indigenous self-determination where health professionals provide culturally appropriate care is essential.

In addition to topics related to cultural safety, various studies highlight a need for community-driven and local knowledge to inform programs and policies related to infant nutrition [ 31 , 47 , 57 , 64 , 75 ]. Several studies also focus on infant feeding specific programs and behavioral changes in their recommendations [ 47 , 50 , 65 ]; however, many of these studies also highlight the need to expand beyond the individual’s role in decision making and address the broader social and environmental factors such as the workplace, healthcare infrastructure, social perceptions, among others, that influence infant feeding decisions. For example, Eni et al. [ 36 ] note that there are a complexity of factors resulting in various breastfeeding environments. These structural, social and cultural contexts are discussed throughout several of the grey literature texts as well [ 32 , 33 ]. It is also important to note that in the most recently published database paper, maternal mental health is directly addressed in the recommendations and this is the only paper with this focus for next steps [ 65 ]. Interventions that target socio-ecological factors based on the included papers’ recommendations for infant feeding are summarized in Fig.  7 .

(Adapted from Rollins et al. 2016)

The components of Indigenous infant feeding environments informed by community-based interventions

This scoping review presents and summarizes the findings reporting Indigenous infant feeding experiences within the qualitative literature in Canada, the US, Australia, and Aotearoa. Twelve themes were identified which summarize the literature including culture and traditionality , colonization, family, environment, social perceptions, professional influences, milk substitutes, breastfeeding initiation, cultural safety, survivance, infant feeding knowledge, and autonomy. The most prevalent themes discussed by caregivers and researchers in the included papers were family and culture/traditionality . The frequency of these two themes highlight the significant impact of family and culture/traditionality on infant nutrition decision-making for Indigenous caregivers and overlaps with components of the socio-ecological model [ 77 ]. This focus on family and culture/traditionality also emphasizes the importance of familial relationships and a collective mentality within traditional life ways for many Indigenous communities in these regions on infant nutrition and care practices.

In their informative global breastfeeding paper, Rollins and colleagues’ [ 1 ] conceptualize the components that contribute to the breastfeeding environment at multiple levels, overlapping with the social determinants of health. In this review, we observed that caregivers report similar components of the breastfeeding environment; however, these components seem to be described collectively, rather than as separate contexts. This is evident in the recommendations proposed by authors with a large focus on local and community-specific leadership, multidisciplinary interventions, and cultural safety in response to historical traumas, particularly within the healthcare system (Fig.  7 ). This aligns with Indigenous epistemology with an emphasis on the collective and interconnectedness of all things where power is manifested together, not over one another, and is based in local land-based knowledge [ 78 , 79 ].

A primary recommendation echoed within many of these studies was the need for community engagement in program and policy development [ 34 , 47 , 50 , 64 ]. This may need to be expanded upon to support Indigenous self-determination of policy and programs related to infant feeding where community members are not only engaged but leading the way forward in maternal and infant health. It is important to note that there have been changes over time in how these recommendations and perspectives are discussed and the role of the health professional, particularly related to cultural safety. For example, although similar concepts are discussed in Neander and Morse’s paper published in 1989, ‘cultural safety’ is not used as the terminology, which has been expanded upon in recent years by Indigenous and non-Indigenous scholars [ 37 , 80 , 81 ].

Related to this focus on health professionals and cultural safety, it’s important to distinguish that in many of the positive experiences expressed by participants in the studies, these interactions seemed to be primarily with professionals interacting closely with families. For example, midwives, who make home visits, were often included as part of positive experiences. In the literature, there is an emphasis on including practitioners who can build strong relationships with families through home visits and regular community engagement in routine services, which supports cultural safety within the healthcare system [ 82 , 83 ]. Health professional regulatory bodies should consider implementing practice competencies that support professionals to build and navigate strong and ethical relationships with clients/patients. Similarly, healthcare settings that serve Indigenous peoples should consider processes and therefore, facility infrastructures that enable close family-client-professional interactions. An example of this implementation with positive client experiences is the Toronto Birthing Centre, which uses an Indigenous framework and has birthing rooms with space for family [ 84 ].

The studies in this review are written within various fields of research; therefore, there were differences in methodological reporting. Future qualitative work should be thorough in reporting theoretical foundations to provide clarity of how the analyses and overall projects are approached (Fig.  4 ) [ 85 ]. Given the limited studies that report author/researcher positionality (Fig.  2 ), this may be an important addition in forthcoming work as a means of respecting Indigenous and qualitative literature conventions where we recognize that positionality influences ontological origins [ 86 ]. We challenge the academy to recognize that Indigenous and local knowledges are required within Indigenous health research and dissemination practices, while acknowledging our own limitation in this review of a single country authorship team.

This systematic scoping review utilized a rigorous search strategy that limited the possibility of missing relevant publications; however, it was time intensive. PRISMA-ScR guidelines were followed with two independent reviewers at each stage, enabling reproducibility of this review. The inclusion of the grey literature is a strength in this study as it captured important papers that were not published in peer-reviewed journals, often from Indigenous authors and communities (many of which were graduate dissertations), which was a priority in this review. A possible limitation is the exclusion of work that only discussed the introduction to solid foods; it is possible that this excluded an important conversation about the differences of introducing solids, like traditional foods from an Indigenous group’s perspective. In addition, the topic of this review is multidisciplinary; therefore, it is possible that although effort was made to include a broad range of research field databases in the search, relevant sources may have been missed.

In conclusion, this scoping review highlights important considerations for infant feeding environments within Indigenous communities with a focus on family and culture. Based on caregiver experiences, Indigenous breastfeeding supports must be community led with a focus on local capacity and traditional teachings. An emphasis on an intergenerational perspective that considers structural and systems approaches including cultural safety within healthcare, addressing maternal mental health, and consideration of sustainability over time is encouraged. Future work should focus on these key areas through strength-based research approaches, grounded in strong relationships and long-term follow-up.

Data availability

All data generated or analysed during this study are available from the corresponding author on reasonable request.

Rollins NC, Bhandari N, Hajeebhoy N, Horton S, Lutter CK, Martines JC, et al. Why invest, and what it will take to improve breastfeeding practices? Lancet. 2016;387:491–504. https://doi.org/10.1016/S0140-6736(15)01044-2

Article   PubMed   Google Scholar  

Victora CG, Bahl R, Barros AJD, França GVA, Horton S, Krasevec J, et al. Breastfeeding in the 21st century: epidemiology, mechanisms, and lifelong effect. Lancet. 2016;387:475–90. https://doi.org/10.1016/S0140-6736(15)01024-7

Ip S, Chung M, Raman G, Chew P, Magula N, DeVine D et al. Breastfeeding and maternal and infant health outcomes in developed countries. Evid ReportTechnology Assess 2007:1–186.

Dieterich CM, Felice JP, O’Sullivan E, Rasmussen KM. Breastfeeding and Health outcomes for the Mother-Infant Dyad. Pediatr Clin North Am. 2013;60:31–48. https://doi.org/10.1016/j.pcl.2012.09.010

Binns C, Lee M, Low WY. The Long-Term Public Health Benefits of Breastfeeding. Asia Pac J Public Health. 2016;28:7–14. https://doi.org/10.1177/1010539515624964

World Health Organization. Infant and young child feeding. World Health Organ 2021. https://www.who.int/news-room/fact-sheets/detail/infant-and-young-child-feeding (accessed April 28, 2022).

Manhire KM, Williams SM, Tipene-Leach D, Baddock SA, Abel S, Tangiora A, et al. Predictors of breastfeeding duration in a predominantly Māori population in New Zealand. BMC Pediatr. 2018;18. https://doi.org/10.1186/s12887-018-1274-9

Australian Government. 2.20 Breastfeeding practices. Aborig Torres Strait Isl Health Perform Framew 2021. https://www.indigenoushpf.gov.au/measures/2-20-breastfeeding-practices (accessed April 28, 2022).

Health Canada. Duration of Exclusive Breastfeeding in Canada: Key Statistics and Graphics (2009–2010). Gov Can 2012. https://www.canada.ca/en/health-canada/services/food-nutrition/food-nutrition-surveillance/health-nutrition-surveys/canadian-community-health-survey-cchs/duration-exclusive-breastfeeding-canada-key-statistics-graphics-2009-2010.html (accessed April 28, 2022).

Smylie J, Crengle S, Freemantle J, Taualii M. Indigenous birth outcomes in Australia, Canada, New Zealand and the United States – an overview. Open Womens Health J 2010:7–17. https://doi.org/10.2174/1874291201004010007

King M, Smith A, Gracey M. Indigenous health part 2: the underlying causes of the health gap. Lancet. 2009;374:76–85. https://doi.org/10.1016/S0140-6736(09)60827-8

Houghton MD, Graybeal TE. Breast-feeding practices of native American mothers participating in WIC. J Am Diet Assoc. 2001;101:245–7.

Article   CAS   PubMed   Google Scholar  

Rhodes KL, Hellerstedt WL, Davey CS, Pirie PL, Daly KA. American Indian breastfeeding attitudes and practices in Minnesota. Matern Child Health J. 2008;12(Suppl 1):46–54. https://doi.org/10.1007/s10995-008-0310-z

Eckhardt CL, Lutz T, Karanja N, Jobe JB, Maupome G, Ritenbaugh C. Knowledge, attitudes, and beliefs that can influence infant feeding practices in American Indian mothers. J Acad Nutr Diet. 2014;114:1587–93. https://doi.org/10.1016/j.jand.2014.04.021

Article   PubMed   PubMed Central   Google Scholar  

Clarke AM, Jack B. The benefits of using qualitative research. Prof Nurse Lond Engl. 1998;13:845–7.

CAS   Google Scholar  

van’t Riet A, Berg M, Hiddema F, Sol K. Meeting patients’ needs with patient information systems: potential benefits of qualitative research methods. Int J Med Inf. 2001;64:1–14. https://doi.org/10.1016/S1386-5056(01)00185-X

Article   Google Scholar  

Kovach M. Indigenous methodologies: characteristics, conversations, and contexts. Canada: University of Toronto; 2009.

Google Scholar  

Anderson I, Crengle S, Kamaka ML, Chen T-H, Palafox N, Jackson-Pulver L. Indigenous health in Australia, New Zealand, and the Pacific. Lancet Lond Engl. 2006;367:1775–85. https://doi.org/10.1016/S0140-6736(06)68773-4

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169:467–73. https://doi.org/mundel.

Peters M, Godfrey C, McInerney P, Munn Z, Tricco A, Khalil H. Chapter 11: Scoping Reviews (2020 Version). Aromataris E Munn Z Ed. Joanna Briggs Inst. Rev. Man., JBI; 2020.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8:19–32. https://doi.org/10.1080/1364557032000119616

Monteith H, Galloway T, Hanley AJ. Protocol for a scoping review of the qualitative literature on indigenous infant feeding experiences. BMJ Open. 2021;11:e043476. https://doi.org/10.1136/bmjopen-2020-043476

Yeates KE, Cass A, Sequist TD, McDonald SP, Jardine MJ, Trpeski L, et al. Indigenous people in Australia, Canada, New Zealand and the United States are less likely to receive renal transplantation. Kidney Int. 2009;76:659–64. https://doi.org/10.1038/ki.2009.236

Canadian Agency for Drugs and Technologies in Health (CADTH). Grey Matters: a practical tool for searching health-related grey literature 2015.

Stapleton J. Grey Literature Search Plan Template 2018.

Godin K, Stapleton J, Kirkpatrick SI, Hanning RM, Leatherdale ST. Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada. Syst Rev. 2015;4:138. https://doi.org/10.1186/s13643-015-0125-0

Covidence. Systematic Review Software n.d.

Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009;9:59. https://doi.org/10.1186/1471-2288-9-59

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. https://doi.org/10.1136/bmj.n71

Dodgson JE, Duckett L, Garwick A, Graham BL. An ecological perspective of breastfeeding in an indigenous community. J Nurs Scholarsh. 2002;34:235–41. https://doi.org/10.1111/j.1547-5069.2002.00235.x

Cormier JA. The infant feeding experiences of Mi’kmaw women: a Feminist Phenomenological Inquiry. Dalhousie University; 2014.

Helps C. Exploring Infant Feeding Choices in the Northern NSW Aboriginal Community 2014.

Gauld TK. The infant feeding experiences of urban Aboriginal mothers: implications for universal breastfeeding policy. Lakehead University; 2009.

Cidro J, Zahayko L, Lawrence H, McGregor M, McKay K. Traditional and cultural approaches to childrearing: preventing early childhood caries in Norway House Cree Nation, Manitoba. Rural Remote Health. 2014;14:2968.

PubMed   Google Scholar  

Dodgson J, Struthers R. Traditional breastfeeding practices of the Ojibwe of Northern Minnesota. Health Care Women Int. 2003;24:49–61.

Eni R, Phillips-Beck W, Mehta P. At the edges of embodiment: determinants of breastfeeding for first nations women. Breastfeed Med off J Acad Breastfeed Med. 2014;9:203–14. https://doi.org/10.1089/bfm.2013.0129

Neander WL, Morse JM. Tradition and change in the Northern Alberta Woodlands Cree: implications for infant feeding practices. Can J Public Health Rev Can Sante Publique. 1989;80:190–4.

Helps C, Barclay L. Aboriginal women in rural Australia; a small study of infant feeding behaviour. Women Birth J Aust Coll Midwives. 2015;28:129–36. https://doi.org/10.1016/j.wombi.2014.12.004

Luby B. From milk-medicine to Public (Re)Education Programs: an examination of Anishinabek Mothers’ responses to hydroelectric flooding in the Treaty #3 District, 1900–1975. Can Bull Med Hist Bull Can Hist Med. 2015;32:363–89. https://doi.org/10.3138/cbmh.32.2.363

Tapera R, Harwood M, Anderson A. A qualitative Kaupapa Maori approach to understanding infant and young child feeding practices of Maori and Pacific grandparents in Auckland, New Zealand. Public Health Nutr. 2017;20:1090–8. https://doi.org/10.1017/S1368980016002950

Crosschild C, URBAN INDIGENOUS MOTHERS’ EXPERIENCES WITH POSTNATAL NURSING. CARE IN SOUTHERN ALBERTA: a BLACKFOOT METHODOLOGY. Master of nursing. University of Lethbridge; 2019.

Goudman AM. Many gifts: a narrative Inquiry Study into Urban Aboriginal women’s experiences of Breastfeeding. Master of Nursing. University of Alberta; 2014.

Phillips-Beck W. Development of a Framework of Improved Childbirth Care for First Nation women in Manitoba: A First Nation Family Centred Approach. Master of Science. University of Manitoba; 2010.

Jones H, Barber CC, Nikora LW, Middlemiss W. Māori child rearing and infant sleep practices. N Z J Psychol. 2017;46:30–7.

Bauer MC, Wright AL. Integrating quantitative and quantitative methods to model infant feeding behavior among Navajo mothers. Hum Organ. 1996;55:183–92. https://doi.org/10.17730/humo.55.2.p55g316v70572732

WRIGHT AL, BAUER M, CLARK C, MORGAN F, BEGISHE K. Cultural interpretations and intracultural variability in Navajo beliefs about breastfeeding. Am Ethnol. 1993;20:781–96. https://doi.org/10.1525/ae.1993.20.4.02a00060

Horodynski MA, Calcatera M, Carpenter A. Infant feeding practices: perceptions of native American mothers and health paraprofessionals. Health Educ J. 2012;71:327–39. https://doi.org/10.1177/0017896911398814

Wright AL, Naylor A, Wester R, Bauer M, Sutcliffe E. Using cultural knowledge in health promotion: breastfeeding among the Navajo. Health Educ Behav off Publ Soc Public Health Educ. 1997;24:625–39.

MacQuarrie ME. Determination of the antecedents of infant feeding practices and the effect of choice of infant feeding on subsequent health status of Canadian ojibway infants and young children. Diss Abstr Int. 1984;44:3722.

McCalman J, Searles A, Bainbridge R, Ham R, Mein J, Neville J, et al. Empowering families by engaging and relating Murri way: a grounded theory study of the implementation of the Cape York Baby Basket program. BMC Pregnancy Childbirth. 2015. https://doi.org/10.1186/s12884-015-0543-y

Abel S, Park J, Tipene-Leach D, Finau S, Lennan M. Infant care practices in New Zealand: a cross-cultural qualitative study. Soc Sci Med 1982. 2001;53:1135–48.

Cidro J, Zahayko L, Lawrence HP, Folster S, McGregor M, McKay K. Breast feeding practices as cultural interventions for early childhood caries in Cree communities. BMC Oral Health. 2015;15:49. https://doi.org/10.1186/s12903-015-0027-5

Moffitt P, Dickinson R. Creating exclusive breastfeeding knowledge translation tools with First Nations mothers in Northwest territories, Canada. Int J Circumpolar Health. 2016;75:32989. https://doi.org/10.3402/ijch.v75.32989

Holmes W, Thorpe L, Phillips J. Influences on infant-feeding beliefs and practices in an urban aboriginal community. Aust N Z J Public Health. 1997;21:504–10.

Kruske S, Belton S, Wardaguga M, Narjic C. Growing up our way: the first year of life in remote Aboriginal Australia. Qual Health Res. 2012;22:777–87. https://doi.org/10.1177/1049732311432717

Tipene-Leach D, Abel S, Finau SA, Park J, Lenna M. Maori infant care practices: implications for health messages, infant care services and SIDS prevention in maori communities. Pac Health Dialog. 2000;7:29–37.

CAS   PubMed   Google Scholar  

Moffitt P. Learning from mothers, grandmothers & great-grandmothers about breastfeeding in the northwest territories 2018.

Wagner M. The infant feeding experiences and decision-making influences of Aboriginal women in Saskatoon. Undefined. 2005.

Archibald L. Teenage Pregnancy in Inuit Communities: Issues and Perspectives. 2004.

Glover M, Manaena-Biddle H, Waldon J. The role of Whānau in Māori women’s decisions about breast feeding. Altern Int J Indig Peoples. 2006;3:140–57. https://doi.org/10.1177/117718010600300108

Urban Indian Health Institute. Seattle Indian Health Board. Discussions with urban American Indian and Alaska native parents. Keeping Babies Healthy and Safe; 2011.

Ross Leitenberger KA. Aboriginal Midwifery and Traditional Birthing systems Revisited and revitalized: interviews with First Nations elders in the Northwest Region of British Columbia. Master of arts. University of Northern British Columbia; 1998.

Moffitt P, Lakhani S-A, Cruz S. Chapter 21: infant feeding teachings from indigenous grandmothers: Generating Knowledge through sharing circles. North. Indig. Health Healthc. Saskatoon, Saskatchewan: University of Saskatchewan; 2018. pp. 169–75.

Glover M, Waldon J, Manaena-Biddle H, Holdaway M, Cunningham C. Barriers to best outcomes in breastfeeding for Maori: mothers’ perceptions, whanau perceptions, and services. J Hum Lact off J Int Lact Consult Assoc. 2009;25:307–16. https://doi.org/10.1177/0890334409332436

Houghtaling B, Byker Shanks C, Ahmed S, Rink E. Grandmother and health care professional breastfeeding perspectives provide opportunities for health promotion in an American Indian community. Soc Sci Med. 2018;208:80–8. https://doi.org/10.1016/j.socscimed.2018.05.017

Foley W, Schubert L, Denaro T. Breastfeeding experiences of Aboriginal and Torres Strait Islander mothers in an urban setting in Brisbane. Breastfeed Rev Prof Publ Nurs Mothers Assoc Aust. 2013;21:53–61.

Myers J, Thorpe S, Browne J, Gibbons K, Brown S. Early childhood nutrition concerns, resources and services for Aboriginal families in Victoria. Aust N Z J Public Health. 2014;38:370–6. https://doi.org/10.1111/1753-6405.12206

Glover M, Manaena-Biddle H, Waldon J. Influences that affect maori women breastfeeding. Breastfeed Rev Prof Publ Nurs Mothers Assoc Aust. 2007;15:5–14.

Reinfelds MA, Kia Mau, Kia U. Supporting the breastfeeding journey of maori women and their whanau in Taranaki. Massey University; 2015.

Victorian Aboriginal Community Controlled Health Organisation. Feeding our Future: Aboriginal Early Childhood Nutrition & Physical Activity Needs Assessment Report. Fitzroy, Australia: 2012.

Oneha MFM, Dodgson JE. Community influences on Breastfeeding described by native hawaiian mothers. Pimatisiwin J Aborig Inidgenous Community Health. 2009;7:75–97.

Martens PJ. Evaluating the effectiveness of a breastfeeding promotion community strategy in Sagkeeng First Nation. University of Manitoba; 1999.

Foley W, Schubert L. Ethics of care and urban Aboriginal and Torres Strait Islander Breastfeeding beyond three months, 2016. https://doi.org/10.15226/2471-6529/2/1/00110

Wright AL, Clark C, Bauer M. Maternal employment and infant feeding practices among the Navajo. Med Anthropol Q. 1993;7:260–80. https://doi.org/10.1525/maq.1993.7.3.02a00020

Weinstein P, Troyer R, Jacobi D, Moccasin M. Dental experiences and parenting practices of native American mothers and caretakers: what we can learn for the prevention of baby bottle tooth decay. ASDC J Dent Child. 1999;66:120–85.

Survivance. Narratives of Native Presence. Lincoln: 2008.

Willows ND, Hanley AJG, Delormier T. A socioecological framework to understand weight-related issues in Aboriginal children in Canada. Appl Physiol Nutr Metab. 2012;37:1–13. https://doi.org/10.1139/h11-128

Hickey D. Indigenous epistemologies, worldviews and theories of Power. Turt Isl J Indig Health. 2020;1:14–25. https://doi.org/10.33137/tijih.v1i1.34021

Battiste MA. Reclaiming indigenous voice and vision. Vancouver, BC: UBC; 2000.

Wylie L, McConkey S, Corrado AM. It’s a Journey not a check Box: Indigenous Cultural Safety from Training to Transformation. Int J Indig Health. 2021;16. https://doi.org/10.32799/ijih.v16i1.33240

Browne AJ, Varcoe C, Ward C. San’yas Indigenous Cultural Safety Training as an Educational intervention: promoting Anti-racism and Equity in Health Systems, policies, and practices. Int Indig Policy J. 2021;12:1–26. https://doi.org/10.18584/iipj.2021.12.3.8204

Corcoran PM, Catling C, Homer CSE. Models of midwifery care for indigenous women and babies: a meta-synthesis. Women Birth. 2016;0. https://doi.org/10.1016/j.wombi.2016.08.003

Churchill ME, Smylie JK, Wolfe SH, Bourgeois C, Moeller H, Firestone M. Conceptualising cultural safety at an indigenous-focused midwifery practice in Toronto, Canada: qualitative interviews with indigenous and non-indigenous clients. BMJ Open. 2020;10:e038168. https://doi.org/10.1136/bmjopen-2020-038168

Reszel J, Weiss D, Darling EK, Sidney D, Van Wagner V, Soderstrom B, et al. Client experience with the Ontario Birth Center Demonstration Project. J Midwifery Womens Health. 2021;66:174–84. https://doi.org/10.1111/jmwh.13164

Eakin JM, Mykhalovskiy E. Reframing the evaluation of qualitative health research: reflections on a review of appraisal guidelines in the health sciences. J Eval Clin Pract. 2003;9:187–94. https://doi.org/10.1046/j.1365-2753.2003.00392.x

Holmes AGD. Researcher positionality -- a consideration of its influence and place in qualitative research -- a new researcher guide. Shanlax Int J Educ. 2020;8:1–10.

Download references

Acknowledgements

We wish to acknowledge the important contribution of Halima Abubakar in the review process. Given the knowledge specific to Indigenous communities discussed in this scoping review and out of respect for Indigenous research conventions, the authors position themselves within the research to explain the lens from which they approach and understand the research process. TG and AH are non-Indigenous scholars and faculty members based at the University of Toronto, which rests on lands that are the traditional home of the Huron-Wendat, the Seneca, and the Mississaugas of the Credit. All other authors have had student or supporting roles throughout this work and situate themselves as follows: HM is a settler of Scottish, Irish, French, German, and English ancestry residing in Haudenosaunee and Anishinaabe territory, which is part of the dish with one spoon agreement; CC is a settler living in Treaty 7 Territory, with ancestral roots in Germany, Scotland, and the Ukraine; AS is an Odawa Kwe from Wikwemikong, Manitoulin Island, Ontario. Currently, residing in the Tiohtià:ke in Kanien’kéha unceded territory; and HS is living in Treaty 13 territory with ancestral roots in Afghanistan. The remaining co-authors identify as non-Indigenous scholars.

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Author information

Authors and affiliations.

Department of Nutritional Sciences, Temerty Faculty of Medicine, University of Toronto, University of Toronto Medical, King’s College Circle, Sciences Building, 5th Floor, Room 5253A, Toronto, ON, M5S 1A8, Canada

Hiliary Monteith, Hosna Sahak, Christina Liu & Anthony J. G. Hanley

Department of Anthropology, University of Toronto Mississauga Campus, Terrence Donnelly Health Sciences Complex, Room 354, 3359 Mississauga Rd, Mississauga, ON, L5L 1C6, Canada

Carly Checholik & Tracey Galloway

Department of Family Medicine, McGill University, 5858, chemin de la Côte-des-Neiges, 3rd floor, Montreal, QC, H3S 1Z1, Canada

Amy Shawanda

Epidemiology Division, University of Toronto, Dalla Lana School of Public Health, Toronto, ON, Canada

Anthony J. G. Hanley

Leadership Sinai Centre for Diabetes, Mount Sinai Hospital, Toronto, ON, Canada

You can also search for this author in PubMed   Google Scholar

Contributions

As the first author, HM conceptualized this work and provided leadership throughout. She participated in every aspect of this review, wrote the initial manuscript, and completed revisions. CC contributed to the screening and full text review of this work. She also contributed to the analysis, and the writing and review of the manuscript. TG supported the protocol of this review and provided guidance throughout analysis. She also contributed to the final manuscript. HS supported screening and full text review. She also provided edits for the manuscript. AS provided feedback on the analysis for this review and contributed to the writing of the manuscript. CL supported screening of papers and provided edits to the final manuscript. AH provided guidance throughout the duration of this review, supported decision making, and provided edits on the manuscript. All authors approved the final manuscript.

Corresponding author

Correspondence to Anthony J. G. Hanley .

Ethics declarations

Ethics approval and consent to participate.

As this review involves material previously published or in the public domain, ethical approval is not required.

Consent for publication

Not applicable.

Competing interests

Authors have no competing interests to report.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Monteith, H., Checholik, C., Galloway, T. et al. Infant feeding experiences among Indigenous communities in Canada, the United States, Australia, and Aotearoa: a scoping review of the qualitative literature. BMC Public Health 24 , 1583 (2024). https://doi.org/10.1186/s12889-024-19060-1

Download citation

Received : 04 January 2024

Accepted : 05 June 2024

Published : 13 June 2024

DOI : https://doi.org/10.1186/s12889-024-19060-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Indigenous Health
• Infant feeding
• Breastfeeding
• Qualitative
• Maternal and child health
• Scoping review

BMC Public Health

ISSN: 1471-2458

• General enquiries: [email protected]