what is a population in a research framework

Population and Target Population in Research Methodology

• Golden Ratio of Social Science and Education 4(1):75-79
• CC BY-SA 4.0

• Council for Medical Schemes,South Africa

Abstract and Figures

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• Alex Casteel

• Suyash Shringarpure
• Eric P. Xing

• James P Donnelly
• J W Creswell
• J D Creswell
• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

Study Population

• Reference work entry
• pp 6412–6414
• Cite this reference work entry

3435 Accesses

1 Citations

Study population is a subset of the target population from which the sample is actually selected. It is broader than the concept sample frame . It may be appropriate to say that sample frame is an operationalized form of study population. For example, suppose that a study is going to conduct a survey of high school students on their social well-being . High school students all over the world might be considered as the target population. Because of practicalities, researchers decide to only recruit high school students studying in China who are the study population in this example. Suppose there is a list of high school students of China, this list is used as the sample frame .

Description

Study population is the operational definition of target population (Henry, 1990 ; Bickman & Rog, 1998 ). Researchers are seldom in a position to study the entire target population, which is not always readily accessible. Instead, only part of it—respondents who are both eligible for the study...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Babbie, E. R. (2010). The practice of social research . Belmont, CA: Wadsworth Publishing Company.

Google Scholar  

Bickman, L., & Rog, D. J. (1998). Handbook of applied social research methods . Thousand Oaks, CA: Sage Publications.

Friedman, L. M., Furberg, C. D., & DeMets, D. L. (2010). Fundamentals of clinical trials . New York: Springer.

Gerrish, K., & Lacey, A. (2010). The research process in nursing . West Sussex: Wiley-Blackwell.

Henry, G. T. (1990). Practical sampling . Newbury Park, CA: Sage Publications.

Kumar, R. (2011). Research methodology: A step-by-step guide for beginners . London: Sage Publications Limited.

Riegelman, R. K. (2005). Studying a study and testing a test: How to read the medical evidence . Philadelphia: Lippincott Williams & Wilkins.

Download references

Author information

Authors and affiliations.

Sociology Department, National University of Singapore, 11 Arts Link, 117570, Singapore, Singapore

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Shu Hu .

Editor information

Editors and affiliations.

University of Northern British Columbia, Prince George, BC, Canada

Alex C. Michalos

(residence), Brandon, MB, Canada

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media Dordrecht

About this entry

Cite this entry.

Hu, S. (2014). Study Population. In: Michalos, A.C. (eds) Encyclopedia of Quality of Life and Well-Being Research. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-0753-5_2893

Download citation

DOI : https://doi.org/10.1007/978-94-007-0753-5_2893

Publisher Name : Springer, Dordrecht

Print ISBN : 978-94-007-0752-8

Online ISBN : 978-94-007-0753-5

eBook Packages : Humanities, Social Sciences and Law Reference Module Humanities and Social Sciences Reference Module Business, Economics and Social Sciences

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Jump to navigation

Cochrane Training

Chapter 3: defining the criteria for including studies and how they will be grouped for the synthesis.

Joanne E McKenzie, Sue E Brennan, Rebecca E Ryan, Hilary J Thomson, Renea V Johnston, James Thomas

Key Points:

• The scope of a review is defined by the types of population (participants), types of interventions (and comparisons), and the types of outcomes that are of interest. The acronym PICO (population, interventions, comparators and outcomes) helps to serve as a reminder of these.
• The population, intervention and comparison components of the question, with the additional specification of types of study that will be included, form the basis of the pre-specified eligibility criteria for the review. It is rare to use outcomes as eligibility criteria: studies should be included irrespective of whether they report outcome data, but may legitimately be excluded if they do not measure outcomes of interest, or if they explicitly aim to prevent a particular outcome.
• Cochrane Reviews should include all outcomes that are likely to be meaningful and not include trivial outcomes. Critical and important outcomes should be limited in number and include adverse as well as beneficial outcomes.
• Review authors should plan at the protocol stage how the different populations, interventions, outcomes and study designs within the scope of the review will be grouped for analysis.

Cite this chapter as: McKenzie JE, Brennan SE, Ryan RE, Thomson HJ, Johnston RV, Thomas J. Chapter 3: Defining the criteria for including studies and how they will be grouped for the synthesis. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated August 2023). Cochrane, 2023. Available from www.training.cochrane.org/handbook .

3.1 Introduction

One of the features that distinguishes a systematic review from a narrative review is that systematic review authors should pre-specify criteria for including and excluding studies in the review (eligibility criteria, see MECIR Box 3.2.a ).

When developing the protocol, one of the first steps is to determine the elements of the review question (including the population, intervention(s), comparator(s) and outcomes, or PICO elements) and how the intervention, in the specified population, produces the expected outcomes (see Chapter 2, Section 2.5.1 and Chapter 17, Section 17.2.1 ). Eligibility criteria are based on the PICO elements of the review question plus a specification of the types of studies that have addressed these questions. The population, interventions and comparators in the review question usually translate directly into eligibility criteria for the review, though this is not always a straightforward process and requires a thoughtful approach, as this chapter shows. Outcomes usually are not part of the criteria for including studies, and a Cochrane Review would typically seek all sufficiently rigorous studies (most commonly randomized trials) of a particular comparison of interventions in a particular population of participants, irrespective of the outcomes measured or reported. It should be noted that some reviews do legitimately restrict eligibility to specific outcomes. For example, the same intervention may be studied in the same population for different purposes; or a review may specifically address the adverse effects of an intervention used for several conditions (see Chapter 19 ).

Eligibility criteria do not exist in isolation, but should be specified with the synthesis of the studies they describe in mind. This will involve making plans for how to group variants of the PICO elements for synthesis. This chapter describes the processes by which the structure of the synthesis can be mapped out at the beginning of the review, and the interplay between the review question, considerations for the analysis and their operationalization in terms of eligibility criteria. Decisions about which studies to include (and exclude), and how they will be combined in the review’s synthesis, should be documented and justified in the review protocol.

A distinction between three different stages in the review at which the PICO construct might be used is helpful for understanding the decisions that need to be made. In Chapter 2, Section 2.3 , we introduced the ideas of a review PICO (on which eligibility of studies is based), the PICO for each synthesis (defining the question that each specific synthesis aims to answer) and the PICO of the included studies (what was actually investigated in the included studies). In this chapter, we focus on the review PICO and the PICO for each synthesis as a basis for specifying which studies should be included in the review and planning its syntheses. These PICOs should relate clearly and directly to the questions or hypotheses that are posed when the review is formulated (see Chapter 2 ) and will involve specifying the population in question, and a set of comparisons between the intervention groups.

An integral part of the process of setting up the review is to specify which characteristics of the interventions (e.g. individual compounds of a drug), populations (e.g. acute and chronic conditions), outcomes (e.g. different depression measurement scales) and study designs, will be grouped together. Such decisions should be made independent of knowing which studies will be included and the methods of synthesis that will be used (e.g. meta-analysis). There may be a need to modify the comparisons and even add new ones at the review stage in light of the data that are collected. For example, important variations in the intervention may be discovered only after data are collected, or modifying the comparison may facilitate the possibility of synthesis when only one or few studies meet the comparison PICO. Planning for the latter scenario at the protocol stage may lead to less post-hoc decision making ( Chapter 2, Section 2.5.3 ) and, of course, any changes made during the conduct of the review should be recorded and documented in the final report.

3.2 Articulating the review and comparison PICO

3.2.1 defining types of participants: which people and populations.

The criteria for considering types of people included in studies in a review should be sufficiently broad to encompass the likely diversity of studies and the likely scenarios in which the interventions will be used, but sufficiently narrow to ensure that a meaningful answer can be obtained when studies are considered together; they should be specified in advance (see MECIR Box 3.2.a ). As discussed in Chapter 2, Section 2.3.1 , the degree of breadth will vary, depending on the question being asked and the analytical approach to be employed. A range of evidence may inform the choice of population characteristics to examine, including theoretical considerations, evidence from other interventions that have a similar mechanism of action, and in vitro or animal studies. Consideration should be given to whether the population characteristic is at the level of the participant (e.g. age, severity of disease) or the study (e.g. care setting, geographical location), since this has implications for grouping studies and for the method of synthesis ( Chapter 10, Section 10.11.5 ). It is often helpful to consider the types of people that are of interest in three steps.

MECIR Box 3.2.a Relevant expectations for conduct of intervention reviews

First, the diseases or conditions of interest should be defined using explicit criteria for establishing their presence (or absence). Criteria that will force the unnecessary exclusion of studies should be avoided. For example, diagnostic criteria that were developed more recently – which may be viewed as the current gold standard for diagnosing the condition of interest – will not have been used in earlier studies. Expensive or recent diagnostic tests may not be available in many countries or settings, and time-consuming tests may not be practical in routine healthcare settings.

Second, the broad population and setting of interest should be defined . This involves deciding whether a specific population group is within scope, determined by factors such as age, sex, race, educational status or the presence of a particular condition such as angina or shortness of breath. Interest may focus on a particular setting such as a community, hospital, nursing home, chronic care institution, or outpatient setting. Box 3.2.a outlines some factors to consider when developing population criteria.

Whichever criteria are used for defining the population and setting of interest, it is common to encounter studies that only partially overlap with the review’s population. For example, in a review focusing on children, a cut-point of less than 16 years might be desirable, but studies may be identified with participants aged from 12 to 18. Unless the study reports separate data from the eligible section of the population (in which case data from the eligible participants can be included in the review), review authors will need a strategy for dealing with these studies (see MECIR Box 3.2.a ). This will involve balancing concerns about reduced applicability by including participants who do not meet the eligibility criteria, against the loss of data when studies are excluded. Arbitrary rules (such as including a study if more than 80% of the participants are under 16) will not be practical if detailed information is not available from the study. A less stringent rule, such as ‘the majority of participants are under 16’ may be sufficient. Although there is a risk of review authors’ biases affecting post-hoc inclusion decisions (which is why many authors endeavour to pre-specify these rules), this may be outweighed by a common-sense strategy in which eligibility decisions keep faith with the objectives of the review rather than with arbitrary rules. Difficult decisions should be documented in the review, checked with the advisory group (if available, see Chapter 1 ), and sensitivity analyses can assess the impact of these decisions on the review’s findings (see Chapter 10, Section 10.14 and MECIR Box 3.2.b ).

Box 3.2.a Factors to consider when developing criteria for ‘Types of participants’

MECIR Box 3.2.b Relevant expectations for conduct of intervention reviews

Third, there should be consideration of whether there are population characteristics that might be expected to modify the size of the intervention effects (e.g. different severities of heart failure). Identifying subpopulations may be important for implementation of the intervention. If relevant subpopulations are identified, two courses of action are possible: limiting the scope of the review to exclude certain subpopulations; or maintaining the breadth of the review and addressing subpopulations in the analysis.

Restricting the review with respect to specific population characteristics or settings should be based on a sound rationale. It is important that Cochrane Reviews are globally relevant, so the rationale for the exclusion of studies based on population characteristics should be justified. For example, focusing a review of the effectiveness of mammographic screening on women between 40 and 50 years old may be justified based on biological plausibility, previously published systematic reviews and existing controversy. On the other hand, focusing a review on a particular subgroup of people on the basis of their age, sex or ethnicity simply because of personal interests, when there is no underlying biologic or sociological justification for doing so, should be avoided, as these reviews will be less useful to decision makers and readers of the review.

Maintaining the breadth of the review may be best when it is uncertain whether there are important differences in effects among various subgroups of people, since this allows investigation of these differences (see Chapter 10, Section 10.11.5 ). Review authors may combine the results from different subpopulations in the same synthesis, examining whether a given subdivision explains variation (heterogeneity) among the intervention effects. Alternatively, the results may be synthesized in separate comparisons representing different subpopulations. Splitting by subpopulation risks there being too few studies to yield a useful synthesis (see Table 3.2.a and Chapter 2, Section 2.3.2 ). Consideration needs to be given to the subgroup analysis method, particularly for population characteristics measured at the participant level (see Chapter 10 and Chapter 26 , Fisher et al 2017). All subgroup analyses should ideally be planned a priori and stated as a secondary objective in the protocol, and not driven by the availability of data.

In practice, it may be difficult to assign included studies to defined subpopulations because of missing information about the population characteristic, variability in how the population characteristic is measured across studies (e.g. variation in the method used to define the severity of heart failure), or because the study does not wholly fall within (or report the results separately by) the defined subpopulation. The latter issue mainly applies for participant characteristics but can also arise for settings or geographic locations where these vary within studies. Review authors should consider planning for these scenarios (see example reviews Hetrick et al 2012, Safi et al 2017; Table 3.2.b , column 3).

Table 3.2.a Examples of population attributes and characteristics

3.2.2 Defining interventions and how they will be grouped

In some reviews, predefining the intervention ( MECIR Box 3.2.c ) may be straightforward. For example, in a review of the effect of a given anticoagulant on deep vein thrombosis, the intervention can be defined precisely. A more complicated definition might be required for a multi-component intervention composed of dietary advice, training and support groups to reduce rates of obesity in a given population.

The inherent complexity present when defining an intervention often comes to light when considering how it is thought to achieve its intended effect and whether the effect is likely to differ when variants of the intervention are used. In the first example, the anticoagulant warfarin is thought to reduce blood clots by blocking an enzyme that depends on vitamin K to generate clotting factors. In the second, the behavioural intervention is thought to increase individuals’ self-efficacy in their ability to prepare healthy food. In both examples, we cannot assume that all forms of the intervention will work in the same way. When defining drug interventions, such as anticoagulants, factors such as the drug preparation, route of administration, dose, duration, and frequency should be considered. For multi-component interventions (such as interventions to reduce rates of obesity), the common or core features of the interventions must be defined, so that the review authors can clearly differentiate them from other interventions not included in the review.

MECIR Box 3.2.c Relevant expectations for conduct of intervention reviews

In general, it is useful to consider exactly what is delivered, who delivers it, how it is delivered, where it is delivered, when and how much is delivered, and whether the intervention can be adapted or tailored , and to consider this for each type of intervention included in the review (see the TIDieR checklist (Hoffmann et al 2014)). As argued in Chapter 17 , separating interventions into ‘simple’ and ‘complex’ is a false dichotomy; all interventions can be complex in some ways. The critical issue for review authors is to identify the most important factors to be considered in a specific review. Box 3.2.b outlines some factors to consider when developing broad criteria for the ‘Types of interventions’ (and comparisons).

Box 3.2.b Factors to consider when developing criteria for ‘Types of interventions’

Once interventions eligible for the review have been broadly defined, decisions should be made about how variants of the intervention will be handled in the synthesis. Differences in intervention characteristics across studies occur in all reviews. If these reflect minor differences in the form of the intervention used in practice (such as small differences in the duration or content of brief alcohol counselling interventions), then an overall synthesis can provide useful information for decision makers. Where differences in intervention characteristics are more substantial (such as delivery of brief alcohol counselling by nurses versus doctors), and are expected to have a substantial impact on the size of intervention effects, these differences should be examined in the synthesis. What constitutes an important difference requires judgement, but in general differences that alter decisions about how an intervention is implemented or whether the intervention is used or not are likely to be important. In such circumstances, review authors should consider specifying separate groups (or subgroups) to examine in their synthesis.

Clearly defined intervention groups serve two main purposes in the synthesis. First, the way in which interventions are grouped for synthesis (meta-analysis or other synthesis) is likely to influence review findings. Careful planning of intervention groups makes best use of the available data, avoids decisions that are influenced by study findings (which may introduce bias), and produces a review focused on questions relevant to decision makers. Second, the intervention groups specified in a protocol provide a standardized terminology for describing the interventions throughout the review, overcoming the varied descriptions used by study authors (e.g. where different labels are used for the same intervention, or similar labels used for different techniques) (Michie et al 2013). This standardization enables comparison and synthesis of information about intervention characteristics across studies (common characteristics and differences) and provides a consistent language for reporting that supports interpretation of review findings.

Table 3.2.b   outlines a process for planning intervention groups as a basis for/precursor to synthesis, and the decision points and considerations at each step. The table is intended to guide, rather than to be prescriptive and, although it is presented as a sequence of steps, the process is likely to be iterative, and some steps may be done concurrently or in a different sequence. The process aims to minimize data-driven approaches that can arise once review authors have knowledge of the findings of the included studies. It also includes principles for developing a flexible plan that maximizes the potential to synthesize in circumstances where there are few studies, many variants of an intervention, or where the variants are difficult to anticipate. In all stages, review authors should consider how to categorize studies whose reports contain insufficient detail.

Table 3.2.b A process for planning intervention groups for synthesis

3.2.3 Defining which comparisons will be made

When articulating the PICO for each synthesis, defining the intervention groups alone is not sufficient for complete specification of the planned syntheses. The next step is to define the comparisons that will be made between the intervention groups. Setting aside for a moment more complex analyses such as network meta-analyses, which can simultaneously compare many groups ( Chapter 11 ), standard meta-analysis ( Chapter 10 ) aims to draw conclusions about the comparative effects of two groups at a time (i.e. which of two intervention groups is more effective?). These comparisons form the basis for the syntheses that will be undertaken if data are available. Cochrane Reviews sometimes include one comparison, but most often include multiple comparisons. Three commonly identified types of comparisons include the following (Davey et al 2011).

• newer generation antidepressants versus placebo (Hetrick et al 2012); and
• vertebroplasty for osteoporotic vertebral compression fractures versus placebo (sham procedure) (Buchbinder et al 2018).
• chemotherapy or targeted therapy plus best supportive care (BSC) versus BSC for palliative treatment of esophageal and gastroesophageal-junction carcinoma (Janmaat et al 2017); and
• personalized care planning versus usual care for people with long-term conditions (Coulter et al 2015).
• early (commenced at less than two weeks of age) versus late (two weeks of age or more) parenteral zinc supplementation in term and preterm infants (Taylor et al 2017);
• high intensity versus low intensity physical activity or exercise in people with hip or knee osteoarthritis (Regnaux et al 2015);
• multimedia education versus other education for consumers about prescribed and over the counter medications (Ciciriello et al 2013).

The first two types of comparisons aim to establish the effectiveness of an intervention, while the last aims to compare the effectiveness of two interventions. However, the distinction between the placebo and control is often arbitrary, since any differences in the care provided between trials with a control arm and those with a placebo arm may be unimportant , especially where ‘usual care’ is provided to both. Therefore, placebo and control groups may be determined to be similar enough to be combined for synthesis.

In reviews including multiple intervention groups, many comparisons are possible. In some of these reviews, authors seek to synthesize evidence on the comparative effectiveness of all their included interventions, including where there may be only indirect comparison of some interventions across the included studies ( Chapter 11, Section 11.2.1 ). However, in many reviews including multiple intervention groups, a limited subset of the possible comparisons will be selected. The chosen subset of comparisons should address the most important clinical and research questions. For example, if an established intervention (or dose of an intervention) is used in practice, then the synthesis would ideally compare novel or alternative interventions to this established intervention, and not, for example, to no intervention.

3.2.3.1 Dealing with co-interventions

Planning is needed for the special case where the same supplementary intervention is delivered to both the intervention and comparator groups. A supplementary intervention is an additional intervention delivered alongside the intervention of interest, such as massage in a review examining the effects of aromatherapy (i.e. aromatherapy plus massage versus massage alone). In many cases, the supplementary intervention will be unimportant and can be ignored. In other situations, the effect of the intervention of interest may differ according to whether participants receive the supplementary therapy. For example, the effect of aromatherapy among people who receive a massage may differ from the effect of the aromatherapy given alone. This will be the case if the intervention of interest interacts with the supplementary intervention leading to larger (synergistic) or smaller (dysynergistic/antagonistic) effects than the intervention of interest alone (Squires et al 2013). While qualitative interactions are rare (where the effect of the intervention is in the opposite direction when combined with the supplementary intervention), it is possible that there will be more variation in the intervention effects (heterogeneity) when supplementary interventions are involved, and it is important to plan for this. Approaches for dealing with this in the statistical synthesis may include fitting a random-effects meta-analysis model that encompasses heterogeneity ( Chapter 10, Section 10.10.4 ), or investigating whether the intervention effect is modified by the addition of the supplementary intervention through subgroup analysis ( Chapter 10, Section 10.11.2 ).

3.2.4 Selecting, prioritizing and grouping review outcomes

3.2.4.1 selecting review outcomes.

Broad outcome domains are decided at the time of setting up the review PICO (see Chapter 2 ). Once the broad domains are agreed, further specification is required to define the domains to facilitate reporting and synthesis (i.e. the PICO for comparison) (see Chapter 2, Section 2.3 ). The process for specifying and grouping outcomes largely parallels that used for specifying intervention groups.

Reporting of outcomes should rarely determine study eligibility for a review. In particular, studies should not be excluded because they do not report results of an outcome they may have measured, or provide ‘no usable data’ ( MECIR Box 3.2.d ). This is essential to avoid bias arising from selective reporting of findings by the study authors (see Chapter 13 ). However, in some circumstances, the measurement of certain outcomes may be a study eligibility criterion. This may be the case, for example, when the review addresses the potential for an intervention to prevent a particular outcome, or when the review addresses a specific purpose of an intervention that can be used in the same population for different purposes (such as hormone replacement therapy, or aspirin).

MECIR Box 3.2.d Relevant expectations for conduct of intervention reviews

In general, systematic reviews should aim to include outcomes that are likely to be meaningful to the intended users and recipients of the reviewed evidence. This may include clinicians, patients (consumers), the general public, administrators and policy makers. Outcomes may include survival (mortality), clinical events (e.g. strokes or myocardial infarction), behavioural outcomes (e.g. changes in diet, use of services), patient-reported outcomes (e.g. symptoms, quality of life), adverse events, burdens (e.g. demands on caregivers, frequency of tests, restrictions on lifestyle) and economic outcomes (e.g. cost and resource use). It is critical that outcomes used to assess adverse effects as well as outcomes used to assess beneficial effects are among those addressed by a review (see Chapter 19 ).

Outcomes that are trivial or meaningless to decision makers should not be included in Cochrane Reviews. Inclusion of outcomes that are of little or no importance risks overwhelming and potentially misleading readers. Interim or surrogate outcomes measures, such as laboratory results or radiologic results (e.g. loss of bone mineral content as a surrogate for fractures in hormone replacement therapy), while potentially helpful in explaining effects or determining intervention integrity (see Chapter 5, Section 5.3.4.1 ), can also be misleading since they may not predict clinically important outcomes accurately. Many interventions reduce the risk for a surrogate outcome but have no effect or have harmful effects on clinically relevant outcomes, and some interventions have no effect on surrogate measures but improve clinical outcomes.

Various sources can be used to develop a list of relevant outcomes, including input from consumers and advisory groups (see Chapter 2 ), the clinical experiences of the review authors, and evidence from the literature (including qualitative research about outcomes important to those affected (see Chapter 21 )). A further driver of outcome selection is consideration of outcomes used in related reviews. Harmonization of outcomes across reviews addressing related questions facilitates broader evidence synthesis questions being addressed through the use of Overviews of reviews (see Chapter V ).

Outcomes considered to be meaningful, and therefore addressed in a review, may not have been reported in the primary studies. For example, quality of life is an important outcome, perhaps the most important outcome, for people considering whether or not to use chemotherapy for advanced cancer, even if the available studies are found to report only survival (see Chapter 18 ). A further example arises with timing of the outcome measurement, where time points determined as clinically meaningful in a review are not measured in the primary studies. Including and discussing all important outcomes in a review will highlight gaps in the primary research and encourage researchers to address these gaps in future studies.

3.2.4.2 Prioritizing review outcomes

Once a full list of relevant outcomes has been compiled for the review, authors should prioritize the outcomes and select the outcomes of most relevance to the review question. The GRADE approach to assessing the certainty of evidence (see Chapter 14 ) suggests that review authors separate outcomes into those that are ‘critical’, ‘important’ and ‘not important’ for decision making.

The critical outcomes are the essential outcomes for decision making, and are those that would form the basis of a ‘Summary of findings’ table or other summary versions of the review, such as the Abstract or Plain Language Summary. ‘Summary of findings’ tables provide key information about the amount of evidence for important comparisons and outcomes, the quality of the evidence and the magnitude of effect (see Chapter 14, Section 14.1 ). There should be no more than seven outcomes included in a ‘Summary of findings’ table, and those outcomes that will be included in summaries should be specified at the protocol stage. They should generally not include surrogate or interim outcomes. They should not be chosen on the basis of any anticipated or observed magnitude of effect, or because they are likely to have been addressed in the studies to be reviewed. Box 3.2.c summarizes the principal factors to consider when selecting and prioritizing review outcomes.

Box 3.2.c Factors to consider when selecting and prioritizing review outcomes

3.2.4.3 Defining and grouping outcomes for synthesis

Table 3.2.c outlines a process for planning for the diversity in outcome measurement that may be encountered in the studies included in a review and which can complicate, and sometimes prevent, synthesis. Research has repeatedly documented inconsistency in the outcomes measured across trials in the same clinical areas (Harrison et al 2016, Williamson et al 2017). This inconsistency occurs across all aspects of outcome measurement, including the broad domains considered, the outcomes measured, the way these outcomes are labelled and defined, and the methods and timing of measurement. For example, a review of outcome measures used in 563 studies of interventions for dementia and mild cognitive impairment found that 321 unique measurement methods were used for 1278 assessments of cognitive outcomes (Harrison et al 2016). Initiatives like COMET ( Core Outcome Measures in Effectiveness Trials ) aim to encourage standardization of outcome measurement across trials (Williamson et al 2017), but these initiatives are comparatively new and review authors will inevitably encounter diversity in outcomes across studies.

The process begins by describing the scope of each outcome domain in sufficient detail to enable outcomes from included studies to be categorized ( Table 3.2.c Step 1). This step may be straightforward in areas for which core outcome sets (or equivalent systems) exist ( Table 3.2.c Step 2). The methods and timing of outcome measurement also need to be specified, giving consideration to how differences across studies will be handled ( Table 3.2.c Steps 3 and 4). Subsequent steps consider options for dealing with studies that report multiple measures within an outcome domain ( Table 3.2.c Step 5), planning how outcome domains will be used in synthesis ( Table 3.2.c Step 6), and building in contingencies to maximize potential to synthesize ( Table 3.2.c Step 7).

Table 3.2.c A process for planning outcome groups for synthesis

3.3 Determining which study designs to include

Some study designs are more appropriate than others for answering particular questions. Authors need to consider a priori what study designs are likely to provide reliable data with which to address the objectives of their review ( MECIR Box 3.3.a ). Sections 3.3.1 and 3.3.2 cover randomized and non-randomized designs for assessing treatment effects; Chapter 17, Section 17.2.5  discusses other study designs in the context of addressing intervention complexity.

MECIR Box 3.3.a Relevant expectations for conduct of intervention reviews

3.3.1 Including randomized trials

Because Cochrane Reviews address questions about the effects of health care, they focus primarily on randomized trials and randomized trials should be included if they are feasible for the interventions of interest ( MECIR Box 3.3.b ). Randomization is the only way to prevent systematic differences between baseline characteristics of participants in different intervention groups in terms of both known and unknown (or unmeasured) confounders (see Chapter 8 ), and claims about cause and effect can be based on their findings with far more confidence than almost any other type of study. For clinical interventions, deciding who receives an intervention and who does not is influenced by many factors, including prognostic factors. Empirical evidence suggests that, on average, non-randomized studies produce effect estimates that indicate more extreme benefits of the effects of health care than randomized trials. However, the extent, and even the direction, of the bias is difficult to predict. These issues are discussed at length in Chapter 24 , which provides guidance on when it might be appropriate to include non-randomized studies in a Cochrane Review.

Practical considerations also motivate the restriction of many Cochrane Reviews to randomized trials. In recent decades there has been considerable investment internationally in establishing infrastructure to index and identify randomized trials. Cochrane has contributed to these efforts, including building up and maintaining a database of randomized trials, developing search filters to aid their identification, working with MEDLINE to improve tagging and identification of randomized trials, and using machine learning and crowdsourcing to reduce author workload in identifying randomized trials ( Chapter 4, Section 4.6.6.2 ). The same scale of organizational investment has not (yet) been matched for the identification of other types of studies. Consequently, identifying and including other types of studies may require additional efforts to identify studies and to keep the review up to date, and might increase the risk that the result of the review will be influenced by publication bias. This issue and other bias-related issues that are important to consider when defining types of studies are discussed in detail in Chapter 7 and Chapter 13 .

Specific aspects of study design and conduct should be considered when defining eligibility criteria, even if the review is restricted to randomized trials. For example, whether cluster-randomized trials ( Chapter 23, Section 23.1 ) and crossover trials ( Chapter 23, Section 23.2 ) are eligible, as well as other criteria for eligibility such as use of a placebo comparison group, evaluation of outcomes blinded to allocation sequence, or a minimum period of follow-up. There will always be a trade-off between restrictive study design criteria (which might result in the inclusion of studies that are at low risk of bias, but very few in number) and more liberal design criteria (which might result in the inclusion of more studies, but at a higher risk of bias). Furthermore, excessively broad criteria might result in the inclusion of misleading evidence. If, for example, interest focuses on whether a therapy improves survival in patients with a chronic condition, it might be inappropriate to look at studies of very short duration, except to make explicit the point that they cannot address the question of interest.

MECIR Box 3.3.b Relevant expectations for conduct of intervention reviews

3.3.2 Including non-randomized studies

The decision of whether non-randomized studies (and what type) will be included is decided alongside the formulation of the review PICO. The main drivers that may lead to the inclusion of non-randomized studies include: (i) when randomized trials are unable to address the effects of the intervention on harm and long-term outcomes or in specific populations or settings; or (ii) for interventions that cannot be randomized (e.g. policy change introduced in a single or small number of jurisdictions) (see Chapter 24 ). Cochrane, in collaboration with others, has developed guidance for review authors to support their decision about when to look for and include non-randomized studies (Schünemann et al 2013).

Non-randomized designs have the commonality of not using randomization to allocate units to comparison groups, but their different design features mean that they are variable in their susceptibility to bias. Eligibility criteria should be based on explicit study design features, and not the study labels applied by the primary researchers (e.g. case-control, cohort), which are often used inconsistently (Reeves et al 2017; see Chapter 24 ).

When non-randomized studies are included, review authors should consider how the studies will be grouped and used in the synthesis. The Cochrane Non-randomized Studies Methods Group taxonomy of design features (see Chapter 24 ) may provide a basis for grouping together studies that are expected to have similar inferential strength and for providing a consistent language for describing the study design.

Once decisions have been made about grouping study designs, planning of how these will be used in the synthesis is required. Review authors need to decide whether it is useful to synthesize results from non-randomized studies and, if so, whether results from randomized trials and non-randomized studies should be included in the same synthesis (for the purpose of examining whether study design explains heterogeneity among the intervention effects), or whether the effects should be synthesized in separate comparisons (Valentine and Thompson 2013). Decisions should be made for each of the different types of non-randomized studies under consideration. Review authors might anticipate increased heterogeneity when non-randomized studies are synthesized, and adoption of a meta-analysis model that encompasses heterogeneity is wise (Valentine and Thompson 2013) (such as a random effects model, see Chapter 10, Section 10.10.4 ). For further discussion of non-randomized studies, see Chapter 24 .

3.4 Eligibility based on publication status and language

Chapter 4 contains detailed guidance on how to identify studies from a range of sources including, but not limited to, those in peer-reviewed journals. In general, a strategy to include studies reported in all types of publication will reduce bias ( Chapter 7 ). There would need to be a compelling argument for the exclusion of studies on the basis of their publication status ( MECIR Box 3.4.a ), including unpublished studies, partially published studies, and studies published in ‘grey’ literature sources. Given the additional challenge in obtaining unpublished studies, it is possible that any unpublished studies identified in a given review may be an unrepresentative subset of all the unpublished studies in existence. However, the bias this introduces is of less concern than the bias introduced by excluding all unpublished studies, given what is known about the impact of reporting biases (see Chapter 13 on bias due to missing studies, and Chapter 4, Section 4.3 for a more detailed discussion of searching for unpublished and grey literature).

Likewise, while searching for, and analysing, studies in any language can be extremely resource-intensive, review authors should consider carefully the implications for bias (and equity, see Chapter 16 ) if they restrict eligible studies to those published in one specific language (usually English). See Chapter 4, Section 4.4.5 , for further discussion of language and other restrictions while searching.

MECIR Box 3.4.a Relevant expectations for conduct of intervention reviews

3.5 Chapter information

Authors: Joanne E McKenzie, Sue E Brennan, Rebecca E Ryan, Hilary J Thomson, Renea V Johnston, James Thomas

Acknowledgements: This chapter builds on earlier versions of the Handbook . In particular, Version 5, Chapter 5 , edited by Denise O’Connor, Sally Green and Julian Higgins.

Funding: JEM is supported by an Australian National Health and Medical Research Council (NHMRC) Career Development Fellowship (1143429). SEB and RER’s positions are supported by the NHMRC Cochrane Collaboration Funding Program. HJT is funded by the UK Medical Research Council (MC_UU_12017-13 and MC_UU_12017-15) and Scottish Government Chief Scientist Office (SPHSU13 and SPHSU15). RVJ’s position is supported by the NHMRC Cochrane Collaboration Funding Program and Cabrini Institute. JT is supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care North Thames at Barts Health NHS Trust. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

3.6 References

Bailey JV, Murray E, Rait G, Mercer CH, Morris RW, Peacock R, Cassell J, Nazareth I. Interactive computer-based interventions for sexual health promotion. Cochrane Database of Systematic Reviews 2010; 9 : CD006483.

Bender R, Bunce C, Clarke M, Gates S, Lange S, Pace NL, Thorlund K. Attention should be given to multiplicity issues in systematic reviews. Journal of Clinical Epidemiology 2008; 61 : 857–865.

Buchbinder R, Johnston RV, Rischin KJ, Homik J, Jones CA, Golmohammadi K, Kallmes DF. Percutaneous vertebroplasty for osteoporotic vertebral compression fracture. Cochrane Database of Systematic Reviews 2018; 4 : CD006349.

Caldwell DM, Welton NJ. Approaches for synthesising complex mental health interventions in meta-analysis. Evidence-Based Mental Health 2016; 19 : 16–21.

Chamberlain C, O’Mara-Eves A, Porter J, Coleman T, Perlen S, Thomas J, McKenzie J. Psychosocial interventions for supporting women to stop smoking in pregnancy. Cochrane Database of Systematic Reviews 2017; 2 : CD001055.

Ciciriello S, Johnston RV, Osborne RH, Wicks I, deKroo T, Clerehan R, O’Neill C, Buchbinder R. Multimedia educational interventions for consumers about prescribed and over-the-counter medications. Cochrane Database of Systematic Reviews 2013; 4 : CD008416.

Cochrane Consumers & Communication Group. Outcomes of Interest to the Cochrane Consumers & Communication Group: taxonomy. http://cccrg.cochrane.org/ .

COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiative. COSMIN database of systematic reviews of outcome measurement instruments. https://database.cosmin.nl/ .

Coulter A, Entwistle VA, Eccles A, Ryan S, Shepperd S, Perera R. Personalised care planning for adults with chronic or long-term health conditions. Cochrane Database of Systematic Reviews 2015; 3 : CD010523.

Davey J, Turner RM, Clarke MJ, Higgins JPT. Characteristics of meta-analyses and their component studies in the Cochrane Database of Systematic Reviews: a cross-sectional, descriptive analysis. BMC Medical Research Methodology 2011; 11 : 160.

Desroches S, Lapointe A, Ratte S, Gravel K, Legare F, Turcotte S. Interventions to enhance adherence to dietary advice for preventing and managing chronic diseases in adults. Cochrane Database of Systematic Reviews 2013; 2 : CD008722.

Deyo RA, Dworkin SF, Amtmann D, Andersson G, Borenstein D, Carragee E, Carrino J, Chou R, Cook K, DeLitto A, Goertz C, Khalsa P, Loeser J, Mackey S, Panagis J, Rainville J, Tosteson T, Turk D, Von Korff M, Weiner DK. Report of the NIH Task Force on research standards for chronic low back pain. Journal of Pain 2014; 15 : 569–585.

Dodd S, Clarke M, Becker L, Mavergames C, Fish R, Williamson PR. A taxonomy has been developed for outcomes in medical research to help improve knowledge discovery. Journal of Clinical Epidemiology 2018; 96 : 84–92.

Fisher DJ, Carpenter JR, Morris TP, Freeman SC, Tierney JF. Meta-analytical methods to identify who benefits most from treatments: daft, deluded, or deft approach? BMJ 2017; 356 : j573.

Fransen M, McConnell S, Harmer AR, Van der Esch M, Simic M, Bennell KL. Exercise for osteoarthritis of the knee. Cochrane Database of Systematic Reviews 2015; 1 : CD004376.

Guise JM, Chang C, Viswanathan M, Glick S, Treadwell J, Umscheid CA. Systematic reviews of complex multicomponent health care interventions. Report No. 14-EHC003-EF . Rockville, MD: Agency for Healthcare Research and Quality; 2014.

Harrison JK, Noel-Storr AH, Demeyere N, Reynish EL, Quinn TJ. Outcomes measures in a decade of dementia and mild cognitive impairment trials. Alzheimer’s Research and Therapy 2016; 8 : 48.

Hedges LV, Tipton E, Johnson M, C. Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods 2010; 1 : 39–65.

Hetrick SE, McKenzie JE, Cox GR, Simmons MB, Merry SN. Newer generation antidepressants for depressive disorders in children and adolescents. Cochrane Database of Systematic Reviews 2012; 11 : CD004851.

Higgins JPT, López-López JA, Becker BJ, Davies SR, Dawson S, Grimshaw JM, McGuinness LA, Moore THM, Rehfuess E, Thomas J, Caldwell DM. Synthesizing quantitative evidence in systematic reviews of complex health interventions. BMJ Global Health 2019; 4 : e000858.

Hoffmann T, Glasziou P, Barbour V, Macdonald H. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ 2014; 1687 : 1-13.

Hollands GJ, Shemilt I, Marteau TM, Jebb SA, Lewis HB, Wei Y, Higgins JPT, Ogilvie D. Portion, package or tableware size for changing selection and consumption of food, alcohol and tobacco. Cochrane Database of Systematic Reviews 2015; 9 : CD011045.

Howe TE, Shea B, Dawson LJ, Downie F, Murray A, Ross C, Harbour RT, Caldwell LM, Creed G. Exercise for preventing and treating osteoporosis in postmenopausal women. Cochrane Database of Systematic Reviews 2011; 7 : CD000333.

ICHOM. The International Consortium for Health Outcomes Measurement 2018. http://www.ichom.org/ .

IPDAS. International Patient Decision Aid Standards Collaboration (IPDAS) standards. www.ipdas.ohri.ca .

Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O’Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews 2012; 6 : CD000259.

Janmaat VT, Steyerberg EW, van der Gaast A, Mathijssen RH, Bruno MJ, Peppelenbosch MP, Kuipers EJ, Spaander MC. Palliative chemotherapy and targeted therapies for esophageal and gastroesophageal junction cancer. Cochrane Database of Systematic Reviews 2017; 11 : CD004063.

Kendrick D, Kumar A, Carpenter H, Zijlstra GAR, Skelton DA, Cook JR, Stevens Z, Belcher CM, Haworth D, Gawler SJ, Gage H, Masud T, Bowling A, Pearl M, Morris RW, Iliffe S, Delbaere K. Exercise for reducing fear of falling in older people living in the community. Cochrane Database of Systematic Reviews 2014; 11 : CD009848.

Kirkham JJ, Gargon E, Clarke M, Williamson PR. Can a core outcome set improve the quality of systematic reviews? A survey of the Co-ordinating Editors of Cochrane Review Groups. Trials 2013; 14 : 21.

Konstantopoulos S. Fixed effects and variance components estimation in three-level meta-analysis. Research Synthesis Methods 2011; 2 : 61–76.

Lamb SE, Becker C, Gillespie LD, Smith JL, Finnegan S, Potter R, Pfeiffer K. Reporting of complex interventions in clinical trials: development of a taxonomy to classify and describe fall-prevention interventions. Trials 2011; 12 : 125.

Lewin S, Hendry M, Chandler J, Oxman AD, Michie S, Shepperd S, Reeves BC, Tugwell P, Hannes K, Rehfuess EA, Welch V, Mckenzie JE, Burford B, Petkovic J, Anderson LM, Harris J, Noyes J. Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR). BMC Medical Research Methodology 2017; 17 : 76.

López-López JA, Page MJ, Lipsey MW, Higgins JPT. Dealing with multiplicity of effect sizes in systematic reviews and meta-analyses. Research Synthesis Methods 2018; 9 : 336–351.

Mavridis D, Salanti G. A practical introduction to multivariate meta-analysis. Statistical Methods in Medical Research 2013; 22 : 133–158.

Michie S, van Stralen M, West R. The Behaviour Change Wheel: a new method for characterising and designing behaviour change interventions. Implementation Science 2011; 6 : 42.

Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, Eccles MP, Cane J, Wood CE. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Annals of Behavioral Medicine 2013; 46 : 81–95.

Moraes VY, Lenza M, Tamaoki MJ, Faloppa F, Belloti JC. Platelet-rich therapies for musculoskeletal soft tissue injuries. Cochrane Database of Systematic Reviews 2014; 4 : CD010071.

O'Neill J, Tabish H, Welch V, Petticrew M, Pottie K, Clarke M, Evans T, Pardo Pardo J, Waters E, White H, Tugwell P. Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. Journal of Clinical Epidemiology 2014; 67 : 56–64.

Pompoli A, Furukawa TA, Imai H, Tajika A, Efthimiou O, Salanti G. Psychological therapies for panic disorder with or without agoraphobia in adults: a network meta-analysis. Cochrane Database of Systematic Reviews 2016; 4 : CD011004.

Pompoli A, Furukawa TA, Efthimiou O, Imai H, Tajika A, Salanti G. Dismantling cognitive-behaviour therapy for panic disorder: a systematic review and component network meta-analysis. Psychological Medicine 2018; 48 : 1–9.

Reeves BC, Wells GA, Waddington H. Quasi-experimental study designs series-paper 5: a checklist for classifying studies evaluating the effects on health interventions – a taxonomy without labels. Journal of Clinical Epidemiology 2017; 89 : 30–42.

Regnaux J-P, Lefevre-Colau M-M, Trinquart L, Nguyen C, Boutron I, Brosseau L, Ravaud P. High-intensity versus low-intensity physical activity or exercise in people with hip or knee osteoarthritis. Cochrane Database of Systematic Reviews 2015; 10 : CD010203.

Richards SH, Anderson L, Jenkinson CE, Whalley B, Rees K, Davies P, Bennett P, Liu Z, West R, Thompson DR, Taylor RS. Psychological interventions for coronary heart disease. Cochrane Database of Systematic Reviews 2017; 4 : CD002902.

Safi S, Korang SK, Nielsen EE, Sethi NJ, Feinberg J, Gluud C, Jakobsen JC. Beta-blockers for heart failure. Cochrane Database of Systematic Reviews 2017; 12 : CD012897.

Santesso N, Carrasco-Labra A, Brignardello-Petersen R. Hip protectors for preventing hip fractures in older people. Cochrane Database of Systematic Reviews 2014; 3 : CD001255.

Shepherd E, Gomersall JC, Tieu J, Han S, Crowther CA, Middleton P. Combined diet and exercise interventions for preventing gestational diabetes mellitus. Cochrane Database of Systematic Reviews 2017; 11 : CD010443.

Squires J, Valentine J, Grimshaw J. Systematic reviews of complex interventions: framing the review question. Journal of Clinical Epidemiology 2013; 66 : 1215–1222.

Stacey D, Légaré F, Lewis K, Barry MJ, Bennett CL, Eden KB, Holmes-Rovner M, Llewellyn-Thomas H, Lyddiatt A, Thomson R, Trevena L. Decision aids for people facing health treatment or screening decisions. Cochrane Database of Systematic Reviews 2017; 4 : CD001431.

Stroke Unit Trialists Collaboration. Organised inpatient (stroke unit) care for stroke. Cochrane Database of Systematic Reviews 2013; 9 : CD000197.

Taylor AJ, Jones LJ, Osborn DA. Zinc supplementation of parenteral nutrition in newborn infants. Cochrane Database of Systematic Reviews 2017; 2 : CD012561.

Valentine JC, Thompson SG. Issues relating to confounding and meta-analysis when including non-randomized studies in systematic reviews on the effects of interventions. Research Synthesis Methods 2013; 4 : 26–35.

Vaona A, Banzi R, Kwag KH, Rigon G, Cereda D, Pecoraro V, Tramacere I, Moja L. E-learning for health professionals. Cochrane Database of Systematic Reviews 2018; 1 : CD011736.

Verheyden GSAF, Weerdesteyn V, Pickering RM, Kunkel D, Lennon S, Geurts ACH, Ashburn A. Interventions for preventing falls in people after stroke. Cochrane Database of Systematic Reviews 2013; 5 : CD008728.

Weisz JR, Kuppens S, Ng MY, Eckshtain D, Ugueto AM, Vaughn-Coaxum R, Jensen-Doss A, Hawley KM, Krumholz Marchette LS, Chu BC, Weersing VR, Fordwood SR. What five decades of research tells us about the effects of youth psychological therapy: a multilevel meta-analysis and implications for science and practice. American Psychologist 2017; 72 : 79–117.

Welch V, Petkovic J, Simeon R, Presseau J, Gagnon D, Hossain A, Pardo Pardo J, Pottie K, Rader T, Sokolovski A, Yoganathan M, Tugwell P, DesMeules M. Interactive social media interventions for health behaviour change, health outcomes, and health equity in the adult population. Cochrane Database of Systematic Reviews 2018; 2 : CD012932.

Welton NJ, Caldwell DM, Adamopoulos E, Vedhara K. Mixed treatment comparison meta-analysis of complex interventions: psychological interventions in coronary heart disease. American Journal of Epidemiology 2009; 169 : 1158–1165.

Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, Clarke M, Gargon E, Gorst S, Harman N, Kirkham JJ, McNair A, Prinsen CAC, Schmitt J, Terwee CB, Young B. The COMET Handbook: version 1.0. Trials 2017; 18 : 280.

For permission to re-use material from the Handbook (either academic or commercial), please see here for full details.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• PMC10863900

Frameworks for measuring population health: A scoping review

Sze ling chan.

1 Health Services Research Centre, SingHealth, Singapore, Singapore

2 Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore

Clement Zhong Hao Ho

3 Centre for Population Health Research and Implementation, SingHealth Regional Health System, Singapore, Singapore

Nang Ei Ei Khaing

4 Health Services Research, Changi General Hospital, Singapore, Singapore

Candelyn Pong

Jia sheng guan.

5 School of Biological Sciences, Nanyang Technological University, Singapore, Singapore

Calida Chua

6 Care and Health Integration, Changi General Hospital, Singapore, Singapore

7 Preventive Medicine Residency, National University Health System, Singapore, Singapore

8 School of Computing and Information Systems, Singapore Management University, Singapore, Singapore

Sean Shao Wei Lam

Lian leng low.

9 Post-Acute and Continuing Care, Outram Community Hospital, Singapore, Singapore

10 Department of Family Medicine and Continuing Care, Singapore General Hospital, Singapore, Singapore

11 SingHealth Duke-NUS Family Medicine Academic Clinical Program, Singapore, Singapore

Choon How How

12 SingHealth Office of Regional Health, Changi General Hospital, Singapore, Singapore

Associated Data

All relevant data are within the paper and S2 File .

Introduction

Many regions in the world are using the population health approach and require a means to measure the health of their population of interest. Population health frameworks provide a theoretical grounding for conceptualization of population health and therefore a logical basis for selection of indicators. The aim of this scoping review was to provide an overview and summary of the characteristics of existing population health frameworks that have been used to conceptualize the measurement of population health.

We used the Population, Concept and Context (PCC) framework to define eligibility criteria of frameworks. We were interested in frameworks applicable for general populations, that contained components of measurement of health with or without its antecedents and applied at the population level or used a population health approach. Eligible reports of eligible frameworks should include at least domains and subdomains, purpose, or indicators. We searched 5 databases (Pubmed, EMBASE, Web of Science, NYAM Grey Literature Report, and OpenGrey), governmental and organizational sites on Google and websites of selected organizations using keywords from the PCC framework. Characteristics of the frameworks were summarized descriptively and narratively.

Fifty-seven frameworks were included. The majority originated from the US (46%), Europe (23%) and Canada (19%). Apart from 1 framework developed for rural populations and 2 for indigenous populations, the rest were for general urban populations. The numbers of domains, subdomains and indicators were highly variable. Health status and social determinants of health were the most common domains across all frameworks. Different frameworks had different priorities and therefore focus on different domains.

Key domains common across frameworks other than health status were social determinants of health, health behaviours and healthcare system performance. The results in this review serve as a useful resource for governments and healthcare organizations for informing their population health measurement efforts.

Population health has become an increasingly prominent concept in public health discourse, governance, and research in recent years. In their seminal paper, Kindig and Stoddart defines population health as an approach to understanding health that transcends the individual, focusing on interrelated factors and conditions shaping the health of a population. These includes the environment, social and cultural forces, lifestyle choices and government policies [ 1 ]. In other words, health cannot be fully understood without a contextualisation of socioeconomic and other factors, such as lifestyle, that are shaped by environments and communities [ 2 ]. This change in focus and understanding of health originated during the 1970s-80s in response to the growing body of evidence on social determinants of health, and increasing advocacy for social justice and equity [ 3 ]. In contrast to the traditional biomedical model that focused on individual risk factors of diseases, such as obesity, alcohol consumption or family history, a population health approach adopts an upstream preventive approach by addressing root causes, rather than symptoms, to achieve health outcomes.

Population health indicators provide a means for government agencies and Non-Governmental Organisations (NGO) to monitor public health, evaluate interventions, and guide population health policies. Summary measures such as life-expectancy are commonly used to measure the health of a population and for benchmarking against others but are limited on their own, as they do not provide information on other aspects of health [ 4 ]. With health and its antecedents being complex and multifaceted constructs, so is the selection of relevant population health indicators. In a scoping review of population health indices, only 7 out of 27 indices had a theoretical or conceptual foundation guiding the aggregation of indicators in a meaningful way [ 5 ].

A framework should therefore precede indicator selection [ 4 ]. Frameworks provide a structure by which to organise the dynamic and interrelated factors between individuals and their environment, and through which to develop hypotheses about how such relationships affect health outcomes over time [ 6 ]. For instance, the widely accepted Canadian Institutes of Health Research population health framework provides an integrated view of health through upstream forces (a whole spectrum of cultural, economic, social and other forces), proximal causes of heath (such as physiological risk factors), lifespan processes, disparities across sub-populations, health services, and health outcomes, as well as the indicators and indices used to measure them [ 7 ]. Others may differ depending on their purpose and definition of health and population health.

The usage of a population health framework is necessary as it provides a theoretical grounding and context for selection of indicators and clarifies the role of each indicator [ 5 ]. Indeed, this is a step many government agencies and NGOs have taken in their population health efforts. There have been reviews on population health indicators [ 5 , 7 , 8 ]. However, to our knowledge there is no work that organises and clarifies this growing body of literature.

In this paper, we conducted a scoping review with the aim of providing an overview and summary of the characteristics of existing population health frameworks that have been used to conceptualize the measurement of population health. Specific aims were to understand what domains were included in the frameworks, how or why they were chosen, and what some representative indicators under each domain were.

This scoping review follows the guidelines described by the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist, a minimum set of items for reporting of scoping reviews to promote transparent reporting of scoping reviews [ 9 ] ( S1 File ).

Eligibility criteria

The eligibility criteria of population health frameworks were guided by the elements of the Population, Concept, and Context (PCC) framework. In the population element, we were interested in frameworks that were applied to general populations, which included subsets by demographic variables (e.g. age or ethnicity). However, we excluded populations which were defined by illnesses or diseases (e.g. stroke or mental health patients), or institutional settings (e.g. workplace, schools).

For the Concept element, frameworks should contain components of measurement of health, with or without its antecedents. Frameworks by definition convey structure, at least in the form of categorization [ 6 ]. Therefore, eligible frameworks should fulfil this definition. Simple lists of indicators without categories are excluded. Frameworks should also be novel, so mere representations of known literature or frameworks with insufficient explanation, and logic models for specific programs were excluded. For context, frameworks should be applied at the macrolevel, or use a population health approach.

Eligible reports of eligible frameworks would need to include at least one of the following dimensions– 1) Domains and subdomains; 2) purpose of the framework; or 3) population health indicators used. Where there were more than 1 report for the same framework, we selected the one with the most relevant and comprehensive information. If another report supplemented information not found in this primary report, we would include both. We included primary articles of any study design, reviews and selected grey literature. Conference abstracts, theses and dissertations, letters to editors, commentaries, non-English articles, and articles published before 1990 were excluded.

Information sources

We searched MEDLINE (PubMed), EMBASE, Web of Science, NYAM Grey Literature Report and OpenGrey databases. In addition, we searched governmental and organizational sites on Google (site:.gov OR site:.org OR site:.net OR site:.eu) and websites of the following government agencies and NGOs known to have population health initiatives and/or frameworks:

• UK National Health Service (NHS)
• Agency for Healthcare Research and Quality (AHRQ)
• Centres for Disease Control (CDC)
• US Department of Health and Human Services
• Public Health Agency of Canada
• Australian Government Department of Health
• World Health Organization (WHO)
• Organisation for Economic Co-operation and Development (OECD)
• Public Health England
• European Union (EU) CDC
• National Quality Forum (NQF)
• Health Information Technology, Evaluation, and Quality Center (HITEQ)
• The King’s Fund
• Africa Population and Health Research Centre
• Canterbury District Health Board

Search strategy

We used the keywords ‘framework’ and ‘population health’ from the concept and context elements as search terms, respectively. Depending on the database, we used these terms as keywords or also included controlled vocabulary that corresponded to them. The keywords or controlled vocabulary were combined using the BOOLEAN operator ‘OR’ and ‘AND’ within and across the PCC elements, respectively. The search terms are given in S2 File . Where possible, filters were applied to select only human studies and English articles. The search of the databases was performed from 1 Jan 1990 to 5 May 2023. For some databases (Pubmed, EMBASE, Web of Science) we further applied a ‘title/abstract’ filter to improve the specificity of the search results. If we came across reports that mention an eligible framework but did not contain the relevant details to be included, we then searched for reports on that particular framework. We also searched reference lists of included reports.

Selection of sources of evidence

Three reviewers (SLC, CZHH, NEEK) developed and piloted the search strategy. Two stages of screenings were performed to select the sources of evidence. At the first stage, the titles and abstracts of each source was screened and selected for full text review by two reviewers independently. In the second stage, the full texts of articles selected in the first stage were also reviewed by 2 reviewers independently. In both stages, a third reviewer would make the final decision in the event of a conflict.

Data charting process

A data charting form to extract data of interest was developed by one reviewer (SLC) and piloted by another (CZHH). Data from each report was extracted by one reviewer and reviewed by a second reviewer. Any discrepancies were resolved by consensus between the data extractor and reviewer.

The data items included citation details, details on the framework (e.g. name, country of origin, organization that developed it, type of population it is applicable to, approach to development, dimensions in framework apart from domains, if framework assessed indicators by certain cross-cutting variables such as life stages, socioeconomic factors, and/or health-related sectors), and the domains and indicators used in the framework, including definitions or descriptions where available. For domains, we recorded up to 2 further levels of sub-domains (total 3 levels).

Synthesis of results

To facilitate summary and presentation of results, some variables were reduced to a smaller number of categories manually by a single reviewer (SLC). These variables were the type of organization developing the frameworks, types of population the framework was applicable to, and dimensions of the framework. Types of organizations were broadly categorized into governmental, academic, non-government organizations, non-profit organizations, intergovernmental organizations, and private foundations. Populations were grouped in to general, rural and indigenous populations. Finally, dimensions cut across domains and indicators and we focused mainly on a lifespan, health equity and sector approach. For the lifespan approach, this generally involve diving into indicators relevant for different life stages and/or breaking down indicators by age groups. For the equity approach this typically involves examining indicators by certain socioeconomic factors, such as education level, income, and ethnicity. For the sector approach, this involves looking at indicators specific for different health-related sectors such as clinical care, public health, and community and social services. We categorized frameworks under ‘dimensions’ into lifespan, equity and/or other specific dimensions mentioned.

The characteristics of the frameworks were then summarized descriptively using counts and proportions, and median and ranges, as appropriate. Domains were aggregated by concept using hierarchical clustering and manual refinement for purposes of visualization. The final clustering was agreed on by 3 reviewers (SLC, CP, JSG). The domain concepts, and number of domains, subdomains and indicators were visualized using a word cloud and heatmap, respectively. Other aspects of the frameworks were summarized narratively.

Search results

A total of 57 population health frameworks were included in this review ( Fig 1 ). The characteristics of the frameworks and their details are shown in Tables ​ Tables1 1 and ​ and2, 2 , respectively. The full list of the domains, subdomains and indicators are provided in S3 File .

The PRISMA diagram shows the numbers of reports retrieved from various sources and flow through the stages of the scoping review. A total of 57 reports were included in this review. The diagram was generated using an open source R shiny app [ 10 ].

*Percentages may not add up to 100% due to rounding

*These are websites and the year is based on the date of access,

† not all frameworks explicitly mentioned a dimension.

Acad: academic, Gov: government, Inter-gov: inter-government, NGO: non-government organisation, NPO: non-profit organisation

Characteristics of population health frameworks

Majority of the frameworks originated from the US (45.6%), Europe (22.8%) and Canada (19.3%). None were from Asia. Most were published between 2001 and 2020 (64.9%). Governmental (including intergovernmental) and academic organizations accounted for majority of framework development (84.2%). Only three frameworks were developed for specific populations (2 for indigenous and 1 for rural), while the rest were for the general or urban population. Two-thirds of the frameworks mentioned some dimension, and these were slightly more frameworks using the lifespan approach compared to the equity approach (29.8% vs. 21.1%).

Domains and subdomains

Majority of the frameworks have between 1 to 5 domains (70.2%) but have more level 2 sub-domains (26.3% have 6–10, 29.8% have 11–20 and 19.3% have >20). The median number of domains and level 2 subdomains are 4 (range 2–16) and 10 (range 0–65), respectively ( S1 Fig ). Half of the frameworks do not have level 3 subdomains. Of those that do, most have >10 (72.4%). The median number of indicators is 18 (range 0–255). Twenty-six frameworks did not have indicators (45.6%). Of those that do, majority have >20 indicators (83.9%).

The most common concepts were health, (social) determinants of health, healthcare system and health behaviours ( S2 Fig ). The myriad of domains has gradually accumulated over the years. In frameworks published before 2000, health was the key domain, social determinants of health emerged in the next 2 decades (2001–2020) followed by healthcare system, health behaviours, functional limitations and activities of daily living in the recent frameworks ( S3 Fig ).

For health, most frameworks used summary indicators of health such as mortality and life-expectancy, and indicators of a few selected health conditions. However, four frameworks had longer lists of indicators for specific communicable and non-communicable diseases [ 12 , 26 , 27 , 43 , 50 ]. Of note, psychological or mental health risk factors and/or outcomes feature in 31 (54%) of the frameworks, highlighting its emerging importance [ 12 , 17 – 19 , 22 , 25 – 30 , 32 – 35 , 38 , 39 , 41 – 46 , 48 – 50 , 54 , 56 , 58 , 59 , 62 ].

Social determinants of health, which encompasses the full set of social conditions in which people live and work [ 66 ], were present under some label or other in all except 7 frameworks [ 16 , 34 , 42 , 50 , 52 , 54 , 60 ]. Some of the frameworks elaborate on these factors, with sub-domains and indicators on the physical environment, social environment, and even politics, national and global trends [ 12 , 21 – 23 , 26 , 29 , 35 , 43 , 53 , 55 – 59 , 63 , 64 , 67 ]. For example, the conceptual framework for urban health measures sub-domains such as immigration, globalization and the changing role of government [ 21 ]. The framework for community contextual characteristics, one of the two frameworks with the largest number of indicators, also measures the economic, employment, education, political, environmental, housing, governmental, transport aspects in the region where the population of interest is located [ 29 ]. Interestingly, crime and violence features in 16 frameworks, as this affects the physical safety of people in a community [ 12 , 15 , 26 , 29 , 30 , 33 , 41 , 43 – 45 , 53 , 56 , 59 , 62 , 63 , 67 ]. Many frameworks also measure lifestyle and health-related behaviours. Apart from the common ones like diet, physical activity, smoking and alcohol use, some frameworks include sexual behaviour, use of illicit drugs, seatbelt behaviour, immunization or health screening, breastfeeding and induced abortion [ 12 , 15 , 27 – 30 , 32 , 33 , 39 , 45 , 55 , 58 , 59 , 62 ]. One even included measures of parenting practices [ 43 ].

Almost a third (31.6%) of the frameworks have domains that pertain to the healthcare system or healthcare performance. One example is the OECD framework, which assesses health system performance within the context of other contextual determinants of health [ 46 ]. Within the construct of healthcare performance, common subdomains are accessibility, capacity, quality, patient-centeredness, cost and effectiveness [ 11 , 16 , 19 , 24 , 31 , 32 , 34 , 39 , 43 , 46 , 48 , 51 , 53 , 54 ].

A few of the frameworks had specific focuses and therefore unique domains and indicators that are relevant largely for their setting. For example, the reporting framework for indigenous adolescents in Australia contained domains that were largely relevant for that community, such as ‘family, kinship and community health’, which explored family roles and responsibilities, contact with extended family, removal from family, participation in community events and sense of belonging to the community [ 12 ]. Another example is the Ghana’s Holistic Assessment Tool, which contains indicators for health-related United Nations sustainable development goals (SDGs) such as proportion of deliveries attended by a trained health worker, proportion of children under 5 years sleeping under insecticide treated net, and tuberculosis treatment success rate, and certain endemic communicable diseases such as non-acute flaccid paralysis polio rate [ 42 ].

Approach to framework development

Evans and Stoddart developed a population health framework in 1990 [ 20 ] based on a much earlier 1974 Whitepaper titled “A new perspective on the health of Canadians”, which recognized the limitations of the healthcare system on improving health status and presented a preliminary framework of the ‘health field’ [ 68 ]. Subsequent frameworks were mostly developed from one or a combination of four approaches: 1) adaptation from an existing framework [ 11 , 12 , 33 , 45 , 46 , 48 – 51 , 56 , 58 – 60 , 63 , 65 ], 2) environmental scan of existing frameworks and literature review to summarize current knowledge of health determinants [ 7 , 14 , 16 – 20 , 24 , 25 , 29 , 32 , 36 , 37 , 44 , 48 , 52 , 57 , 61 , 63 ], 3) consulting and getting inputs from experts and stakeholders [ 12 , 17 , 19 , 24 , 26 – 29 , 35 , 39 , 41 , 48 , 52 – 55 , 62 , 63 ] and 4) basing on past work (e.g. primary data collection, drawing on secondary data, past population health efforts, etc), priorities and goals of the organization developing it [ 7 , 11 , 21 , 38 , 61 , 64 , 67 ].

Population health has been a popular concept in healthcare for the past 3 decades but interestingly does not have a unanimous definition [ 1 , 2 , 69 ]. The most commonly used definition, which originated from Kindig and Stoddart, defines population health as ‘the health outcomes of a group of individuals, including the distribution of such outcomes within the group” [ 1 ]. Nevertheless, people working on ‘population health’ would have different focuses, goals and populations of interest [ 69 ]. This may explain the large number of population health frameworks we found in this review.

Population health has its roots from recognition of health disparities by socioeconomic factors from as early as the 18 th century to early epidemiological studies that informed public health measures, particularly in Britain and France, and finally to a renewed interest in the last 2 decades due to a range of health problems facing the world [ 70 ]. Development of the population health approach in Canada, driven by the government and healthcare leaders, began in the 1970s [ 71 ]. Improving population health was motivated by the articulation of the Triple Aims as a goal for the US healthcare system in the late 2000s [ 72 ]. It is therefore unsurprising that most of the frameworks originate from US, Europe and Canada. Even with purposive searching of organizations in the Southern hemisphere such as Australia and New Zealand, the results were still dominated by the Northern hemisphere, reflecting the state of development of population health in the world. Similarly, the lack of frameworks from Asia might be because much of the work done in improving the health of populations is ‘public health’ rather than ‘population health’.

Health status and social determinants of health were the most common domains across the frameworks. As seen from the word cloud, there were also many other domains that were closely related to and/or could be considered subdomains of one of these domains. This is because different frameworks have different level of detail, and the hierarchy of domains and subdomains are different in level of detail across frameworks. In other words, a subdomain in one framework could be a domain in another, or an indicator in one framework could be a subdomain in another. It is therefore also difficult to summarize domains and subdomains in a simple way across the frameworks.

The domains and subdomains chosen in different frameworks largely reflects the purpose, information needs of varying stakeholders, and the focus of the organization(s) developing them. It is unsurprising to see that some key domains appear in many frameworks, and domains are branched out to varying degrees in different frameworks. For example, social determinants of health features in all frameworks except 7 frameworks [ 16 , 34 , 42 , 50 , 52 , 54 , 60 ]. Some frameworks have a heavy focus on health status, such as the Healthy Montogomery Core Measures Set, Triple Aim, Euro-REVES 2 and Ohio health priorities, with the Euro-REVES 2 framework even measuring activities of daily living and degree of functional limitations [ 26 , 27 , 50 , 60 ]. Other frameworks break down the social determinants into considerable detail, such as the framework for community contextual characteristics, life course health development framework, Healthy Cities Indicators, and others [ 12 , 22 , 23 , 26 , 29 , 38 , 49 , 53 , 55 , 56 , 59 , 63 , 64 , 67 ]. Several have a heavier focus on healthcare performance, such as the EU Joint Assessment Framework, European Community Health Indicators (ECHI), OECD, the Primary Healthcare Performance Initiative (PHCPI), National scorecard for the US health system and the Ireland HSPA framework [ 19 , 34 , 39 , 46 , 48 , 54 ]. Others are generally more balanced between the domains.

It is also noteworthy that almost half of the frameworks did not have any indicators and these tended to be older frameworks. About 61% of frameworks developed in 2010 and before did not have indicators while the converse is true for those developed after 2010. There was likely stronger focus on understanding the range of factors affecting population health and identifying priorities for improving population health in the earlier period. As organizations started to implement population health management strategies, measurement of population health started to feature more and more recent frameworks tended to include specific indicators. The inclusion of specific indicators also implies the ability to measure them, and therefore the availability of health information systems for data collection. These have generally become more well developed in the recent decade or so, also explaining why more recent frameworks have indicators. Nevertheless, frameworks without indicators can still offer a theoretical basis for selecting indicators that are relevant and feasible for a given setting.

The results of this scoping review can serve as an evidence base for governments and/or health systems developing their own population health frameworks and selecting indicators for their population health initiatives. They can select and adapt from the frameworks available, and assess the relevance of the range of domains, subdomains and indicators in their context. Populations are largely unique as they are shaped by their local and wider contextual factors. As such, no one framework used in one population or healthcare system is likely directly applicable to another population or healthcare system without adaptation. Population health practitioners can derive any level of detail that matches their interests and requirements from this review, from a broad sense of the literature down to specific indicators. The range of subdomains and indicators could also be sources of new hypotheses in a given region or jurisdiction for the purposes of population health research.

Settings which are further ahead in the population health journey with existing indicators can also use these results to assess what domains and subdomains have been covered, and where the gaps are. For example, population health is an increasingly important national priority in Singapore and the Ministry of Health is planning several major initiatives to improve the health of the general population [ 73 , 74 ]. To achieve this, the Ministry is working closely with the three major public healthcare clusters in Singapore to develop a set of population health indicators and the evidence base here can help inform the choices. With an initial set of indicators, practitioners can also interrogate their data systems and medical records to determine if they are available or if they need to build prospective data collection tools. This can also be an iterative process for selecting indicators using the results here as a resource. One constraint of the data in its current form though is the difficulty in navigating the long list of domains, subdomains and indicators. In future work, we aim to design a dashboard that allows for interactive exploration of the scoping review data.

There are limitations to this scoping review. Firstly, some frameworks might have been missed due to our language restriction, especially those in Asia. However, many official documents from this region are available in English, so this might not have impacted the search results significantly. Secondly, there are many terms and concepts in the literature that have overlaps with population health, such as public health, urban health, global health, population health management, health equity, health system performance and social determinants of health. Based on our inclusion criteria, concepts like urban health, rural health, community health and global health would be included as they pertain to general populations albeit in different types of settings. Related concepts such as health equity, social determinants of health and health system performance were not the focus of the search and could be part of the frameworks included. However, if a framework was focused on one of these concepts alone without the measurement of health status, then it would be excluded. Some frameworks also focused more on population health management and if it looked more like a logic model for specific interventions then these would also be excluded [ 75 , 76 ]. Overall, this review represents a useful collection of frameworks used for measuring the health of a population and its key antecedents [ 60 ].

We found 57 frameworks for the measurement of population health with variable numbers of domains, subdomains and indicators, and depth of detail. The key domains apart from health status were social determinants of health, health behaviours and healthcare system performance. These results serve as a useful resource for governments and healthcare organizations for informing their population health measurement efforts. Specifically, when developing their own population health framework and/or selection of population health indicators, they can identify common domains and subdomains that other organizations include, as well as consider others more systematically for relevance in their context.

Supporting information

This file contains the full list of domains, subdomains and indicators from the 57 included population health frameworks.

L2: level 2, L3: level 3, This is a visualization of the numbers of domains, subdomains and indicators in each framework in both figures and shading. Blank cells represent absence of the corresponding subdomain and/or indicators.

Level 1 domains in all frameworks were clustered by concept using a combination of hierarchical clustering and manual edit. The sizes of the concepts are proportional to the number of domains in each concept.

Level 1 domains in all frameworks were clustered by concept using a combination of hierarchical clustering and manual edit. The sizes of the concepts are proportional to the number of domains in each concept. The concepts are presented by decade when the frameworks were published. A: Before 2000, B: 2001 to 2010, C: 2011 to 2020, D: After 2020.

Acknowledgments

We would like to thank Ms Sabrina Liau for her assistance with article screening.

Funding Statement

This research is supported by the National Medical Research Council (NMRC) through the SingHealth PULSES II Centre Grant (CG21APR1013).

Data Availability

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Frameworks for measuring population health: A scoping review

Roles Data curation, Formal analysis, Methodology, Project administration, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliations Health Services Research Centre, SingHealth, Singapore, Singapore, Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore

Roles Data curation, Investigation, Methodology, Project administration, Validation, Writing – review & editing

Affiliation Centre for Population Health Research and Implementation, SingHealth Regional Health System, Singapore, Singapore

Roles Data curation, Investigation, Methodology, Writing – review & editing

Affiliation Health Services Research, Changi General Hospital, Singapore, Singapore

Roles Data curation, Investigation, Methodology, Validation, Writing – review & editing

Roles Data curation, Investigation, Validation, Writing – review & editing

Roles Formal analysis, Methodology, Validation, Visualization, Writing – review & editing

Affiliation School of Biological Sciences, Nanyang Technological University, Singapore, Singapore

Roles Investigation, Writing – review & editing

Affiliation Care and Health Integration, Changi General Hospital, Singapore, Singapore

Affiliation Preventive Medicine Residency, National University Health System, Singapore, Singapore

Affiliation School of Computing and Information Systems, Singapore Management University, Singapore, Singapore

Roles Conceptualization, Project administration, Resources, Writing – review & editing

Roles Writing – review & editing

Affiliations Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore, Centre for Population Health Research and Implementation, SingHealth Regional Health System, Singapore, Singapore, Post-Acute and Continuing Care, Outram Community Hospital, Singapore, Singapore, Department of Family Medicine and Continuing Care, Singapore General Hospital, Singapore, Singapore, SingHealth Duke-NUS Family Medicine Academic Clinical Program, Singapore, Singapore

Roles Conceptualization, Resources, Supervision, Writing – review & editing

Affiliations SingHealth Duke-NUS Family Medicine Academic Clinical Program, Singapore, Singapore, SingHealth Office of Regional Health, Changi General Hospital, Singapore, Singapore

• Sze Ling Chan, 
• Clement Zhong Hao Ho, 
• Nang Ei Ei Khaing, 
• Ezra Ho, 
• Candelyn Pong, 
• Jia Sheng Guan, 
• Calida Chua, 
• Zongbin Li, 
• Trudi Lim, 

• Published: February 13, 2024
• https://doi.org/10.1371/journal.pone.0278434
• Reader Comments

Introduction

Many regions in the world are using the population health approach and require a means to measure the health of their population of interest. Population health frameworks provide a theoretical grounding for conceptualization of population health and therefore a logical basis for selection of indicators. The aim of this scoping review was to provide an overview and summary of the characteristics of existing population health frameworks that have been used to conceptualize the measurement of population health.

We used the Population, Concept and Context (PCC) framework to define eligibility criteria of frameworks. We were interested in frameworks applicable for general populations, that contained components of measurement of health with or without its antecedents and applied at the population level or used a population health approach. Eligible reports of eligible frameworks should include at least domains and subdomains, purpose, or indicators. We searched 5 databases (Pubmed, EMBASE, Web of Science, NYAM Grey Literature Report, and OpenGrey), governmental and organizational sites on Google and websites of selected organizations using keywords from the PCC framework. Characteristics of the frameworks were summarized descriptively and narratively.

Fifty-seven frameworks were included. The majority originated from the US (46%), Europe (23%) and Canada (19%). Apart from 1 framework developed for rural populations and 2 for indigenous populations, the rest were for general urban populations. The numbers of domains, subdomains and indicators were highly variable. Health status and social determinants of health were the most common domains across all frameworks. Different frameworks had different priorities and therefore focus on different domains.

Key domains common across frameworks other than health status were social determinants of health, health behaviours and healthcare system performance. The results in this review serve as a useful resource for governments and healthcare organizations for informing their population health measurement efforts.

Citation: Chan SL, Ho CZH, Khaing NEE, Ho E, Pong C, Guan JS, et al. (2024) Frameworks for measuring population health: A scoping review. PLoS ONE 19(2): e0278434. https://doi.org/10.1371/journal.pone.0278434

Editor: Angela Mendes Freitas, University of Coimbra: Universidade de Coimbra, PORTUGAL

Received: November 15, 2022; Accepted: October 3, 2023; Published: February 13, 2024

Copyright: © 2024 Chan et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper and S2 File .

Funding: This research is supported by the National Medical Research Council (NMRC) through the SingHealth PULSES II Centre Grant (CG21APR1013).

Competing interests: The authors have declared that no competing interests exist.

Population health has become an increasingly prominent concept in public health discourse, governance, and research in recent years. In their seminal paper, Kindig and Stoddart defines population health as an approach to understanding health that transcends the individual, focusing on interrelated factors and conditions shaping the health of a population. These includes the environment, social and cultural forces, lifestyle choices and government policies [ 1 ]. In other words, health cannot be fully understood without a contextualisation of socioeconomic and other factors, such as lifestyle, that are shaped by environments and communities [ 2 ]. This change in focus and understanding of health originated during the 1970s-80s in response to the growing body of evidence on social determinants of health, and increasing advocacy for social justice and equity [ 3 ]. In contrast to the traditional biomedical model that focused on individual risk factors of diseases, such as obesity, alcohol consumption or family history, a population health approach adopts an upstream preventive approach by addressing root causes, rather than symptoms, to achieve health outcomes.

Population health indicators provide a means for government agencies and Non-Governmental Organisations (NGO) to monitor public health, evaluate interventions, and guide population health policies. Summary measures such as life-expectancy are commonly used to measure the health of a population and for benchmarking against others but are limited on their own, as they do not provide information on other aspects of health [ 4 ]. With health and its antecedents being complex and multifaceted constructs, so is the selection of relevant population health indicators. In a scoping review of population health indices, only 7 out of 27 indices had a theoretical or conceptual foundation guiding the aggregation of indicators in a meaningful way [ 5 ].

A framework should therefore precede indicator selection [ 4 ]. Frameworks provide a structure by which to organise the dynamic and interrelated factors between individuals and their environment, and through which to develop hypotheses about how such relationships affect health outcomes over time [ 6 ]. For instance, the widely accepted Canadian Institutes of Health Research population health framework provides an integrated view of health through upstream forces (a whole spectrum of cultural, economic, social and other forces), proximal causes of heath (such as physiological risk factors), lifespan processes, disparities across sub-populations, health services, and health outcomes, as well as the indicators and indices used to measure them [ 7 ]. Others may differ depending on their purpose and definition of health and population health.

The usage of a population health framework is necessary as it provides a theoretical grounding and context for selection of indicators and clarifies the role of each indicator [ 5 ]. Indeed, this is a step many government agencies and NGOs have taken in their population health efforts. There have been reviews on population health indicators [ 5 , 7 , 8 ]. However, to our knowledge there is no work that organises and clarifies this growing body of literature.

In this paper, we conducted a scoping review with the aim of providing an overview and summary of the characteristics of existing population health frameworks that have been used to conceptualize the measurement of population health. Specific aims were to understand what domains were included in the frameworks, how or why they were chosen, and what some representative indicators under each domain were.

This scoping review follows the guidelines described by the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist, a minimum set of items for reporting of scoping reviews to promote transparent reporting of scoping reviews [ 9 ] ( S1 File ).

Eligibility criteria

The eligibility criteria of population health frameworks were guided by the elements of the Population, Concept, and Context (PCC) framework. In the population element, we were interested in frameworks that were applied to general populations, which included subsets by demographic variables (e.g. age or ethnicity). However, we excluded populations which were defined by illnesses or diseases (e.g. stroke or mental health patients), or institutional settings (e.g. workplace, schools).

For the Concept element, frameworks should contain components of measurement of health, with or without its antecedents. Frameworks by definition convey structure, at least in the form of categorization [ 6 ]. Therefore, eligible frameworks should fulfil this definition. Simple lists of indicators without categories are excluded. Frameworks should also be novel, so mere representations of known literature or frameworks with insufficient explanation, and logic models for specific programs were excluded. For context, frameworks should be applied at the macrolevel, or use a population health approach.

Eligible reports of eligible frameworks would need to include at least one of the following dimensions– 1) Domains and subdomains; 2) purpose of the framework; or 3) population health indicators used. Where there were more than 1 report for the same framework, we selected the one with the most relevant and comprehensive information. If another report supplemented information not found in this primary report, we would include both. We included primary articles of any study design, reviews and selected grey literature. Conference abstracts, theses and dissertations, letters to editors, commentaries, non-English articles, and articles published before 1990 were excluded.

Information sources

We searched MEDLINE (PubMed), EMBASE, Web of Science, NYAM Grey Literature Report and OpenGrey databases. In addition, we searched governmental and organizational sites on Google (site:.gov OR site:.org OR site:.net OR site:.eu) and websites of the following government agencies and NGOs known to have population health initiatives and/or frameworks:

• UK National Health Service (NHS)
• Agency for Healthcare Research and Quality (AHRQ)
• Centres for Disease Control (CDC)
• US Department of Health and Human Services
• Public Health Agency of Canada
• Australian Government Department of Health
• World Health Organization (WHO)
• Organisation for Economic Co-operation and Development (OECD)
• Public Health England
• European Union (EU) CDC
• National Quality Forum (NQF)
• Health Information Technology, Evaluation, and Quality Center (HITEQ)
• The King’s Fund
• Africa Population and Health Research Centre
• Canterbury District Health Board

Search strategy

We used the keywords ‘framework’ and ‘population health’ from the concept and context elements as search terms, respectively. Depending on the database, we used these terms as keywords or also included controlled vocabulary that corresponded to them. The keywords or controlled vocabulary were combined using the BOOLEAN operator ‘OR’ and ‘AND’ within and across the PCC elements, respectively. The search terms are given in S2 File . Where possible, filters were applied to select only human studies and English articles. The search of the databases was performed from 1 Jan 1990 to 5 May 2023. For some databases (Pubmed, EMBASE, Web of Science) we further applied a ‘title/abstract’ filter to improve the specificity of the search results. If we came across reports that mention an eligible framework but did not contain the relevant details to be included, we then searched for reports on that particular framework. We also searched reference lists of included reports.

Selection of sources of evidence

Three reviewers (SLC, CZHH, NEEK) developed and piloted the search strategy. Two stages of screenings were performed to select the sources of evidence. At the first stage, the titles and abstracts of each source was screened and selected for full text review by two reviewers independently. In the second stage, the full texts of articles selected in the first stage were also reviewed by 2 reviewers independently. In both stages, a third reviewer would make the final decision in the event of a conflict.

Data charting process

A data charting form to extract data of interest was developed by one reviewer (SLC) and piloted by another (CZHH). Data from each report was extracted by one reviewer and reviewed by a second reviewer. Any discrepancies were resolved by consensus between the data extractor and reviewer.

The data items included citation details, details on the framework (e.g. name, country of origin, organization that developed it, type of population it is applicable to, approach to development, dimensions in framework apart from domains, if framework assessed indicators by certain cross-cutting variables such as life stages, socioeconomic factors, and/or health-related sectors), and the domains and indicators used in the framework, including definitions or descriptions where available. For domains, we recorded up to 2 further levels of sub-domains (total 3 levels).

Synthesis of results

To facilitate summary and presentation of results, some variables were reduced to a smaller number of categories manually by a single reviewer (SLC). These variables were the type of organization developing the frameworks, types of population the framework was applicable to, and dimensions of the framework. Types of organizations were broadly categorized into governmental, academic, non-government organizations, non-profit organizations, intergovernmental organizations, and private foundations. Populations were grouped in to general, rural and indigenous populations. Finally, dimensions cut across domains and indicators and we focused mainly on a lifespan, health equity and sector approach. For the lifespan approach, this generally involve diving into indicators relevant for different life stages and/or breaking down indicators by age groups. For the equity approach this typically involves examining indicators by certain socioeconomic factors, such as education level, income, and ethnicity. For the sector approach, this involves looking at indicators specific for different health-related sectors such as clinical care, public health, and community and social services. We categorized frameworks under ‘dimensions’ into lifespan, equity and/or other specific dimensions mentioned.

The characteristics of the frameworks were then summarized descriptively using counts and proportions, and median and ranges, as appropriate. Domains were aggregated by concept using hierarchical clustering and manual refinement for purposes of visualization. The final clustering was agreed on by 3 reviewers (SLC, CP, JSG). The domain concepts, and number of domains, subdomains and indicators were visualized using a word cloud and heatmap, respectively. Other aspects of the frameworks were summarized narratively.

Search results

A total of 57 population health frameworks were included in this review ( Fig 1 ). The characteristics of the frameworks and their details are shown in Tables 1 and 2 , respectively. The full list of the domains, subdomains and indicators are provided in S3 File .

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

The PRISMA diagram shows the numbers of reports retrieved from various sources and flow through the stages of the scoping review. A total of 57 reports were included in this review. The diagram was generated using an open source R shiny app [ 10 ].

https://doi.org/10.1371/journal.pone.0278434.g001

https://doi.org/10.1371/journal.pone.0278434.t001

https://doi.org/10.1371/journal.pone.0278434.t002

Characteristics of population health frameworks

Majority of the frameworks originated from the US (45.6%), Europe (22.8%) and Canada (19.3%). None were from Asia. Most were published between 2001 and 2020 (64.9%). Governmental (including intergovernmental) and academic organizations accounted for majority of framework development (84.2%). Only three frameworks were developed for specific populations (2 for indigenous and 1 for rural), while the rest were for the general or urban population. Two-thirds of the frameworks mentioned some dimension, and these were slightly more frameworks using the lifespan approach compared to the equity approach (29.8% vs. 21.1%).

Domains and subdomains

Majority of the frameworks have between 1 to 5 domains (70.2%) but have more level 2 sub-domains (26.3% have 6–10, 29.8% have 11–20 and 19.3% have >20). The median number of domains and level 2 subdomains are 4 (range 2–16) and 10 (range 0–65), respectively ( S1 Fig ). Half of the frameworks do not have level 3 subdomains. Of those that do, most have >10 (72.4%). The median number of indicators is 18 (range 0–255). Twenty-six frameworks did not have indicators (45.6%). Of those that do, majority have >20 indicators (83.9%).

The most common concepts were health, (social) determinants of health, healthcare system and health behaviours ( S2 Fig ). The myriad of domains has gradually accumulated over the years. In frameworks published before 2000, health was the key domain, social determinants of health emerged in the next 2 decades (2001–2020) followed by healthcare system, health behaviours, functional limitations and activities of daily living in the recent frameworks ( S3 Fig ).

For health, most frameworks used summary indicators of health such as mortality and life-expectancy, and indicators of a few selected health conditions. However, four frameworks had longer lists of indicators for specific communicable and non-communicable diseases [ 12 , 26 , 27 , 43 , 50 ]. Of note, psychological or mental health risk factors and/or outcomes feature in 31 (54%) of the frameworks, highlighting its emerging importance [ 12 , 17 – 19 , 22 , 25 – 30 , 32 – 35 , 38 , 39 , 41 – 46 , 48 – 50 , 54 , 56 , 58 , 59 , 62 ].

Social determinants of health, which encompasses the full set of social conditions in which people live and work [ 66 ], were present under some label or other in all except 7 frameworks [ 16 , 34 , 42 , 50 , 52 , 54 , 60 ]. Some of the frameworks elaborate on these factors, with sub-domains and indicators on the physical environment, social environment, and even politics, national and global trends [ 12 , 21 – 23 , 26 , 29 , 35 , 43 , 53 , 55 – 59 , 63 , 64 , 67 ]. For example, the conceptual framework for urban health measures sub-domains such as immigration, globalization and the changing role of government [ 21 ]. The framework for community contextual characteristics, one of the two frameworks with the largest number of indicators, also measures the economic, employment, education, political, environmental, housing, governmental, transport aspects in the region where the population of interest is located [ 29 ]. Interestingly, crime and violence features in 16 frameworks, as this affects the physical safety of people in a community [ 12 , 15 , 26 , 29 , 30 , 33 , 41 , 43 – 45 , 53 , 56 , 59 , 62 , 63 , 67 ]. Many frameworks also measure lifestyle and health-related behaviours. Apart from the common ones like diet, physical activity, smoking and alcohol use, some frameworks include sexual behaviour, use of illicit drugs, seatbelt behaviour, immunization or health screening, breastfeeding and induced abortion [ 12 , 15 , 27 – 30 , 32 , 33 , 39 , 45 , 55 , 58 , 59 , 62 ]. One even included measures of parenting practices [ 43 ].

Almost a third (31.6%) of the frameworks have domains that pertain to the healthcare system or healthcare performance. One example is the OECD framework, which assesses health system performance within the context of other contextual determinants of health [ 46 ]. Within the construct of healthcare performance, common subdomains are accessibility, capacity, quality, patient-centeredness, cost and effectiveness [ 11 , 16 , 19 , 24 , 31 , 32 , 34 , 39 , 43 , 46 , 48 , 51 , 53 , 54 ].

A few of the frameworks had specific focuses and therefore unique domains and indicators that are relevant largely for their setting. For example, the reporting framework for indigenous adolescents in Australia contained domains that were largely relevant for that community, such as ‘family, kinship and community health’, which explored family roles and responsibilities, contact with extended family, removal from family, participation in community events and sense of belonging to the community [ 12 ]. Another example is the Ghana’s Holistic Assessment Tool, which contains indicators for health-related United Nations sustainable development goals (SDGs) such as proportion of deliveries attended by a trained health worker, proportion of children under 5 years sleeping under insecticide treated net, and tuberculosis treatment success rate, and certain endemic communicable diseases such as non-acute flaccid paralysis polio rate [ 42 ].

Approach to framework development

Evans and Stoddart developed a population health framework in 1990 [ 20 ] based on a much earlier 1974 Whitepaper titled “A new perspective on the health of Canadians”, which recognized the limitations of the healthcare system on improving health status and presented a preliminary framework of the ‘health field’ [ 68 ]. Subsequent frameworks were mostly developed from one or a combination of four approaches: 1) adaptation from an existing framework [ 11 , 12 , 33 , 45 , 46 , 48 – 51 , 56 , 58 – 60 , 63 , 65 ], 2) environmental scan of existing frameworks and literature review to summarize current knowledge of health determinants [ 7 , 14 , 16 – 20 , 24 , 25 , 29 , 32 , 36 , 37 , 44 , 48 , 52 , 57 , 61 , 63 ], 3) consulting and getting inputs from experts and stakeholders [ 12 , 17 , 19 , 24 , 26 – 29 , 35 , 39 , 41 , 48 , 52 – 55 , 62 , 63 ] and 4) basing on past work (e.g. primary data collection, drawing on secondary data, past population health efforts, etc), priorities and goals of the organization developing it [ 7 , 11 , 21 , 38 , 61 , 64 , 67 ].

Population health has been a popular concept in healthcare for the past 3 decades but interestingly does not have a unanimous definition [ 1 , 2 , 69 ]. The most commonly used definition, which originated from Kindig and Stoddart, defines population health as ‘the health outcomes of a group of individuals, including the distribution of such outcomes within the group” [ 1 ]. Nevertheless, people working on ‘population health’ would have different focuses, goals and populations of interest [ 69 ]. This may explain the large number of population health frameworks we found in this review.

Population health has its roots from recognition of health disparities by socioeconomic factors from as early as the 18 th century to early epidemiological studies that informed public health measures, particularly in Britain and France, and finally to a renewed interest in the last 2 decades due to a range of health problems facing the world [ 70 ]. Development of the population health approach in Canada, driven by the government and healthcare leaders, began in the 1970s [ 71 ]. Improving population health was motivated by the articulation of the Triple Aims as a goal for the US healthcare system in the late 2000s [ 72 ]. It is therefore unsurprising that most of the frameworks originate from US, Europe and Canada. Even with purposive searching of organizations in the Southern hemisphere such as Australia and New Zealand, the results were still dominated by the Northern hemisphere, reflecting the state of development of population health in the world. Similarly, the lack of frameworks from Asia might be because much of the work done in improving the health of populations is ‘public health’ rather than ‘population health’.

Health status and social determinants of health were the most common domains across the frameworks. As seen from the word cloud, there were also many other domains that were closely related to and/or could be considered subdomains of one of these domains. This is because different frameworks have different level of detail, and the hierarchy of domains and subdomains are different in level of detail across frameworks. In other words, a subdomain in one framework could be a domain in another, or an indicator in one framework could be a subdomain in another. It is therefore also difficult to summarize domains and subdomains in a simple way across the frameworks.

The domains and subdomains chosen in different frameworks largely reflects the purpose, information needs of varying stakeholders, and the focus of the organization(s) developing them. It is unsurprising to see that some key domains appear in many frameworks, and domains are branched out to varying degrees in different frameworks. For example, social determinants of health features in all frameworks except 7 frameworks [ 16 , 34 , 42 , 50 , 52 , 54 , 60 ]. Some frameworks have a heavy focus on health status, such as the Healthy Montogomery Core Measures Set, Triple Aim, Euro-REVES 2 and Ohio health priorities, with the Euro-REVES 2 framework even measuring activities of daily living and degree of functional limitations [ 26 , 27 , 50 , 60 ]. Other frameworks break down the social determinants into considerable detail, such as the framework for community contextual characteristics, life course health development framework, Healthy Cities Indicators, and others [ 12 , 22 , 23 , 26 , 29 , 38 , 49 , 53 , 55 , 56 , 59 , 63 , 64 , 67 ]. Several have a heavier focus on healthcare performance, such as the EU Joint Assessment Framework, European Community Health Indicators (ECHI), OECD, the Primary Healthcare Performance Initiative (PHCPI), National scorecard for the US health system and the Ireland HSPA framework [ 19 , 34 , 39 , 46 , 48 , 54 ]. Others are generally more balanced between the domains.

It is also noteworthy that almost half of the frameworks did not have any indicators and these tended to be older frameworks. About 61% of frameworks developed in 2010 and before did not have indicators while the converse is true for those developed after 2010. There was likely stronger focus on understanding the range of factors affecting population health and identifying priorities for improving population health in the earlier period. As organizations started to implement population health management strategies, measurement of population health started to feature more and more recent frameworks tended to include specific indicators. The inclusion of specific indicators also implies the ability to measure them, and therefore the availability of health information systems for data collection. These have generally become more well developed in the recent decade or so, also explaining why more recent frameworks have indicators. Nevertheless, frameworks without indicators can still offer a theoretical basis for selecting indicators that are relevant and feasible for a given setting.

The results of this scoping review can serve as an evidence base for governments and/or health systems developing their own population health frameworks and selecting indicators for their population health initiatives. They can select and adapt from the frameworks available, and assess the relevance of the range of domains, subdomains and indicators in their context. Populations are largely unique as they are shaped by their local and wider contextual factors. As such, no one framework used in one population or healthcare system is likely directly applicable to another population or healthcare system without adaptation. Population health practitioners can derive any level of detail that matches their interests and requirements from this review, from a broad sense of the literature down to specific indicators. The range of subdomains and indicators could also be sources of new hypotheses in a given region or jurisdiction for the purposes of population health research.

Settings which are further ahead in the population health journey with existing indicators can also use these results to assess what domains and subdomains have been covered, and where the gaps are. For example, population health is an increasingly important national priority in Singapore and the Ministry of Health is planning several major initiatives to improve the health of the general population [ 73 , 74 ]. To achieve this, the Ministry is working closely with the three major public healthcare clusters in Singapore to develop a set of population health indicators and the evidence base here can help inform the choices. With an initial set of indicators, practitioners can also interrogate their data systems and medical records to determine if they are available or if they need to build prospective data collection tools. This can also be an iterative process for selecting indicators using the results here as a resource. One constraint of the data in its current form though is the difficulty in navigating the long list of domains, subdomains and indicators. In future work, we aim to design a dashboard that allows for interactive exploration of the scoping review data.

There are limitations to this scoping review. Firstly, some frameworks might have been missed due to our language restriction, especially those in Asia. However, many official documents from this region are available in English, so this might not have impacted the search results significantly. Secondly, there are many terms and concepts in the literature that have overlaps with population health, such as public health, urban health, global health, population health management, health equity, health system performance and social determinants of health. Based on our inclusion criteria, concepts like urban health, rural health, community health and global health would be included as they pertain to general populations albeit in different types of settings. Related concepts such as health equity, social determinants of health and health system performance were not the focus of the search and could be part of the frameworks included. However, if a framework was focused on one of these concepts alone without the measurement of health status, then it would be excluded. Some frameworks also focused more on population health management and if it looked more like a logic model for specific interventions then these would also be excluded [ 75 , 76 ]. Overall, this review represents a useful collection of frameworks used for measuring the health of a population and its key antecedents [ 60 ].

We found 57 frameworks for the measurement of population health with variable numbers of domains, subdomains and indicators, and depth of detail. The key domains apart from health status were social determinants of health, health behaviours and healthcare system performance. These results serve as a useful resource for governments and healthcare organizations for informing their population health measurement efforts. Specifically, when developing their own population health framework and/or selection of population health indicators, they can identify common domains and subdomains that other organizations include, as well as consider others more systematically for relevance in their context.

Supporting information

S1 file. prisma-scr checklist..

https://doi.org/10.1371/journal.pone.0278434.s001

S2 File. Search strategy.

https://doi.org/10.1371/journal.pone.0278434.s002

S3 File. Domains, subdomains and indicators.

This file contains the full list of domains, subdomains and indicators from the 57 included population health frameworks.

https://doi.org/10.1371/journal.pone.0278434.s003

S1 Fig. Heatmap of number of domains, subdomains and indicators.

L2: level 2, L3: level 3, This is a visualization of the numbers of domains, subdomains and indicators in each framework in both figures and shading. Blank cells represent absence of the corresponding subdomain and/or indicators.

https://doi.org/10.1371/journal.pone.0278434.s004

S2 Fig. Wordcloud for framework domains.

Level 1 domains in all frameworks were clustered by concept using a combination of hierarchical clustering and manual edit. The sizes of the concepts are proportional to the number of domains in each concept.

https://doi.org/10.1371/journal.pone.0278434.s005

S3 Fig. Wordcloud for framework domains by year of publication.

Level 1 domains in all frameworks were clustered by concept using a combination of hierarchical clustering and manual edit. The sizes of the concepts are proportional to the number of domains in each concept. The concepts are presented by decade when the frameworks were published. A: Before 2000, B: 2001 to 2010, C: 2011 to 2020, D: After 2020.

https://doi.org/10.1371/journal.pone.0278434.s006

Acknowledgments

We would like to thank Ms Sabrina Liau for her assistance with article screening.

• View Article
• PubMed/NCBI
• Google Scholar
• 3. Valles SA. Philosophy of Population Health. Gronfeldt R, editor. New York: Routledge; 2018.
• 15. U.S. Centers for Disease Control and Prevention. Community Health Assessment for Population Health Improvement: Resource of Most Frequently Recommended Health Outcomes and Determinants. Atlanta, GA, USA; 2013.
• 16. Canterbury Health System. System level measures improvement plan 2021–2022 [Internet]. 2021. https://www.cdhb.health.nz/wp-content/uploads/ed30b6b8-canterbury-system-level-measures-improvement-plan-2021-2022.pdf
• 17. Canadian Institute for Health Information. A Performance Measurement Framework for the Canadian Health System [Internet]. 2013. https://secure.cihi.ca/free_products/HSP_Framework_Technical_Report_EN.pdf
• 19. European Commission. Towards a Joint Assessment Framework in the Area of Health. Work in progress: 2015 update [Internet]. 2015. https://ec.europa.eu/social/BlobServlet?docId=17033&langId=en
• 25. Canada H. Strategies for Population Health—Investing in the Health of Canadians [Internet]. Ottawa; 1994. https://publications.gc.ca/collections/collection_2016/sc-hc/H39-316-1994-eng.pdf
• 26. Health Policy Institute of Ohio. Improving Population Health Planning in Ohio [Internet]. 2016. https://regroup-production.s3.amazonaws.com/documents/ReviewReference/699737076/Improving population health planning in Ohio.pdf?response-content-type=application%2Fpdf&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYSFKCAWYQ4D5IUHG%2F20230703%2Fus
• 27. Healthy Montgomery. HEALTHY MONTGOMERY 2016 COMMUNITY HEALTH NEEDS ASSESSMENT. 2016; https://www.montgomerycountymd.gov/healthymontgomery/Resources/Files/Reports/2016_HM_CHNA_Final_June_2_2016.pdf
• 28. Government of Ireland. Healthy Ireland Outcomes Framework [Internet]. 2019. https://assets.gov.ie/7626/cb95e0dbb01e4a9fb7ce7affd609507e.pdf
• 31. Inf-Act. A Distributed Infrastructure on Population Health (DIPoH) [Internet]. 2020. https://www.inf-act.eu/sites/inf-act.eu/files/2020-01/Booklet.pdf
• 32. Institute of Medicine (IOM). State of the USA Health Indicators: Letter Report. State of the USA Health Indicators. 2009 Jan 1;
• 33. IOM (Institute of Medicine). Toward quality measures for population health and the leading health indicators. [Internet]. Washington, DC; 2012. https://www.ncbi.nlm.nih.gov/books/NBK202180/pdf/Bookshelf_NBK202180.pdf
• 34. Ireland Department of Health. Health System Performance (HSPA) Framework [Internet]. 2021 [cited 2023 Jul 3]. https://www.gov.ie/en/publication/6660a-health-system-performance-assessment-hspa-framework/#
• 43. Los Angeles County Department of Public Health. Key Indicators of Health by Service Planning Area [Internet]. 2017. http://publichealth.lacounty.gov/ha/docs/2015LACHS/KeyIndicator/PH-KIH_2017-sec UPDATED.pdf
• 44. Levene LS, Bankart J, Walker N, Wilson A, Baker R. How health care may modify the effects of illness determinants on population outcomes: the Leicester SEARCH conceptual framework for primary care. 2018;
• 45. National Quality Forum. Finding Common Ground for Healthcare Priorities: Families of Measures for Assessing Affordability, Population Health, and Person- and Family-Centered Care [Internet]. 2014. https://www.qualityforum.org/WorkArea/linkit.aspx?LinkIdentifier=id&ItemID=77001
• 46. OECD. Health at a Glance 2021: OECD Indicators. Paris: OECD Publishing; 2021.
• 47. Oleske DM. An Epidemiologic Framework for the Delivery of Health Care Services. Epidemiology and the Delivery of Health Care Services. 2009;3–30.
• 48. Primary Health Care Performance Initiative. The PHCPI Conceptual Framework [Internet]. 2018 [cited 2022 Jul 15]. https://improvingphc.org/phcpi-conceptual-framework
• 49. Health England P. Addressing health inequalities through collaborative action: briefing note.
• 50. Robine J, Jagger C, Romieu I. Selection of a Coherent Set of Health Indicators for the European Union. Phase II: Final report [Internet]. 2002. https://reves.site.ined.fr/fichier/s_rubrique/20027/userguide2.en.pdf
• 52. Sadana R, Tandon A, Murray CJ, Serdobova I, Cao Y, Xie WJ, et al. Describing population health in six domains: comparable results from 66 household surveys. 2002.
• 57. County of San Diego Health and Human Services Agency. Equity Framework [Internet]. 2020. https://regroup-production.s3.amazonaws.com/documents/ReviewReference/699737125/Equity Framework Overview_FINAL.pdf?response-content-type=application%252Fpdf&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYSFKCAWYQ4D5IUHG%252F20230703%252Fus-east-1%252Fs3%25
• 58. SfHIP. San Francisco Framework for Assessing Population Health and Equity [Internet]. [cited 2022 May 27]. http://www.sfhip.org/chna/community-health-data/san-francisco-framework-for-assessing-population-health-and-equity/
• 60. Stiefel M, Nolan K. A Guide to Measuring the Triple Aim: Population Health, Experience of Care, and Per Capita Cost. IHI Innovation Series white paper [Internet]. 2012. https://www.ihi.org/resources/Pages/IHIWhitePapers/AGuidetoMeasuringTripleAim.aspx
• 62. UK Department of Health. Public Health Outcomes Framework [Internet]. [cited 2022 Jul 12]. https://fingertips.phe.org.uk/profile/public-health-outcomes-framework
• 63. Vila PM, Angela Kempf BM, Bridget Booske MC, Athens JK, Patrick Remington ML. 2006 Wisconsin County Health Rankings Full Report. 2006; https://uwphi.pophealth.wisc.edu/wp-content/uploads/sites/316/2017/11/WCHR_2006_FullReport.pdf
• 66. Solar O, Irwin A. A conceptual framework for action on the social determinants of health. Social Determinants of Health Discussion Paper 2 (Policy and Practice). 2010.
• 67. Institute for People, Place, & Possibility. How the Vital Conditions Framework and IP3 ASSESS Support West Virginia Communities [Internet]. 2017 [cited 2023 Jul 3]. https://www.i-p3.org/post/how-the-vital-conditions-framework-and-ip3-assess-support-west-virginia-communities
• 68. Lalonde M. A new perspective on the health of Canadians [Internet]. 1974. http://www.phac-aspc.gc.ca/ph-sp/pdf/perspect-eng.pdf
• 73. Ministry of Health Singpaore. Promoting overall healthier living while targeting specific sub-populations [Internet]. 2022 [cited 2022 Aug 29]. https://www.moh.gov.sg/news-highlights/details/promoting-overall-healthier-living-while-targeting-specific-sub-populations
• 74. Ministry of Health Singpaore. Healthier you, with Healthier SG [Internet]. 2022 [cited 2022 Aug 29]. https://www.healthiersg.gov.sg/