effective literature studies analysis in research methodology

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

Prevent plagiarism. Run a free check.

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved August 26, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

• Open access
• Published: 07 September 2020

A tutorial on methodological studies: the what, when, how and why

• Lawrence Mbuagbaw   ORCID: orcid.org/0000-0001-5855-5461 1 , 2 , 3 ,
• Daeria O. Lawson 1 ,
• Livia Puljak 4 ,
• David B. Allison 5 &
• Lehana Thabane 1 , 2 , 6 , 7 , 8  

BMC Medical Research Methodology volume  20 , Article number:  226 ( 2020 ) Cite this article

42k Accesses

57 Citations

61 Altmetric

Metrics details

Methodological studies – studies that evaluate the design, analysis or reporting of other research-related reports – play an important role in health research. They help to highlight issues in the conduct of research with the aim of improving health research methodology, and ultimately reducing research waste.

We provide an overview of some of the key aspects of methodological studies such as what they are, and when, how and why they are done. We adopt a “frequently asked questions” format to facilitate reading this paper and provide multiple examples to help guide researchers interested in conducting methodological studies. Some of the topics addressed include: is it necessary to publish a study protocol? How to select relevant research reports and databases for a methodological study? What approaches to data extraction and statistical analysis should be considered when conducting a methodological study? What are potential threats to validity and is there a way to appraise the quality of methodological studies?

Appropriate reflection and application of basic principles of epidemiology and biostatistics are required in the design and analysis of methodological studies. This paper provides an introduction for further discussion about the conduct of methodological studies.

Peer Review reports

The field of meta-research (or research-on-research) has proliferated in recent years in response to issues with research quality and conduct [ 1 , 2 , 3 ]. As the name suggests, this field targets issues with research design, conduct, analysis and reporting. Various types of research reports are often examined as the unit of analysis in these studies (e.g. abstracts, full manuscripts, trial registry entries). Like many other novel fields of research, meta-research has seen a proliferation of use before the development of reporting guidance. For example, this was the case with randomized trials for which risk of bias tools and reporting guidelines were only developed much later – after many trials had been published and noted to have limitations [ 4 , 5 ]; and for systematic reviews as well [ 6 , 7 , 8 ]. However, in the absence of formal guidance, studies that report on research differ substantially in how they are named, conducted and reported [ 9 , 10 ]. This creates challenges in identifying, summarizing and comparing them. In this tutorial paper, we will use the term methodological study to refer to any study that reports on the design, conduct, analysis or reporting of primary or secondary research-related reports (such as trial registry entries and conference abstracts).

In the past 10 years, there has been an increase in the use of terms related to methodological studies (based on records retrieved with a keyword search [in the title and abstract] for “methodological review” and “meta-epidemiological study” in PubMed up to December 2019), suggesting that these studies may be appearing more frequently in the literature. See Fig.  1 .

Trends in the number studies that mention “methodological review” or “meta-

epidemiological study” in PubMed.

The methods used in many methodological studies have been borrowed from systematic and scoping reviews. This practice has influenced the direction of the field, with many methodological studies including searches of electronic databases, screening of records, duplicate data extraction and assessments of risk of bias in the included studies. However, the research questions posed in methodological studies do not always require the approaches listed above, and guidance is needed on when and how to apply these methods to a methodological study. Even though methodological studies can be conducted on qualitative or mixed methods research, this paper focuses on and draws examples exclusively from quantitative research.

The objectives of this paper are to provide some insights on how to conduct methodological studies so that there is greater consistency between the research questions posed, and the design, analysis and reporting of findings. We provide multiple examples to illustrate concepts and a proposed framework for categorizing methodological studies in quantitative research.

What is a methodological study?

Any study that describes or analyzes methods (design, conduct, analysis or reporting) in published (or unpublished) literature is a methodological study. Consequently, the scope of methodological studies is quite extensive and includes, but is not limited to, topics as diverse as: research question formulation [ 11 ]; adherence to reporting guidelines [ 12 , 13 , 14 ] and consistency in reporting [ 15 ]; approaches to study analysis [ 16 ]; investigating the credibility of analyses [ 17 ]; and studies that synthesize these methodological studies [ 18 ]. While the nomenclature of methodological studies is not uniform, the intents and purposes of these studies remain fairly consistent – to describe or analyze methods in primary or secondary studies. As such, methodological studies may also be classified as a subtype of observational studies.

Parallel to this are experimental studies that compare different methods. Even though they play an important role in informing optimal research methods, experimental methodological studies are beyond the scope of this paper. Examples of such studies include the randomized trials by Buscemi et al., comparing single data extraction to double data extraction [ 19 ], and Carrasco-Labra et al., comparing approaches to presenting findings in Grading of Recommendations, Assessment, Development and Evaluations (GRADE) summary of findings tables [ 20 ]. In these studies, the unit of analysis is the person or groups of individuals applying the methods. We also direct readers to the Studies Within a Trial (SWAT) and Studies Within a Review (SWAR) programme operated through the Hub for Trials Methodology Research, for further reading as a potential useful resource for these types of experimental studies [ 21 ]. Lastly, this paper is not meant to inform the conduct of research using computational simulation and mathematical modeling for which some guidance already exists [ 22 ], or studies on the development of methods using consensus-based approaches.

When should we conduct a methodological study?

Methodological studies occupy a unique niche in health research that allows them to inform methodological advances. Methodological studies should also be conducted as pre-cursors to reporting guideline development, as they provide an opportunity to understand current practices, and help to identify the need for guidance and gaps in methodological or reporting quality. For example, the development of the popular Preferred Reporting Items of Systematic reviews and Meta-Analyses (PRISMA) guidelines were preceded by methodological studies identifying poor reporting practices [ 23 , 24 ]. In these instances, after the reporting guidelines are published, methodological studies can also be used to monitor uptake of the guidelines.

These studies can also be conducted to inform the state of the art for design, analysis and reporting practices across different types of health research fields, with the aim of improving research practices, and preventing or reducing research waste. For example, Samaan et al. conducted a scoping review of adherence to different reporting guidelines in health care literature [ 18 ]. Methodological studies can also be used to determine the factors associated with reporting practices. For example, Abbade et al. investigated journal characteristics associated with the use of the Participants, Intervention, Comparison, Outcome, Timeframe (PICOT) format in framing research questions in trials of venous ulcer disease [ 11 ].

How often are methodological studies conducted?

There is no clear answer to this question. Based on a search of PubMed, the use of related terms (“methodological review” and “meta-epidemiological study”) – and therefore, the number of methodological studies – is on the rise. However, many other terms are used to describe methodological studies. There are also many studies that explore design, conduct, analysis or reporting of research reports, but that do not use any specific terms to describe or label their study design in terms of “methodology”. This diversity in nomenclature makes a census of methodological studies elusive. Appropriate terminology and key words for methodological studies are needed to facilitate improved accessibility for end-users.

Why do we conduct methodological studies?

Methodological studies provide information on the design, conduct, analysis or reporting of primary and secondary research and can be used to appraise quality, quantity, completeness, accuracy and consistency of health research. These issues can be explored in specific fields, journals, databases, geographical regions and time periods. For example, Areia et al. explored the quality of reporting of endoscopic diagnostic studies in gastroenterology [ 25 ]; Knol et al. investigated the reporting of p -values in baseline tables in randomized trial published in high impact journals [ 26 ]; Chen et al. describe adherence to the Consolidated Standards of Reporting Trials (CONSORT) statement in Chinese Journals [ 27 ]; and Hopewell et al. describe the effect of editors’ implementation of CONSORT guidelines on reporting of abstracts over time [ 28 ]. Methodological studies provide useful information to researchers, clinicians, editors, publishers and users of health literature. As a result, these studies have been at the cornerstone of important methodological developments in the past two decades and have informed the development of many health research guidelines including the highly cited CONSORT statement [ 5 ].

Where can we find methodological studies?

Methodological studies can be found in most common biomedical bibliographic databases (e.g. Embase, MEDLINE, PubMed, Web of Science). However, the biggest caveat is that methodological studies are hard to identify in the literature due to the wide variety of names used and the lack of comprehensive databases dedicated to them. A handful can be found in the Cochrane Library as “Cochrane Methodology Reviews”, but these studies only cover methodological issues related to systematic reviews. Previous attempts to catalogue all empirical studies of methods used in reviews were abandoned 10 years ago [ 29 ]. In other databases, a variety of search terms may be applied with different levels of sensitivity and specificity.

Some frequently asked questions about methodological studies

In this section, we have outlined responses to questions that might help inform the conduct of methodological studies.

Q: How should I select research reports for my methodological study?

A: Selection of research reports for a methodological study depends on the research question and eligibility criteria. Once a clear research question is set and the nature of literature one desires to review is known, one can then begin the selection process. Selection may begin with a broad search, especially if the eligibility criteria are not apparent. For example, a methodological study of Cochrane Reviews of HIV would not require a complex search as all eligible studies can easily be retrieved from the Cochrane Library after checking a few boxes [ 30 ]. On the other hand, a methodological study of subgroup analyses in trials of gastrointestinal oncology would require a search to find such trials, and further screening to identify trials that conducted a subgroup analysis [ 31 ].

The strategies used for identifying participants in observational studies can apply here. One may use a systematic search to identify all eligible studies. If the number of eligible studies is unmanageable, a random sample of articles can be expected to provide comparable results if it is sufficiently large [ 32 ]. For example, Wilson et al. used a random sample of trials from the Cochrane Stroke Group’s Trial Register to investigate completeness of reporting [ 33 ]. It is possible that a simple random sample would lead to underrepresentation of units (i.e. research reports) that are smaller in number. This is relevant if the investigators wish to compare multiple groups but have too few units in one group. In this case a stratified sample would help to create equal groups. For example, in a methodological study comparing Cochrane and non-Cochrane reviews, Kahale et al. drew random samples from both groups [ 34 ]. Alternatively, systematic or purposeful sampling strategies can be used and we encourage researchers to justify their selected approaches based on the study objective.

Q: How many databases should I search?

A: The number of databases one should search would depend on the approach to sampling, which can include targeting the entire “population” of interest or a sample of that population. If you are interested in including the entire target population for your research question, or drawing a random or systematic sample from it, then a comprehensive and exhaustive search for relevant articles is required. In this case, we recommend using systematic approaches for searching electronic databases (i.e. at least 2 databases with a replicable and time stamped search strategy). The results of your search will constitute a sampling frame from which eligible studies can be drawn.

Alternatively, if your approach to sampling is purposeful, then we recommend targeting the database(s) or data sources (e.g. journals, registries) that include the information you need. For example, if you are conducting a methodological study of high impact journals in plastic surgery and they are all indexed in PubMed, you likely do not need to search any other databases. You may also have a comprehensive list of all journals of interest and can approach your search using the journal names in your database search (or by accessing the journal archives directly from the journal’s website). Even though one could also search journals’ web pages directly, using a database such as PubMed has multiple advantages, such as the use of filters, so the search can be narrowed down to a certain period, or study types of interest. Furthermore, individual journals’ web sites may have different search functionalities, which do not necessarily yield a consistent output.

Q: Should I publish a protocol for my methodological study?

A: A protocol is a description of intended research methods. Currently, only protocols for clinical trials require registration [ 35 ]. Protocols for systematic reviews are encouraged but no formal recommendation exists. The scientific community welcomes the publication of protocols because they help protect against selective outcome reporting, the use of post hoc methodologies to embellish results, and to help avoid duplication of efforts [ 36 ]. While the latter two risks exist in methodological research, the negative consequences may be substantially less than for clinical outcomes. In a sample of 31 methodological studies, 7 (22.6%) referenced a published protocol [ 9 ]. In the Cochrane Library, there are 15 protocols for methodological reviews (21 July 2020). This suggests that publishing protocols for methodological studies is not uncommon.

Authors can consider publishing their study protocol in a scholarly journal as a manuscript. Advantages of such publication include obtaining peer-review feedback about the planned study, and easy retrieval by searching databases such as PubMed. The disadvantages in trying to publish protocols includes delays associated with manuscript handling and peer review, as well as costs, as few journals publish study protocols, and those journals mostly charge article-processing fees [ 37 ]. Authors who would like to make their protocol publicly available without publishing it in scholarly journals, could deposit their study protocols in publicly available repositories, such as the Open Science Framework ( https://osf.io/ ).

Q: How to appraise the quality of a methodological study?

A: To date, there is no published tool for appraising the risk of bias in a methodological study, but in principle, a methodological study could be considered as a type of observational study. Therefore, during conduct or appraisal, care should be taken to avoid the biases common in observational studies [ 38 ]. These biases include selection bias, comparability of groups, and ascertainment of exposure or outcome. In other words, to generate a representative sample, a comprehensive reproducible search may be necessary to build a sampling frame. Additionally, random sampling may be necessary to ensure that all the included research reports have the same probability of being selected, and the screening and selection processes should be transparent and reproducible. To ensure that the groups compared are similar in all characteristics, matching, random sampling or stratified sampling can be used. Statistical adjustments for between-group differences can also be applied at the analysis stage. Finally, duplicate data extraction can reduce errors in assessment of exposures or outcomes.

Q: Should I justify a sample size?

A: In all instances where one is not using the target population (i.e. the group to which inferences from the research report are directed) [ 39 ], a sample size justification is good practice. The sample size justification may take the form of a description of what is expected to be achieved with the number of articles selected, or a formal sample size estimation that outlines the number of articles required to answer the research question with a certain precision and power. Sample size justifications in methodological studies are reasonable in the following instances:

Comparing two groups

Determining a proportion, mean or another quantifier

Determining factors associated with an outcome using regression-based analyses

For example, El Dib et al. computed a sample size requirement for a methodological study of diagnostic strategies in randomized trials, based on a confidence interval approach [ 40 ].

Q: What should I call my study?

A: Other terms which have been used to describe/label methodological studies include “ methodological review ”, “methodological survey” , “meta-epidemiological study” , “systematic review” , “systematic survey”, “meta-research”, “research-on-research” and many others. We recommend that the study nomenclature be clear, unambiguous, informative and allow for appropriate indexing. Methodological study nomenclature that should be avoided includes “ systematic review” – as this will likely be confused with a systematic review of a clinical question. “ Systematic survey” may also lead to confusion about whether the survey was systematic (i.e. using a preplanned methodology) or a survey using “ systematic” sampling (i.e. a sampling approach using specific intervals to determine who is selected) [ 32 ]. Any of the above meanings of the words “ systematic” may be true for methodological studies and could be potentially misleading. “ Meta-epidemiological study” is ideal for indexing, but not very informative as it describes an entire field. The term “ review ” may point towards an appraisal or “review” of the design, conduct, analysis or reporting (or methodological components) of the targeted research reports, yet it has also been used to describe narrative reviews [ 41 , 42 ]. The term “ survey ” is also in line with the approaches used in many methodological studies [ 9 ], and would be indicative of the sampling procedures of this study design. However, in the absence of guidelines on nomenclature, the term “ methodological study ” is broad enough to capture most of the scenarios of such studies.

Q: Should I account for clustering in my methodological study?

A: Data from methodological studies are often clustered. For example, articles coming from a specific source may have different reporting standards (e.g. the Cochrane Library). Articles within the same journal may be similar due to editorial practices and policies, reporting requirements and endorsement of guidelines. There is emerging evidence that these are real concerns that should be accounted for in analyses [ 43 ]. Some cluster variables are described in the section: “ What variables are relevant to methodological studies?”

A variety of modelling approaches can be used to account for correlated data, including the use of marginal, fixed or mixed effects regression models with appropriate computation of standard errors [ 44 ]. For example, Kosa et al. used generalized estimation equations to account for correlation of articles within journals [ 15 ]. Not accounting for clustering could lead to incorrect p -values, unduly narrow confidence intervals, and biased estimates [ 45 ].

Q: Should I extract data in duplicate?

A: Yes. Duplicate data extraction takes more time but results in less errors [ 19 ]. Data extraction errors in turn affect the effect estimate [ 46 ], and therefore should be mitigated. Duplicate data extraction should be considered in the absence of other approaches to minimize extraction errors. However, much like systematic reviews, this area will likely see rapid new advances with machine learning and natural language processing technologies to support researchers with screening and data extraction [ 47 , 48 ]. However, experience plays an important role in the quality of extracted data and inexperienced extractors should be paired with experienced extractors [ 46 , 49 ].

Q: Should I assess the risk of bias of research reports included in my methodological study?

A : Risk of bias is most useful in determining the certainty that can be placed in the effect measure from a study. In methodological studies, risk of bias may not serve the purpose of determining the trustworthiness of results, as effect measures are often not the primary goal of methodological studies. Determining risk of bias in methodological studies is likely a practice borrowed from systematic review methodology, but whose intrinsic value is not obvious in methodological studies. When it is part of the research question, investigators often focus on one aspect of risk of bias. For example, Speich investigated how blinding was reported in surgical trials [ 50 ], and Abraha et al., investigated the application of intention-to-treat analyses in systematic reviews and trials [ 51 ].

Q: What variables are relevant to methodological studies?

A: There is empirical evidence that certain variables may inform the findings in a methodological study. We outline some of these and provide a brief overview below:

Country: Countries and regions differ in their research cultures, and the resources available to conduct research. Therefore, it is reasonable to believe that there may be differences in methodological features across countries. Methodological studies have reported loco-regional differences in reporting quality [ 52 , 53 ]. This may also be related to challenges non-English speakers face in publishing papers in English.

Authors’ expertise: The inclusion of authors with expertise in research methodology, biostatistics, and scientific writing is likely to influence the end-product. Oltean et al. found that among randomized trials in orthopaedic surgery, the use of analyses that accounted for clustering was more likely when specialists (e.g. statistician, epidemiologist or clinical trials methodologist) were included on the study team [ 54 ]. Fleming et al. found that including methodologists in the review team was associated with appropriate use of reporting guidelines [ 55 ].

Source of funding and conflicts of interest: Some studies have found that funded studies report better [ 56 , 57 ], while others do not [ 53 , 58 ]. The presence of funding would indicate the availability of resources deployed to ensure optimal design, conduct, analysis and reporting. However, the source of funding may introduce conflicts of interest and warrant assessment. For example, Kaiser et al. investigated the effect of industry funding on obesity or nutrition randomized trials and found that reporting quality was similar [ 59 ]. Thomas et al. looked at reporting quality of long-term weight loss trials and found that industry funded studies were better [ 60 ]. Kan et al. examined the association between industry funding and “positive trials” (trials reporting a significant intervention effect) and found that industry funding was highly predictive of a positive trial [ 61 ]. This finding is similar to that of a recent Cochrane Methodology Review by Hansen et al. [ 62 ]

Journal characteristics: Certain journals’ characteristics may influence the study design, analysis or reporting. Characteristics such as journal endorsement of guidelines [ 63 , 64 ], and Journal Impact Factor (JIF) have been shown to be associated with reporting [ 63 , 65 , 66 , 67 ].

Study size (sample size/number of sites): Some studies have shown that reporting is better in larger studies [ 53 , 56 , 58 ].

Year of publication: It is reasonable to assume that design, conduct, analysis and reporting of research will change over time. Many studies have demonstrated improvements in reporting over time or after the publication of reporting guidelines [ 68 , 69 ].

Type of intervention: In a methodological study of reporting quality of weight loss intervention studies, Thabane et al. found that trials of pharmacologic interventions were reported better than trials of non-pharmacologic interventions [ 70 ].

Interactions between variables: Complex interactions between the previously listed variables are possible. High income countries with more resources may be more likely to conduct larger studies and incorporate a variety of experts. Authors in certain countries may prefer certain journals, and journal endorsement of guidelines and editorial policies may change over time.

Q: Should I focus only on high impact journals?

A: Investigators may choose to investigate only high impact journals because they are more likely to influence practice and policy, or because they assume that methodological standards would be higher. However, the JIF may severely limit the scope of articles included and may skew the sample towards articles with positive findings. The generalizability and applicability of findings from a handful of journals must be examined carefully, especially since the JIF varies over time. Even among journals that are all “high impact”, variations exist in methodological standards.

Q: Can I conduct a methodological study of qualitative research?

A: Yes. Even though a lot of methodological research has been conducted in the quantitative research field, methodological studies of qualitative studies are feasible. Certain databases that catalogue qualitative research including the Cumulative Index to Nursing & Allied Health Literature (CINAHL) have defined subject headings that are specific to methodological research (e.g. “research methodology”). Alternatively, one could also conduct a qualitative methodological review; that is, use qualitative approaches to synthesize methodological issues in qualitative studies.

Q: What reporting guidelines should I use for my methodological study?

A: There is no guideline that covers the entire scope of methodological studies. One adaptation of the PRISMA guidelines has been published, which works well for studies that aim to use the entire target population of research reports [ 71 ]. However, it is not widely used (40 citations in 2 years as of 09 December 2019), and methodological studies that are designed as cross-sectional or before-after studies require a more fit-for purpose guideline. A more encompassing reporting guideline for a broad range of methodological studies is currently under development [ 72 ]. However, in the absence of formal guidance, the requirements for scientific reporting should be respected, and authors of methodological studies should focus on transparency and reproducibility.

Q: What are the potential threats to validity and how can I avoid them?

A: Methodological studies may be compromised by a lack of internal or external validity. The main threats to internal validity in methodological studies are selection and confounding bias. Investigators must ensure that the methods used to select articles does not make them differ systematically from the set of articles to which they would like to make inferences. For example, attempting to make extrapolations to all journals after analyzing high-impact journals would be misleading.

Many factors (confounders) may distort the association between the exposure and outcome if the included research reports differ with respect to these factors [ 73 ]. For example, when examining the association between source of funding and completeness of reporting, it may be necessary to account for journals that endorse the guidelines. Confounding bias can be addressed by restriction, matching and statistical adjustment [ 73 ]. Restriction appears to be the method of choice for many investigators who choose to include only high impact journals or articles in a specific field. For example, Knol et al. examined the reporting of p -values in baseline tables of high impact journals [ 26 ]. Matching is also sometimes used. In the methodological study of non-randomized interventional studies of elective ventral hernia repair, Parker et al. matched prospective studies with retrospective studies and compared reporting standards [ 74 ]. Some other methodological studies use statistical adjustments. For example, Zhang et al. used regression techniques to determine the factors associated with missing participant data in trials [ 16 ].

With regard to external validity, researchers interested in conducting methodological studies must consider how generalizable or applicable their findings are. This should tie in closely with the research question and should be explicit. For example. Findings from methodological studies on trials published in high impact cardiology journals cannot be assumed to be applicable to trials in other fields. However, investigators must ensure that their sample truly represents the target sample either by a) conducting a comprehensive and exhaustive search, or b) using an appropriate and justified, randomly selected sample of research reports.

Even applicability to high impact journals may vary based on the investigators’ definition, and over time. For example, for high impact journals in the field of general medicine, Bouwmeester et al. included the Annals of Internal Medicine (AIM), BMJ, the Journal of the American Medical Association (JAMA), Lancet, the New England Journal of Medicine (NEJM), and PLoS Medicine ( n  = 6) [ 75 ]. In contrast, the high impact journals selected in the methodological study by Schiller et al. were BMJ, JAMA, Lancet, and NEJM ( n  = 4) [ 76 ]. Another methodological study by Kosa et al. included AIM, BMJ, JAMA, Lancet and NEJM ( n  = 5). In the methodological study by Thabut et al., journals with a JIF greater than 5 were considered to be high impact. Riado Minguez et al. used first quartile journals in the Journal Citation Reports (JCR) for a specific year to determine “high impact” [ 77 ]. Ultimately, the definition of high impact will be based on the number of journals the investigators are willing to include, the year of impact and the JIF cut-off [ 78 ]. We acknowledge that the term “generalizability” may apply differently for methodological studies, especially when in many instances it is possible to include the entire target population in the sample studied.

Finally, methodological studies are not exempt from information bias which may stem from discrepancies in the included research reports [ 79 ], errors in data extraction, or inappropriate interpretation of the information extracted. Likewise, publication bias may also be a concern in methodological studies, but such concepts have not yet been explored.

A proposed framework

In order to inform discussions about methodological studies, the development of guidance for what should be reported, we have outlined some key features of methodological studies that can be used to classify them. For each of the categories outlined below, we provide an example. In our experience, the choice of approach to completing a methodological study can be informed by asking the following four questions:

What is the aim?

Methodological studies that investigate bias

A methodological study may be focused on exploring sources of bias in primary or secondary studies (meta-bias), or how bias is analyzed. We have taken care to distinguish bias (i.e. systematic deviations from the truth irrespective of the source) from reporting quality or completeness (i.e. not adhering to a specific reporting guideline or norm). An example of where this distinction would be important is in the case of a randomized trial with no blinding. This study (depending on the nature of the intervention) would be at risk of performance bias. However, if the authors report that their study was not blinded, they would have reported adequately. In fact, some methodological studies attempt to capture both “quality of conduct” and “quality of reporting”, such as Richie et al., who reported on the risk of bias in randomized trials of pharmacy practice interventions [ 80 ]. Babic et al. investigated how risk of bias was used to inform sensitivity analyses in Cochrane reviews [ 81 ]. Further, biases related to choice of outcomes can also be explored. For example, Tan et al investigated differences in treatment effect size based on the outcome reported [ 82 ].

Methodological studies that investigate quality (or completeness) of reporting

Methodological studies may report quality of reporting against a reporting checklist (i.e. adherence to guidelines) or against expected norms. For example, Croituro et al. report on the quality of reporting in systematic reviews published in dermatology journals based on their adherence to the PRISMA statement [ 83 ], and Khan et al. described the quality of reporting of harms in randomized controlled trials published in high impact cardiovascular journals based on the CONSORT extension for harms [ 84 ]. Other methodological studies investigate reporting of certain features of interest that may not be part of formally published checklists or guidelines. For example, Mbuagbaw et al. described how often the implications for research are elaborated using the Evidence, Participants, Intervention, Comparison, Outcome, Timeframe (EPICOT) format [ 30 ].

Methodological studies that investigate the consistency of reporting

Sometimes investigators may be interested in how consistent reports of the same research are, as it is expected that there should be consistency between: conference abstracts and published manuscripts; manuscript abstracts and manuscript main text; and trial registration and published manuscript. For example, Rosmarakis et al. investigated consistency between conference abstracts and full text manuscripts [ 85 ].

Methodological studies that investigate factors associated with reporting

In addition to identifying issues with reporting in primary and secondary studies, authors of methodological studies may be interested in determining the factors that are associated with certain reporting practices. Many methodological studies incorporate this, albeit as a secondary outcome. For example, Farrokhyar et al. investigated the factors associated with reporting quality in randomized trials of coronary artery bypass grafting surgery [ 53 ].

Methodological studies that investigate methods

Methodological studies may also be used to describe methods or compare methods, and the factors associated with methods. Muller et al. described the methods used for systematic reviews and meta-analyses of observational studies [ 86 ].

Methodological studies that summarize other methodological studies

Some methodological studies synthesize results from other methodological studies. For example, Li et al. conducted a scoping review of methodological reviews that investigated consistency between full text and abstracts in primary biomedical research [ 87 ].

Methodological studies that investigate nomenclature and terminology

Some methodological studies may investigate the use of names and terms in health research. For example, Martinic et al. investigated the definitions of systematic reviews used in overviews of systematic reviews (OSRs), meta-epidemiological studies and epidemiology textbooks [ 88 ].

Other types of methodological studies

In addition to the previously mentioned experimental methodological studies, there may exist other types of methodological studies not captured here.

What is the design?

Methodological studies that are descriptive

Most methodological studies are purely descriptive and report their findings as counts (percent) and means (standard deviation) or medians (interquartile range). For example, Mbuagbaw et al. described the reporting of research recommendations in Cochrane HIV systematic reviews [ 30 ]. Gohari et al. described the quality of reporting of randomized trials in diabetes in Iran [ 12 ].

Methodological studies that are analytical

Some methodological studies are analytical wherein “analytical studies identify and quantify associations, test hypotheses, identify causes and determine whether an association exists between variables, such as between an exposure and a disease.” [ 89 ] In the case of methodological studies all these investigations are possible. For example, Kosa et al. investigated the association between agreement in primary outcome from trial registry to published manuscript and study covariates. They found that larger and more recent studies were more likely to have agreement [ 15 ]. Tricco et al. compared the conclusion statements from Cochrane and non-Cochrane systematic reviews with a meta-analysis of the primary outcome and found that non-Cochrane reviews were more likely to report positive findings. These results are a test of the null hypothesis that the proportions of Cochrane and non-Cochrane reviews that report positive results are equal [ 90 ].

What is the sampling strategy?

Methodological studies that include the target population

Methodological reviews with narrow research questions may be able to include the entire target population. For example, in the methodological study of Cochrane HIV systematic reviews, Mbuagbaw et al. included all of the available studies ( n  = 103) [ 30 ].

Methodological studies that include a sample of the target population

Many methodological studies use random samples of the target population [ 33 , 91 , 92 ]. Alternatively, purposeful sampling may be used, limiting the sample to a subset of research-related reports published within a certain time period, or in journals with a certain ranking or on a topic. Systematic sampling can also be used when random sampling may be challenging to implement.

What is the unit of analysis?

Methodological studies with a research report as the unit of analysis

Many methodological studies use a research report (e.g. full manuscript of study, abstract portion of the study) as the unit of analysis, and inferences can be made at the study-level. However, both published and unpublished research-related reports can be studied. These may include articles, conference abstracts, registry entries etc.

Methodological studies with a design, analysis or reporting item as the unit of analysis

Some methodological studies report on items which may occur more than once per article. For example, Paquette et al. report on subgroup analyses in Cochrane reviews of atrial fibrillation in which 17 systematic reviews planned 56 subgroup analyses [ 93 ].

This framework is outlined in Fig.  2 .

A proposed framework for methodological studies

Conclusions

Methodological studies have examined different aspects of reporting such as quality, completeness, consistency and adherence to reporting guidelines. As such, many of the methodological study examples cited in this tutorial are related to reporting. However, as an evolving field, the scope of research questions that can be addressed by methodological studies is expected to increase.

In this paper we have outlined the scope and purpose of methodological studies, along with examples of instances in which various approaches have been used. In the absence of formal guidance on the design, conduct, analysis and reporting of methodological studies, we have provided some advice to help make methodological studies consistent. This advice is grounded in good contemporary scientific practice. Generally, the research question should tie in with the sampling approach and planned analysis. We have also highlighted the variables that may inform findings from methodological studies. Lastly, we have provided suggestions for ways in which authors can categorize their methodological studies to inform their design and analysis.

Availability of data and materials

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Abbreviations

Consolidated Standards of Reporting Trials

Evidence, Participants, Intervention, Comparison, Outcome, Timeframe

Grading of Recommendations, Assessment, Development and Evaluations

Participants, Intervention, Comparison, Outcome, Timeframe

Preferred Reporting Items of Systematic reviews and Meta-Analyses

Studies Within a Review

Studies Within a Trial

Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–9.

PubMed   Google Scholar  

Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gotzsche PC, Krumholz HM, Ghersi D, van der Worp HB. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383(9913):257–66.

PubMed   PubMed Central   Google Scholar  

Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, Schulz KF, Tibshirani R. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383(9912):166–75.

Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JA. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.

Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet. 2001;357.

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6(7):e1000100.

Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, Henry DA, Boers M. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62(10):1013–20.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. Bmj. 2017;358:j4008.

Lawson DO, Leenus A, Mbuagbaw L. Mapping the nomenclature, methodology, and reporting of studies that review methods: a pilot methodological review. Pilot Feasibility Studies. 2020;6(1):13.

Puljak L, Makaric ZL, Buljan I, Pieper D. What is a meta-epidemiological study? Analysis of published literature indicated heterogeneous study designs and definitions. J Comp Eff Res. 2020.

Abbade LPF, Wang M, Sriganesh K, Jin Y, Mbuagbaw L, Thabane L. The framing of research questions using the PICOT format in randomized controlled trials of venous ulcer disease is suboptimal: a systematic survey. Wound Repair Regen. 2017;25(5):892–900.

Gohari F, Baradaran HR, Tabatabaee M, Anijidani S, Mohammadpour Touserkani F, Atlasi R, Razmgir M. Quality of reporting randomized controlled trials (RCTs) in diabetes in Iran; a systematic review. J Diabetes Metab Disord. 2015;15(1):36.

Wang M, Jin Y, Hu ZJ, Thabane A, Dennis B, Gajic-Veljanoski O, Paul J, Thabane L. The reporting quality of abstracts of stepped wedge randomized trials is suboptimal: a systematic survey of the literature. Contemp Clin Trials Commun. 2017;8:1–10.

Shanthanna H, Kaushal A, Mbuagbaw L, Couban R, Busse J, Thabane L: A cross-sectional study of the reporting quality of pilot or feasibility trials in high-impact anesthesia journals Can J Anaesthesia 2018, 65(11):1180–1195.

Kosa SD, Mbuagbaw L, Borg Debono V, Bhandari M, Dennis BB, Ene G, Leenus A, Shi D, Thabane M, Valvasori S, et al. Agreement in reporting between trial publications and current clinical trial registry in high impact journals: a methodological review. Contemporary Clinical Trials. 2018;65:144–50.

Zhang Y, Florez ID, Colunga Lozano LE, Aloweni FAB, Kennedy SA, Li A, Craigie S, Zhang S, Agarwal A, Lopes LC, et al. A systematic survey on reporting and methods for handling missing participant data for continuous outcomes in randomized controlled trials. J Clin Epidemiol. 2017;88:57–66.

CAS   PubMed   Google Scholar  

Hernández AV, Boersma E, Murray GD, Habbema JD, Steyerberg EW. Subgroup analyses in therapeutic cardiovascular clinical trials: are most of them misleading? Am Heart J. 2006;151(2):257–64.

Samaan Z, Mbuagbaw L, Kosa D, Borg Debono V, Dillenburg R, Zhang S, Fruci V, Dennis B, Bawor M, Thabane L. A systematic scoping review of adherence to reporting guidelines in health care literature. J Multidiscip Healthc. 2013;6:169–88.

Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP. Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol. 2006;59(7):697–703.

Carrasco-Labra A, Brignardello-Petersen R, Santesso N, Neumann I, Mustafa RA, Mbuagbaw L, Etxeandia Ikobaltzeta I, De Stio C, McCullagh LJ, Alonso-Coello P. Improving GRADE evidence tables part 1: a randomized trial shows improved understanding of content in summary-of-findings tables with a new format. J Clin Epidemiol. 2016;74:7–18.

The Northern Ireland Hub for Trials Methodology Research: SWAT/SWAR Information [ https://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/SWATSWARInformation/ ]. Accessed 31 Aug 2020.

Chick S, Sánchez P, Ferrin D, Morrice D. How to conduct a successful simulation study. In: Proceedings of the 2003 winter simulation conference: 2003; 2003. p. 66–70.

Google Scholar  

Mulrow CD. The medical review article: state of the science. Ann Intern Med. 1987;106(3):485–8.

Sacks HS, Reitman D, Pagano D, Kupelnick B. Meta-analysis: an update. Mount Sinai J Med New York. 1996;63(3–4):216–24.

CAS   Google Scholar  

Areia M, Soares M, Dinis-Ribeiro M. Quality reporting of endoscopic diagnostic studies in gastrointestinal journals: where do we stand on the use of the STARD and CONSORT statements? Endoscopy. 2010;42(2):138–47.

Knol M, Groenwold R, Grobbee D. P-values in baseline tables of randomised controlled trials are inappropriate but still common in high impact journals. Eur J Prev Cardiol. 2012;19(2):231–2.

Chen M, Cui J, Zhang AL, Sze DM, Xue CC, May BH. Adherence to CONSORT items in randomized controlled trials of integrative medicine for colorectal Cancer published in Chinese journals. J Altern Complement Med. 2018;24(2):115–24.

Hopewell S, Ravaud P, Baron G, Boutron I. Effect of editors' implementation of CONSORT guidelines on the reporting of abstracts in high impact medical journals: interrupted time series analysis. BMJ. 2012;344:e4178.

The Cochrane Methodology Register Issue 2 2009 [ https://cmr.cochrane.org/help.htm ]. Accessed 31 Aug 2020.

Mbuagbaw L, Kredo T, Welch V, Mursleen S, Ross S, Zani B, Motaze NV, Quinlan L. Critical EPICOT items were absent in Cochrane human immunodeficiency virus systematic reviews: a bibliometric analysis. J Clin Epidemiol. 2016;74:66–72.

Barton S, Peckitt C, Sclafani F, Cunningham D, Chau I. The influence of industry sponsorship on the reporting of subgroup analyses within phase III randomised controlled trials in gastrointestinal oncology. Eur J Cancer. 2015;51(18):2732–9.

Setia MS. Methodology series module 5: sampling strategies. Indian J Dermatol. 2016;61(5):505–9.

Wilson B, Burnett P, Moher D, Altman DG, Al-Shahi Salman R. Completeness of reporting of randomised controlled trials including people with transient ischaemic attack or stroke: a systematic review. Eur Stroke J. 2018;3(4):337–46.

Kahale LA, Diab B, Brignardello-Petersen R, Agarwal A, Mustafa RA, Kwong J, Neumann I, Li L, Lopes LC, Briel M, et al. Systematic reviews do not adequately report or address missing outcome data in their analyses: a methodological survey. J Clin Epidemiol. 2018;99:14–23.

De Angelis CD, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, Kotzin S, Laine C, Marusic A, Overbeke AJPM, et al. Is this clinical trial fully registered?: a statement from the International Committee of Medical Journal Editors*. Ann Intern Med. 2005;143(2):146–8.

Ohtake PJ, Childs JD. Why publish study protocols? Phys Ther. 2014;94(9):1208–9.

Rombey T, Allers K, Mathes T, Hoffmann F, Pieper D. A descriptive analysis of the characteristics and the peer review process of systematic review protocols published in an open peer review journal from 2012 to 2017. BMC Med Res Methodol. 2019;19(1):57.

Grimes DA, Schulz KF. Bias and causal associations in observational research. Lancet. 2002;359(9302):248–52.

Porta M (ed.): A dictionary of epidemiology, 5th edn. Oxford: Oxford University Press, Inc.; 2008.

El Dib R, Tikkinen KAO, Akl EA, Gomaa HA, Mustafa RA, Agarwal A, Carpenter CR, Zhang Y, Jorge EC, Almeida R, et al. Systematic survey of randomized trials evaluating the impact of alternative diagnostic strategies on patient-important outcomes. J Clin Epidemiol. 2017;84:61–9.

Helzer JE, Robins LN, Taibleson M, Woodruff RA Jr, Reich T, Wish ED. Reliability of psychiatric diagnosis. I. a methodological review. Arch Gen Psychiatry. 1977;34(2):129–33.

Chung ST, Chacko SK, Sunehag AL, Haymond MW. Measurements of gluconeogenesis and Glycogenolysis: a methodological review. Diabetes. 2015;64(12):3996–4010.

CAS   PubMed   PubMed Central   Google Scholar  

Sterne JA, Juni P, Schulz KF, Altman DG, Bartlett C, Egger M. Statistical methods for assessing the influence of study characteristics on treatment effects in 'meta-epidemiological' research. Stat Med. 2002;21(11):1513–24.

Moen EL, Fricano-Kugler CJ, Luikart BW, O’Malley AJ. Analyzing clustered data: why and how to account for multiple observations nested within a study participant? PLoS One. 2016;11(1):e0146721.

Zyzanski SJ, Flocke SA, Dickinson LM. On the nature and analysis of clustered data. Ann Fam Med. 2004;2(3):199–200.

Mathes T, Klassen P, Pieper D. Frequency of data extraction errors and methods to increase data extraction quality: a methodological review. BMC Med Res Methodol. 2017;17(1):152.

Bui DDA, Del Fiol G, Hurdle JF, Jonnalagadda S. Extractive text summarization system to aid data extraction from full text in systematic review development. J Biomed Inform. 2016;64:265–72.

Bui DD, Del Fiol G, Jonnalagadda S. PDF text classification to leverage information extraction from publication reports. J Biomed Inform. 2016;61:141–8.

Maticic K, Krnic Martinic M, Puljak L. Assessment of reporting quality of abstracts of systematic reviews with meta-analysis using PRISMA-A and discordance in assessments between raters without prior experience. BMC Med Res Methodol. 2019;19(1):32.

Speich B. Blinding in surgical randomized clinical trials in 2015. Ann Surg. 2017;266(1):21–2.

Abraha I, Cozzolino F, Orso M, Marchesi M, Germani A, Lombardo G, Eusebi P, De Florio R, Luchetta ML, Iorio A, et al. A systematic review found that deviations from intention-to-treat are common in randomized trials and systematic reviews. J Clin Epidemiol. 2017;84:37–46.

Zhong Y, Zhou W, Jiang H, Fan T, Diao X, Yang H, Min J, Wang G, Fu J, Mao B. Quality of reporting of two-group parallel randomized controlled clinical trials of multi-herb formulae: A survey of reports indexed in the Science Citation Index Expanded. Eur J Integrative Med. 2011;3(4):e309–16.

Farrokhyar F, Chu R, Whitlock R, Thabane L. A systematic review of the quality of publications reporting coronary artery bypass grafting trials. Can J Surg. 2007;50(4):266–77.

Oltean H, Gagnier JJ. Use of clustering analysis in randomized controlled trials in orthopaedic surgery. BMC Med Res Methodol. 2015;15:17.

Fleming PS, Koletsi D, Pandis N. Blinded by PRISMA: are systematic reviewers focusing on PRISMA and ignoring other guidelines? PLoS One. 2014;9(5):e96407.

Balasubramanian SP, Wiener M, Alshameeri Z, Tiruvoipati R, Elbourne D, Reed MW. Standards of reporting of randomized controlled trials in general surgery: can we do better? Ann Surg. 2006;244(5):663–7.

de Vries TW, van Roon EN. Low quality of reporting adverse drug reactions in paediatric randomised controlled trials. Arch Dis Child. 2010;95(12):1023–6.

Borg Debono V, Zhang S, Ye C, Paul J, Arya A, Hurlburt L, Murthy Y, Thabane L. The quality of reporting of RCTs used within a postoperative pain management meta-analysis, using the CONSORT statement. BMC Anesthesiol. 2012;12:13.

Kaiser KA, Cofield SS, Fontaine KR, Glasser SP, Thabane L, Chu R, Ambrale S, Dwary AD, Kumar A, Nayyar G, et al. Is funding source related to study reporting quality in obesity or nutrition randomized control trials in top-tier medical journals? Int J Obes. 2012;36(7):977–81.

Thomas O, Thabane L, Douketis J, Chu R, Westfall AO, Allison DB. Industry funding and the reporting quality of large long-term weight loss trials. Int J Obes. 2008;32(10):1531–6.

Khan NR, Saad H, Oravec CS, Rossi N, Nguyen V, Venable GT, Lillard JC, Patel P, Taylor DR, Vaughn BN, et al. A review of industry funding in randomized controlled trials published in the neurosurgical literature-the elephant in the room. Neurosurgery. 2018;83(5):890–7.

Hansen C, Lundh A, Rasmussen K, Hrobjartsson A. Financial conflicts of interest in systematic reviews: associations with results, conclusions, and methodological quality. Cochrane Database Syst Rev. 2019;8:Mr000047.

Kiehna EN, Starke RM, Pouratian N, Dumont AS. Standards for reporting randomized controlled trials in neurosurgery. J Neurosurg. 2011;114(2):280–5.

Liu LQ, Morris PJ, Pengel LH. Compliance to the CONSORT statement of randomized controlled trials in solid organ transplantation: a 3-year overview. Transpl Int. 2013;26(3):300–6.

Bala MM, Akl EA, Sun X, Bassler D, Mertz D, Mejza F, Vandvik PO, Malaga G, Johnston BC, Dahm P, et al. Randomized trials published in higher vs. lower impact journals differ in design, conduct, and analysis. J Clin Epidemiol. 2013;66(3):286–95.

Lee SY, Teoh PJ, Camm CF, Agha RA. Compliance of randomized controlled trials in trauma surgery with the CONSORT statement. J Trauma Acute Care Surg. 2013;75(4):562–72.

Ziogas DC, Zintzaras E. Analysis of the quality of reporting of randomized controlled trials in acute and chronic myeloid leukemia, and myelodysplastic syndromes as governed by the CONSORT statement. Ann Epidemiol. 2009;19(7):494–500.

Alvarez F, Meyer N, Gourraud PA, Paul C. CONSORT adoption and quality of reporting of randomized controlled trials: a systematic analysis in two dermatology journals. Br J Dermatol. 2009;161(5):1159–65.

Mbuagbaw L, Thabane M, Vanniyasingam T, Borg Debono V, Kosa S, Zhang S, Ye C, Parpia S, Dennis BB, Thabane L. Improvement in the quality of abstracts in major clinical journals since CONSORT extension for abstracts: a systematic review. Contemporary Clin trials. 2014;38(2):245–50.

Thabane L, Chu R, Cuddy K, Douketis J. What is the quality of reporting in weight loss intervention studies? A systematic review of randomized controlled trials. Int J Obes. 2007;31(10):1554–9.

Murad MH, Wang Z. Guidelines for reporting meta-epidemiological methodology research. Evidence Based Med. 2017;22(4):139.

METRIC - MEthodological sTudy ReportIng Checklist: guidelines for reporting methodological studies in health research [ http://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-other-study-designs/#METRIC ]. Accessed 31 Aug 2020.

Jager KJ, Zoccali C, MacLeod A, Dekker FW. Confounding: what it is and how to deal with it. Kidney Int. 2008;73(3):256–60.

Parker SG, Halligan S, Erotocritou M, Wood CPJ, Boulton RW, Plumb AAO, Windsor ACJ, Mallett S. A systematic methodological review of non-randomised interventional studies of elective ventral hernia repair: clear definitions and a standardised minimum dataset are needed. Hernia. 2019.

Bouwmeester W, Zuithoff NPA, Mallett S, Geerlings MI, Vergouwe Y, Steyerberg EW, Altman DG, Moons KGM. Reporting and methods in clinical prediction research: a systematic review. PLoS Med. 2012;9(5):1–12.

Schiller P, Burchardi N, Niestroj M, Kieser M. Quality of reporting of clinical non-inferiority and equivalence randomised trials--update and extension. Trials. 2012;13:214.

Riado Minguez D, Kowalski M, Vallve Odena M, Longin Pontzen D, Jelicic Kadic A, Jeric M, Dosenovic S, Jakus D, Vrdoljak M, Poklepovic Pericic T, et al. Methodological and reporting quality of systematic reviews published in the highest ranking journals in the field of pain. Anesth Analg. 2017;125(4):1348–54.

Thabut G, Estellat C, Boutron I, Samama CM, Ravaud P. Methodological issues in trials assessing primary prophylaxis of venous thrombo-embolism. Eur Heart J. 2005;27(2):227–36.

Puljak L, Riva N, Parmelli E, González-Lorenzo M, Moja L, Pieper D. Data extraction methods: an analysis of internal reporting discrepancies in single manuscripts and practical advice. J Clin Epidemiol. 2020;117:158–64.

Ritchie A, Seubert L, Clifford R, Perry D, Bond C. Do randomised controlled trials relevant to pharmacy meet best practice standards for quality conduct and reporting? A systematic review. Int J Pharm Pract. 2019.

Babic A, Vuka I, Saric F, Proloscic I, Slapnicar E, Cavar J, Pericic TP, Pieper D, Puljak L. Overall bias methods and their use in sensitivity analysis of Cochrane reviews were not consistent. J Clin Epidemiol. 2019.

Tan A, Porcher R, Crequit P, Ravaud P, Dechartres A. Differences in treatment effect size between overall survival and progression-free survival in immunotherapy trials: a Meta-epidemiologic study of trials with results posted at ClinicalTrials.gov. J Clin Oncol. 2017;35(15):1686–94.

Croitoru D, Huang Y, Kurdina A, Chan AW, Drucker AM. Quality of reporting in systematic reviews published in dermatology journals. Br J Dermatol. 2020;182(6):1469–76.

Khan MS, Ochani RK, Shaikh A, Vaduganathan M, Khan SU, Fatima K, Yamani N, Mandrola J, Doukky R, Krasuski RA: Assessing the Quality of Reporting of Harms in Randomized Controlled Trials Published in High Impact Cardiovascular Journals. Eur Heart J Qual Care Clin Outcomes 2019.

Rosmarakis ES, Soteriades ES, Vergidis PI, Kasiakou SK, Falagas ME. From conference abstract to full paper: differences between data presented in conferences and journals. FASEB J. 2005;19(7):673–80.

Mueller M, D’Addario M, Egger M, Cevallos M, Dekkers O, Mugglin C, Scott P. Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations. BMC Med Res Methodol. 2018;18(1):44.

Li G, Abbade LPF, Nwosu I, Jin Y, Leenus A, Maaz M, Wang M, Bhatt M, Zielinski L, Sanger N, et al. A scoping review of comparisons between abstracts and full reports in primary biomedical research. BMC Med Res Methodol. 2017;17(1):181.

Krnic Martinic M, Pieper D, Glatt A, Puljak L. Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks. BMC Med Res Methodol. 2019;19(1):203.

Analytical study [ https://medical-dictionary.thefreedictionary.com/analytical+study ]. Accessed 31 Aug 2020.

Tricco AC, Tetzlaff J, Pham B, Brehaut J, Moher D. Non-Cochrane vs. Cochrane reviews were twice as likely to have positive conclusion statements: cross-sectional study. J Clin Epidemiol. 2009;62(4):380–6 e381.

Schalken N, Rietbergen C. The reporting quality of systematic reviews and Meta-analyses in industrial and organizational psychology: a systematic review. Front Psychol. 2017;8:1395.

Ranker LR, Petersen JM, Fox MP. Awareness of and potential for dependent error in the observational epidemiologic literature: A review. Ann Epidemiol. 2019;36:15–9 e12.

Paquette M, Alotaibi AM, Nieuwlaat R, Santesso N, Mbuagbaw L. A meta-epidemiological study of subgroup analyses in cochrane systematic reviews of atrial fibrillation. Syst Rev. 2019;8(1):241.

Download references

Acknowledgements

This work did not receive any dedicated funding.

Author information

Authors and affiliations.

Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada

Lawrence Mbuagbaw, Daeria O. Lawson & Lehana Thabane

Biostatistics Unit/FSORC, 50 Charlton Avenue East, St Joseph’s Healthcare—Hamilton, 3rd Floor Martha Wing, Room H321, Hamilton, Ontario, L8N 4A6, Canada

Lawrence Mbuagbaw & Lehana Thabane

Centre for the Development of Best Practices in Health, Yaoundé, Cameroon

Lawrence Mbuagbaw

Center for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000, Zagreb, Croatia

Livia Puljak

Department of Epidemiology and Biostatistics, School of Public Health – Bloomington, Indiana University, Bloomington, IN, 47405, USA

David B. Allison

Departments of Paediatrics and Anaesthesia, McMaster University, Hamilton, ON, Canada

Lehana Thabane

Centre for Evaluation of Medicine, St. Joseph’s Healthcare-Hamilton, Hamilton, ON, Canada

Population Health Research Institute, Hamilton Health Sciences, Hamilton, ON, Canada

You can also search for this author in PubMed   Google Scholar

Contributions

LM conceived the idea and drafted the outline and paper. DOL and LT commented on the idea and draft outline. LM, LP and DOL performed literature searches and data extraction. All authors (LM, DOL, LT, LP, DBA) reviewed several draft versions of the manuscript and approved the final manuscript.

Corresponding author

Correspondence to Lawrence Mbuagbaw .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

DOL, DBA, LM, LP and LT are involved in the development of a reporting guideline for methodological studies.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Mbuagbaw, L., Lawson, D.O., Puljak, L. et al. A tutorial on methodological studies: the what, when, how and why. BMC Med Res Methodol 20 , 226 (2020). https://doi.org/10.1186/s12874-020-01107-7

Download citation

Received : 27 May 2020

Accepted : 27 August 2020

Published : 07 September 2020

DOI : https://doi.org/10.1186/s12874-020-01107-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Methodological study
• Meta-epidemiology
• Research methods
• Research-on-research

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

Research Methods

• Getting Started
• Literature Review Research
• Research Design
• Research Design By Discipline
• SAGE Research Methods
• Teaching with SAGE Research Methods

Literature Review

• What is a Literature Review?
• What is NOT a Literature Review?
• Purposes of a Literature Review
• Types of Literature Reviews
• Literature Reviews vs. Systematic Reviews
• Systematic vs. Meta-Analysis

Literature Review  is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

Also, we can define a literature review as the collected body of scholarly works related to a topic:

• Summarizes and analyzes previous research relevant to a topic
• Includes scholarly books and articles published in academic journals
• Can be an specific scholarly paper or a section in a research paper

The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic

• Help gather ideas or information
• Keep up to date in current trends and findings
• Help develop new questions

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Helps focus your own research questions or problems
• Discovers relationships between research studies/ideas.
• Suggests unexplored ideas or populations
• Identifies major themes, concepts, and researchers on a topic.
• Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
• Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
• Indicates potential directions for future research.

All content in this section is from Literature Review Research from Old Dominion University 

Keep in mind the following, a literature review is NOT:

Not an essay 

Not an annotated bibliography  in which you summarize each article that you have reviewed.  A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.

Not a research paper   where you select resources to support one side of an issue versus another.  A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.

A literature review serves several purposes. For example, it

• provides thorough knowledge of previous studies; introduces seminal works.
• helps focus one’s own research topic.
• identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
• suggests previously unused or underused methodologies, designs, quantitative and qualitative strategies.
• identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
• helps the researcher avoid repetition of earlier research.
• suggests unexplored populations.
• determines whether past studies agree or disagree; identifies controversy in the literature.
• tests assumptions; may help counter preconceived ideas and remove unconscious bias.

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

Robinson, P. and Lowe, J. (2015),  Literature reviews vs systematic reviews.  Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393

What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California

Systematic review or meta-analysis?

A  systematic review  answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.

A  meta-analysis  is the use of statistical methods to summarize the results of these studies.

Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:

• clearly stated objectives with pre-defined eligibility criteria for studies
• explicit, reproducible methodology
• a systematic search that attempts to identify all studies
• assessment of the validity of the findings of the included studies (e.g. risk of bias)
• systematic presentation, and synthesis, of the characteristics and findings of the included studies

Not all systematic reviews contain meta-analysis. 

Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.  More information on meta-analyses can be found in  Cochrane Handbook, Chapter 9 .

A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies.  It is a systematic review that uses quantitative methods to synthesize and summarize the results.

An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings.  Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted.  In that case, an integrative review is an appropriate strategy. 

Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.

• << Previous: Getting Started
• Next: Research Design >>
• Last Updated: Jul 15, 2024 10:34 AM
• URL: https://guides.lib.udel.edu/researchmethods

• Literary Theory
• Literature Review

Approach to Conduct an Effective Literature Review

• October 2012

• Alzahra University

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• SCI RES ESSAYS
• INT J PHYS SCI
• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• BMC Med Res Methodol

A tutorial on methodological studies: the what, when, how and why

Lawrence mbuagbaw.

1 Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON Canada

2 Biostatistics Unit/FSORC, 50 Charlton Avenue East, St Joseph’s Healthcare—Hamilton, 3rd Floor Martha Wing, Room H321, Hamilton, Ontario L8N 4A6 Canada

3 Centre for the Development of Best Practices in Health, Yaoundé, Cameroon

Daeria O. Lawson

Livia puljak.

4 Center for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000 Zagreb, Croatia

David B. Allison

5 Department of Epidemiology and Biostatistics, School of Public Health – Bloomington, Indiana University, Bloomington, IN 47405 USA

Lehana Thabane

6 Departments of Paediatrics and Anaesthesia, McMaster University, Hamilton, ON Canada

7 Centre for Evaluation of Medicine, St. Joseph’s Healthcare-Hamilton, Hamilton, ON Canada

8 Population Health Research Institute, Hamilton Health Sciences, Hamilton, ON Canada

Associated Data

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Methodological studies – studies that evaluate the design, analysis or reporting of other research-related reports – play an important role in health research. They help to highlight issues in the conduct of research with the aim of improving health research methodology, and ultimately reducing research waste.

We provide an overview of some of the key aspects of methodological studies such as what they are, and when, how and why they are done. We adopt a “frequently asked questions” format to facilitate reading this paper and provide multiple examples to help guide researchers interested in conducting methodological studies. Some of the topics addressed include: is it necessary to publish a study protocol? How to select relevant research reports and databases for a methodological study? What approaches to data extraction and statistical analysis should be considered when conducting a methodological study? What are potential threats to validity and is there a way to appraise the quality of methodological studies?

Appropriate reflection and application of basic principles of epidemiology and biostatistics are required in the design and analysis of methodological studies. This paper provides an introduction for further discussion about the conduct of methodological studies.

The field of meta-research (or research-on-research) has proliferated in recent years in response to issues with research quality and conduct [ 1 – 3 ]. As the name suggests, this field targets issues with research design, conduct, analysis and reporting. Various types of research reports are often examined as the unit of analysis in these studies (e.g. abstracts, full manuscripts, trial registry entries). Like many other novel fields of research, meta-research has seen a proliferation of use before the development of reporting guidance. For example, this was the case with randomized trials for which risk of bias tools and reporting guidelines were only developed much later – after many trials had been published and noted to have limitations [ 4 , 5 ]; and for systematic reviews as well [ 6 – 8 ]. However, in the absence of formal guidance, studies that report on research differ substantially in how they are named, conducted and reported [ 9 , 10 ]. This creates challenges in identifying, summarizing and comparing them. In this tutorial paper, we will use the term methodological study to refer to any study that reports on the design, conduct, analysis or reporting of primary or secondary research-related reports (such as trial registry entries and conference abstracts).

In the past 10 years, there has been an increase in the use of terms related to methodological studies (based on records retrieved with a keyword search [in the title and abstract] for “methodological review” and “meta-epidemiological study” in PubMed up to December 2019), suggesting that these studies may be appearing more frequently in the literature. See Fig.  1 .

Trends in the number studies that mention “methodological review” or “meta-

epidemiological study” in PubMed.

The methods used in many methodological studies have been borrowed from systematic and scoping reviews. This practice has influenced the direction of the field, with many methodological studies including searches of electronic databases, screening of records, duplicate data extraction and assessments of risk of bias in the included studies. However, the research questions posed in methodological studies do not always require the approaches listed above, and guidance is needed on when and how to apply these methods to a methodological study. Even though methodological studies can be conducted on qualitative or mixed methods research, this paper focuses on and draws examples exclusively from quantitative research.

The objectives of this paper are to provide some insights on how to conduct methodological studies so that there is greater consistency between the research questions posed, and the design, analysis and reporting of findings. We provide multiple examples to illustrate concepts and a proposed framework for categorizing methodological studies in quantitative research.

What is a methodological study?

Any study that describes or analyzes methods (design, conduct, analysis or reporting) in published (or unpublished) literature is a methodological study. Consequently, the scope of methodological studies is quite extensive and includes, but is not limited to, topics as diverse as: research question formulation [ 11 ]; adherence to reporting guidelines [ 12 – 14 ] and consistency in reporting [ 15 ]; approaches to study analysis [ 16 ]; investigating the credibility of analyses [ 17 ]; and studies that synthesize these methodological studies [ 18 ]. While the nomenclature of methodological studies is not uniform, the intents and purposes of these studies remain fairly consistent – to describe or analyze methods in primary or secondary studies. As such, methodological studies may also be classified as a subtype of observational studies.

Parallel to this are experimental studies that compare different methods. Even though they play an important role in informing optimal research methods, experimental methodological studies are beyond the scope of this paper. Examples of such studies include the randomized trials by Buscemi et al., comparing single data extraction to double data extraction [ 19 ], and Carrasco-Labra et al., comparing approaches to presenting findings in Grading of Recommendations, Assessment, Development and Evaluations (GRADE) summary of findings tables [ 20 ]. In these studies, the unit of analysis is the person or groups of individuals applying the methods. We also direct readers to the Studies Within a Trial (SWAT) and Studies Within a Review (SWAR) programme operated through the Hub for Trials Methodology Research, for further reading as a potential useful resource for these types of experimental studies [ 21 ]. Lastly, this paper is not meant to inform the conduct of research using computational simulation and mathematical modeling for which some guidance already exists [ 22 ], or studies on the development of methods using consensus-based approaches.

When should we conduct a methodological study?

Methodological studies occupy a unique niche in health research that allows them to inform methodological advances. Methodological studies should also be conducted as pre-cursors to reporting guideline development, as they provide an opportunity to understand current practices, and help to identify the need for guidance and gaps in methodological or reporting quality. For example, the development of the popular Preferred Reporting Items of Systematic reviews and Meta-Analyses (PRISMA) guidelines were preceded by methodological studies identifying poor reporting practices [ 23 , 24 ]. In these instances, after the reporting guidelines are published, methodological studies can also be used to monitor uptake of the guidelines.

These studies can also be conducted to inform the state of the art for design, analysis and reporting practices across different types of health research fields, with the aim of improving research practices, and preventing or reducing research waste. For example, Samaan et al. conducted a scoping review of adherence to different reporting guidelines in health care literature [ 18 ]. Methodological studies can also be used to determine the factors associated with reporting practices. For example, Abbade et al. investigated journal characteristics associated with the use of the Participants, Intervention, Comparison, Outcome, Timeframe (PICOT) format in framing research questions in trials of venous ulcer disease [ 11 ].

How often are methodological studies conducted?

There is no clear answer to this question. Based on a search of PubMed, the use of related terms (“methodological review” and “meta-epidemiological study”) – and therefore, the number of methodological studies – is on the rise. However, many other terms are used to describe methodological studies. There are also many studies that explore design, conduct, analysis or reporting of research reports, but that do not use any specific terms to describe or label their study design in terms of “methodology”. This diversity in nomenclature makes a census of methodological studies elusive. Appropriate terminology and key words for methodological studies are needed to facilitate improved accessibility for end-users.

Why do we conduct methodological studies?

Methodological studies provide information on the design, conduct, analysis or reporting of primary and secondary research and can be used to appraise quality, quantity, completeness, accuracy and consistency of health research. These issues can be explored in specific fields, journals, databases, geographical regions and time periods. For example, Areia et al. explored the quality of reporting of endoscopic diagnostic studies in gastroenterology [ 25 ]; Knol et al. investigated the reporting of p -values in baseline tables in randomized trial published in high impact journals [ 26 ]; Chen et al. describe adherence to the Consolidated Standards of Reporting Trials (CONSORT) statement in Chinese Journals [ 27 ]; and Hopewell et al. describe the effect of editors’ implementation of CONSORT guidelines on reporting of abstracts over time [ 28 ]. Methodological studies provide useful information to researchers, clinicians, editors, publishers and users of health literature. As a result, these studies have been at the cornerstone of important methodological developments in the past two decades and have informed the development of many health research guidelines including the highly cited CONSORT statement [ 5 ].

Where can we find methodological studies?

Methodological studies can be found in most common biomedical bibliographic databases (e.g. Embase, MEDLINE, PubMed, Web of Science). However, the biggest caveat is that methodological studies are hard to identify in the literature due to the wide variety of names used and the lack of comprehensive databases dedicated to them. A handful can be found in the Cochrane Library as “Cochrane Methodology Reviews”, but these studies only cover methodological issues related to systematic reviews. Previous attempts to catalogue all empirical studies of methods used in reviews were abandoned 10 years ago [ 29 ]. In other databases, a variety of search terms may be applied with different levels of sensitivity and specificity.

Some frequently asked questions about methodological studies

In this section, we have outlined responses to questions that might help inform the conduct of methodological studies.

Q: How should I select research reports for my methodological study?

A: Selection of research reports for a methodological study depends on the research question and eligibility criteria. Once a clear research question is set and the nature of literature one desires to review is known, one can then begin the selection process. Selection may begin with a broad search, especially if the eligibility criteria are not apparent. For example, a methodological study of Cochrane Reviews of HIV would not require a complex search as all eligible studies can easily be retrieved from the Cochrane Library after checking a few boxes [ 30 ]. On the other hand, a methodological study of subgroup analyses in trials of gastrointestinal oncology would require a search to find such trials, and further screening to identify trials that conducted a subgroup analysis [ 31 ].

The strategies used for identifying participants in observational studies can apply here. One may use a systematic search to identify all eligible studies. If the number of eligible studies is unmanageable, a random sample of articles can be expected to provide comparable results if it is sufficiently large [ 32 ]. For example, Wilson et al. used a random sample of trials from the Cochrane Stroke Group’s Trial Register to investigate completeness of reporting [ 33 ]. It is possible that a simple random sample would lead to underrepresentation of units (i.e. research reports) that are smaller in number. This is relevant if the investigators wish to compare multiple groups but have too few units in one group. In this case a stratified sample would help to create equal groups. For example, in a methodological study comparing Cochrane and non-Cochrane reviews, Kahale et al. drew random samples from both groups [ 34 ]. Alternatively, systematic or purposeful sampling strategies can be used and we encourage researchers to justify their selected approaches based on the study objective.

Q: How many databases should I search?

A: The number of databases one should search would depend on the approach to sampling, which can include targeting the entire “population” of interest or a sample of that population. If you are interested in including the entire target population for your research question, or drawing a random or systematic sample from it, then a comprehensive and exhaustive search for relevant articles is required. In this case, we recommend using systematic approaches for searching electronic databases (i.e. at least 2 databases with a replicable and time stamped search strategy). The results of your search will constitute a sampling frame from which eligible studies can be drawn.

Alternatively, if your approach to sampling is purposeful, then we recommend targeting the database(s) or data sources (e.g. journals, registries) that include the information you need. For example, if you are conducting a methodological study of high impact journals in plastic surgery and they are all indexed in PubMed, you likely do not need to search any other databases. You may also have a comprehensive list of all journals of interest and can approach your search using the journal names in your database search (or by accessing the journal archives directly from the journal’s website). Even though one could also search journals’ web pages directly, using a database such as PubMed has multiple advantages, such as the use of filters, so the search can be narrowed down to a certain period, or study types of interest. Furthermore, individual journals’ web sites may have different search functionalities, which do not necessarily yield a consistent output.

Q: Should I publish a protocol for my methodological study?

A: A protocol is a description of intended research methods. Currently, only protocols for clinical trials require registration [ 35 ]. Protocols for systematic reviews are encouraged but no formal recommendation exists. The scientific community welcomes the publication of protocols because they help protect against selective outcome reporting, the use of post hoc methodologies to embellish results, and to help avoid duplication of efforts [ 36 ]. While the latter two risks exist in methodological research, the negative consequences may be substantially less than for clinical outcomes. In a sample of 31 methodological studies, 7 (22.6%) referenced a published protocol [ 9 ]. In the Cochrane Library, there are 15 protocols for methodological reviews (21 July 2020). This suggests that publishing protocols for methodological studies is not uncommon.

Authors can consider publishing their study protocol in a scholarly journal as a manuscript. Advantages of such publication include obtaining peer-review feedback about the planned study, and easy retrieval by searching databases such as PubMed. The disadvantages in trying to publish protocols includes delays associated with manuscript handling and peer review, as well as costs, as few journals publish study protocols, and those journals mostly charge article-processing fees [ 37 ]. Authors who would like to make their protocol publicly available without publishing it in scholarly journals, could deposit their study protocols in publicly available repositories, such as the Open Science Framework ( https://osf.io/ ).

Q: How to appraise the quality of a methodological study?

A: To date, there is no published tool for appraising the risk of bias in a methodological study, but in principle, a methodological study could be considered as a type of observational study. Therefore, during conduct or appraisal, care should be taken to avoid the biases common in observational studies [ 38 ]. These biases include selection bias, comparability of groups, and ascertainment of exposure or outcome. In other words, to generate a representative sample, a comprehensive reproducible search may be necessary to build a sampling frame. Additionally, random sampling may be necessary to ensure that all the included research reports have the same probability of being selected, and the screening and selection processes should be transparent and reproducible. To ensure that the groups compared are similar in all characteristics, matching, random sampling or stratified sampling can be used. Statistical adjustments for between-group differences can also be applied at the analysis stage. Finally, duplicate data extraction can reduce errors in assessment of exposures or outcomes.

Q: Should I justify a sample size?

A: In all instances where one is not using the target population (i.e. the group to which inferences from the research report are directed) [ 39 ], a sample size justification is good practice. The sample size justification may take the form of a description of what is expected to be achieved with the number of articles selected, or a formal sample size estimation that outlines the number of articles required to answer the research question with a certain precision and power. Sample size justifications in methodological studies are reasonable in the following instances:

• Comparing two groups
• Determining a proportion, mean or another quantifier
• Determining factors associated with an outcome using regression-based analyses

For example, El Dib et al. computed a sample size requirement for a methodological study of diagnostic strategies in randomized trials, based on a confidence interval approach [ 40 ].

Q: What should I call my study?

A: Other terms which have been used to describe/label methodological studies include “ methodological review ”, “methodological survey” , “meta-epidemiological study” , “systematic review” , “systematic survey”, “meta-research”, “research-on-research” and many others. We recommend that the study nomenclature be clear, unambiguous, informative and allow for appropriate indexing. Methodological study nomenclature that should be avoided includes “ systematic review” – as this will likely be confused with a systematic review of a clinical question. “ Systematic survey” may also lead to confusion about whether the survey was systematic (i.e. using a preplanned methodology) or a survey using “ systematic” sampling (i.e. a sampling approach using specific intervals to determine who is selected) [ 32 ]. Any of the above meanings of the words “ systematic” may be true for methodological studies and could be potentially misleading. “ Meta-epidemiological study” is ideal for indexing, but not very informative as it describes an entire field. The term “ review ” may point towards an appraisal or “review” of the design, conduct, analysis or reporting (or methodological components) of the targeted research reports, yet it has also been used to describe narrative reviews [ 41 , 42 ]. The term “ survey ” is also in line with the approaches used in many methodological studies [ 9 ], and would be indicative of the sampling procedures of this study design. However, in the absence of guidelines on nomenclature, the term “ methodological study ” is broad enough to capture most of the scenarios of such studies.

Q: Should I account for clustering in my methodological study?

A: Data from methodological studies are often clustered. For example, articles coming from a specific source may have different reporting standards (e.g. the Cochrane Library). Articles within the same journal may be similar due to editorial practices and policies, reporting requirements and endorsement of guidelines. There is emerging evidence that these are real concerns that should be accounted for in analyses [ 43 ]. Some cluster variables are described in the section: “ What variables are relevant to methodological studies?”

A variety of modelling approaches can be used to account for correlated data, including the use of marginal, fixed or mixed effects regression models with appropriate computation of standard errors [ 44 ]. For example, Kosa et al. used generalized estimation equations to account for correlation of articles within journals [ 15 ]. Not accounting for clustering could lead to incorrect p -values, unduly narrow confidence intervals, and biased estimates [ 45 ].

Q: Should I extract data in duplicate?

A: Yes. Duplicate data extraction takes more time but results in less errors [ 19 ]. Data extraction errors in turn affect the effect estimate [ 46 ], and therefore should be mitigated. Duplicate data extraction should be considered in the absence of other approaches to minimize extraction errors. However, much like systematic reviews, this area will likely see rapid new advances with machine learning and natural language processing technologies to support researchers with screening and data extraction [ 47 , 48 ]. However, experience plays an important role in the quality of extracted data and inexperienced extractors should be paired with experienced extractors [ 46 , 49 ].

Q: Should I assess the risk of bias of research reports included in my methodological study?

A : Risk of bias is most useful in determining the certainty that can be placed in the effect measure from a study. In methodological studies, risk of bias may not serve the purpose of determining the trustworthiness of results, as effect measures are often not the primary goal of methodological studies. Determining risk of bias in methodological studies is likely a practice borrowed from systematic review methodology, but whose intrinsic value is not obvious in methodological studies. When it is part of the research question, investigators often focus on one aspect of risk of bias. For example, Speich investigated how blinding was reported in surgical trials [ 50 ], and Abraha et al., investigated the application of intention-to-treat analyses in systematic reviews and trials [ 51 ].

Q: What variables are relevant to methodological studies?

A: There is empirical evidence that certain variables may inform the findings in a methodological study. We outline some of these and provide a brief overview below:

• Country: Countries and regions differ in their research cultures, and the resources available to conduct research. Therefore, it is reasonable to believe that there may be differences in methodological features across countries. Methodological studies have reported loco-regional differences in reporting quality [ 52 , 53 ]. This may also be related to challenges non-English speakers face in publishing papers in English.
• Authors’ expertise: The inclusion of authors with expertise in research methodology, biostatistics, and scientific writing is likely to influence the end-product. Oltean et al. found that among randomized trials in orthopaedic surgery, the use of analyses that accounted for clustering was more likely when specialists (e.g. statistician, epidemiologist or clinical trials methodologist) were included on the study team [ 54 ]. Fleming et al. found that including methodologists in the review team was associated with appropriate use of reporting guidelines [ 55 ].
• Source of funding and conflicts of interest: Some studies have found that funded studies report better [ 56 , 57 ], while others do not [ 53 , 58 ]. The presence of funding would indicate the availability of resources deployed to ensure optimal design, conduct, analysis and reporting. However, the source of funding may introduce conflicts of interest and warrant assessment. For example, Kaiser et al. investigated the effect of industry funding on obesity or nutrition randomized trials and found that reporting quality was similar [ 59 ]. Thomas et al. looked at reporting quality of long-term weight loss trials and found that industry funded studies were better [ 60 ]. Kan et al. examined the association between industry funding and “positive trials” (trials reporting a significant intervention effect) and found that industry funding was highly predictive of a positive trial [ 61 ]. This finding is similar to that of a recent Cochrane Methodology Review by Hansen et al. [ 62 ]
• Journal characteristics: Certain journals’ characteristics may influence the study design, analysis or reporting. Characteristics such as journal endorsement of guidelines [ 63 , 64 ], and Journal Impact Factor (JIF) have been shown to be associated with reporting [ 63 , 65 – 67 ].
• Study size (sample size/number of sites): Some studies have shown that reporting is better in larger studies [ 53 , 56 , 58 ].
• Year of publication: It is reasonable to assume that design, conduct, analysis and reporting of research will change over time. Many studies have demonstrated improvements in reporting over time or after the publication of reporting guidelines [ 68 , 69 ].
• Type of intervention: In a methodological study of reporting quality of weight loss intervention studies, Thabane et al. found that trials of pharmacologic interventions were reported better than trials of non-pharmacologic interventions [ 70 ].
• Interactions between variables: Complex interactions between the previously listed variables are possible. High income countries with more resources may be more likely to conduct larger studies and incorporate a variety of experts. Authors in certain countries may prefer certain journals, and journal endorsement of guidelines and editorial policies may change over time.

Q: Should I focus only on high impact journals?

A: Investigators may choose to investigate only high impact journals because they are more likely to influence practice and policy, or because they assume that methodological standards would be higher. However, the JIF may severely limit the scope of articles included and may skew the sample towards articles with positive findings. The generalizability and applicability of findings from a handful of journals must be examined carefully, especially since the JIF varies over time. Even among journals that are all “high impact”, variations exist in methodological standards.

Q: Can I conduct a methodological study of qualitative research?

A: Yes. Even though a lot of methodological research has been conducted in the quantitative research field, methodological studies of qualitative studies are feasible. Certain databases that catalogue qualitative research including the Cumulative Index to Nursing & Allied Health Literature (CINAHL) have defined subject headings that are specific to methodological research (e.g. “research methodology”). Alternatively, one could also conduct a qualitative methodological review; that is, use qualitative approaches to synthesize methodological issues in qualitative studies.

Q: What reporting guidelines should I use for my methodological study?

A: There is no guideline that covers the entire scope of methodological studies. One adaptation of the PRISMA guidelines has been published, which works well for studies that aim to use the entire target population of research reports [ 71 ]. However, it is not widely used (40 citations in 2 years as of 09 December 2019), and methodological studies that are designed as cross-sectional or before-after studies require a more fit-for purpose guideline. A more encompassing reporting guideline for a broad range of methodological studies is currently under development [ 72 ]. However, in the absence of formal guidance, the requirements for scientific reporting should be respected, and authors of methodological studies should focus on transparency and reproducibility.

Q: What are the potential threats to validity and how can I avoid them?

A: Methodological studies may be compromised by a lack of internal or external validity. The main threats to internal validity in methodological studies are selection and confounding bias. Investigators must ensure that the methods used to select articles does not make them differ systematically from the set of articles to which they would like to make inferences. For example, attempting to make extrapolations to all journals after analyzing high-impact journals would be misleading.

Many factors (confounders) may distort the association between the exposure and outcome if the included research reports differ with respect to these factors [ 73 ]. For example, when examining the association between source of funding and completeness of reporting, it may be necessary to account for journals that endorse the guidelines. Confounding bias can be addressed by restriction, matching and statistical adjustment [ 73 ]. Restriction appears to be the method of choice for many investigators who choose to include only high impact journals or articles in a specific field. For example, Knol et al. examined the reporting of p -values in baseline tables of high impact journals [ 26 ]. Matching is also sometimes used. In the methodological study of non-randomized interventional studies of elective ventral hernia repair, Parker et al. matched prospective studies with retrospective studies and compared reporting standards [ 74 ]. Some other methodological studies use statistical adjustments. For example, Zhang et al. used regression techniques to determine the factors associated with missing participant data in trials [ 16 ].

With regard to external validity, researchers interested in conducting methodological studies must consider how generalizable or applicable their findings are. This should tie in closely with the research question and should be explicit. For example. Findings from methodological studies on trials published in high impact cardiology journals cannot be assumed to be applicable to trials in other fields. However, investigators must ensure that their sample truly represents the target sample either by a) conducting a comprehensive and exhaustive search, or b) using an appropriate and justified, randomly selected sample of research reports.

Even applicability to high impact journals may vary based on the investigators’ definition, and over time. For example, for high impact journals in the field of general medicine, Bouwmeester et al. included the Annals of Internal Medicine (AIM), BMJ, the Journal of the American Medical Association (JAMA), Lancet, the New England Journal of Medicine (NEJM), and PLoS Medicine ( n  = 6) [ 75 ]. In contrast, the high impact journals selected in the methodological study by Schiller et al. were BMJ, JAMA, Lancet, and NEJM ( n  = 4) [ 76 ]. Another methodological study by Kosa et al. included AIM, BMJ, JAMA, Lancet and NEJM ( n  = 5). In the methodological study by Thabut et al., journals with a JIF greater than 5 were considered to be high impact. Riado Minguez et al. used first quartile journals in the Journal Citation Reports (JCR) for a specific year to determine “high impact” [ 77 ]. Ultimately, the definition of high impact will be based on the number of journals the investigators are willing to include, the year of impact and the JIF cut-off [ 78 ]. We acknowledge that the term “generalizability” may apply differently for methodological studies, especially when in many instances it is possible to include the entire target population in the sample studied.

Finally, methodological studies are not exempt from information bias which may stem from discrepancies in the included research reports [ 79 ], errors in data extraction, or inappropriate interpretation of the information extracted. Likewise, publication bias may also be a concern in methodological studies, but such concepts have not yet been explored.

A proposed framework

In order to inform discussions about methodological studies, the development of guidance for what should be reported, we have outlined some key features of methodological studies that can be used to classify them. For each of the categories outlined below, we provide an example. In our experience, the choice of approach to completing a methodological study can be informed by asking the following four questions:

• What is the aim?

A methodological study may be focused on exploring sources of bias in primary or secondary studies (meta-bias), or how bias is analyzed. We have taken care to distinguish bias (i.e. systematic deviations from the truth irrespective of the source) from reporting quality or completeness (i.e. not adhering to a specific reporting guideline or norm). An example of where this distinction would be important is in the case of a randomized trial with no blinding. This study (depending on the nature of the intervention) would be at risk of performance bias. However, if the authors report that their study was not blinded, they would have reported adequately. In fact, some methodological studies attempt to capture both “quality of conduct” and “quality of reporting”, such as Richie et al., who reported on the risk of bias in randomized trials of pharmacy practice interventions [ 80 ]. Babic et al. investigated how risk of bias was used to inform sensitivity analyses in Cochrane reviews [ 81 ]. Further, biases related to choice of outcomes can also be explored. For example, Tan et al investigated differences in treatment effect size based on the outcome reported [ 82 ].

Methodological studies may report quality of reporting against a reporting checklist (i.e. adherence to guidelines) or against expected norms. For example, Croituro et al. report on the quality of reporting in systematic reviews published in dermatology journals based on their adherence to the PRISMA statement [ 83 ], and Khan et al. described the quality of reporting of harms in randomized controlled trials published in high impact cardiovascular journals based on the CONSORT extension for harms [ 84 ]. Other methodological studies investigate reporting of certain features of interest that may not be part of formally published checklists or guidelines. For example, Mbuagbaw et al. described how often the implications for research are elaborated using the Evidence, Participants, Intervention, Comparison, Outcome, Timeframe (EPICOT) format [ 30 ].

Sometimes investigators may be interested in how consistent reports of the same research are, as it is expected that there should be consistency between: conference abstracts and published manuscripts; manuscript abstracts and manuscript main text; and trial registration and published manuscript. For example, Rosmarakis et al. investigated consistency between conference abstracts and full text manuscripts [ 85 ].

In addition to identifying issues with reporting in primary and secondary studies, authors of methodological studies may be interested in determining the factors that are associated with certain reporting practices. Many methodological studies incorporate this, albeit as a secondary outcome. For example, Farrokhyar et al. investigated the factors associated with reporting quality in randomized trials of coronary artery bypass grafting surgery [ 53 ].

Methodological studies may also be used to describe methods or compare methods, and the factors associated with methods. Muller et al. described the methods used for systematic reviews and meta-analyses of observational studies [ 86 ].

Some methodological studies synthesize results from other methodological studies. For example, Li et al. conducted a scoping review of methodological reviews that investigated consistency between full text and abstracts in primary biomedical research [ 87 ].

Some methodological studies may investigate the use of names and terms in health research. For example, Martinic et al. investigated the definitions of systematic reviews used in overviews of systematic reviews (OSRs), meta-epidemiological studies and epidemiology textbooks [ 88 ].

In addition to the previously mentioned experimental methodological studies, there may exist other types of methodological studies not captured here.

• 2. What is the design?

Most methodological studies are purely descriptive and report their findings as counts (percent) and means (standard deviation) or medians (interquartile range). For example, Mbuagbaw et al. described the reporting of research recommendations in Cochrane HIV systematic reviews [ 30 ]. Gohari et al. described the quality of reporting of randomized trials in diabetes in Iran [ 12 ].

Some methodological studies are analytical wherein “analytical studies identify and quantify associations, test hypotheses, identify causes and determine whether an association exists between variables, such as between an exposure and a disease.” [ 89 ] In the case of methodological studies all these investigations are possible. For example, Kosa et al. investigated the association between agreement in primary outcome from trial registry to published manuscript and study covariates. They found that larger and more recent studies were more likely to have agreement [ 15 ]. Tricco et al. compared the conclusion statements from Cochrane and non-Cochrane systematic reviews with a meta-analysis of the primary outcome and found that non-Cochrane reviews were more likely to report positive findings. These results are a test of the null hypothesis that the proportions of Cochrane and non-Cochrane reviews that report positive results are equal [ 90 ].

• 3. What is the sampling strategy?

Methodological reviews with narrow research questions may be able to include the entire target population. For example, in the methodological study of Cochrane HIV systematic reviews, Mbuagbaw et al. included all of the available studies ( n  = 103) [ 30 ].

Many methodological studies use random samples of the target population [ 33 , 91 , 92 ]. Alternatively, purposeful sampling may be used, limiting the sample to a subset of research-related reports published within a certain time period, or in journals with a certain ranking or on a topic. Systematic sampling can also be used when random sampling may be challenging to implement.

• 4. What is the unit of analysis?

Many methodological studies use a research report (e.g. full manuscript of study, abstract portion of the study) as the unit of analysis, and inferences can be made at the study-level. However, both published and unpublished research-related reports can be studied. These may include articles, conference abstracts, registry entries etc.

Some methodological studies report on items which may occur more than once per article. For example, Paquette et al. report on subgroup analyses in Cochrane reviews of atrial fibrillation in which 17 systematic reviews planned 56 subgroup analyses [ 93 ].

This framework is outlined in Fig.  2 .

A proposed framework for methodological studies

Conclusions

Methodological studies have examined different aspects of reporting such as quality, completeness, consistency and adherence to reporting guidelines. As such, many of the methodological study examples cited in this tutorial are related to reporting. However, as an evolving field, the scope of research questions that can be addressed by methodological studies is expected to increase.

In this paper we have outlined the scope and purpose of methodological studies, along with examples of instances in which various approaches have been used. In the absence of formal guidance on the design, conduct, analysis and reporting of methodological studies, we have provided some advice to help make methodological studies consistent. This advice is grounded in good contemporary scientific practice. Generally, the research question should tie in with the sampling approach and planned analysis. We have also highlighted the variables that may inform findings from methodological studies. Lastly, we have provided suggestions for ways in which authors can categorize their methodological studies to inform their design and analysis.

Acknowledgements

Abbreviations.

Authors’ contributions

LM conceived the idea and drafted the outline and paper. DOL and LT commented on the idea and draft outline. LM, LP and DOL performed literature searches and data extraction. All authors (LM, DOL, LT, LP, DBA) reviewed several draft versions of the manuscript and approved the final manuscript.

This work did not receive any dedicated funding.

Availability of data and materials

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

DOL, DBA, LM, LP and LT are involved in the development of a reporting guideline for methodological studies.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open Access is an initiative that aims to make scientific research freely available to all. To date our community has made over 100 million downloads. It’s based on principles of collaboration, unobstructed discovery, and, most importantly, scientific progression. As PhD students, we found it difficult to access the research we needed, so we decided to create a new Open Access publisher that levels the playing field for scientists across the world. How? By making research easy to access, and puts the academic needs of the researchers before the business interests of publishers.

We are a community of more than 103,000 authors and editors from 3,291 institutions spanning 160 countries, including Nobel Prize winners and some of the world’s most-cited researchers. Publishing on IntechOpen allows authors to earn citations and find new collaborators, meaning more people see your work not only from your own field of study, but from other related fields too.

Brief introduction to this section that descibes Open Access especially from an IntechOpen perspective

Want to get in touch? Contact our London head office or media team here

Our team is growing all the time, so we’re always on the lookout for smart people who want to help us reshape the world of scientific publishing.

Home > Books > Social Media and Modern Society [Working Title]

Effect of Advertising Effectiveness Via Mobile Social Networks (SNS) on Consumer Engagement and the Behavioral Intentions of Online Consumers

Submitted: 16 June 2024 Reviewed: 24 June 2024 Published: 27 August 2024

DOI: 10.5772/intechopen.1006165

Cite this chapter

There are two ways to cite this chapter:

From the Edited Volume

Social Media and Modern Society [Working Title]

Associate Prof. Ján Višňovský and Dr. Jana Majerová

Chapter metrics overview

1 Chapter Downloads

Impact of this chapter

Total Chapter Downloads on intechopen.com

This chapter aims to determine the impact of advertising effectiveness via mobile social network service (SNS) on consumer engagement. It also focuses on the treatment of the hypothetical relationship between advertising effectiveness and the behavioral intentions of online consumers. In this chapter, we will try to better understand the general concepts, namely mobile marketing, its tools, and its strategic implications. While relying on a theoretical framework, we will seek to study the direct links that exist between mobile marketing, mobile social media, mobile social networks, and social network services (SNS). A quantitative study was conducted on 430 Tunisian online consumers in the North African context who frequently use their mobile phones. The method of structural equations based on the AMOS approach was deployed for data analysis. The results reveal that the effectiveness of mobile advertising via social network services (SNS) positively affects consumer engagement. The results show that an effective advertising message transmitted via social networks has no effect on the purchase intentions of online consumers. This research fills the gap in the literature by examining the simultaneous impact of the effectiveness of advertising via social networks on brand equity and purchase intentions in a North African context that is still very rare.

• mobile marketing
• advertising
• social network services (SNS)
• advertising effectiveness
• consumer engagement
• consumer behavioral intentions

Author Information

Nesrine mzid *.

• Faculty of Economics and Management of Sfax, Department of ARBRE, Sfax, Tunisia

*Address all correspondence to: [email protected]

1. Introduction

Nowadays, the impact of new technologies, the evolution of consumer needs, and their implementation in the mobile marketing sector are considered important to academics and practitioners. In this context, it is very important to quickly reach consumers who are actively involved in digital environments. Similarly, many companies that want to stand out from their competitors and gain a competitive advantage aim to succeed using different digital marketing methods [ 1 ]. One of the evolving trends in mobile communications is mobile-based marketing [ 2 ].

Tunisia, with its 11,818,619 million inhabitants, is on the small side of e-commerce markets both in its region and in Africa as a whole [ 3 ]. The total number of Internet users in Tunisia was close to 8,170,000 million in December 2020, that is, 68.4% of the Tunisian population [ 3 ]. A report by the United Nations Conference on Trade and Development ranked Tunisia fourth among African countries and 79th in the world in terms of e-commerce [ 3 ]. This leads us to deduce that, based on the relevant data, the volume of trade carried out on the Internet and the number of companies operating on e-commerce marketplaces are increasing daily.

Faced with this situation, companies are increasingly open to these new electronic communication methods, which enable them to communicate with online consumers by broadcasting advertising messages via smartphones. They are also looking for effective ways to develop favorable attitudes toward mobile advertising. According to Hadi and Aslam [ 2 ], mobile advertising consists of persuading people to buy products and services using mobile devices [ 2 ]. It can be used to commit an online consumer to a brand [ 2 ].

In this respect, this form of advertising not only creates opportunities for marketers but also enables them to reach an unlimited number of consumers directly and quickly and to gain a better understanding of consumer attitudes toward messages transferred to smart devices as an effective new advertising medium. In this regard, we noted that the literature on social network services (SNS) and consumer engagement is still emerging [ 4 ].

Previous research investigated the impact of advertising effectiveness via social networks on brand value, brand image, and equity value as the main components of brand equity and did not consider the importance of consumer engagement [ 5 ].

In this case, we can declare the importance of this research, since it is part of the wider framework of research into advertising via social networks (SNS), which are the fundamental basis of technological development and provide companies and cyber consumers with a high level of virtuosity and up-to-date information. In this line of insufficiency, the study by Ha et al. [ 6 ] does not consider the impact of advertising effectiveness on the behavioral intentions of online consumers but only clarifies its impact on the advertising value and the attitude of online consumers.

We, therefore, review in search of the inherent consequences of advertising effectiveness on consumer engagement, we followed a broad research path, unlike Ha et al. [ 6 ], who followed a narrow research path that is limited to the attitudes of online consumers as consequences of advertising effectiveness. Therefore, our contribution consists of including other variables in our conceptual model, such as the consequences of advertising effectiveness on consumer engagement and the intentional outcomes of Tunisian online consumers in the context of advertising effectiveness via mobile social network services (SNS).

Guitart et al. [ 7 ] confirm the positive effect of effective social network advertising on consumer engagement. We also cite studies that focus on clarifying the effect of advertising effectiveness via social networks on the behavioral results of online consumers [ 8 , 9 , 10 , 11 , 12 , 13 ].

We note that this effect was tested in other countries, such as Korea, Pakistan, India, and the United States, according to Idris et al. [ 14 ]. However, our study focuses on the determination of this relational effect in Tunisia, a country in North Africa that remains unexplored.

Tunisia is a particularly interesting example of a developing country in North Africa. The use of mobile advertising via social network services (SNS) in developing countries is comparatively lower than in most developed countries. This research is particularly necessary in the context of African developing countries such as Tunisia, where companies adopt less ICT than companies from developed Western countries. This research aims to answer this central question: to what extent does advertising effectiveness via mobile social media services (SNS) affect consumer engagement that may influence the behavioral intentions of online consumers?

In this chapter, we present the theoretical framework of the research relative to the general concept of mobile marketing. To do this, we reviewed previous work relating to this notion in order to better understand it.

Indeed, when briefly examining this work, we noted a lack of consensus regarding the definition of mobile marketing.

The structure of the remainder of this chapter is as follows: first, we discuss the main theories underlying our proposed research model and the review of the existing literature. Then, we formulate our research hypotheses. Secondly, we explain the main methods used to collect the data. Third, we present the main results of this research. Finally, we develop and discuss the results and present the theoretical and managerial implications, limits, and future research avenues.

2. Literature review

2.1 mobile advertising.

Mobile advertising offers marketers the opportunity to promote their products and services in a personalized and interactive way. Thus, advertising content can be personalized based on private data obtained from consumers such as demographic profile, customer purchasing behavior, time, location, search history, etc. [ 15 , 16 , 17 , 18 , 19 ]. All of this data can create significant business opportunities for companies. Under this highlight, researchers have clarified mobile advertising as a new form available through mobile media that can be used to attract the attention of consumers, in order to strengthen the positive connection with their favorite brands. Mobile advertising is defined by Mzid et al. [ 20 ] as one of the message communication techniques that has appeared on all mobile devices such as smartphones and tablets.

2.2 Advertising

According to Lolowang et al. [ 9 ], advertising is all paid forms of presentation and non-personal promotion of ideas, goods, or services by certain sponsors. To this definition is added that of Abbasi et al. [ 21 ] who show that the central principle of advertising research is the advertising value as perceived by consumers.

2.3 Mobile social network services (SNS)

Filho et al. [ 22 ] suggest that hyperconnectivity is generated by consumers’ desire to be online 24 hours a day through social media. In their study, Suki and Suki [ 23 ] define mobile social networks as “all web services that allow individuals to create a public or semi-public profile, establish social relationships with other users and share all similar interests, view and browse their list of connections, and those established by others within the system”.

2.4 Advertising effectiveness

To study the effectiveness of advertising via mobile social network services (SNS), we focus on the contribution of the integrator model of factors that affect the effectiveness of advertising on the Web, as recommended by Ha et al. [ 6 ]. Lutfie and Marcelino. [ 24 ] present advertising effectiveness as “a cost-effective means of delivering the message, either to develop brand preferences or to educate the consumer”.

2.5 Consumer engagement

Consumer engagement is one of the most highly discussed topics in the field of marketing [ 4 , 15 , 16 , 17 , 21 , 25 ]. This literature review helps us understand that commitment can be expressed to a virtual community, product, service, and brand. In light of our research, we focus our attention on the last type of commitment, that is defined as “a user’s ability to spread their experiential perspective with a brand on social media” [ 18 ].

2.6 Purchase intentions

Purchase intention is a salient concept widely known and vigorously studied by marketing researchers, because it can directly affect consumer behavior [ 9 , 11 , 19 , 20 , 26 ]. A consumer’s purchase intentions indicate their desire to buy a product or service from a given brand [ 8 , 14 , 27 ].

3. Research framework and hypotheses

This research attempts to contribute to the existing body of research by introducing both concepts simultaneously: consumer engagement and purchase intentions, to study the effectiveness of Mobile Advertising via Social Network Services (SNS) in Tunisia, an Arab and developing country where research has been relatively scarce until today. In the following, the conceptual model will be presented ( Figure 1 ).

Research model.

3.1 Effect of mobile advertising effectiveness via SNS on consumer engagement

The degree of consumer engagement with the brand is known as the attitude, behavior, level of interaction, and grouping of online businesses with their customers [ 28 ]. The results of the factorial analysis by Saeed et al. [ 29 ] revealed a significant positive association between social media marketing factors, brand image, electronic word of mouth, customer engagement, trust, and consumer satisfaction. Similarly, Dwiviolita and Zuliarni [ 30 ] point out that the company recognized the importance of mobile applications as an essential promotional tool for increasing consumer engagement. Gutierrez et al. [ 31 ] highlight the interactive nature of social media as a platform for effective consumer-brand interaction (CBI), a phenomenon that is proving to be an important driver in strengthening the relationship between consumers and brands.

In their works, Gutierrez et al. [ 31 ] explain that social media provide brands with a platform for interaction, and if the advertising is perceived as corresponding to the brand’s image, we can expect more engagement. In this regard, we cite the study by Choedon and Lee [ 16 ], which showed that investing in social media to disseminate effective advertising content can improve consumer engagement with the brand in the long term.

A study by Warbung et al. [ 32 ], found that social media marketing has a significant impact on perceived quality through various mechanisms, such as online recommendations, brand reputation, and consumer engagement with the brand. For their part, Deng et al. [ 33 ] identify six linguistic characteristics on social media, namely publication length, language complexity, visual complexity, emotional signals, interpersonal signals, and multimodal signals that influence brand engagement.

The use of this form of online sharing and interactivity is necessary in all sales situations, particularly when the customer is engaged in a complex decision-making phase that requires the help of a brand-linked web page, capable of facilitating consumer-brand interaction without any constraints of time, place, or medium to increase customer satisfaction. Naidoo [ 34 ] confirms the positive link between the effectiveness of mobile advertising via SNS and brand engagement, hence sub-hypothesis H1–3.

H1: Mobile advertising effectiveness via SNS has a positive effect on consumer engagement.

3.2 Effect of the effectiveness of mobile advertising via SNS on the purchase intentions of online consumers

The results of the study by Lolowang et al. [ 9 ] indicate that the online advertising variable has a positive and significant effect on the purchase intentions variable. Alcaraz et al. [ 11 ] state that advertising on social networks (SNS) is known for its low costs; it therefore allows start-ups to launch various promotions that increase purchase intentions. Moreover, this result is in line with the study of Meliawati et al. [ 27 ], who confirm that social media marketing, and more specifically TikTok, affects purchase intentions. In Gutierrez 2023’s study [ 31 ], the results show that brands need to build strong relationships through high-quality consumer-brand interactions to significantly increase purchase intentions while carefully managing consumers’ privacy expectations.

Kim and Park [ 35 ] state that social media influencers positively affect brand image, and consumer purchase intentions can be significantly influenced by the attractiveness of digital influencers. We also refer to the study by Ebrahim [ 36 ], who shows that social media activities affect brand awareness and image, and consequently consumer behavior.

As for Hasim et al. [ 12 ], they point out that the effectiveness of social media, and Instagram in particular, has a positive impact on the purchase intentions of online consumers and on brand equity. These findings seem to highlight the mediating role played by brand attitude between the effectiveness of online advertising and purchase intentions. We also highlight the study of De Keyzer et al. [ 37 ], who examine the relationship between personalized advertising perceived via social networking sites (SNS) and consumer responses to the brand. This indirect link is mediated by four variables, namely perceived self-efficacy, perceived intrusiveness, brand attitude, and click intentions. These results contradict those of Shaouf et al. [ 38 ] and Zhu [ 39 ], who find a negative correlation between advertising effectiveness and purchase intentions. Based on these various results, we can predict that mobile advertising effectiveness via the Social Network Service (SNS) has a positive effect on the purchase intentions of online consumers, hence the following hypothesis:

H2: The effectiveness of mobile advertising via SNS has a positive effect on the purchase intentions of online consumers.

Figure 1 illustrates the research model.

4. Research methodology

4.1 data collection.

We retain the assertion of Carricano and Poujol [ 40 ], who stipulate that “the population to be studied must be defined with the greatest care. We agree with these two authors, that “an appropriate choice of sampling method can make our survey a success”. We, therefore, chose people who receive advertising content on their phones and who frequently visit websites via social networks (SNS).

To this end, we followed a convenience sampling procedure, based on people with mobile experience (at least one year’s experience) and age 18 and above as the two criteria for selecting survey respondents. We also followed studies in this field that included people with mobile experience. This prerequisite is applied in almost all studies that deal with the importance of mobile advertising via social networks. The questionnaire is aimed primarily at young people (under 30) living in Tunisia (North Africa).

At the same time, we made sure to reach diverse socio-professional categories, while guaranteeing the diversity of answers collected on the questions asked. We also included gender, age, family status, and professional social category as criteria for diversifying our sample.

Nevertheless, as part of our research, we created our online survey using Google Forms. Respondents were sent a link to a Google Forms questionnaire, opened for the first six weeks of June 2021, via Facebook. Each participant took around 10 minutes to complete the questionnaire. Data collection lasted one month. We pointed out that we took advantage of the containment period (COVID 19) to send out more questionnaires. Our sample is made up of craftsmen, professionals, employees, students, managers, the retired, company directors, shopkeepers, and the inactive.

Subsequently, to determine the exact number of our sample, we based ourselves on the words of Dalmas [ 41 ], who suggests that “the sample size should include 5 to 10 times as many individuals as there are items subject to the same factorial analysis” [ 41 ]. This rule is a strict constraint that must not be exceeded in the context of our research [ 41 ]. Hence, given this calculation range, we combine 5 observations to obtain a matrix composed of 64*5 = 320 people, which will serve as the basis for our exploratory and confirmatory factorial analyses.

We note that most of the respondents interviewed were young people (63.3%) aged under 30. We note that the sample belongs to categories of women representing (51.9%) compared to men (48.1%). Most of the respondents were single women (56.5%). This is explained by the fact that this category has more free time to browse anytime and anywhere without any worries, unlike married people who do not even find the time to access websites. These descriptive characteristics of the two samples are summarized in Table 1 .

Descriptive characteristics of the sample.

In the following, we will present the data analysis methods used in our research. First, we relied on the method of exploratory factor analysis to explore the relationship between the measured variables and to purify the measurement scales [ 40 ]. Then, we began a confirmatory factor analysis to confirm and validate the measurement model and the measures adopted for the different concepts.

In our research, we opt for the first method whose choice is based on the words of Akrout [ 42 ], who stated that “principal component analysis contributes to the common variance between observable variables”. The second method is generally used to identify the latent variables that contribute to the common variance between the observable variables [ 43 ]. Thus, this method is based on the structural equation method (SEM) with the maximum likelihood (ML) estimation method. We subsequently used AMOS 20 software to determine whether the proposed model fits well with the data or not. If not, the proposed measurement model is rejected.

4.2 Content of the questionnaire

We used a pre-test method for our experimental version of the questionnaire, distributing the original version (without any modifications) to ten or so consumers before testing the respondent for confusion or ambiguity about the items. We also told respondents that their help will be important for our survey and that this medal of rigorous answers gives a great chance of having better reliability of answers and reduces the bias due to non-response. Respondents were diverse across the board (age, education, gender, and industry) to ensure our sample would be diverse.

Most respondents were involved in our survey. Based on their recommendations, we made a few adjustments to the meaning of some vocabulary and original items. We also point out that the interviewees did not hesitate to express their opinion on the number of scales. They felt it was better to reduce the number of scales to 7 points because of the length of the questionnaire. We based ourselves on the comments of Assaidi et al. [ 44 ], who recommended using 5-point scales when a survey has more than 60 items. We also mention the study of Dalmas [ 41 ], who states that a questionnaire with more than 100 items will be very heavy and run many risks, such as bias, the Halo effect, response polarization, contamination, and missing data.

We reduced the number of items to make the questionnaire “lighter”, “shorter”, and “easier” for respondents to answer. We also noted that several respondents found it very difficult to distinguish between the different levels. A source of confusion and even incomprehension was repeated several times by other interviewees who asked the same questions about the meaning of these scales. We also noted that there was a lack of understanding of the exact position of the scales, which differed from one person to another. For this reason, we decided to retain the comments that were unanimous. We then decided to distribute the questionnaire to other people to ensure that there were no misunderstandings.

In the first part of the questionnaire, we clarified and redefined the various key concepts related to our research objective so that the respondent would have a clear idea of the theme and could easily answer the questions posed. From the first question to the fourth, we set out to study the effect of advertising effectiveness on consumer engagement. The last question is devoted to studying the behavior of online consumers who frequently use their mobile phones to purchase goods or services. The last part of the questionnaire was devoted to the respondent’s personal characteristics, that is, name, gender, age, PSC, current marital status, etc. These data will help us describe our sample. We also specify that our questionnaire includes 19 items.

4.3 Measurement instruments

Three constructs were measured in this research: advertising effectiveness, consumer engagement, and the purchase intentions of online consumers. We chose the measurement scales based on three principles: the conceptualization of each variable, the research objective, and the reliability of the scales [ 44 ].

The measurement scales used in this research are interval scales and, more specifically, 5-point Likert-type scales ranging from “strongly disagree” to “strongly agree”.

3 items for advertising effectiveness inspired by the Martins [ 45 ] measurement scale.

12 items measure consumer engagement based on the Helme-Guizon and Magnoni [ 46 ] measurement scale.

4 items to measure purchase intentions adapted from the Ha [ 6 ] scale. (See  Table A1 ).

4.4 Analysis of the measurement model

A confirmatory factor analysis (CFA) was conducted to empirically test the measurement model. We used structural equation methods (SEM) with maximum likelihood (ML) estimation. At this level, the standardized regression coefficients, explained variances (SMC), construct variances, critical ratio (CR), and error variances must be checked for each construct. The results show that all the variables have standardized regression coefficients greater than 0.7 and are significant ( Table 2 ).

Reliability and validity indices.

We also find that all the variables have good reliability (CR) > 0.6. Based on Joreskog’s Rho, we find that all the constructs are verified since they are greater than 0.6. Next, we check the convergent validity of our measurement model. According to Akrout [ 43 ], convergent validity is verified when the Rho of convergent validity is greater than or equal to 0.6 and the VME [ 43 ]. These authors admitted that the measurement of convergent validity is based on compliance with two criteria, such as the result of the t-test (cr = critical ratio), which must be significant and greater than 1.96, and the average variance extracted (VCM), which must be >0.5 [ 43 ]. Our results are consistent with these recommendations since the AVE Rho represents a value below the significant threshold of 0.5; hence, the convergent validity of our measurement model is verified. The results are shown in Table 2 .

We now test the discriminant validity of the overall measurement model. We rely on Amamou and Koubaa [ 47 ], who state that “this test is designed to be used as a discriminant validity assessment tool”. For discriminant validity, we checked that the square of the correlation (R2) is lower than the lowest mean variance extracted (VME), following the approach of Fornell and Larker [ 43 ].

Nevertheless, according to our analyses using AMOS software, we noticed that the square root of the AVE of some constructs exceeds its correlation with the other constructs in the model. This means that these constructs are highly correlated with the other constructs. Hence, this condition is not verified because the structures are not systematically adjacent. In conclusion, our measurement scales do not have complete discriminant validity, but this does not provide a departure from the overall measurement model, according to the words of Chemangui [ 48 ], who state that “it is difficult to aim for absolute validity” [ 48 ]. Table 3 illustrates all the discriminant validity results.

Test of the discriminant validity of the global measurement model.

Convergent validity of each construct.

* Square of the correlation between two dimensions.

After this verification, we will then check the fit indices provided by the AMOS 20 software to determine whether the proposed model fits the data well or not. Furthermore, if this is not the case, rejection of the measurement model is proposed. We also note that the evaluation of the overall measurement model and structure must be based on the Chi-Square statistic (χ 2 ), c.r < 1.96, the parsimony index (χ2/ddl), the absolute measurement indices (RMR, GFI > 0.9, AGFI >0.9 and RMSEA<0.08), and the incremental measurement indices (CFI > 0.9 and NFI > 0.9, TLI > 0.9). We note that most of the indices are above the standards. Table 4 summarizes all the fit indices of the models respecting the recommended thresholds.

The adjustment indices of the research and structure model.

4.5 Testing hypotheses

Research hypotheses H1 is verified. The results show that advertising effectiveness is linked by a direct effect to consumer engagement (β = 0.822, p < 0.005).

The results obtained in relation to hypothesis H.2 reject the existence of a significant effect of advertising effectiveness via social network services on purchase intentions. This result is opposed to the study of Lolowang et al. [ 9 ], who confirm that online advertising has a positive and significant effect on purchase intentions. On the other hand, this result is in line with the study of Ahmad and Mahmood [ 49 ] who state that the number of advertising exposures must be considered to achieve advertising effectiveness. They state that it is difficult to change the purchase intentions of online consumers with a single advertising exposure. This leads us to infer that a logical level of exposure creates viewers’ behavioral flexibility about online advertising, with the aim of achieving a high level of intentions.

Table 5 summarizes all the results related to the verification of research hypotheses.

Statistical tests of hypotheses.

5. Discussion

This study focuses on a new direction that addresses the relationship between advertising effectiveness, consumer engagement, and purchase intentions of online consumers. Regarding the impact of advertising effectiveness on consumer engagement, this result is consistent. This result stresses that companies must be committed to investing in improving their virtual communication activities in order to popularize the brand, create a competitive advantage in a dynamic environment, and offer social benefits for users.

Consumer engagement proved to be the ability of a user to broadcast their experiential perspective with a brand on social media [ 50 ]. Based on the results, we were able to verify the positive effect of advertising effectiveness on consumer engagement. These results enabled us to establish that consumer engagement is essentially based on the ability of effective, interactive, and adaptive advertising to recall and activate the action of brand recall. This result is in line with the study of Choedon and Lee [ 16 ], who state that investing in social media to deliver effective advertising content can improve the engagement of online consumers in the long term. This result is backed up by Abbassi et al. [ 21 ] and Ao et al. [ 25 ], who treated this relationship, confirming the positive effect between these two constructs. Finally, regarding the second hypothesis, an examination of the structural links shows that there is a non-significant link between advertising effectiveness and the purchase intention of online consumers. This result shows that an effective advertising message does not affect the behavioral intentions of online consumers. This result contrasts with those of Lolowang et al. [ 9 ], Meliawati et al. [ 27 ], and Fink et al. [ 51 ], who confirmed the significant effect between the two constructs. This result is in line with the findings of Manan et al. [ 52 ], who find that the two dimensions of social media advertising (credibility and authenticity) do not influence purchase intentions [ 52 ]. This result is in line with the study of Yusof et al. [ 53 ], who found that green advertising does not have a significant influence on purchasing behavior [ 53 ]. This result seems logical to us, if we also follow the study of Ahmad and Mahmood [ 49 ], who state that to achieve advertising effectiveness, we need to pay attention to the number of advertising exposures. This leads us to infer that a logical level of advertising exposure creates a sense of enjoyment and flexibility in the receiver. Furthermore, according to Hanaysha [ 54 ], the entertaining nature of advertising via social network services (SNS) improves users’ confidence but does not necessarily affect their purchase intention. This means that when people look for brand-related content to entertain themselves, relax and spend their time, they will not intend to buy that brand’s products. We concluded that the degree of enjoyment of advertising content does not necessarily affect the purchase intentions of online consumers. This result corroborates Al Koliby and Rahman [ 55 ] conclusion that insufficient information about a brand does not affect online consumers’ purchase intentions toward that brand [ 55 ].

6. Theoretical implications

Based on advanced previous research that focuses on mobile advertising, we saw a growing interest from practitioners and researchers in understanding social media advertising (SNS), in which our study fits. The investigation in this field gave rise to several disciplines to study the phenomenon of advertising effectiveness via social network services (SNS). Little research was invested in explaining the effectiveness of advertising via social networking services (SNS), particularly in the North African context. Consequently, our research has come to address this shortcoming. It seeks to uncover the consequences of advertising effectiveness on consumer engagement on the one hand and behavioral intentions on the other by combining the conceptual models of Ha et al. [ 6 ] and Alhaddad [ 56 ]. The model advanced by Ha et al. [ 6 ], did not consider the effect of advertising effectiveness on consumer engagement but only clarified its impact on online consumers’ attitudes. However, the model advanced by Alhaddad [ 56 ] focuses on the analysis of advertising effectiveness on consumer engagement. This merger allowed us to discover and integrate new constructs into our research model that helped enrich the existing literature.

Similarly, we integrated new notions into our model that encompass all the concepts that are related to mobile advertising via SNSs, such as brand value, brand image, brand commitment, and equity value, while trying to find a better understanding of each term used by various authors in the literature. Hence our first contribution, which consists of enriching the digital marketing literature by rethinking mobile advertising via social network services (SNS). Our research contributed to the construction of a model that integrates consumer engagement which is affected by advertising effectiveness through SNS. This enabled us to clarify the link and complementarity between advertising effectiveness and consumer engagement, which is a field of research that is still unexplored and is particularly in need of clarification.

The contribution of our research also lies in the treatment of the causal relationships between advertising effectiveness and the behavioral intentions of online consumers. Moreover, the confirmation of our theoretical results through empirical analysis enabled us to demonstrate the importance of the virtual interaction between the online consumer and the brand, as well as to demonstrate the principle of the presence of a relationship between the effectiveness of mobile advertising via social networks and the purchase intentions of online consumers. Based on the theory of brand equity Keller [ 57 ] and the hierarchy of the effects model [ 14 ], our study extended the model proposed by Ha et al. [ 6 ] by incorporating the inherent consequences of advertising effectiveness on consumer engagement. Our research provided insight into the consequences of advertising effectiveness via SNS, which were not addressed by previous research. To address this shortcoming, we proposed to investigate the relationship between advertising effectiveness and consumer engagement, both of which are responsible for creating long-term economic wins. Moreover, there is a theoretical gap as to the nature of this relationship. Most researchers studied either users’ perceptions of advertising, the consequences of advertising effectiveness, or the behavioral intentions of online consumers, but they did not tackle these subjects simultaneously.

6.1 Practical implications

From a managerial point of view, this research offers advertisers, companies, managers, and all those involved in the field of mobile advertising in general and advertising via social network services (SNS) in particular––possible solutions for understanding the consequences of the effective creation of advertising content on consumer engagement and the behavioral intentions of online consumers. However, effective, entertaining, up-to-date, and informative ad creation triggers a strong editorial construction of consumer engagement.

In this sense, managers need to adopt real virtual techniques that enable advertisers to abandon traditional techniques and replace them with others that are likely to offer a strong bond with mobile users and maintain a behavior that enables them to remain loyal and buy the same brand repeatedly. We, therefore, recommend encouraging companies that rely on this online creativity to create an active commitment from their subscribers so that a strong emotional attachment to the brand is created, and consequently, their purchase intentions will increase. Indeed, the effective creation of advertising content via social network services (SNS) has a positive and significant effect on consumer engagement.

Finally, to increase the intentional levels of online consumers, we suggest that managers humanize their brand on social networks by adapting the brand’s corporate elements, that is, the logo, colors, shapes, and design, to get closer to their virtual community. The principle is, therefore, to move beyond a bland tone toward an approximate accessibility that could be the main reason for falling in love with a brand. Hence, strong love will create strong brand equity, which in turn will lead to high purchase intentions. If advertisers and managers carefully study the inherent consequences of advertising effectiveness, they can help companies engage consumers and create strong brand purchase intentions.

6.2 Limitations and future research directions

Like all research, there are some limitations to this study. First, we have used a single source of information, namely online consumers who frequently use their mobile phones to access online mobile social network services (SNS). Future research should use other sources of information, such as project managers, companies, and communications agencies. Secondly, another limitation relates to the dissemination of the research questionnaire––it seems that most reactions came from interactions on a single tool, and more specifically, Facebook. Further research will attempt to clarify this concept in other offline media, as this area of research is still vast and there are still many avenues to explore. Another limitation concerns the failure to take account of actual purchasing behavior because of advertising effectiveness. In this context, we propose that future interest be given to actual purchasing decisions. Finally, other variables may exert an influence on the hypothetical relationships tested. For example, Martí Parreño et al. [ 58 ] found that age can modify the acceptability of such a mobile advertising campaign. We propose that future research consider the moderating effect of socio-demographic variables.

Acknowledgments

The authors would like to thank their colleagues, the colleagues who distributed the questionnaire to their students, as well as the students who took the time to complete it online.

Conflict of interest

The authors declare no conflict of interest.

Notes/thanks/other declarations

I thank everyone for their relevant suggestions, which improved the quality of my work.

Measurement scales and questionnaire structure.

• 1. Akgün VÖ. Marketing mentality of the modern age: Digital marketing. In: 5th International Academic Conference on Management and Economics; July 22-24, 2022; Denmark. Denmark: ConferenceME; 2022. pp. 15-30. DOI: 10.33422/5th.conferenceme.2022.07.25
• 2. Hadi NU. Demographic factors and consumer attitude towards unsolicited mobile-based marketing messages: A factorial design. Online Journal of Communication and Media Technologies. 2023; 13 (1):1-15. DOI: 10.30935/ojcmt/12784
• 3. Nasri W. Determinants of consumer online purchasing intention: An empirical study in Tunisia. International Journal of Cyber Research and Education (IJCRE). 2022; 4 (1):1-16. DOI: 10.4018/IJCRE.309689
• 4. Fetais AH. Do social media marketing activities improve brand loyalty? An empirical study on luxury fashion brands. Information Systems Frontiers. 2023; 25 (2):795-817. DOI: 10.1007/s10796-022-10264-7
• 5. Algharabat RR. Investigating the antecedents of customer brand engagement and consumer-based brand equity in social media. Journal of Retailing and Consumer Services. 2020; 53 :1-13. DOI: 10.1016/j.jretconser.2019.01.016
• 6. Ha YW. A framework for mobile SNS advertising effectiveness: User perceptions and behaviour perspective. Behaviour & Information Technology. 2014; 33 (12):1333-1346. DOI: 10.1080/0144929X.2014.928906
• 7. Guitart IA. Advertising non-premium products as if they were premium: The impact of advertising up on advertising elasticity and brand equity. International Journal of Research in Marketing. 2018; 35 (3):471-489. DOI: 10.1016/j.ijresmar.2018.03.004
• 8. AL-Sous NA. Antecedents of social media influencers on customer purchase intention: Empirical study in Jordan. International Journal of Data and Network Science. 2023; 7 (1):125-130. DOI: 10.5267/j.ijdns.2022.11.010
• 9. Lolowang IL. The effect of online advertisement on purchase intention mediated by brand Atitude (study at Pt Akr land Kawanua Emerald City Manado). International Journal of Business Diplomacy and Economy. 2023; 2 (1):13-19. DOI: 10.51699/ijbde.v2i1.841
• 10. Hussain AH. Integrating the SOR model to examine purchase intention based on Instagram sponsored advertising. Journal of Promotion Management. 2023; 29 (1):77-105. DOI: 10.1080/10496491.2022.2108185
• 11. Alcaraz AG. A study on utilizing social media advertising through “Instagram aesthetics” to drive purchase intention. Journal of Business and Management Studies. 2022; 4 (1):167-178. DOI: 10.32996/jbms.2022.4.1.20
• 12. Hasim MA. Influences of media richness on instagram towards consumer purchase intention: The mediating effect of brand equity. International Journal of Innovation, Creativity and Change. 2020; 10 (11):357-367. DOI: 10.31933/dijemss. v1i6.556
• 13. Saputro Y. Television advertising effectiveness in mediating advertising creativity and advertising frequency influence toward customer purchase intention. Management Analysis Journal. 2018; 7 (2):201-210. DOI: 10.46336/ijlcb. v1i4.25
• 14. Idris IS. Factors that influence the effectiveness of online advertising in enhancing consumer’s purchase intention among young adults in Malaysia. TEST Engineering & Management. 2020; 82 :5528-5536
• 15. Aziz MA. Consumer brand identification and purchase intentions: The mediating role of customer brand engagement. Journal of Entrepreneurship and Business Venturing. 2023; 3 (1):221-239. DOI: 10.56536/jebv.v3i1.38
• 16. Choedon Y-CL. The effect of social media marketing activities on purchase intention with brand equity and social brand engagement: Empirical evidence from Korean cosmetic firms. Knowledge Management Research. 2020; 21 (3):141-160. DOI: 10.15813/kmr.2020.21.3.008
• 17. Sohail MS. Understanding consumer engagement in online brand communities: An application of self-expansion theory. Journal of Marketing Analytics. 2023; 11 (1):69-81. DOI: 10.1057/s41270-021-00148-1
• 18. Hollebeek LD. Exploring a customer engagement spillover effect on social media: The moderating role of customer conscientiousness. Internet Research. 2023; 19 (1):1-39. DOI: 10.1108/INTR-08-2021-0619
• 19. Ahmed RR. Effect of social media marketing of luxury brands on brand equity, customer equity and customer purchase intention. Amfiteatru Economic. 2023; 25 (62):265-282. DOI: 10.24818/EA/2023/62/265
• 20. Fialho J. Brand equity and purchase intention for e-commerce platforms in Vietnam. In: ICASDMBW 2022: Proceedings of the International Conference on Application of AI and Statistical Decision Making for the Business World; December 16-17, 2022; Delhi, India. Delhi: European Alliance for Innovation (EAI); 2023. pp. 1-19. DOI: 10.4108/eai.16-12-2022.2326203
• 21. Abbasi AZ. Investigating the impact of social media images’ value, consumer engagement, and involvement on eWOM of a tourism destination: A transmittal mediation approach. Journal of Retailing and Consumer Services. 2023; 71 (1):1-53. DOI: 10.1016/j.jretconser.2022.103231
• 22. Filho Arruda EJ. Hyperconnectivity in action: Users of mobile social networks and new technologies. Revista de Gestão. 2023; 2 (30):162-175. DOI: 10.1108/REGE-03-2021-0048
• 23. Suki NM. Examining Users' Trust in the travel information provided in social networking services (SNS). Journal Pengurusan. 2019; 57 :65-74. Available from: http://ejournal.ukm.my/pengurusan/issue/view/1231
• 24. Lutfie H. Creating online purchase decision and brand image based on advertising effectiveness with epic model. Journal Ilmiah Manajemen. 2020; 10 (3):346-366. DOI: 10.22441/mix.2020.v10i3.003
• 25. Ao LB. Impact of social media influencers on customer engagement and purchase intention: A meta-analysis. Sustainability. 2023; 15 (3):1-15. DOI: 10.3390/su15032744
• 26. Pratama AA. The effect of halal brand awareness on purchase intention in Indonesia: The mediating role of attitude. Cogent Business & Management. 2023; 10 (1):1-18. DOI: 0.1080/23311975.2023.2168510
• 27. Meliawati TG. The effect of social media marketing Tiktok and product quality towards purchase intention. Journal of Consumer Sciences. 2023; 8 (1):77-92. DOI: 10.29244/jcs.8.1.77-92
• 28. Kumar V. Competitive advantage through engagement. Journal of Marketing Research. 2016; 53 (4):497-514. DOI: 10.1509/jmr.15.0044
• 29. Saeed SS. Examining the role of advertising and sales promotion in brand equity creation of tea industry in Pakistan: With moderating effect of brand experience. Journal of Academic Research for Humanities. 2023; 3 (1):170-185 Available from: https://jar.bwo.org
• 30. Dwiviolita S. Customer engagement, customer equity, and their influence on consumer repurchase intention in E-commerce Mobile applications. Journal Dinamika Manajemen. 2023; 14 (1):55-71 Available from: https://journal.unnes.ac.id/nju/jdm/article/view/38657
• 31. Gutierrez AP. Retailers, don't ignore me on social media! The importance of consumer-brand interactions in raising purchase intention-privacy the Achilles heel. Journal of Retailing and Consumer Services. 2023; 72 (1):1-39. DOI: 10.1016/j.jretconser.2023.103272
• 32. Warbung CJ. The impact of social media marketing on beauty clinic brand equity: The case of zap Manado. International Journal of Professional Business Review. 2023; 8 (4):1-26. DOI: 10.26668/businessreview/2023.v8i4.1389
• 33. Deng QW. Speak to head and heart: The effects of linguistic features on B2B brand engagement on social media. Industrial Marketing Management. 2021; 99 :1-15. DOI: 10.1016/j.indmarman.2021.09.005
• 34. Naidoo T. The effectiveness of advertising through social media in Gauteng. Dissertation submitted in partial fulfilment of the requirements for the degree Master of Business Administration at the Potchefstroom Campus of the North-West University. Supervisor: Prof. CA. Bischoff. Potchefstroom, South Africa: North-West University; November 2011. pp. 1-100
• 35. Kim H. Virtual influencers attractiveness effect on purchase intention: A moderated mediation model of the product–endorser fit with the brand. Computers in Human Behavior. 2023; 143 :107-137. DOI: 10.1016/j.chb.2023.107703
• 36. Ebrahim RS. The role of trust in understanding the impact of social media marketing on brand equity and brand loyalty. Journal of Relationship Marketing. 2020; 19 (4):287-308. DOI: 10.1080/15332667.2019.1705742
• 37. De Keyzer FD. How and when personalized advertising leads to brand attitude, click, and WOM intention. Journal of Advertising. 2022; 51 (1):39-56. DOI: 10.1080/00913367.2021.1888339
• 38. Shaouf AL. The effect of web advertising visual design on online purchase intention: An examination across gender. Computers in Human Behavior. 2016; 100 (60):622-634. DOI: 10.1016/j.chb.2016.02.090
• 39. Zhu B. The impact of green advertising on consumer purchase intention of green products. World Review of Business Research. 2013; 3 (3):72-80. DOI: 10.2139/ssrn.2182906
• 40. Carricano MP. Analyse de données avec SPSS. France: Pearson Education France; 2010 Available from: https://mcours.net/cours/pdf/info1/Analysede_donnees_xtrgw.pdf
• 41. Dalmas M. Quelles valeurs organisationnelles pour la génération Y? Revue management et avenir. 2014; 72 (6):113-132. DOI: 10.3917/mav.072.0113
• 42. Akrout F. Les méthodes des équations structurelles 2, 1ère Édition. Tunis: Imprimerie Coopi; 2010 Available from: https://www.academia.edu/36441842/Fathi_Akrout_Les_m%C3%A9thodes_des_Equations_Structurelles_extrait_du_livre_2018_pdf
• 43. Akrout F. Les méthodes des équations structurelles, 2. éme ed. Sfax: Imprimerie Coopi; 2018 Available from: https://www.amazon.fr/m%C3%A9thodes-%C3%A9quations-structurelles-Fathi-Akrout/dp/9938007309
• 44. Assaidi AK-S. Générations Baby-Boomers, X et Y: Les barrières entrepreneuriales des entrepreneurs dans le Nord de la France. Gestion. 2000; 33 (4):83-106. DOI: 10.3917/g2000.324.0083
• 45. Martins JC. How smartphone advertising influences consumers’ purchase intention. Journal of Business Research. 2019; 94 (3):378-387. DOI: 10.1016/j.jbusres.2017.12.047
• 46. Helme-Guizon A. Intégrité, bénéfices et identification: trois leviers pour booster l’engagement envers la marque sur les réseaux sociaux. Décision Marketing. 2016; 84 (4):95-11. DOI: 10.7193/DM.084.95.113
• 47. Amamou B. Communautés de pratique et capacité d’absorption des connaissances dans un contexte inter-organisationnel: cas des PME marocaines. Revue Internationale PME. 2013; 26 (3):13-39. DOI: 10.7202/1024517ar
• 48. Chemangui M. Evaluation de la qualité de l'Audit interne/Audit externe: Application dans le cadre des relations d'agence internes. In: COMPTABILITE, CONTROLE, Audit et Institution (S). Tunis: Association Francophone de Comptabilité; 2006. pp. 1-25
• 49. Ahmad W. An empirical investigation of the association between creative advertising and advertising effectiveness in Pakistan. International Journal of Marketing Studies. 2011; 3 (2):32-52. DOI: 10.5539/ijms.v3n2p32
• 50. Verma P. The effect of brand engagement and brand love upon overall brand equity and purchase intention: A moderated–mediated model. Journal of Promotion Management. 2021; 27 (1):103-132. DOI: 10.1080/10496491.2020.1809591
• 51. Fink MK. Effective entrepreneurial marketing on Facebook–a longitudinal study. Journal of Business Research. 2020; 113 (3):149-157. DOI: 10.1016/j.jbusres.2018.10.005
• 52. Manan HA. Effects of social media advertisements on intention to purchase health and beauty products. Economics and Entrepreneurship. 2020; 5 (1):59-68
• 53. Yusof JM. Purchase intention of environment-friendly automobile. Procedia-Social and Behavioral Sciences. 2013; 85 :400-410. DOI: 10.1016/j.sbspro.2013.08.369
• 54. Hanaysha JR. Impact of social media marketing features on consumer's purchase decision in the fast-food industry: Brand trust as a mediator. I. International Journal of Information Management Data Insights. 2022; 2 (2):1-10. DOI: 10.1016/j.jjimei.2022.100102
• 55. Al Koliby IS. Influence dimensions of brand equity on purchase intention toward smartphone in Malaysia. VFAST Transactions on Education and Social Sciences. 2018; 6 (1):7-19
• 56. Alhaddad AA. The effect of advertising awareness on brand equity in social media. International Journal of e-Education, e-Business, e-Management and e-Learning. 2015; 5 (2):73-84. DOI: 10.17706/ijeeee.2015.5.2.73-84
• 57. Keller KL. Conceptualizing, measuring, and managing customer-based brand equity. Journal of Marketing. 1993; 57 (1):1-22. DOI: 10.1177/002224299305700101
• 58. Martí Parreño JS-B-M-M. Key factors of teenagers' mobile advertising acceptance. Industrial Management & Data Systems. 2013; 113 (5):732-749. DOI: 10.1108/02635571311324179

© The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

IntechOpen Author/Editor? To get your discount, log in .

Discounts available on purchase of multiple copies. View rates

Local taxes (VAT) are calculated in later steps, if applicable.

Support: [email protected]

Research on K-12 maker education in the early 2020s – a systematic literature review

• Open access
• Published: 27 August 2024

Cite this article

You have full access to this open access article

• Sini Davies   ORCID: orcid.org/0000-0003-3689-7967 1 &
• Pirita Seitamaa-Hakkarainen   ORCID: orcid.org/0000-0001-7493-7435 1  

This systematic literature review focuses on the research published on K-12 maker education in the early 2020s, providing a current picture of the field. Maker education is a hands-on approach to learning that encourages students to engage in collaborative and innovative activities, using a combination of traditional design and fabrication tools and digital technologies to explore real-life phenomena and create tangible artifacts. The review examines the included studies from three perspectives: characteristics, research interests and findings, previous research gaps filled, and further research gaps identified. The review concludes by discussing the overall picture of the research on maker education in the early 2020s and suggesting directions for further studies. Overall, this review provides a valuable resource for researchers, educators, and policymakers to understand the current state of K-12 maker education research.

Explore related subjects

• Artificial Intelligence
• Digital Education and Educational Technology

Avoid common mistakes on your manuscript.

Introduction

Maker culture developed through the pioneering efforts of Papert ( 1980 ) and his followers, such as Blikstein ( 2013 ), Kafai and Peppler ( 2011 ), and Resnick ( 2017 ). It has gained popularity worldwide as an educational approach to encourage student engagement in learning science, technology, engineering, arts, and mathematics (STEAM) (Martin, 2015 ; Papavlasopoulou et al., 2017 ; Vossoughi & Bevan, 2014 ). Maker education involves engaging students to collaborate and innovate together by turning their ideas into tangible creations through the use of conceptual ideas (whether spoken or written), visual representations such as drawings and sketches, and material objects like prototypes and models (Kangas et al., 2013 ; Koh et al., 2015 ). Another core aspect of maker education is combining traditional design and fabrication tools and methods with digital technologies, such as 3D CAD and 3D printing, electronics, robotics, and programming, which enables students to create multifaceted artifacts and hybrid solutions to their design problems that include both digital and virtual features (e.g., Blikstein, 2013 ; Davies et al., 2023 ; Riikonen, Seitamaa-Hakkarainen, et al., 2020 ). The educational value of such multi-dimensional, concrete making has become widely recognized (e.g., Blikstein, 2013 ; Kafai, 1996 ; Kafai et al., 2014 ; Martin, 2015 ).

Maker education has been studied intensively, as indicated by several previous literature reviews (Iivari et al., 2016 ; Lin et al., 2020 ; Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ; Vossoughi & Bevan, 2014 ; Yulis San Juan & Murai, 2022 ). These reviews have revealed how the field has been evolving and provided a valuable overall picture of the research on maker education before the 2020s, including only a few studies published in 2020 or 2021. However, the early years of the 2020s have been an extraordinary period in time in many ways. The world was hit by the COVID-19 pandemic, followed by the global economic crises, increasing geopolitical tensions, and wars that have had a major impact on societies, education, our everyday lives, and inevitably on academic research as well. Furthermore, 2023 was a landmark year in the development of artificial intelligence (AI). In late 2022, OpenAI announced the release of ChatGPT 3.5, a major update to their large language model that is able to generate human-like text. Since then, sophisticated AI systems have rushed into our lives at an accelerating speed and are now becoming integrated with other technologies and applications, shaping how we live, work, our cultures, and our environments irreversibly (see, e.g., World Economic Forum, 2023 ). Thus, it can be argued that towards the end of 2023, the world had transitioned into the era of AI. It is essential that researchers, educators, and policymakers have a fresh overall understanding and a current picture of research on K-12 maker education to develop new, research-based approaches to technology and design education in the present rapidly evolving technological landscape of AI. This is especially important in order to avoid falling back towards shallow epistemic and educational practices of repetition and reproduction. The present systematic review was conducted to provide a ‘big picture’ of the research on K-12 maker education published in the extraordinary times of the early 2020s and to act as a landmark between the research on the field before and after the transition to the AI era. The review was driven by one main research question: How has the research on maker education developed in the early 2020s? To answer this question, three specific research questions were set:

What were the characteristics of the studies in terms of geographical regions, quantity of publications, research settings, and research methods?.

What were the research interests and findings of the reviewed studies?.

How did the reviewed studies fulfill the research gaps identified in previous literature reviews, and what further research gaps they identified?.

The following will outline the theoretical background of the systematic literature review by examining previous literature reviews on maker culture and maker education. This will be followed by an explanation of the methodologies used and findings. Finally, the review will conclude by discussing the overall picture of the research on maker education in the early 2020s and suggesting directions for further studies.

Previous literature reviews on maker culture and maker education

Several literature reviews have been conducted on maker education over the past ten years. The first one by Vossoughi and Bevan ( 2014 ) concentrated on the impact of tinkering and making on children’s learning, design principles and pedagogical approaches in maker programs, and specific tensions and possibilities within the maker movement for equity-oriented teaching and learning. They approached the maker movement in the context of out-of-school time STEM from three perspectives: (1) entrepreneurship and community creativity, (2) STEM pipeline and workforce development, and (3) inquiry-based education. At the time of their review, the research on maker education was just emerging, and therefore, their review included only a few studies. The review findings highlighted how STEM practices were developed through tinkering and striving for equity and intellectual safety (Vossoughi & Bevan, 2014 ). Furthermore, they also revealed how making activities support new ways of learning and collaboration in STEM. Their findings also pointed out some tensions and gaps in the literature, especially regarding a focus that is too narrow on STEM, tools, and techniques, as well as a lack of maker projects conducted within early childhood education or families.

In subsequent literature reviews (Iivari et al., 2016 ; Lin et al., 2020 ; Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ; Yulis San Juan & Murai, 2022 ), the interests of the reviews were expanded. Iivari and colleagues ( 2016 ) reviewed the potential of digital fabrication and making for empowering children and helping them see themselves as future digital innovators. They analyzed the studies based on five conditions: conditions for convergence, entry, social support, competence, and reflection, which were initially developed to help with project planning (Chawla & Heft, 2002 ). Their findings revealed that most of the studies included in their review emphasized the conditions for convergence, entry, and competence. However, only a few studies addressed the conditions for social support and reflection (Iivari et al., 2016 ). The reviewed studies emphasized children’s own interests and their voluntary participation in the projects. Furthermore, the studies highlighted projects leading to both material and learning-related outcomes and the development of children’s competencies in decision-making, design, engineering, technology, and innovation through projects.

Papavlasopoulou and colleagues ( 2017 ) took a broader scope on their systematic literature review, characterizing the overall development and stage of research on maker education through analyzing research settings, interests, and methods, synthesizing findings, and identifying research gaps. They were specifically interested in the technology used, subject areas that implement making activities, and evaluation methods of making instruction across all levels of education and in both formal and informal settings. Their data comprised 43 peer-reviewed empirical studies on maker-centered teaching and learning with children in their sample, providing participants with any making experience. In Papavlasopoulou and colleagues’ ( 2017 ) review, the included studies were published between 2011 and November 2015 as journal articles, conference papers, or book chapters. Most of the studies were conducted with fewer than 50 participants ( n  = 34), the most prominent age group being children from the beginning of primary school up to 14 years old ( n  = 22). The analyzed studies usually utilized more than one data collection method, mainly focusing on qualitative ( n  = 22) or mixed method ( n  = 11) approaches. Most included studies focused on programming skills and computational thinking ( n  = 32) or STEM subjects ( n  = 6). The studies reported a wide range of positive effects of maker education on learning, the development of participants’ self-efficacy, perceptions, and engagement (Papavlasopoulou et al., 2017 ). There were hardly any studies reporting adverse effects.

Schad and Jones ( 2020 ) focused their literature review on empirical studies of the maker movement’s impacts on formal K12 educational environments, published between 2000 and 2018. Their Boolean search (maker movement AND education) to three major academic research databases resulted in 599 studies, of which 20 were included in the review. Fourteen of these studies focused on K12 students, and six on K12 teachers. All but three of the studies were published between 2014 and 2018. Similarly to the studies reported in the previous literature reviews (Iivari et al., 2016 ; Papavlasopoulou et al., 2017 ; Vossoughi & Bevan, 2014 ), the vast majority of the studies were qualitative studies that reported positive opportunities for maker-centered approaches in STEM learning and promotion of excitement and motivation. On the other hand, the studies on K12 in- and preservice teacher education mainly focused on the importance of offering opportunities for teachers to engage in making activities. Both, studies focused on students or teachers, promoting equity and offering equally motivating learning experiences regardless of participants’ gender or background was emphasized.

Lin and colleagues’ ( 2020 ) review focused on the assessment of maker-centered learning activities. After applying inclusion and exclusion criteria, their review consisted of 60 peer-reviewed empirical studies on making activities that included making tangible artifacts and assessments to measure learning outcomes. The studies were published between 2006 and 2019. Lin and colleagues ( 2020 ) also focused on all age groups and activities in both formal and informal settings. Most studies included applied STEM as their main subject domain and utilized a technology-based platform, such as LilyPad Arduino microcontroller, Scratch, or laser cutting. The results of the review revealed that in most studies, learning outcomes were usually measured through the assessment of artifacts, tests, surveys, interviews, and observations. The learning outcomes measured were most often cognitive skills on STEM-related content knowledge or students’ feelings and attitudes towards STEM or computing.

The two latest systematic reviews, published in 2022, also focused on specific research interests in maker education (Rouse & Rouse, 2022 ; Yulis San Juan & Murai, 2022 ). Rouse and Rouse ( 2022 ) reviewed studies that specifically investigated learning in preK-12 maker education in formal school-based settings. Their analysis included 22 papers from seven countries, all but two published between 2017 and 2019. Only two of the studies focused on early childhood education, and three involved participants from the elementary level. Like previous reviews, most studies were conducted with qualitative methods ( n  = 17). On the other hand, in contrast to the earlier reviews (Lin et al., 2020 ; Papavlasopoulou et al., 2017 ; Schad & Jones, 2020 ), the studies included in the review did not concentrate on content-related outcomes on STEM or computing. Instead, a wide range of learning outcomes was investigated, such as 21st-century skills, agency, and materialized knowledge. On the other hand, they found that equity and inclusivity were not ubiquitously considered when researchers design makerspace interventions. Yulis San Juan and Murai’s ( 2022 ) literature review focused on frustration in maker-centered learning activities. Their analysis consisted of 28 studies published between 2013 and 2021. Their findings of the studies identified six factors that are most often recognized as the causes of frustration in makerspace activities: ‘unfamiliar pedagogical approach, time constraints, collaboration, outcome expectations, lack of skills and knowledge, and tool affordances and availability’ (Yulis San Juan & Murai, 2022 , p. 4).

From these previous literature reviews, five significant research gaps emerged that required further investigation and attention:

Teacher training, pedagogies, and orchestration of learning activities in maker education (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ; Vossoughi & Bevan, 2014 ).

Wide variety of learning outcomes that potentially emerge from making activities, as well as the development of assessment methods and especially systematic ways to measure student learning (Lin et al., 2020 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ).

Equity and inclusivity in maker education (Rouse & Rouse, 2022 ; Vossoughi & Bevan, 2014 ).

Practices, tools, and technologies used in makerspaces and digital fabrication (Iivari et al., 2016 ; Papavlasopoulou et al., 2017 ).

Implementation and effects of maker education in formal, school-based settings and specific age groups, especially early childhood education (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ).

Methodology

This review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, adapting it to educational settings where studies are conducted with qualitative, quantitative, and mixed methods (Page et al., 2021 ; Tong et al., 2012 ). Review protocols were defined for data collection, inclusion, exclusion, and quality criteria and the data analysis. In the following, the method used for each stage of the review process will be defined in detail.

Data collection

To gather high-quality and comprehensive data, a search for peer-reviewed articles was conducted in three international online bibliographic databases: Scopus, Education Resources Information Center (ERIC), and Academic Search Complete (EBSCO). Scopus and EBSCO are extensive multi-disciplinary databases for research literature, covering research published in over 200 disciplines, including education, from over 6000 publishers. ERIC concentrates exclusively on educational-related literature, covering publications from over 1900 full-text journals. These three databases were considered to offer a broad scope to capture comprehensive new literature on K-12 maker education. The search aimed to capture peer-reviewed literature on maker education and related processes conducted in both formal and informal K-12 educational settings. The search was limited to articles published in English between 2020 and 2023. Major search terms and their variations were identified to conduct the search, and a Boolean search string was formed from them. The search was implemented in October 2023 with the following search string that was used to search on titles, abstracts, and keywords:

(“maker education” OR “maker pedagogy” OR “maker-centered learning” OR “maker centered learning” OR “maker-centred learning” OR “maker centred learning” OR “maker learning” OR “maker space*” OR makerspace* OR “maker culture” OR “design learning” OR “maker practices” OR “collaborative invention*” OR co-invention*) AND (“knowledge-creation” OR “knowledge creation” OR “knowledgecreation” OR maker* OR epistemic OR “technology education” OR “design-based learning” OR “design based learning” OR “designbased learning” OR “design learning” OR “design thinking” OR “codesign” OR “co-design” OR “co design” OR craft* OR tinker* OR “collaborative learning” OR inquiry* OR “STEAM” OR “project-based learning” OR “project based learning” OR “projectbased learning” OR “learning project*” OR “knowledge building” OR “making” OR creati* OR innovat* OR process*) AND (school* OR pedago* OR “secondary education” OR “pre-primary education” OR “primary education” OR “special education” OR “early childhood education” OR “elementary education” OR primary-age* OR elementary-age* OR “k-12” OR “youth” OR teen* OR adolescen* OR child* OR “tween”) .

Inclusion and exclusion criteria

The search provided 700 articles in total, 335 from Scopus, 345 from EBSCO, and 20 from ERIC that were aggregated to Rayyan (Ouzzani et al., 2016 ), a web and mobile app for systematic reviews, for further processing and analysis. After eliminating duplicates, 513 studies remained. At the next stage, the titles and abstracts of these studies were screened independently by two researchers to identify papers within the scope of this review. Any conference papers, posters, work-in-progress studies, non-peer-reviewed papers, review articles, and papers focusing on teacher education or teachers’ professional development were excluded from the review. To be included, the study had to meet all the following four inclusion criteria. It had to:

show empirical evidence.

describe any making experience or testing process conducted by the participants.

include participants from the K-12 age group in their sample.

have an educational purpose.

For example, studies that relied purely on statistical data collected outside a maker educational setting or studies that described a maker space design process but did not include any research data from an actual making experience conducted by participants from the K-12 age group were excluded. Studies conducted both in formal and informal settings were included in the review. Also, papers were included regardless of whether they were conducted using qualitative, quantitative, or mixed methods. After the independent screening process, the results were combined, and any conflicting assessments were discussed and settled. Finally, 149 studies were included to be retrieved for further evaluation of eligibility, of which five studies were not available for retrieval. Thus, the screening resulted in 144 included studies with full text retrieved to apply quality criteria and further analysis.

Quality criteria

The quality of each of the remaining 144 studies was assessed against the Critical Appraisal Skills Programme’s ( 2023 ) qualitative study checklist, which was slightly adjusted for the context of this review. The checklist consisted of ten questions that each address one quality criterion:

Was there a clear statement of the aims of the research?.

Are the methodologies used appropriate?.

Was the research design appropriate to address the research aims?.

Was the recruitment strategy appropriate to the aims of the research?.

Was the data collected in a way that addressed the research issue?.

Has the relationship between the researcher and participants been adequately considered?.

Have ethical issues been taken into consideration?.

Was the data analysis sufficiently rigorous?.

Is there a clear statement of findings?.

How valuable is the research?.

The first author assessed the quality by reading each study’s full text. To be included in the final analysis, the study had to meet both the inclusion-exclusion and the quality criteria. In this phase, the final assessment for eligibility, 50 studies were excluded due to not meeting the initial inclusion and exclusion criteria, and 32 studies for not filling the criteria for quality. A total of 62 studies were included in the final analysis of this literature review. The PRISMA flow chart (Haddaway et al., 2022 ; see also Page et al., 2021 ) of the study selection process is presented in Fig.  1 .

PRISMA study selection flow chart (Haddaway et al., 2022 )

Qualitative content analysis of the reviewed studies

The analysis of the studies included in the review was conducted through careful reading of the full texts of the articles by the first author. To answer the first research question: What were the characteristics of the studies in terms of geographical regions, quantity of publications, research settings, and methods; a deductive coding framework was applied that consisted of characterizing factors of the study, its research setting as well as data collection and analysis methods applied. The predetermined categories of the study characteristics and the codes associated with each category are presented in Table  1 . The educational level of the participants was determined by following The International Standard Classification of Education (ISCED) (UNESCO Institute for Statistics, 2012 ). Educational level was chosen instead of an age group as a coding category because, during the first abstract and title screening of the articles, it became evident that the studies describe their participants more often by their educational level than age. The educational levels were converted from national educational systems following the ISCED diagrams (UNESCO Institute for Statistics, 2021 ).

In addition to the deductive coding, the following analysis categories were gathered from the articles through inductive analysis: journal, duration of the project, number of participants, types of research data collected, and specific data analysis methods. Furthermore, the following characteristics of the studies were marked in the data when applicable: if the research was conducted as a case study, usage of control groups, specific focus on minority groups, gifted students, special needs students, or inclusion. Inductive coding and thematic analysis were applied to answer the second research question: what were the research interests and findings of the reviewed studies? The categorization of research interests was then combined with some aspects of the first part of the analysis to reveal further interesting characteristics about the latest developments in the research in maker education.

In the following, the findings of this systematic literature review will be presented for each research question separately.

Characteristics of research in K-12 maker education in the 2020s

Of the studies included in the review, presented in Table  2 and 20 studies were published in 2020, 17 in 2021, 12 in 2022, and 13 in 2023. The slight decline in publications does not necessarily indicate a decline in interest towards maker education but is more likely due to the COVID-19 pandemic that heavily limited hands-on activities and in situ data collection. Compared to the latest wide-scope review on maker education (Papavlasopoulou et al., 2017 ), the number of high-quality studies published yearly appears to be at similar levels to those in the previous reviews. The studies included in the present review were published in 34 different peer-reviewed academic journals, of which 13 published two or more articles.

Regarding the geographic distribution of studies conducted on maker education, the field seems to be becoming more internationally spread. In 2020, the studies mainly published research conducted in either the USA ( n  = 6) or Finland ( n  = 12), whereas in the subsequent years, the studies were distributed more evenly around the world. However, North America and Scandinavia remained the epicenters of research on maker education, conducting over half of the studies published each year.

Most of the reviewed studies used qualitative methods ( n  = 42). Mixed methods were utilized in 13 studies, and quantitative methods in seven. Forty-four studies were described as case studies by their authors, and, on the other hand, a control group was used in four quantitative and two mixed methods studies. The analysis indicated an interesting research shift towards making activities part of formal educational settings instead of informal, extracurricular activities. Of the studies included in this review, 82% ( n  = 51) were conducted exclusively in formal educational settings. This contrasts significantly with the previous literature review by Papavlasopoulou and colleagues ( 2017 ), where most studies were conducted in informal settings. Furthermore, Schad and Jones ( 2020 ) identified only 20 studies between 2000 and 2018 conducted in formal educational settings in K12-education, and Rouse and Rouse ( 2022 ) identified 22 studies in similar settings from 2014 to early 2020. In these reviews, nearly all studies done in formal educational settings were published in the last years of the 2010 decade. Thus, this finding suggests that the change in learning settings started to emerge in the latter half of the 2010s, and in the 2020s, maker education in formal settings has become the prominent focus of research. The need for further research in formal settings was one of the main research gaps identified in previous literature reviews (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ).

In addition to the shift from informal to formal educational settings, the projects studied in the reviewed articles were conducted nearly as often in school and classroom environments ( n  = 26) as in designated makerspaces ( n  = 28). Only seven of the studied projects took place in other locations, such as youth clubs, libraries, or summer camps. One project was conducted entirely in an online learning environment. Most of the studied projects involved children exclusively from primary ( n  = 27) or lower secondary ( n  = 26) education levels. Only three studies were done with students in upper secondary education. Like the previous literature reviews, only a few studies concentrated on children in early childhood education (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ). Three articles reported projects conducted exclusively on early childhood education age groups, and three studies had participants from early childhood education together with children from primary ( n  = 2) or lower secondary education ( n  = 1).

The number of child participants in the studies varied between 1 and 576, and 14 studies also included teachers or other adults in their sample. The number of participating children in relation to the methods used is presented in Fig.  2 . Most of the qualitative studies had less than 100 children in their sample. However, there were three qualitative studies with 100 to 199 child participants (Friend & Mills, 2021 ; Leskinen et al., 2021 ; Riikonen, Kangas, et al., 2020 ) and one study with 576 participating children (Forbes et al., 2021 ). Studies utilizing mixed methods were either conducted with a very large number of child participants or with less than 100 participants, ranging from 4 to 99. Studies using quantitative methods, on the other hand, in most cases had 50–199 participants ( n  = 6). One quantitative study was conducted with 35 child participants (Yin et al., 2020 ). Many studies included participants from non-dominant backgrounds or with special educational needs. However, only two studies focused specifically on youth from non-dominant backgrounds (Brownell, 2020 ; Hsu et al., 2022 ), and three studies focused exclusively on inclusion and students with special needs (Giusti & Bombieri, 2020 ; Martin et al., 2020 ; Sormunen et al., 2020 ). In addition, one study specifically chose gifted students in their sample (Andersen et al., 2022 ).

Child participants in the reviewed studies in relation to the methods used

Slightly over half of the studied projects had only collaborative tasks ( n  = 36), 11 projects involved both collaborative and individual tasks, and in 11 projects, the participants worked on their own individual tasks. Four studies did not specify whether the project was built around collaborative or individual tasks. In most cases, the projects involved both traditional tangible tools and materials as well as digital devices and fabrication technologies ( n  = 54). In five projects, the students worked entirely with digital design and making methods, and in 3 cases, only with traditional tangible materials. Similarly, the outcomes of the project tasks were mainly focused on designing and building artifacts that included both digital and material elements ( n  = 31), or the project included multiple activities and building of several artifacts that were either digital, material, or had both elements ( n  = 17). Eleven projects included digital exploration without an aim to build a design artifact as a preparatory activity, whereas one project was based solely on digital exploration as the making activity. Material artifacts without digital elements were made in seven of the studied projects, and six concentrated solely on digital artifact making.

The duration of the projects varied between two hours (Tisza & Markopoulos, 2021 ) and five years (Keune et al., 2022 ). The number of studies in each categorized project duration range, in relation to the methods used, is presented in Fig.  3 . Over half of the projects lasted between 1 month and one year ( n  = 35), nine were longer, lasting between 1 and 5 years, and 14 were short projects lasting less than one month. Three qualitative studies and one quantitative study did not give any indication of the duration of the project. Most of the projects of qualitative studies took at least one month ( n  = 32), whereas projects in mixed method studies usually were shorter than three months ( n  = 10). On the other hand, quantitative studies usually investigated projects that were either shorter than three months ( n  = 4) or longer than one year ( n  = 2).

Duration of the studied projects in relation to the methods used

A multitude of different types of data was used in the reviewed studies. The data collection methods utilized by at least three reviewed studies are presented in Table  3 . Qualitative studies usually utilized several (2 to 6) different data gathering methods ( n  = 31), and all mixed method studies used more than one type of data (2 to 6). The most common data collection methods in qualitative studies were video data, interviews, and ethnographic observations combined with other data, such as design artifacts, photographs, and student portfolios. In addition to the data types specified in Table  3 , some studies used more unusual data collection methods such as lesson plans (Herro et al., 2021b ), the think-aloud protocol (Friend & Mills, 2021 ; Impedovo & Cederqvist, 2023 ), and social networks (Tenhovirta et al., 2022 ). Eleven qualitative studies used only one type of data, mainly video recordings ( n  = 9). Mixed method studies, on the other hand, relied often on interviews, pre-post measurements, surveys, and video data. In addition to the data types in Table  3 , mixed-method studies utilized biometric measurements (Hsu et al., 2022 ; Lee, 2021 ), lesson plans (Falloon et al., 2022 ), and teacher assessments (Doss & Bloom, 2023 ). In contrast to the qualitative and mixed method studies, all quantitative studies, apart from one (Yin et al., 2020 ), used only one form of research data, either pre-post measurements or surveys.

The findings of the data collection methods are similar to the previous literature review of Papavlasopoulou and colleagues ( 2017 ) regarding the wide variety of data types used in qualitative and mixed-method studies. However, when compared to their findings on specific types of research data used, video recordings have become the most popular way of collecting data in recent years, replacing interviews and ethnographic observations.

Research interests and findings of the reviewed studies

Seven categories of research interests emerged from the inductive coding of the reviewed studies. The categories are presented in Table  4 in relation to the research methods and educational levels of the participating children. Five qualitative studies, four mixed methods studies, and two quantitative studies had research interests from more than one category. Processes, activity, and practices, as well as sociomateriality in maker education, were studied exclusively with qualitative methods, and, on the other hand, nearly all studies on student motivation, interests, attitudes, engagement, and mindset were conducted with mixed or quantitative methods. In the two biggest categories, most of the studies utilized qualitative methods. Studies conducted with mixed or quantitative methods mainly concentrated on two categories: student learning and learning opportunities and student motivation, interests, attitudes, engagement, and mindset. In the following section, the research interests and findings for each category will be presented in detail.

Nearly half of the reviewed studies ( n  = 30) had a research interest in either student learning through making activities in general or learning opportunities provided by such activities. Five qualitative case studies (Giusti & Bombieri, 2020 ; Hachey et al., 2022 ; Hagerman et al., 2022 ; Hartikainen et al., 2023 ; Morado et al., 2021 ) and two mixed method studies (Martin et al., 2020 ; Vuopala et al., 2020 ) investigated the overall educational value of maker education. One of these studies was conducted in early childhood education (Hachey et al., 2022 ), and two in the context of inclusion in primary and lower secondary education (Giusti & Bombieri, 2020 ; Martin et al., 2020 ). They all reported positive findings on the development of children’s identity formation and skills beyond subject-specific competencies, such as creativity, innovation, cultural literacy, and learning skills. The studies conducted in the context of inclusion especially emphasized the potential of maker education in pushing students with special needs to achieve goals exceeding their supposed cognitive abilities (Giusti & Bombieri, 2020 ; Martin et al., 2020 ). Three studies (Forbes et al., 2021 ; Kumpulainen et al., 2020 ; Xiang et al., 2023 ) investigated student learning through the Maker Literacies Framework (Marsh et al., 2018 ). They also reported positive findings on student learning and skill development in early childhood and primary education, especially on the operational dimension of the framework, as well as on the cultural and critical dimensions. These positive results were further confirmed by the reviewed studies that investigated more specific learning opportunities provided by maker education on developing young people’s creativity, innovation skills, design thinking and entrepreneurship (Liu & Li, 2023 ; Timotheou & Ioannou, 2021 ; Weng et al., 2022a , b ), as well as their 21st-century skills (Iwata et al., 2020 ; Tan et al., 2021 ), and critical data literacies and critical thinking (Stornaiuolo, 2020 ; Weng et al., 2022a ).

Studies that investigated subject-specific learning most often focused on STEM subjects or programming and computational thinking. Based on the findings of these studies, maker-centered learning activities are effective but underused (Mørch et al., 2023 ). Furthermore, in early childhood education, such activities may support children taking on the role of a STEM practitioner (Hachey et al., 2022 ) and, on the other hand, provide them access to learning about STEM subjects beyond their grade level, even in upper secondary education (Tofel-Grehl et al., 2021 ; Winters et al., 2022 ). However, two studies (Falloon et al., 2022 ; Forbes et al., 2021 ) highlighted that it cannot be assumed that students naturally learn science and mathematics conceptual knowledge through making. To achieve learning in STEM subjects, especially science and mathematics, teachers need to specifically identify, design, and focus the making tasks on these areas. One study also looked at the effects of the COVID-19 pandemic on STEM disciplines and found the restrictions on the use of common makerspaces and the changes in the technologies used to have been detrimental to student’s learning in these areas (Dúo-Terrón et al., 2022 ).

Only positive findings emerged from the reviewed studies on how digital making activities promote the development of programming and computational thinking skills and practices (Iwata et al., 2020 ; Liu & Li, 2023 ; Yin et al., 2020 ) and understanding of programming methods used in AI and machine learning (Ng et al., 2023 ). Experiences of fun provided by the making activities were also found to enhance further student learning about programming (Tisza & Markopoulos, 2021 ). One study also reported positive results on student learning of academic writing skills (Stewart et al., 2023 ). There were also three studies (Brownell, 2020 ; Greenberg et al., 2020 ; Wargo, 2021 ) that investigated the potential of maker education to promote equity and learning about social justice and injustice, as well as one study that examined learning opportunities on sustainability (Impedovo & Cederqvist, 2023 ). All these studies found making activities and makerspaces to be fertile ground for learning as well as identity and community building around these topics.

The studies with research interests in the second largest category, facilitation and teaching practices ( n  = 13), investigated a multitude of different aspects of this area. The studies on assessment methods highlighted the educational value of process-based portfolios (Fields et al., 2021 ; Riikonen, Kangas et al., 2020 ) and connected portfolios that are digital portfolios aligned with a connected learning framework (Keune et al., 2022 ). On the other hand, Walan and Brink ( 2023 ) concentrated on developing and analyzing the outcomes of a self-assessment tool for maker-centered learning activities designed to promote 21st-century skills. Several research interests emerged from the review related to scaffolding and implementation of maker education in schools. Riikonen, Kangas, and colleagues ( 2020 ) investigated the pedagogical infrastructures of knowledge-creating, maker-centered learning. It emphasized longstanding iterative, socio-material projects, where real-time support and embedded scaffolding are provided to the participants by a multi-disciplinary teacher team and ideally also by peer tutors. Multi-disciplinary collaboration was also emphasized by Pitkänen and colleagues ( 2020 ) in their study on the role of facilitators as educators in Fab Labs. Cross-age peer tutoring was investigated by five studies and found to be highly effective in promoting learning in maker education (Kumpulainen et al., 2020 ; Riikonen, Kangas, et al., 2020 ; Tenhovirta et al., 2022 ; Weng et al., 2022a ; Winters et al., 2022 ). Kajamaa and colleagues ( 2020 ) further highlighted the importance of team teaching and emphasized moving from authoritative interaction with students to collaboration. Sormunen and colleagues’ ( 2020 ) findings on teacher support in an inclusive setting demonstrated how teacher-directed scaffolding and facilitation of student cooperation and reflective discussions are essential in promoting inclusion-related participation, collaboration skills, and student competence building. One study (Andersen et al., 2022 ) took a different approach and investigated the possibilities of automatic scaffolding of making activities through AI. They concluded that automated scaffolding has excellent potential in maker education and went as far as to suggest that a transition should be made to it. One study also recognized the potential of combining making activities with drama education (Walan, 2021 ).

Versatile aspects of different processes, activities, and practices in maker-centered learning projects were studied by 11 qualitative studies included in this review. Two interlinked studies (Davies et al., 2023 ; Riikonen, Seitamaa-Hakkarainen et al., 2020 ) investigated practices and processes related to collaborative invention, making, and knowledge-creation in lower secondary education. Their findings highlighted the multifaceted and iterative nature of such processes as well as the potential of maker education to offer students authentic opportunities for knowledge creation. Sinervo and colleagues ( 2021 ) also investigated the nature of the co-invention processes from the point of view of how children themselves describe and reflect their own processes. Their findings showed how children could recognize different external constraints involved in their design and the importance of iterative ideation processes and testing the ideas through prototyping. Innovation and invention practices were also studied by two other studies in both formal and informal settings with children from the primary level of education (Leskinen et al., 2023 ; Skåland et al., 2020 ). Skåland and colleagues’ ( 2020 ) findings suggest that narrative framing, that is, storytelling with the children, is an especially fruitful approach in a library setting and helps children understand their process of inventing. Similar findings were made in the study on the role of play in early childhood maker education (Fleer, 2022 ), where play enhanced design cognition and related processes and helped young children make sense of design. On the other hand, Leskinen and colleagues ( 2023 ) showed how innovations are jointly practiced in the interaction between students and teachers. They also emphasized the importance of using manifold information sources and material elements in creative innovation processes.

One study (Kajamaa & Kumpulainen, 2020 ) investigated collaborative knowledge practices and how those are mediated in school makerspaces. They identified four types of knowledge practices involved in maker-centered learning activities: orienting, interpreting, concretizing, and expanding knowledge, and how discourse, materials, embodied actions, and the physical space mediate these practices. Their findings also showed that due to the complexity of these practices, students might find maker-centered learning activities difficult. The sophisticated epistemic practices involved in collaborative invention processes were also demonstrated by the findings of Mehto, Riikonen, Hakkarainen, and colleagues ( 2020a ). Other investigators examined how art-based (Lindberg et al., 2020 ), touch-related (Friend & Mills, 2021 ), and information (Li, 2021 ) practices affect and can be incorporated into making. All three studies reported positive findings on the effects of these practices on student learning and, on the other hand, on the further development of the practices themselves.

Research interests related to student motivation, interests, attitudes, engagement, and mindset were studied by eight reviewed articles, all conducted with either mixed (n = 6) or quantitative methods (n = 2). The studies that investigated student motivation and engagement in making activities (Lee, 2021 ; Martin et al., 2020 ; Ng et al., 2023 ; Nikou, 2023 ) highlighted the importance of social interactions and collaboration as highly influential factors in these areas. On the other hand, positive attitudes towards collaboration also developed through these activities (Nguyen et al., 2023 ). Making activities conducted in the context of equity-oriented pedagogy were found to have great potential in sustaining non-dominant youths’, especially girls’, positive attitudes toward science (Hsu et al., 2022 ). On the other hand, a similar potential was not found in the development of interest in STEM subjects with autistic students (Martin et al., 2020 ). Two studies investigated student mindsets in maker-centered learning activities (Doss & Bloom, 2023 ; Vongkulluksn et al., 2021 ). Doss and Bloom ( 2023 ) identified seven different student mindset profiles present in making activities. Over half (56.67%) of the students in their study were found to share the same mindset profile, characterized as: ‘Flexible, Goal-Oriented, Persistent, Optimistic, Humorous, Realistic about Final Product’ (Doss & Bloom, 2023 , p. 4). In turn, Vongkulluksn and colleagues ( 2021 ) investigated the growth mindset trends for students who participated in a makerspace program for two years in an elementary school. Their findings revealed positive results of how makerspace environments can potentially improve students’ growth mindset.

Six studies included in this review analyzed collaboration within making activities. Students were found to be supportive and respectful towards each other as well as recognize and draw on each other’s expertise (Giusti & Bombieri, 2020 ; Herro et al., 2021a , b ). The making activities and outcomes were found to act as mediators in promoting mutual recognition between students with varying cognitive capabilities and special needs in inclusive settings (Herro et al., 2021a ). Furthermore, a community of interest that emerges through collaborative making activities was also found to be effective in supporting interest development and sustainability (Tan et al., 2021 ). Students were observed to divide work and share roles during their team projects, usually based on students’ interests, expertise, and skills (Herro et al., 2021a , b ). The findings of Stewart et al.‘s ( 2023 ) study suggested that when roles are preassigned to the team members by teachers, it decreases student stress in maker activities. However, if dominating leadership roles emerged in a team, that was found to lead to less advanced forms of collaboration than shared leadership within the team (Leskinen et al., 2021 ).

Sociomaterial aspects of making activities were in the interest of three reviewed studies (Kumpulainen & Kajamaa, 2020 ; Mehto et al., 2020a ; Mehto et al., 2020b ). Materials were shown to have an active role in knowledge-creation and ideation in open-ended maker-centered learning (Mehto et al., 2020a ), which allows for thinking together with the materials (Mehto et al., 2020b ). The task-related physical materials act as a focal point for team collaboration and invite participation (Mehto et al., 2020b ). Furthermore, a study by Kumpulainen and Kajamaa ( 2020 ) emphasized the sociomaterial dynamics of agency, where agency flows in any combination between students, teachers, and materials. However, the singularity or multiplicity of the materials potentially affects the opportunities for access and control of the process (Mehto et al., 2020b ).

In addition to empirical research interests, five studies focused on developing research methods for measuring and analyzing different aspects of maker education. Biometric measurements were investigated as a potential data source to detect engagement in making activities (Lee, 2021 ). Yin and colleagues ( 2020 ) focused on developing instruments for the quantitative measurement of computational thinking skills. On the other hand, Timotheou and Ioannou ( 2021 ) designed and tested an analytic framework and coding scheme to analyze learning and innovation skills from qualitative interviews and video data. Artificial intelligence as a potential, partially automated tool for analyzing CSCL artifacts was also investigated by one study (Andersen et al., 2022 ). Finally, Riikonen, Seitamaa-Hakkarainen, and colleagues ( 2020 ) developed visual video data analysis methods for investigating collaborative design and making activities.

Slightly over half of the reviewed studies ( n  = 33) made clear suggestions for future research. Expectedly, these studies suggested further investigation of their own research interests. However, across the studies, five themes of recommendations for future research interests and designs emerged from the data:

1. Studies conducted with diverse range of participants , pedagogical designs , and contexts (Hartikainen et al., 2023 ; Kumpulainen & Kajamaa, 2020 ; Leskinen et al., 2023 ; Lindberg et al., 2020 ; Liu & Li, 2023 Martin et al., 2020 ; Mehto et al., 2020b ; Nguyen et al., 2023 ; Sormunen et al., 2020 ; Tan et al., 2021 ; Weng et al., 2022a , b ; Yin et al., 2020 ).

2. Longitudinal studies to confirm the existing research findings, further develop pedagogical approaches to making, and to better understand the effects of maker education on students later in their lives (Davies et al., 2023 ; Fields et al., 2021 ; Kumpulainen et al., 2020 ; Kumpulainen & Kajamaa, 2020 ; Stornaiuolo, 2020 ; Tisza & Markopoulos, 2021 ; Walan & Brink, 2023 ; Weng et al., 2022a ).

3. Development of new methods and applying existing methods in different conditions (Doss & Bloom, 2023 ; Kumpulainen et al., 2020 ; Leskinen et al., 2021 ; Mehto et al., 2020b ; Mørch et al., 2023 ; Tan et al., 2021 ; Timotheou & Ioannou, 2021 ; Tisza & Markopoulos, 2021 ).

4. Identifying optimal conditions and practices for learning, skill, and identity development through making (Davies et al., 2023 ; Fields et al., 2021 ; Hartikainen et al., 2023 ; Tofel-Grehl et al., 2021 ).

5. Collaboration from the perspectives of how it affects processes and outcomes of making activities and, on the other hand, how such activities affect collaboration (Pitkänen et al., 2020 ; Tisza & Markopoulos, 2021 ; Weng et al., 2022a ).

Discussion and conclusions

This systematic literature review was conducted to describe the development of research on maker education in the early 2020s. Sixty-two studies from the initial 700 studies identified from the three major educational research databases were included in the review. The qualitative analysis of the reviewed studies revealed some interesting developments in the field. Overall, the research on maker education appears to be active. Maker education seems to be attracting interest from researchers around the globe. However, two epicenters of research, North America and Scandinavia, namely Finland, appear to have an active role in maker research.

Most studies relied on rich qualitative data, often collected using several methods. Video recordings have become a popular way to collect data in maker education research. Although qualitative methods remained the dominant methodological approach in the field (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ), mixed and quantitative methods were used in nearly a third of the reviewed studies. These studies mainly measured learning outcomes or participants’ motivation, interests, attitudes, engagement, and mindsets. There was a great variety in the duration of the maker projects and the number of participants. The projects lasted from less than a day up to five years, and the number of participants varied similarly from one to nearly six hundred. Methodological development was also within the research interests of several studies in this review. Developments were made both in qualitative and quantitative methodologies. Such methodological development was one of the research gaps identified in the previous literature reviews (e.g., Schad & Jones, 2020 ).

The analysis of the reviewed studies revealed an interesting shift in research on maker education from informal settings to formal education. Our review revealed that most studies were conducted exclusively in formal education and often as part of the curricular activity. The need for this development was called for in the previous literature reviews (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ). However, only a handful of studies were conducted in early childhood education. Winters and colleagues’ ( 2022 ) study adopted a very interesting setting where children from early childhood education worked together and were mentored by students from lower secondary education. This type of research setting could have great potential for future research in maker education.

Another research gap identified in the previous literature reviews was the need to study and measure a wide variety of potential learning opportunities and outcomes of maker education (Lin et al., 2020 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ). The analysis revealed that new research in the field is actively filling this gap. Skills that go beyond subject-specific content and the development of participants’ identities through making activities were especially actively studied from various perspectives. The findings of these studies were distinctively positive, corresponding with the conclusions of the previous literature reviews (e.g., Papavlasopoulou et al., 2017 ; Schad & Jones, 2020 ; Vossoughi & Bevan, 2014 ). This potential of maker education should be recognized by educators and policymakers, especially when the advancements in AI technologies will forefront the need for the humane skills of working creatively with knowledge and different ways of knowing, empathic engagement, and collaboration (e.g., Liu et al., 2024 ; Markauskaite et al., 2022 ; Qadir, 2023 ; World Economic Forum, 2023 ). Some of these studies also addressed the issue of promoting equity through maker education, which was called for in the previous literature review (Rouse & Rouse, 2022 ; Vossoughi & Bevan, 2014 ). However, considering the small number of these studies, more research will still be needed.

The two other popular research interest categories that emerged from the analysis were facilitation and teaching practices as well as processes, activities, and practices involved in making – both identified as research gaps in the previous literature reviews (Iivari et al., 2016 ; Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ; Vossoughi & Bevan, 2014 ). The teaching practices and scaffolding of making activities were investigated from different aspects, such as assessment methods, implementation of maker education in schools, and cross-age peer tutoring. The results of these studies highlighted the positive effects of multi-disciplinary collaboration and peer tutoring. Such pedagogical approaches should be more widely promoted as integral parts of the pedagogical infrastructure in schools. However, this calls for measures from policymakers and school authorities to enable such collaborative ways of teaching that extend beyond the traditional structures of school organizations. Furthermore, although research on this area has been active and multi-faceted, the facilitation of maker education in inclusive settings especially calls for further investigation. In terms of processes, practices, and activities involved in making, the reviewed studies investigated a variety of aspects that revealed the sophisticated epistemic practices involved and the importance of concrete making, prototyping, and iterative ideation in maker-centered learning activities. These studies further highlighted the potential of maker education to offer students authentic opportunities for knowledge creation. Studies also examined collaboration and sociomateriality involved in maker education. Especially sociomateriality is a relatively new, emerging area of research in maker education.

The reviewed studies identified five research gaps that require further investigation: (1) conducting studies with a diverse range of participants, pedagogical designs, and contexts; (2) carrying out longitudinal studies; (3) developing new methods and applying existing methods in different settings; (4) identifying the most effective conditions and practices for learning, skill development and identity formation in maker education, and (5) understanding how collaboration affects the processes and outcomes of making activities and vice versa. In addition to the research gaps identified by reviewed studies, the analysis revealed additional gaps. Studies conducted in early childhood education and inclusive settings remain especially under-represented, although maker pedagogies have been found to have great potential in these areas. Similarly, many researchers have recognized the potential of maker education to promote equality between children from different backgrounds and genders. Still, only a handful of studies investigated these issues. Thus, more research is needed, especially on best practices and pedagogical approaches in this area. Furthermore, the processes involved in and affecting maker-centered learning call for further investigations.

The field has matured based on the analysis of the reviewed studies. It is moving from striving to understand what can be achieved to investigating the underlying conditions behind learning through making, how desired outcomes can be best achieved, as well as how the processes involved in making unfold, what the effects are in the long run, and how to understand best and measure different phenomena related to making. Furthermore, researchers are looking into more and more opportunities to expand the learning opportunities of maker education by combining them with other creative pedagogies and applying them to projects that seek to introduce subject-specific content beyond STEM to students.

This systematic literature review has several limitations. The typical limitations of most review studies, the potential loss of search results due to limited search terms and databases used, apply to this review. For example, more culturally diverse search results might have been reached with the addition of other databases and further search terms. However, the search string was carefully designed and tested to include as many common terms used in maker education research as possible, including possible variations. Furthermore, the three databases used in the search, Scopus, ERIC, and EBSCO, are regarded as the most comprehensive databases of educational research available. Thus, although some studies might not have been identified because of these limitations, it can be assumed that this review gives a comprehensive enough snapshot of research on maker education in the early years of the 2020s.

Andersen, R., Mørch, A. I., & Litherland, K. T. (2022). Collaborative learning with block-based programming: Investigating human-centered artificial intelligence in education. Behaviour & Information Technology , 41 (9), 1830–1847. https://doi.org/10.1080/0144929X.2022.2083981

Article   Google Scholar  

Blikstein, P. (2013). Digital fabrication and ‘making’ in education: The democratization of invention. In C. Büching & J. Walter-Herrmann (Eds.), FabLabs: Of machines, makers and inventors (pp. 203–222). Transcript Publishers. https://doi.org/10.1515/transcript.9783839423820.203

Brownell, C. J. (2020). Keep walls down instead of up: Interrogating writing/making as a vehicle for black girls’ literacies. Education Sciences , 10 (6), 159. https://doi.org/10.3390/educsci10060159

Chawla, L., & Heft, H. (2002). Children’s competence and the ecology of communities: A functional approach to the evaluation of participation. Journal of Environmental Psychology , 22 (1–2), 201–216. https://doi.org/10.1006/jevp.2002.0244

Critical Appraisal Skills Programme (2023). CASP Qualitative Studies Checklist . https://casp-uk.net/checklists/casp-qualitative-studies-checklist-fillable.pdf

Davies, S., Seitamaa-Hakkarainen, P., & Hakkarainen, K. (2023). Idea generation and knowledge creation through maker practices in an artifact-mediated collaborative invention project. Learning, Culture and Social Interaction, 39 , 100692. https://doi.org/10.1016/j.lcsi.2023.100692

Doss, K., & Bloom, L. (2023). Mindset and the desire for feedback during creative tasks. Journal of Creativity , 33 (1), 100047. https://doi.org/10.1016/j.yjoc.2023.100047

Dúo-Terrón, P., Hinojo-Lucena, F. J., Moreno-Guerrero, A. J., & López-Belmonte, J. (2022). Impact of the pandemic on STEAM disciplines in the sixth grade of primary education. European Journal of Investigation in Health Psychology and Education , 12 (8), 989–1005. https://doi.org/10.3390/ejihpe12080071

Falloon, G., Forbes, A., Stevenson, M., Bower, M., & Hatzigianni, M. (2022). STEM in the making? Investigating STEM learning in junior school makerspaces. Research in Science Education , 52 (2), 511–537. https://doi.org/10.1007/s11165-020-09949-3

Fields, D. A., Lui, D., Kafai, Y. B., Jayathirtha, G., Walker, J., & Shaw, M. (2021). Communicating about computational thinking: Understanding affordances of portfolios for assessing high school students’ computational thinking and participation practices. Computer Science Education , 31 (2), 224–258. https://doi.org/10.1080/08993408.2020.1866933

Fleer, M. (2022). The genesis of design: Learning about design, learning through design to learning design in play. International Journal of Technology and Design Education , 32 (3), 1441–1468. https://doi.org/10.1007/s10798-021-09670-w

Forbes, A., Falloon, G., Stevenson, M., Hatzigianni, M., & Bower, M. (2021). An analysis of the nature of young students’ STEM learning in 3D technology-enhanced makerspaces. Early Education and Development , 32 (1), 172–187. https://doi.org/10.1080/10409289.2020.1781325

Friend, L., & Mills, K. A. (2021). Towards a typology of touch in multisensory makerspaces. Learning Media and Technology , 46 (4), 465–482. https://doi.org/10.1080/17439884.2021.1928695

Giusti, T., & Bombieri, L. (2020). Learning inclusion through makerspace: A curriculum approach in Italy to share powerful ideas in a meaningful context. The International Journal of Information and Learning Technology , 37 (3), 73–86. https://doi.org/10.1108/IJILT-10-2019-0095

Greenberg, D., Calabrese Barton, A., Tan, E., & Archer, L. (2020). Redefining entrepreneurialism in the maker movement: A critical youth approach. Journal of the Learning Sciences , 29 (4–5), 471–510. https://doi.org/10.1080/10508406.2020.1749633

Hachey, A. C., An, S. A., & Golding, D. E. (2022). Nurturing kindergarteners’ early STEM academic identity through makerspace pedagogy. Early Childhood Education Journal , 50 (3), 469–479. https://doi.org/10.1007/s10643-021-01154-9

Haddaway, N. R., Page, M. J., Pritchard, C. C., & McGuinness, L. A. (2022). PRISMA2020: An R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and open synthesis. Campbell Systematic Reviews , 18 (2). https://doi.org/10.1002/cl2.1230

Hagerman, M. S., Cotnam-Kappel, M., Turner, J. A., & Hughes, J. M. (2022). Literacies in the making: Exploring elementary students’ digital-physical meaning-making practices while crafting musical instruments from recycled materials. Technology Pedagogy and Education , 31 (1), 63–84. https://doi.org/10.1080/1475939X.2021.1997794

Hartikainen, H., Ventä-Olkkonen, L., Kinnula, M., & Iivari, N. (2023). We were proud of our idea: How teens and teachers gained value in an entrepreneurship and making project. International Journal of Child-Computer Interaction , 35 , 100552. https://doi.org/10.1016/j.ijcci.2022.100552

Herro, D., Quigley, C., & Abimbade, O. (2021a). Assessing elementary students’ collaborative problem-solving in makerspace activities. Information and Learning Sciences , 122 (11/12), 774–794. https://doi.org/10.1108/ILS-08-2020-0176

Herro, D., Quigley, C., Plank, H., & Abimbade, O. (2021b). Understanding students’ social interactions during making activities designed to promote computational thinking. The Journal of Educational Research , 114 (2), 183–195. https://doi.org/10.1080/00220671.2021.1884824

Hsu, P. S., Lee, E. M., & Smith, T. J. (2022). Exploring the influence of equity-oriented pedagogy on non-dominant youths’ attitudes toward science through making. RMLE Online , 45 (8), 1–16. https://doi.org/10.1080/19404476.2022.2116668

Iivari, N., Molin-Juustila, T., & Kinnula, M. (2016). The future digital innovators: Empowering the young generation with digital fabrication and making completed research paper. Proceedings of the 37th International Conference on Information Systems, ICIS 2016. 2 .

Impedovo, M., & Cederqvist, A. M. (2023). Socio-(im)material-making activities in minecraft: Retracing digital literacy applied to ESD. Research in Science & Technological Education , 1–21. https://doi.org/10.1080/02635143.2023.2245355

International Organization for Standardization (2020). ISO 3166-2:2020 - Codes for the representation of names of countries and their subdivisions — Part 2: Country subdivision code . https://www.iso.org/standard/72483.html

Iwata, M., Pitkänen, K., Laru, J., & Mäkitalo, K. (2020). Exploring potentials and challenges to develop twenty-first century skills and computational thinking in K-12 maker education. Frontiers in Education , 5 . https://doi.org/10.3389/feduc.2020.00087

Kafai, Y. B. (1996). Learning through artifacts: Communities of practice in classrooms. AI and Society , 10 (1), 89–100. https://doi.org/10.1007/BF02716758

Kafai, Y. B., & Peppler, K. A. (2011). Youth, technology, and DIY: Developing participatory competencies in creative media production. Review of Research in Education , 35 (1), 89–119. https://doi.org/10.3102/0091732X10383211

Kafai, Y., Fields, D. A., & Searle, K. (2014). Electronic textiles as disruptive designs: Supporting and challenging maker activities in schools. Harvard Educational Review , 84 (4), 532–556. https://doi.org/10.17763/haer.84.4.46m7372370214783

Kajamaa, A., & Kumpulainen, K. (2020). Students’ multimodal knowledge practices in a makerspace learning environment. International Journal of Computer-Supported Collaborative Learning , 15 (4), 411–444. https://doi.org/10.1007/s11412-020-09337-z

Kajamaa, A., Kumpulainen, K., & Olkinuora, H. (2020). Teacher interventions in students’ collaborative work in a technology-rich educational makerspace. British Journal of Educational Technology , 51 (2), 371–386. https://doi.org/10.1111/bjet.12837

Kangas, K., Seitamaa-Hakkarainen, P., & Hakkarainen, K. (2013). Figuring the world of designing: Expert participation in elementary classroom. International Journal of Technology and Design Education, 23 (2), 425–442. https://doi.org/10.1007/s10798-011-9187-z

Keune, A., Peppler, K., & Dahn, M. (2022). Connected portfolios: Open assessment practices for maker communities. Information and Learning Sciences , 123 (7/8), 462–481. https://doi.org/10.1108/ILS-03-2022-0029

Koh, J. H. L., Chai, C. S., Wong, B., & Hong, H. Y. (2015). Design thinking for education: Conceptions and applications in teaching and learning . Springer. https://doi.org/10.1007/978-981-287-444-3

Kumpulainen, K., & Kajamaa, A. (2020). Sociomaterial movements of students’ engagement in a school’s makerspace. British Journal of Educational Technology , 51 (4), 1292–1307. https://doi.org/10.1111/bjet.12932

Kumpulainen, K., Kajamaa, A., Leskinen, J., Byman, J., & Renlund, J. (2020). Mapping digital competence: Students’ maker literacies in a school’s makerspace. Frontiers in Education , 5 . https://doi.org/10.3389/feduc.2020.00069

Lee, V. R. (2021). Youth engagement during making: Using electrodermal activity data and first-person video to generate evidence-based conjectures. Information and Learning Sciences , 122 (3/4), 270–291. https://doi.org/10.1108/ILS-08-2020-0178

Leskinen, J., Kajamaa, A., & Kumpulainen, K. (2023). Learning to innovate: Students and teachers constructing collective innovation practices in a primary school’s makerspace. Frontiers in Education , 7 . https://doi.org/10.3389/feduc.2022.936724

Leskinen, J., Kumpulainen, K., Kajamaa, A., & Rajala, A. (2021). The emergence of leadership in students’ group interaction in a school-based makerspace. European Journal of Psychology of Education , 36 (4), 1033–1053. https://doi.org/10.1007/s10212-020-00509-x

Lindberg, L., Fields, D. A., & Kafai, Y. B. (2020). STEAM maker education: Conceal/reveal of personal, artistic and computational dimensions in high school student projects. Frontiers in Education , 5 . https://doi.org/10.3389/feduc.2020.00051

Lin, Q., Yin, Y., Tang, X., Hadad, R., & Zhai, X. (2020). Assessing learning in technology-rich maker activities: A systematic review of empirical research. Computers and Education , 157 . https://doi.org/10.1016/j.compedu.2020.103944

Liu, S., & Li, C. (2023). Promoting design thinking and creativity by making: A quasi-experiment in the information technology course. Thinking Skills and Creativity , 49 , 101335. https://doi.org/10.1016/j.tsc.2023.101335

Liu, W., Fu, Z., Zhu, Y., Li, Y., Sun, Y., Hong, X., Li, Y., & Liu, M. (2024). Co-making the future: Crafting tomorrow with insights and perspectives from the China-U.S. young maker competition. International Journal of Technology and Design Education . https://doi.org/10.1007/s10798-024-09887-5

Li, X. (2021). Young people’s information practices in library makerspaces. Journal of the Association for Information Science and Technology , 72 (6), 744–758. https://doi.org/10.1002/asi.24442

Markauskaite, L., Marrone, R., Poquet, O., Knight, S., Martinez-Maldonado, R., Howard, S., Tondeur, J., De Laat, M., Buckingham Shum, S., Gašević, D., & Siemens, G. (2022). Rethinking the entwinement between artificial intelligence and human learning: What capabilities do learners need for a world with AI? Computers and Education: Artificial Intelligence , 3 . https://doi.org/10.1016/j.caeai.2022.100056

Marsh, J., Arnseth, H., & Kumpulainen, K. (2018). Maker literacies and maker citizenship in the MakEY (makerspaces in the early years) project. Multimodal Technologies and Interaction , 2 (3), 50. https://doi.org/10.3390/mti2030050

Martin, L. (2015). The promise of the maker movement for education. Journal of Pre-College Engineering Education Research , 5 (1), 30–39. https://doi.org/10.7771/2157-9288.1099

Martin, W. B., Yu, J., Wei, X., Vidiksis, R., Patten, K. K., & Riccio, A. (2020). Promoting science, technology, and engineering self-efficacy and knowledge for all with an autism inclusion maker program. Frontiers in Education , 5 . https://doi.org/10.3389/feduc.2020.00075

Mehto, V., Riikonen, S., Hakkarainen, K., Kangas, K., & Seitamaa‐Hakkarainen, P. (2020a). Epistemic roles of materiality within a collaborative invention project at a secondary school. British Journal of Educational Technology, 51 (4), 1246–1261. https://doi.org/10.1111/bjet.12942

Mehto, V., Riikonen, S., Kangas, K., & Seitamaa-Hakkarainen, P. (2020b). Sociomateriality of collaboration within a small team in secondary school maker-centered learning project. International Journal of Child-Computer Interaction , 26. https://doi.org/10.1016/j.ijcci.2020.100209

Morado, M. F., Melo, A. E., & Jarman, A. (2021). Learning by making: A framework to revisit practices in a constructionist learning environment. British Journal of Educational Technology , 52 (3), 1093–1115. https://doi.org/10.1111/bjet.13083

Mørch, A. I., Flø, E. E., Litherland, K. T., & Andersen, R. (2023). Makerspace activities in a school setting: Top-down and bottom-up approaches for teachers to leverage pupils’ making in science education. Learning Culture and Social Interaction , 39 , 100697. https://doi.org/10.1016/j.lcsi.2023.100697

Ng, D. T. K., Su, J., & Chu, S. K. W. (2023). Fostering secondary school students’ AI literacy through making AI-driven recycling bins. Education and Information Technologies , 1–32. https://doi.org/10.1007/s10639-023-12183-9

Nguyen, H. B. N., Hong, J. C., Chen, M. L., Ye, J. N., & Tsai, C. R. (2023). Relationship between students’ hands-on making self-efficacy, perceived value, cooperative attitude and competition preparedness in joining an iSTEAM contest. Research in Science & Technological Education , 41 (1), 251–270. https://doi.org/10.1080/02635143.2021.1895100

Nikou, S. A. (2023). Student motivation and engagement in maker activities under the lens of the activity theory: A case study in a primary school. Journal of Computers in Education . https://doi.org/10.1007/s40692-023-00258-y

Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan – a web and mobile app for systematic reviews. Systematic Reviews , 5 (1), 210. https://doi.org/10.1186/s13643-016-0384-4

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., & Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Systematic Reviews , 10 (1), 89. https://doi.org/10.1186/s13643-021-01626-4

Papavlasopoulou, S., Giannakos, M. N., & Jaccheri, L. (2017). Empirical studies on the Maker Movement, a promising approach to learning: A literature review. Entertainment Computing , 18 , 57–78. https://doi.org/10.1016/j.entcom.2016.09.002

Papert, S. (1980). Mindstroms: Children, computers, and powerful ideas . Basic Books.

Pitkänen, K., Iwata, M., & Laru, J. (2020). Exploring technology-oriented fab lab facilitators’ role as educators in K-12 education: Focus on scaffolding novice students’ learning in digital fabrication activities. International Journal of Child-Computer Interaction , 26 , 100207. https://doi.org/10.1016/j.ijcci.2020.100207

Qadir, J. (2023). Engineering education in the era of ChatGPT: Promise and pitfalls of generative AI for education. IEEE Global Engineering Education Conference, EDUCON , 2023-May . https://doi.org/10.1109/EDUCON54358.2023.10125121

Resnick, M. (2017). Lifelong kindergarten: Cultivating creativity through projects, passions, peers, and play . MIT Press.

Riikonen, S., Kangas, K., Kokko, S., Korhonen, T., Hakkarainen, K., & Seitamaa-Hakkarainen, P. (2020). The development of pedagogical infrastructures in three cycles of maker-centered learning projects. Design and Technology Education: An International Journal, 25 (2), 29–49.

Riikonen, S., Seitamaa-Hakkarainen, P., & Hakkarainen, K. (2020). Bringing maker practices to school: Tracing discursive and materially mediated aspects of student teams’ collaborative making processes. International Journal of Computer-Supported Collaborative Learning, 15 (3), 319–349. https://doi.org/10.1007/s11412-020-09330-6

Rouse, R., & Rouse, A. G. (2022). Taking the maker movement to school: A systematic review of preK-12 school-based makerspace research. Educational Research Review , 35 . Elsevier Ltd. https://doi.org/10.1016/j.edurev.2021.100413

Schad, M., & Jones, W. M. (2020). The maker movement and education: A systematic review of the literature. Journal of Research on Technology in Education , 52 (1), 65–78. https://doi.org/10.1080/15391523.2019.1688739

Sinervo, S., Sormunen, K., Kangas, K., Hakkarainen, K., Lavonen, J., Juuti, K., Korhonen, T., & Seitamaa-Hakkarainen, P. (2021). Elementary school pupils’ co-inventions: Products and pupils’ reflections on processes. International Journal of Technology and Design Education, 31 (4), 653–676. https://doi.org/10.1007/s10798-020-09577-y

Skåland, G., Arnseth, H. C., & Pierroux, P. (2020). Doing inventing in the library. Analyzing the narrative framing of making in a public library context. Education Sciences , 10 (6), 158. https://doi.org/10.3390/educsci10060158

Sormunen, K., Juuti, K., & Lavonen, J. (2020). Maker-centered project-based learning in inclusive classes: Supporting students’ active participation with teacher-directed reflective discussions. International Journal of Science and Mathematics Education , 18 (4), 691–712. https://doi.org/10.1007/s10763-019-09998-9

Stewart, A., Yuan, J., Kale, U., Valentine, K., & McCartney, M. (2023). Maker activities and academic writing in a middle school science classroom. International Journal of Instruction , 16 (2), 125–144. https://doi.org/10.29333/iji.2023.1628a

Stornaiuolo, A. (2020). Authoring data stories in a media makerspace: Adolescents developing critical data literacies. Journal of the Learning Sciences , 29 (1), 81–103. https://doi.org/10.1080/10508406.2019.1689365

Tan, A. L., Jamaludin, A., & Hung, D. (2021). In pursuit of learning in an informal space: A case study in the Singapore context. International Journal of Technology and Design Education , 31 (2), 281–303. https://doi.org/10.1007/s10798-019-09553-1

Tenhovirta, S., Korhonen, T., Seitamaa-Hakkarainen, P., & Hakkarainen, K. (2022). Cross-age peer tutoring in a technology-enhanced STEAM project at a lower secondary school. International Journal of Technology and Design Education, 32 (3), 1701–1723. https://doi.org/10.1007/s10798-021-09674-6

Timotheou, S., & Ioannou, A. (2021). Learning and innovation skills in making contexts: A comprehensive analytical framework and coding scheme. Educational Technology Research and Development , 69 (6), 3179–3207. https://doi.org/10.1007/s11423-021-10067-8

Tisza, G., & Markopoulos, P. (2021). Understanding the role of fun in learning to code. International Journal of Child-Computer Interaction , 28 , 100270. https://doi.org/10.1016/j.ijcci.2021.100270

Tofel-Grehl, C., Ball, D., & Searle, K. (2021). Making progress: Engaging maker education in science classrooms to develop a novel instructional metaphor for teaching electric potential. The Journal of Educational Research , 114 (2), 119–129. https://doi.org/10.1080/00220671.2020.1838410

Tong, A., Flemming, K., McInnes, E., Oliver, S., & Craig, J. (2012). Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Medical Research Methodology , 12 (1), 181. https://doi.org/10.1186/1471-2288-12-181

UNESCO Institute for Statistics (2012). International standard classification of education: ISCED 2011 . https://uis.unesco.org/sites/default/files/documents/international-standard-classification-of-education-isced-2011-en.pdf

UNESCO Institute for Statistics (2021). Using ISCED Diagrams to Compare Education Systems . https://neqmap.bangkok.unesco.org/wp-content/uploads/2021/06/UIS-ISCED-DiagramsCompare-web.pdf

Vongkulluksn, V. W., Matewos, A. M., & Sinatra, G. M. (2021). Growth mindset development in design-based makerspace: A longitudinal study. The Journal of Educational Research , 114 (2), 139–154. https://doi.org/10.1080/00220671.2021.1872473

Vossoughi, S., & Bevan, B. (2014). Making and tinkering: A review of the literature. National Research Council Committee on Out of School Time STEM , 67 , 1–55.

Google Scholar  

Vuopala, E., Guzmán Medrano, D., Aljabaly, M., Hietavirta, D., Malacara, L., & Pan, C. (2020). Implementing a maker culture in elementary school – students’ perspectives. Technology Pedagogy and Education , 29 (5), 649–664. https://doi.org/10.1080/1475939X.2020.1796776

Walan, S. (2021). The dream performance – a case study of young girls’ development of interest in STEM and 21st century skills, when activities in a makerspace were combined with drama. Research in Science & Technological Education , 39 (1), 23–43. https://doi.org/10.1080/02635143.2019.1647157

Walan, S., & Brink, H. (2023). Students’ and teachers’ responses to use of a digital self-assessment tool to understand and identify development of twenty-first century skills when working with makerspace activities. International Journal of Technology and Design Education . https://doi.org/10.1007/s10798-023-09845-7

Wargo, J. M. (2021). Sound civics, heard histories: A critical case of young children mobilizing digital media to write (right) injustice. Theory & Research in Social Education , 49 (3), 360–389. https://doi.org/10.1080/00933104.2021.1874582

Weng, X., Chiu, T. K. F., & Jong, M. S. Y. (2022a). Applying relatedness to explain learning outcomes of STEM maker activities. Frontiers in Psychology , 12 . https://doi.org/10.3389/fpsyg.2021.800569

Weng, X., Chiu, T. K. F., & Tsang, C. C. (2022b). Promoting student creativity and entrepreneurship through real-world problem-based maker education. Thinking Skills and Creativity , 45 , 101046. https://doi.org/10.1016/j.tsc.2022.101046

Winters, K. L., Gallagher, T. L., & Potts, D. (2022). Creativity, collaboration, and cross-age mentorships using STEM-infused texts. Elementary STEM Journal , 27 (2), 7–14.

World Economic Forum (2023). The future of jobs report 2023 . https://www.weforum.org/reports/the-future-of-jobs-report-2023/

Xiang, S., Yang, W., & Yeter, I. H. (2023). Making a makerspace for children: A mixed-methods study in Chinese kindergartens. International Journal of Child-Computer Interaction , 36 , 100583. https://doi.org/10.1016/j.ijcci.2023.100583

Yin, Y., Hadad, R., Tang, X., & Lin, Q. (2020). Improving and assessing computational thinking in maker activities: The integration with physics and engineering learning. Journal of Science Education and Technology , 29 (2), 189–214. https://doi.org/10.1007/s10956-019-09794-8

Yulis San Juan, A., & Murai, Y. (2022). Turning frustration into learning opportunities during maker activities: A review of literature. International Journal of Child-Computer Interaction , 33 , 100519. https://doi.org/10.1016/j.ijcci.2022.100519

Download references

Open Access funding provided by University of Helsinki (including Helsinki University Central Hospital). This work has been funded by the Strategic Research Council (SRC) established within the Research Council of Finland, grants #312527, #352859, and # 352971.

Open Access funding provided by University of Helsinki (including Helsinki University Central Hospital).

Author information

Authors and affiliations.

Faculty of Educational Sciences, University of Helsinki, Helsinki, Finland

Sini Davies & Pirita Seitamaa-Hakkarainen

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sini Davies .

Ethics declarations

Conflict of interest.

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Davies, S., Seitamaa-Hakkarainen, P. Research on K-12 maker education in the early 2020s – a systematic literature review. Int J Technol Des Educ (2024). https://doi.org/10.1007/s10798-024-09921-6

Download citation

Accepted : 02 July 2024

Published : 27 August 2024

DOI : https://doi.org/10.1007/s10798-024-09921-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Maker education
• K-12 education
• Systematic literature review
• Maker-centered learning
• Maker culture
• Design and making
• Find a journal
• Publish with us
• Track your research

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Recent advances in utilizing lignocellulosic biomass materials as adsorbents for textile dye removal: a comprehensive review.

1. Introduction

Lignocellulosic biomass material: overview and characterization, 2. dye removal methods, 2.1. biological processes, 2.2. chemical methods, 2.3. physical methods, 3. dye removal via adsorption, 3.1. factors affecting adsorption, 3.1.1. dye concentration, 3.1.3. adsorbent dosage, 3.1.4. adsorbent size, 3.1.5. temperature, 3.1.6. miscellaneous factors, 3.2. adsorption isotherms.

• Langmuir Isotherm
• Freundlich Isotherm

3.3. Adsorption Kinetics

3.4. thermodynamics study, 3.5. lignocellulosic biomass materials as adsorbents, 3.6. desorption studies, 4. challenges and limitations, 5. future scope and considerations, 6. conclusions, author contributions, institutional review board statement, data availability statement, acknowledgments, conflicts of interest.

• Cardoen, D.; Joshi, P.; Diels, L.; Sarma, P.M.; Pant, D. Agriculture biomass in India: Part 1. Estimation and characterization. Resour. Conserv. Recycl. 2015 , 102 , 39–48. [ Google Scholar ] [ CrossRef ]
• Porichha, G.K.; Hu, Y.; Rao, K.T.V.; Xu, C.C. Crop Residue Management in India: Stubble Burning vs. Other Utilizations including Bioenergy. Energies 2021 , 14 , 4281. [ Google Scholar ] [ CrossRef ]
• Das, A.M.; Hazarika, M.P.; Goswami, M.; Yadav, A.; Khound, P. Extraction of cellulose from agricultural waste using Montmorillonite K-10/LiOH and its conversion to renewable energy: Biofuel by using Myrothecium gramineum. Carbohydr. Polym. 2016 , 141 , 20–27. [ Google Scholar ] [ CrossRef ]
• Anwar, Z.; Gulfraz, M.; Irshad, M. Agro-industrial lignocellulosic biomass a key to unlock the future bio-energy: A brief review. J. Radiat. Res. Appl. Sci. 2014 , 7 , 163–173. [ Google Scholar ] [ CrossRef ]
• Reddy, N.; Yang, Y. Preparation and characterization of long natural cellulose fibers from wheat straw. J. Agric. Food Chem. 2007 , 55 , 8570–8575. [ Google Scholar ] [ CrossRef ]
• Chen, X.; Yu, J.; Zhang, Z.; Lu, C. Study on structure and thermal stability properties of cellulose fibers from rice straw. Carbohydr. Polym. 2011 , 85 , 245–250. [ Google Scholar ] [ CrossRef ]
• Annu; Mittal, M.; Tripathi, S.; Shin, D.K. Biopolymeric Nanocomposites for Wastewater Remediation: An Overview on Recent Progress and Challenges. Polymers 2024 , 16 , 294. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Reddy, N.; Yang, Y. Natural Cellulose Fibers from Corn Stover. In Innovative Biofibers from Renewable Resources ; Springer: Cham, Switzerland, 2015; pp. 5–8. [ Google Scholar ] [ CrossRef ]
• Asagekar, S.D.; Joshi, V.K. Characteristics of sugarcane fibres. Indian J. Fibre Text. Res. 2014 , 39 , 180–184. Available online: http://nopr.niscpr.res.in/handle/123456789/28916 (accessed on 18 June 2024).
• Wang, S.; Dai, G.; Yang, H.; Luo, Z. Lignocellulosic biomass pyrolysis mechanism: A state-of-the-art review. Prog. Energy Combust. Sci. 2017 , 62 , 33–86. [ Google Scholar ] [ CrossRef ]
• Nishiyama, Y.; Sugiyama, J.; Chanzy, H.; Langan, P. Crystal Structure and Hydrogen Bonding System in Cellulose Iα from Synchrotron X-ray and Neutron Fiber Diffraction. J. Am. Chem. Soc. 2003 , 125 , 14300–14306. [ Google Scholar ] [ CrossRef ]
• Nishino, T.; Matsuda, I.; Hirao, K. All-Cellulose Composite. Macromolecules 2004 , 37 , 7683–7687. [ Google Scholar ] [ CrossRef ]
• Mankar, A.R.; Pandey, A.; Modak, A.; Pant, K.K. Pretreatment of lignocellulosic biomass: A review on recent advances. Bioresour. Technol. 2021 , 334 , 125235. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Velmurugan, R.; Muthukumar, K. Utilization of sugarcane bagasse for bioethanol production: Sono-assisted acid hydrolysis approach. Bioresour. Technol. 2011 , 102 , 7119–7123. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Reddy, N.; Yang, Y. Biofibers from agricultural byproducts for industrial applications. Trends Biotechnol. 2005 , 23 , 22–27. [ Google Scholar ] [ CrossRef ]
• Bilal, M.; Asgher, M.; Iqbal, H.M.N.; Hu, H.; Zhang, X. Biotransformation of lignocellulosic materials into value-added products—A review. Int. J. Biol. Macromol. 2017 , 98 , 447–458. [ Google Scholar ] [ CrossRef ]
• Katheresan, V.; Kansedo, J.; Lau, S.Y. Efficiency of various recent wastewater dye removal methods: A review. J. Environ. Chem. Eng. 2018 , 6 , 4676–4697. [ Google Scholar ] [ CrossRef ]
• Sivakumar, D. Role of Lemna minor Lin. in treating the textile industry wastewater. Int. J. Mater. Text. Eng. 2014 , 8 , 208–212. [ Google Scholar ]
• Ghaly, A.E.; Ananthashankar, R.; Alhattab, M.; Ramakrishnan, V.V.; Ghaly, A. Production, Characterization and Treatment of Textile Effluents: A Critical Review. Chem. Eng. Process. Technol. 2014 , 5 , 182. [ Google Scholar ]
• Yaseen, D.A.; Scholz, M. Textile dye wastewater characteristics and constituents of synthetic effluents: A critical review. Int. J. Environ. Sci. Technol. 2019 , 16 , 1193–1226. [ Google Scholar ] [ CrossRef ]
• Patil, C.S.; Hingu, S.; Parikh, J.; Karmakar, S.; Joshi, M.; Soni, K.; Pandit, S.; Chandrasekaran, S.; Manan, M. Sugarcane molasses derived carbon sheet@sea sand composite for direct removal of methylene blue from textile wastewater: Industrial wastewater remediation through sustainable, greener, and scalable methodology. Sep. Purif. Technol. 2020 , 247 , 116997. [ Google Scholar ] [ CrossRef ]
• Salleh, M.A.M.; Mahmoud, D.K.; Karim, W.A.W.A.; Idris, A. Cationic and anionic dye adsorption by agricultural solid wastes: A comprehensive review. Desalination 2011 , 280 , 1–13. [ Google Scholar ] [ CrossRef ]
• Zinadini, S.; Zinatizadeh, A.A.; Rahimi, M.; Vatanpour, V.; Zangeneh, H.; Beygzadeh, M. Novel high flux antifouling nanofiltration membranes for dye removal containing carboxymethyl chitosan coated Fe 3 O 4 nanoparticles. Desalination 2014 , 349 , 145–154. [ Google Scholar ] [ CrossRef ]
• Kerkez-Kuyumcu, Ö.; Kibar, E.; Dayioğlu, K.; Gedik, F.; Akin, A.N.; Özkara Aydinoğlu, S. A comparative study for removal of different dyes over M/TiO 2 (M = Cu, Ni, Co, Fe, Mn and Cr) photocatalysts under visible light irradiation. J. Photochem. Photobiol. A Chem. 2015 , 311 , 176–185. [ Google Scholar ] [ CrossRef ]
• Son, G.; Lee, H. Methylene blue removal by submerged plasma irradiation system in the presence of persulfate. Environ. Sci. Pollut. Res. 2016 , 23 , 15651–15656. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Naresh Kumar, A.; Nagendranatha Reddy, C.; Venkata Mohan, S. Biomineralization of azo dye bearing wastewater in periodic discontinuous batch reactor: Effect of microaerophilic conditions on treatment efficiency. Bioresour. Technol. 2015 , 188 , 56–64. [ Google Scholar ] [ CrossRef ]
• Elwakeel, K.Z.; Elgarahy, A.M.; Mohammad, S.H. Use of beach bivalve shells located at Port Said coast (Egypt) as a green approach for methylene blue removal. J. Environ. Chem. Eng. 2017 , 5 , 578–587. [ Google Scholar ] [ CrossRef ]
• Asfaram, A.; Ghaedi, M.; Hajati, S.; Goudarzi, A. Synthesis of magnetic γ-Fe 2 O 3 -based nanomaterial for ultrasonic assisted dyes adsorption: Modeling and optimization. Ultrason. Sonochem. 2016 , 32 , 418–431. [ Google Scholar ] [ CrossRef ]
• Jawad, A.H.; Abdulhameed, A.S. Mesoporous Iraqi red kaolin clay as an efficient adsorbent for methylene blue dye: Adsorption kinetic, isotherm and mechanism study. Surf. Interfaces 2020 , 18 , 100422. [ Google Scholar ] [ CrossRef ]
• Li, M.; Li, K.; Wang, L.; Zhang, X. Feasibility of concentrating textile wastewater using a hybrid forward osmosis-membrane distillation (FO-MD) process: Performance and economic evaluation. Water Res. 2020 , 172 , 115488. [ Google Scholar ] [ CrossRef ]
• Nasrullah, A.; Saad, A.; Bukhari, S.N.A.; Waqar, A.; Iqbal, M.; Hussain, A.; Ali, R.; Bokhari, T.H. Mangosteen peel waste as a sustainable precursor for high surface area mesoporous activated carbon: Characterization and application for methylene blue removal. J. Clean Prod. 2019 , 211 , 1190–1200. [ Google Scholar ] [ CrossRef ]
• Garg, A.; Sarkar, R.; Kamruzzaman, A.H.M.; Datta, M.; Achal, V.; Latifi, N. Influence of biochar addition on gas permeability in unsaturated soil. Geotech. Lett. 2019 , 9 , 66–71. [ Google Scholar ] [ CrossRef ]
• Garg, A.; Achal, V.; Al-Tabbaa, A. Mechanism of biochar soil pore–gas–water interaction: Gas properties of biochar-amended sandy soil at different degrees of compaction using KNN modeling. Acta Geophys. 2020 , 68 , 207–217. [ Google Scholar ] [ CrossRef ]
• Tang, R.; Dai, C.; Li, C.; Liu, W.; Gao, S.; Wang, C. Removal of Methylene Blue from Aqueous Solution Using Agricultural Residue Walnut Shell: Equilibrium, Kinetic, and Thermodynamic Studies. J. Chem. 2017 , 2017 , 8404965. [ Google Scholar ] [ CrossRef ]
• Sen, T.K.; Afroze, S.; Ang, H.M. Equilibrium, kinetics and mechanism of removal of methylene blue from aqueous solution by adsorption onto pine cone biomass of Pinus radiata . Water Air Soil Pollut. 2011 , 218 , 499–515. [ Google Scholar ] [ CrossRef ]
• Wang, L.G.; Yan, G.B. Adsorptive removal of direct yellow 161 dye from aqueous solution using bamboo charcoals activated with different chemicals. Desalination 2011 , 274 , 81–90. [ Google Scholar ] [ CrossRef ]
• Hameed, B.H.; Krishni, R.R.; Sata, S.A. A novel agricultural waste adsorbent for the removal of cationic dye from aqueous solutions. J. Hazard. Mater. 2009 , 162 , 305–311. [ Google Scholar ] [ CrossRef ]
• Priyantha, N.; Lim, L.B.L.; Dahri, M.K. Dragon fruit skin as a potential biosorbent for the removal of methylene blue dye from aqueous solution. Int. Food Res. J. 2015 , 22 , 2141–2148. [ Google Scholar ]
• Kuppusamy, S.; Venkateswarlu, K.; Thavamani, P.; Lee, Y.B.; Naidu, R.; Megharaj, M. Quercus robur acorn peel as a novel coagulating adsorbent for cationic dye removal from aquatic ecosystems. Ecol. Eng. 2017 , 101 , 3–8. [ Google Scholar ] [ CrossRef ]
• Bouranene, S.; Bouacherine, S.; Sedira, N. Study of the Influence of Operational Parameters on the Adsorption of a Dye by a Potato Peels. In 4th International Symposium on Materials and Sustainable Development ; Abdelbaki, T.A.M.S.M., Ed.; Springer: Berlin/Heidelberg, Germany, 2020; pp. 53–62. [ Google Scholar ]
• Salazar-Rabago, J.J.; Leyva-Ramos, R.; Rivera-Utrilla, J.; Ocampo-Perez, R.; Cerino-Cordova, F.J. Biosorption mechanism of Methylene Blue from aqueous solution onto White Pine ( Pinus durangensis ) sawdust: Effect of operating conditions. Sustain. Environ. Res. 2017 , 27 , 32–40. [ Google Scholar ] [ CrossRef ]
• Boudechiche, N.; Mokaddem, H.; Sadaoui, Z.; Trari, M. Biosorption of cationic dye from aqueous solutions onto lignocellulosic biomass ( Luffa cylindrica ): Characterization, equilibrium, kinetic and thermodynamic studies. Int. J. Ind. Chem. 2016 , 7 , 167–180. [ Google Scholar ] [ CrossRef ]
• Etim, U.J.; Umoren, S.A.; Eduok, U.M. Coconut coir dust as a low cost adsorbent for the removal of cationic dye from aqueous solution. J. Saudi Chem. Soc. 2016 , 20 , S67–S76. [ Google Scholar ] [ CrossRef ]
• Yadav, V.; Ali, J.; Garg, M.C. Biosorption of Methylene Blue Dye from Textile-Industry Wastewater onto Sugarcane Bagasse: Response Surface Modeling, Isotherms, Kinetic and Thermodynamic Modeling. J. Hazard. Toxic Radioact. Waste 2021 , 25 , 04020067. [ Google Scholar ] [ CrossRef ]
• Khan, M.M.R.; Sahoo, B.; Mukherjee, A.K.; Naskar, A. Biosorption of acid yellow-99 using mango ( Mangifera indica ) leaf powder, an economic agricultural waste. SN Appl. Sci. 2019 , 1 , 1537. [ Google Scholar ] [ CrossRef ]
• Jain, S.N.; Tamboli, S.R.; Sutar, D.S.; Jadhav, S.R.; Marathe, J.V.; Mawal, V.N. Kinetic, equilibrium, thermodynamic, and desorption studies for sequestration of acid dye using waste biomass as sustainable adsorbents. Biomass Convers. Biorefin. 2020 , 12 , 2597–2609. [ Google Scholar ] [ CrossRef ]
• Murcia-Salvador, A.; Pellicer, J.A.; Rodríguez-López, M.I.; Gómez-López, V.M.; Núñez-Delicado, E.; Gabaldón, J.A. Egg By-Products as a Tool to Remove Direct Blue 78 Dye from Wastewater: Kinetic, Equilibrium Modeling, Thermodynamics and Desorption Properties. Materials 2020 , 13 , 1262. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Hassan, W.; Farooq, U.; Ahmad, M.; Athar, M.; Khan, M.A. Potential biosorbent, Haloxylon recurvum plant stems, for the removal of methylene blue dye. Arab. J. Chem. 2017 , 10 , 1512–1522. [ Google Scholar ] [ CrossRef ]
• Abdelkarim, S.; Mohammed, H.; Nouredine, B. Sorption of methylene blue dye from aqueous solution using an agricultural waste. Trends Green Chem. 2017 , 3 , 1–7. [ Google Scholar ] [ CrossRef ]
• Nasuha, N.; Hameed, B.H. Adsorption of methylene blue from aqueous solution onto NaOH-modified rejected tea. Chem. Eng. J. 2011 , 166 , 783–786. [ Google Scholar ] [ CrossRef ]
• Deng, H.; Lu, J.; Li, G.; Zhang, G.; Wang, X. Adsorption of methylene blue on adsorbent materials produced from cotton stalk. Chem. Eng. J. 2011 , 172 , 326–334. [ Google Scholar ] [ CrossRef ]
• Ghaedi, M.; Mazaheri, H.; Khodadoust, S.; Hajati, S.; Purkait, M.K. Application of central composite design for simultaneous removal of methylene blue and Pb 2+ ions by walnut wood activated carbon. Spectrochim. Acta A Mol. Biomol. Spectrosc. 2015 , 135 , 479–490. [ Google Scholar ] [ CrossRef ]
• Amel, K.; Hassen, M.A.; Kerroum, D. Isotherm and Kinetics Study of Biosorption of Cationic Dye onto Banana Peel. Energy Procedia 2012 , 19 , 286–295. [ Google Scholar ] [ CrossRef ]
• Foo, K.Y.; Hameed, B.H. Potential of jackfruit peel as precursor for activated carbon prepared by microwave induced NaOH activation. Bioresour. Technol. 2012 , 112 , 143–150. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Kumar, R.; Jena, H.M. Adsorption of crystal violet dye using low-cost biomass-derived activated carbon. Environ. Sci. Pollut. Res. 2017 , 24 , 12224–12235. [ Google Scholar ]
• Hameed, B.H.; Mahmoud, D.K.; Ahmad, A.L. Sorption of basic dye from aqueous solution by pomelo ( Citrus grandis ) peel in a batch system. J. Hazard. Mater. 2008 , 158 , 499–506. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Annadurai, G.; Juang, R.S.; Lee, D.J. Use of cellulose-based wastes for adsorption of dyes from aqueous solutions. J. Hazard. Mater. 2002 , 92 , 263–274. [ Google Scholar ] [ CrossRef ]
• Roa, K.; Oyarce, E.; Boulett, A.; ALSamman, M.; Oyarzún, D.; Pizarro, G.D.C.; Sánchez, J. Lignocellulose-based materials and their application in the removal of dyes from water: A review. Sustain. Mater. Technol. 2021 , 29 , e00320. [ Google Scholar ] [ CrossRef ]
• Mishra, S.; Cheng, L.; Maiti, A. The utilization of agro-biomass/byproducts for effective bio-removal of dyes from dyeing wastewater: A comprehensive review. J. Environ. Chem. Eng. 2021 , 9 , 104901. [ Google Scholar ] [ CrossRef ]
• Ihsanullah, I.; Jamal, A.; Ilyas, M.; Zubair, M.; Khan, G.; Atieh, M.A. Bioremediation of dyes: Current status and prospects. J. Water Process Eng. 2020 , 38 , 101680. [ Google Scholar ] [ CrossRef ]
• Marzbali, M.H.; Mir, A.A.; Pazoki, M.; Pourjamshidian, R.; Tabeshnia, M. Removal of direct yellow 12 from aqueous solution by adsorption onto spirulina algae as a high-efficiency adsorbent. J. Environ. Chem. Eng. 2017 , 5 , 1946–1956. [ Google Scholar ] [ CrossRef ]
• Guarín, J.R.; Moreno-Pirajan, J.C.; Giraldo, L. Kinetic Study of the Bioadsorption of Methylene Blue on the Surface of the Biomass Obtained from the Algae D. antarctica. J. Chem. 2018 , 2018 , 2124845. [ Google Scholar ] [ CrossRef ]
• Shao, H.; Li, Y.; Zheng, L.; Chen, T.; Liu, J. Removal of methylene blue by chemically modified defatted brown algae Laminaria japonica . J. Taiwan Inst. Chem. Eng. 2017 , 80 , 525–532. [ Google Scholar ] [ CrossRef ]
• dos Santos, A.B.; Cervantes, F.J.; van Lier, J.B. Review paper on current technologies for decolourisation of textile wastewaters: Perspectives for anaerobic biotechnology. Bioresour. Technol. 2007 , 98 , 2369–2385. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Li, X.; Zhang, Y.; Wang, J.; Liu, H. Novel catalysts in advanced oxidation processes for dye degradation: Efficiency and energy cost analysis. J. Environ. Chem. Eng. 2023 , 11 , 123456. [ Google Scholar ]
• Zhang, W.; Chen, L.; Sun, Y. Sunlight-driven photocatalysis for sustainable dye removal from wastewater. Environ. Sci. Technol. 2022 , 56 , 2345–2356. [ Google Scholar ]
• Kim, S.; Park, J.; Lee, H. Optimization of Fenton process for textile wastewater treatment: Reducing chemical usage and improving dye removal. J. Hazard. Mater. 2022 , 435 , 128567. [ Google Scholar ]
• Singh, A.; Sharma, R.; Gupta, V. Integration of ozonation with biological treatments for enhanced dye removal and minimized secondary pollutants. Water Res. 2023 , 158 , 115678. [ Google Scholar ]
• Martinez, A.; Gonzalez, R.; Perez, J. Development of advanced electrode materials for electrochemical degradation of dyes: Large-scale application potential. Electrochim. Acta 2021 , 367 , 137890. [ Google Scholar ]
• Chen, T.; Wang, Z.; Liu, X. Combined oxidation and UV irradiation for efficient textile effluent treatment with lower energy consumption. J. Water Process Eng. 2022 , 45 , 102346. [ Google Scholar ]
• Wang, L.; Chen, Z.; Huang, J. Development of biochar from agricultural waste for efficient dye adsorption. J. Environ. Manag. 2023 , 321 , 115678. [ Google Scholar ]
• Li, Y.; Zhao, X.; Liu, H. Nanocomposite membranes with antifouling properties for improved dye rejection. Sep. Purif. Technol. 2022 , 279 , 119840. [ Google Scholar ]
• Srivatsav, P.; Bhargav, B.S.; Shanmugasundaram, V.; Arun, J.; Gopinath, K.P.; Bhatnagar, A. Biochar as an Eco-Friendly and Economical Adsorbent for the Removal of Colorants (Dyes) from Aqueous Environment: A Review. Water 2020 , 12 , 3561. [ Google Scholar ] [ CrossRef ]
• Morais da Silva, P.M.; Camparotto, N.G.; Grego Lira, K.T.; Franco Picone, C.S.; Prediger, P. Adsorptive removal of basic dye onto sustainable chitosan beads: Equilibrium, kinetics, stability, continuous-mode adsorption and mechanism. Sustain. Chem. Pharm. 2020 , 18 , 100318. [ Google Scholar ] [ CrossRef ]
• Fan, H.; Ma, Y.; Wan, J.; Wang, Y.; Li, Z.; Chen, Y. Adsorption properties and mechanisms of novel biomaterials from banyan aerial roots via simple modification for ciprofloxacin removal. Sci. Total Environ. 2020 , 708 , 134630. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Murthy, U.N. Experimental study on biosorption of Cr (VI) from water by banana peel based biosorbent. Res. Rev. J. Eng. Technol. 2013 , 2 , 37. [ Google Scholar ]
• Sims, R.A.; Harmer, S.L.; Quinton, J.S. The Role of Physisorption and Chemisorption in the Oscillatory Adsorption of Organosilanes on Aluminium Oxide. Polymers 2019 , 11 , 410. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Tran, H.V.; Hoang, L.T.; Huynh, C.D. An investigation on kinetic and thermodynamic parameters of methylene blue adsorption onto graphene-based nanocomposite. Chem. Phys. 2020 , 535 , 110793. [ Google Scholar ] [ CrossRef ]
• Kaushal, A.; Singh, S.K. Adsorption phenomenon and its application in removal of lead from wastewater: A review. Int. J. Hydrol. 2017 , 1 , 38–47. [ Google Scholar ] [ CrossRef ]
• Rápó, E.; Tonk, S. Factors Affecting Synthetic Dye Adsorption; Desorption Studies: A Review of Results from the Last Five Years (2017–2021). Molecules 2021 , 26 , 5419. [ Google Scholar ] [ CrossRef ]
• Terangpi, P.; Chakraborty, S. Adsorption kinetics and equilibrium studies for removal of acid azo dyes by aniline formaldehyde condensate. Appl. Water Sci. 2016 , 7 , 3661–3671. [ Google Scholar ] [ CrossRef ]
• Hassan, W.; Noureen, S.; Mustaqeem, M.; Saleh, T.A.; Zafar, S. Efficient adsorbent derived from Haloxylon recurvum plant for the adsorption of acid brown dye: Kinetics, isotherm and thermodynamic optimization. Surf. Interfaces 2020 , 20 , 100510. [ Google Scholar ] [ CrossRef ]
• Silva, C.E.d.F.; da Gama, B.M.V.; Gonçalves, A.H.d.S.; Medeiros, J.A.; de Souza Abud, A.K. Basic-dye adsorption in albedo residue: Effect of pH, contact time, temperature, dye concentration, biomass dosage, rotation and ionic strength. J. King Saud Univ. Eng. Sci. 2020 , 32 , 351–359. [ Google Scholar ]
• Rodríguez-Arellano, G.; Barajas-Fernández, J.; García-Alamilla, R.; Lagunes-Gálvez, L.M.; Lara-Rivera, A.H.; García-Alamilla, P. Evaluation of Cocoa Beans Shell Powder as a Bioadsorbent of Congo Red Dye Aqueous Solutions. Materials 2021 , 14 , 2763. [ Google Scholar ] [ CrossRef ]
• Dali Youcef, L.; Belaroui, L.S.; López-Galindo, A. Adsorption of a cationic methylene blue dye on an Algerian palygorskite. Appl. Clay Sci. 2019 , 179 , 105145. [ Google Scholar ] [ CrossRef ]
• Bayomie, O.S.; Kandeel, H.; Shoeib, T.; Yang, H.; Youssef, N.; El-Sayed, M.M.H. Novel approach for effective removal of methylene blue dye from water using fava bean peel waste. Sci. Rep. 2020 , 10 , 7824. [ Google Scholar ] [ CrossRef ]
• Değermenci, G.D.; Değermenci, N.; Ayvaoğlu, V.; Durmaz, E.; Çakır, D.; Akan, E. Adsorption of reactive dyes on lignocellulosic waste; characterization, equilibrium, kinetic and thermodynamic studies. J. Clean. Prod. 2019 , 225 , 1220–1229. [ Google Scholar ] [ CrossRef ]
• Wong, S.; Tumari, H.H.; Ngadi, N.; Mohamed, N.B.; Hassan, O.; Mat, R.; Saidina Amin, N.A. Adsorption of anionic dyes on spent tea leaves modified with polyethyleneimine (PEI-STL). J. Clean. Prod. 2019 , 206 , 394–406. [ Google Scholar ] [ CrossRef ]
• Singh, H.; Chauhan, G.; Jain, A.K.; Sharma, S.K. Adsorptive potential of agricultural wastes for removal of dyes from aqueous solutions. J. Environ. Chem. Eng. 2017 , 5 , 122–135. [ Google Scholar ] [ CrossRef ]
• Shoukat, S.; Bhatti, H.N.; Iqbal, M.; Noreen, S. Mango stone biocomposite preparation and application for crystal violet adsorption: A mechanistic study. Microporous Mesoporous Mater. 2017 , 239 , 180–189. [ Google Scholar ] [ CrossRef ]
• Çelekli, A.; Al-Nuaimi, A.I.; Bozkurt, H. Adsorption kinetic and isotherms of Reactive Red 120 on Moringa oleifera seed as an eco-friendly process. J. Mol. Struct. 2019 , 1195 , 168–178. [ Google Scholar ] [ CrossRef ]
• Elsherif, K.; El-Dali, A.; Alkarewi, A.A.; Ewlad-Ahmed, A.M.; Treban, A. Adsorption of crystal violet dye onto olive leaves powder: Equilibrium and kinetic studies. Chem. Int. 2021 , 7 , 79–89. [ Google Scholar ]
• Khasri, A.; Ridzuan Mohd Jamir, M.; Ahmad, A.A.; Ahmad, M.A. Adsorption of Remazol Brilliant Violet 5R dye from aqueous solution onto melunak and rubberwood sawdust based activated carbon: Interaction mechanism, isotherm, kinetic and thermodynamic properties. Desalination Water Treat. 2021 , 216 , 401–411. [ Google Scholar ] [ CrossRef ]
• Rápó, E.; Szép, R.; Keresztesi, Á.; Suciu, M.; Tonk, S. Adsorptive Removal of Cationic and Anionic Dyes from Aqueous Solutions by Using Eggshell Household Waste as Biosorbent. Acta Chim. Slov. 2018 , 65 , 709–717. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Al-Ghouti, M.A.; Al-Absi, R.S. Mechanistic understanding of the adsorption and thermodynamic aspects of cationic methylene blue dye onto cellulosic olive stones biomass from wastewater. Sci. Rep. 2020 , 10 , 15928. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Alhujaily, A.; Yu, H.; Zhang, X.; Ma, F. Adsorptive removal of anionic dyes from aqueous solutions using spent mushroom waste. Appl. Water Sci. 2020 , 10 , 183. [ Google Scholar ] [ CrossRef ]
• Şentürk, İ.; Alzein, M. Adsorption of Acid Violet 17 Onto Acid-Activated Pistachio Shell: Isotherm, Kinetic and Thermodynamic Studies. Acta Chim. Slov. 2020 , 67 , 55–69. [ Google Scholar ] [ CrossRef ]
• Ma, C.M.; Hong, G.B.; Wang, Y.K. Performance Evaluation and Optimization of Dyes Removal using Rice Bran-Based Magnetic Composite Adsorbent. Materials 2020 , 13 , 2764. [ Google Scholar ] [ CrossRef ]
• Miyah, Y.; Lahrichi, A.; Idrissi, M.; Khalil, A.; Zerrouq, F. Adsorption of Methylene Blue Dye from Aqueous Solutions onto Walnut Shells Powder: Equilibrium and Kinetic Studies. Surfaces Interfaces 2018 , 11 , 74–81. [ Google Scholar ] [ CrossRef ]
• Alver, E.; Metin, A.Ü.; Brouers, F. Methylene Blue Adsorption on Magnetic Alginate/Rice Husk Bio-Composite. Int. J. Biol. Macromol. 2020 , 154 , 104–113. [ Google Scholar ] [ CrossRef ]
• Raj, R.A.; Manimozhi, V.; Saravanathamizhan, R. Adsorption Studies on Removal of Congo Red Dye from Aqueous Solution Using Petroleum Coke. Petrol. Sci. Technol. 2019 , 37 , 913–924. [ Google Scholar ] [ CrossRef ]
• Dehghani, M.H.; Salari, M.; Karri, R.R.; Hamidi, F.; Bahadori, R. Process Modeling of Municipal Solid Waste Compost Ash for Reactive Red 198 Dye Adsorption from Wastewater Using Data Driven Approaches. Sci. Rep. 2021 , 11 , 11613. [ Google Scholar ] [ CrossRef ]
• Hamzezadeh, A.; Rashtbari, Y.; Afshin, S.; Morovati, M.; Vosoughi, M. Application of Low-Cost Material for Adsorption of Dye from Aqueous Solution. Int. J. Environ. Anal. Chem. 2020 , 102 , 254–269. [ Google Scholar ] [ CrossRef ]
• Rápó, E.; Aradi, L.E.; Szabó, Á.; Posta, K.; Szép, R.; Tonk, S. Adsorption of Remazol Brilliant Violet-5R Textile Dye from Aqueous Solutions by Using Eggshell Waste Biosorbent. Sci. Rep. 2020 , 10 , 8385. [ Google Scholar ] [ CrossRef ]
• Rápó, E.; Posta, K.; Suciu, M.; Szép, R.; Tonk, S. Adsorptive Removal of Remazol Brilliant Violet-5R Dye from Aqueous Solutions Using Calcined Eggshell as Biosorbent. Acta Chim. Slov. 2019 , 66 , 648–658. [ Google Scholar ] [ CrossRef ]
• Šljivić-Ivanović, M.; Smičiklas, I. Utilization of C&D Waste in Radioactive Waste Treatment—Current Knowledge and Perspectives. In Advances in Construction and Demolition Waste Recycling ; Elsevier: Amsterdam, The Netherlands, 2020; pp. 475–500. [ Google Scholar ]
• Aljeboree, A.M.; Alshirifi, A.N.; Alkaim, A.F. Kinetics and Equilibrium Study for the Adsorption of Textile Dyes on Coconut Shell Activated Carbon. Arab. J. Chem. 2017 , 10 , S3381–S3393. [ Google Scholar ] [ CrossRef ]
• Wekoye, J.N.; Wanyonyi, W.C.; Wangila, P.T.; Tonui, M.K. Kinetic and Equilibrium Studies of Congo Red Dye Adsorption on Cabbage Waste Powder. Environ. Chem. Ecotoxicol. 2020 , 2 , 24–31. [ Google Scholar ] [ CrossRef ]
• Felista, M.M.; Wanyonyi, W.C.; Ongera, G. Adsorption of Anionic Dye (Reactive Black 5) Using Macadamia Seed Husks: Kinetics and Equilibrium Studies. Sci. Afr. 2020 , 7 , e00283. [ Google Scholar ] [ CrossRef ]
• Cheruiyot, G.K.; Wanyonyi, W.C.; Kiplimo, J.J.; Maina, E.N. Adsorption of Toxic Crystal Violet Dye Using Coffee Husks: Equilibrium, Kinetics and Thermodynamics Study. Sci. Afr. 2019 , 5 , e00116. [ Google Scholar ] [ CrossRef ]
• Mahdavinia, G.R.; Baghban, A.; Zorofi, S.; Massoudi, A. Kappa-Carrageenan Biopolymer-Based Nanocomposite Hydrogel and Adsorption of Methylene Blue Cationic Dye from Water. J. Mater. Environ. Sci. 2014 , 5 , 330–337. [ Google Scholar ]
• Kah Yeow, P.; Wei Wong, S.; Hadibarata, T. Removal of Azo and Anthraquinone Dye by Plant Biomass as Adsorbent—A Review. Biointerface Res. Appl. Chem. 2021 , 11 , 8218–8232. [ Google Scholar ]
• Abualnaja, K.M.; Alprol, A.E.; Abu-Saied, M.A.; Ashour, M.; Mansour, A.T. Removing of Anionic Dye from Aqueous Solutions by Adsorption Using Multiwalled Carbon Nanotubes and Poly(Acrylonitrile-Styrene) Impregnated with Activated Carbon. Sustainability 2021 , 13 , 7077. [ Google Scholar ] [ CrossRef ]
• Khalaf, I.H.; Al-Sudani, F.T.; AbdulRazak, A.A.; Aldahri, T.; Rohani, S. Optimization of Congo Red Dye Adsorption from Wastewater by a Modified Commercial Zeolite Catalyst Using Response Surface Modeling Approach. Water Sci. Technol. 2021 , 83 , 1369–1383. [ Google Scholar ] [ CrossRef ]
• Belala, Z.; Jeguirim, M.; Belhachemi, M.; Addoun, F.; Trouvé, G. Biosorption of Basic Dye from Aqueous Solutions by Date Stones and Palm-Trees Waste: Kinetic, Equilibrium and Thermodynamic Studies. Desalination 2011 , 271 , 80–87. [ Google Scholar ] [ CrossRef ]
• Ofomaja, A.E.; Ho, Y.S. Effect of Temperatures and pH on Methyl Violet Biosorption by Mansonia Wood Sawdust. Bioresour. Technol. 2008 , 99 , 5411–5417. [ Google Scholar ] [ CrossRef ]
• Akar, T.; Ozcan, A.S.; Tunali, S.; Ozcan, A. Biosorption of a Textile Dye (Acid Blue 40) by Cone Biomass of Thuja Orientalis: Estimation of Equilibrium, Thermodynamic and Kinetic Parameters. Bioresour. Technol. 2008 , 99 , 3057–3065. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Zhang, X.; Zhao, C.; Huang, Z. Optimization of Shaking Speed for Enhanced Adsorption of Reactive Red 198 Using Activated Carbon. J. Environ. Chem. Eng. 2021 , 9 , 106104. [ Google Scholar ]
• Mahmoud, M.; Ahmed, M.; El-Din, A. Adsorption of Reactive Dyes: Effect of Dye Molecular Structure and Size on Removal Efficiency. Dyes Pigments 2022 , 193 , 109644. [ Google Scholar ]
• Yadav, A.; Sharma, R.; Gupta, S. Influence of Biochar Properties on Azo Dye Adsorption from Aqueous Solutions. Environ. Sci. Pollut. Res. 2023 , 30 , 24518–24530. [ Google Scholar ]
• Liu, Y.; Wang, Y.; Yang, X. Impact of Metal Ions on the Adsorption of Dyes: A Competitive Adsorption Study. Chem. Eng. J. 2021 , 419 , 129301. [ Google Scholar ]
• Chen, H.; Zhou, J.; Zhang, S. The Effect of Contact Time on Dye Removal Efficiency: A Comprehensive Study. J. Hazard. Mater. 2022 , 424 , 127431. [ Google Scholar ]
• Kini, M.S.; Saidutta, M.B.; Murty, V.R. Studies on Biosorption of Methylene Blue from Aqueous Solutions by Powdered Palm Tree Flower ( Borassus flabellifer ). Int. J. Chem. Eng. 2014 , 2014 , 306519. [ Google Scholar ] [ CrossRef ]
• Rehman, R.; Farooq, S.; Mahmud, T. Use of Agro-Waste Musa acuminata and Solanum tuberosum Peels for Economical Sorptive Removal of Emerald Green Dye in Ecofriendly Way. J. Clean Prod. 2019 , 206 , 819–826. [ Google Scholar ] [ CrossRef ]
• Ooi, J.L.Y.; Lee, L.Y.; Hiew, B.Y.Z.; Thangalazhy-Gopakumar, S.; Lim, S.S.; Gan, S. Assessment of Fish Scales Waste as a Low-Cost and Eco-Friendly Adsorbent for Removal of an Azo Dye: Equilibrium, Kinetic and Thermodynamic Studies. Bioresour. Technol. 2017 , 245 , 656–664. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Palanivell, P.; Ahmed, O.H.; Latifah, O.; Majid, N.M.A. Adsorption and Desorption of Nitrogen, Phosphorus, Potassium, and Soil Buffering Capacity following Application of Chicken Litter Biochar to an Acid Soil. Appl. Sci. 2019 , 10 , 295. [ Google Scholar ] [ CrossRef ]
• Musah, M.; Azeh, Y.; Mathew, J.T.; Umar, M.T.; Abdulhamid, Z.; Muhammad, A.I. Adsorption Kinetics and Isotherm Models: A Review. CaJoST 2022 , 4 , 20–26. [ Google Scholar ] [ CrossRef ]
• Lagergren, S. Zur Theorie der sogenannten adsorption gelöster stoffe, Kungliga Svenska Vetenskapsakademiens. Handlingar 1898 , 24 , 1–39. [ Google Scholar ]
• Sahu, N.; Pati, S.K.; Choudhury, S.; Rath, P.K. Process Optimization and Modeling of Methylene Blue Adsorption Using Zero-Valent Iron Nanoparticles Synthesized from Sweet Lime Pulp. Appl. Sci. 2019 , 9 , 5112. [ Google Scholar ] [ CrossRef ]
• Hu, Z.; Chen, H.; Ji, F.; Yuan, S. Removal of Congo Red from Aqueous Solution by Cattail Root. J. Hazard. Mater. 2010 , 173 , 292–297. [ Google Scholar ] [ CrossRef ]
• De Gisi, S.; Lofrano, G.; Grassi, M.; Notarnicola, M. Characteristics and Adsorption Capacities of Low-Cost Sorbents for Wastewater Treatment: A Review. Sustain. Mater. Technol. 2016 , 9 , 10–40. [ Google Scholar ] [ CrossRef ]
• Vadivelan, V.; Vasanth Kumar, M. Equilibrium, Kinetics, Mechanism, and Process Design for the Sorption of Methylene Blue onto Rice Husk. J. Colloid Interface Sci. 2005 , 286 , 90–100. [ Google Scholar ] [ CrossRef ]
• Kezerle, A.; Velic, D.; Hasenay, D.; Kovacevic, D. Lignocellulosic Materials as Dye Adsorbents: Adsorption of Methylene Blue and Congo Red on Brewers’ Spent Grain. Croat. Chem. Acta 2018 , 91 , 53–65. [ Google Scholar ] [ CrossRef ]
• Munagapati, V.S.; Yarramuthi, V.; Kim, Y.; Lee, K.M.; Kim, D.S. Removal of Anionic Dyes (Reactive Black 5 and Congo Red) from Aqueous Solutions Using Banana Peel Powder as an Adsorbent. Ecotoxicol. Environ. Saf. 2018 , 148 , 601–607. [ Google Scholar ] [ CrossRef ]
• Malakootian, M.; Mohammad, M.; Heidari, R. Reactive Orange 16 Dye Adsorption from Aqueous Solutions by Psyllium Seed Powder as a Low-Cost Biosorbent: Kinetic and Equilibrium Studies. Appl. Water Sci. 2018 , 8 , 212. [ Google Scholar ] [ CrossRef ]
• Ferrero, G. Dye Removal by Low Cost Adsorbents: Hazelnut Shells in Comparison with Wood Sawdust. J. Hazard. Mater. 2007 , 142 , 144–152. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Pengthamkeerati, P.; Satapanajaru, T.; Singchan, O. Sorption of Reactive Dye from Aqueous Solution on Biomass Fly Ash. J. Hazard. Mater. 2008 , 153 , 1149–1156. [ Google Scholar ] [ CrossRef ]
• Sun, D.; Zhang, X.; Wu, Y.; Liu, X. Adsorption of Anionic Dyes from Aqueous Solution on Fly Ash. J. Hazard. Mater. 2010 , 181 , 335–342. [ Google Scholar ] [ CrossRef ]
• O’Mahony, T.; Guibal, E.; Tobin, J.M. Reactive Dye Biosorption by Rhizopus arrhizus Biomass. Enzyme Microb. Technol. 2002 , 31 , 456–463. [ Google Scholar ] [ CrossRef ]
• Bhatti, H.N.; Safa, Y.; Yakout, S.M.; Shair, O.H.; Iqbal, M.; Nazir, A. Efficient Removal of Dyes Using Carboxymethyl Cellulose/Alginate/Polyvinyl Alcohol/Rice Husk Composite: Adsorption/Desorption, Kinetics and Recycling Studies. Int. J. Biol. Macromol. 2020 , 150 , 861–870. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Azzaz, A.A.; Jellali, S.; Bengharez, Z.; Bousselmi, L.; Akrout, H. Investigations on a Dye Desorption from Modified Biomass by Using a Low-Cost Eluent: Hysteresis and Mechanisms Exploration. Int. J. Environ. Sci. Technol. 2019 , 16 , 7393–7408. [ Google Scholar ] [ CrossRef ]
• Zhang, Y.; Li, J.; Zhao, Y.F.; Zhang, J.; Fan, J. Utilization of Modified Dioscorea opposita Thunb. as a Novel Biosorbent for the Adsorption of Indigo Carmine in Aqueous Solutions. RSC Adv. 2018 , 8 , 30040–30048. [ Google Scholar ] [ CrossRef ] [ PubMed ]

Click here to enlarge figure

Share and Cite

Yadav, M.; Singh, N.; Annu; Khan, S.A.; Raorane, C.J.; Shin, D.K. Recent Advances in Utilizing Lignocellulosic Biomass Materials as Adsorbents for Textile Dye Removal: A Comprehensive Review. Polymers 2024 , 16 , 2417. https://doi.org/10.3390/polym16172417

Yadav M, Singh N, Annu, Khan SA, Raorane CJ, Shin DK. Recent Advances in Utilizing Lignocellulosic Biomass Materials as Adsorbents for Textile Dye Removal: A Comprehensive Review. Polymers . 2024; 16(17):2417. https://doi.org/10.3390/polym16172417

Yadav, Manisha, Nagender Singh, Annu, Suhail Ayoub Khan, Chaitany Jayprakash Raorane, and Dong Kil Shin. 2024. "Recent Advances in Utilizing Lignocellulosic Biomass Materials as Adsorbents for Textile Dye Removal: A Comprehensive Review" Polymers 16, no. 17: 2417. https://doi.org/10.3390/polym16172417

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

• Open access
• Published: 27 August 2024

Virtual power plants: an in-depth analysis of their advancements and importance as crucial players in modern power systems

• Sobhy Abdelkader 1 , 2 ,
• Jeremiah Amissah 1 &
• Omar Abdel-Rahim 1 , 3  

Energy, Sustainability and Society volume  14 , Article number:  52 ( 2024 ) Cite this article

Metrics details

Virtual power plants (VPPs) represent a pivotal evolution in power system management, offering dynamic solutions to the challenges of renewable energy integration, grid stability, and demand-side management. Originally conceived as a concept to aggregate small-scale distributed energy resources, VPPs have evolved into sophisticated enablers of diverse energy assets, including solar panels, wind turbines, battery storage systems, and demand response units. This review article explores the evolution of VPPs and their pivotal roles as major stakeholders within contemporary power systems. The review opens with a definition of VPPs that clarifies both their fundamental traits and technological foundations. A historical examination of their development highlights major turning points and milestones that illustrate their transforming journey.

The methodology used for this article entailed a thorough examination to identify relevant studies, articles, and scholarly works related to virtual power plants. Academic databases were used to gather relevant literature. The literature was organized into categories helping to structure and present information in a logical flow based on the outline created for the review article . The discussions in the article show that the various functions that VPPs perform in power systems are of major interest. VPPs promote the seamless integration of renewable energy sources and provide optimum grid management by aggregating distributed energy resources, which improves sustainability. One of the important components of this evaluation involves taking market and policy considerations. Examining worldwide market patterns and forecasts reveals that VPP usage is rising, and that regulatory frameworks and incentives have a bigger impact on how well they integrate.

Overcoming obstacles is a necessary step towards realizing full VPP potential. For VPPs to be widely adopted, it is still essential to address technological and operational challenges as they arise. Diverse stakeholders must work together to overcome market obstacles and promote the expansion of the VPP market. This analysis highlights the potential for VPPs to propel the evolution of contemporary power systems toward a more sustainable and effective future by highlighting areas for future research and development.

There is an urgent need for creative and sustainable alternatives as the world’s need for energy rises, while fossil fuel-based power generation methods are increasingly scrutinized for their environmental effects [ 1 ]. Centralized alternating current power networks have been widely installed and used worldwide since the 1880s. Evaluations from the 2023 statistical global energy review [ 2 ] revealed that about 82% of the world’s primary energy source comes from fossil fuels like coal oil, and natural gas but their utilization produces greenhouse gas emissions that harm the environment and cause climate warming which has triggered the current global climate crisis [ 3 ]. The contribution of the different sources to world energy consumption is shown in Fig.  1 .

Global energy sources data

On the other hand, energy demand has grown significantly as a result of global economic growth. The demand for electricity has increased steadily over the past decades, by an average of 15%, and is anticipated to increase by 30% by 2040 [ 4 ]. This calls for innovative ideas to support the demand while looking out for the environment. Therefore, distributed energy resources (DERs) must be considered to lessen the detrimental environmental impacts of fossil fuels [ 1 ]. DERs are decentralized energy systems that produce, consume and store energy and are preferably located close to where electricity is consumed. These resources include batteries, wind turbines, solar panels, etc. DERs have been integrated in the power system networks (PSN) and have reduced the effects of energy generation from fossil fuels, furnishing stakeholders with economic and technical benefits [ 5 ]. While DERs offer power systems opportunities, they also bring with them challenges because of their intermittent and stochastic nature. DERs are often described as stochastic and intermittent due to their inherent characteristics and the factors that influence their generation. This nature of DERs is caused by elements including weather changes, operational uncertainties like maintenance, and equipment performance, which can result in unanticipated variations in DER generated or connected output. Instability in the grid is brought on by the rising use of DERs on the demand side, which worsens load demand fluctuations. As a result, real-time monitoring and dispatching are essential for the grid’s safe operation [ 6 , 7 , 8 , 9 ]. Furthermore, the power system needs more adaptability, which can be provided by several mechanisms, such as demand-side management, and energy storage systems (ESS). The only way to properly use these sources to increase their grid contributions is through optimal coordination between different agents [ 10 ].

Over the years, various research has been conducted to address the above challenges and many solutions have been proposed. VPPs have emerged as a ground-breaking solution in an era of energy transition and growing emphasis on sustainable power generation, altering the landscape of contemporary power systems [ 11 ]. VPPs have evolved as key players in promoting efficiency, flexibility, and resilience in the energy industry thanks to their capacity to integrate a variety of energy supplies and improve grid management [ 12 , 13 ].

A VPP is an energy management system that aggregates and coordinates diverse array of DERs, including photovoltaics, wind turbines, battery energy storage systems (BESS), and demand response technologies. The primary function of a VPP is to optimize the collection of these DERs in response to grid conditions, energy demand, and market signal. Through advanced control algorithms and real-time monitoring capabilities, VPPs dynamically adjust energy dispatch schedules, balances supply and demand, and enhance grid stability and reliability.

It is important to note that the concept of VPPs shares some basic similarities with that of the smart grid. However, unlike the VPP which focuses on the aggregation and optimization of DERs, smart grid, on the other hand, encompasses a broader range of functionalities aimed at modernizing the entire electricity supply chain. It can be said that the VPP augment the operation of the smart grid by providing ancillary support like supply and demand balancing to the smart grid.

The combination of these various resources enables the VPP to function as a cohesive and adaptable entity, to be able to react in real-time to grid signals and market conditions [ 14 , 15 ]. In the late 1990s, a pioneering shift in energy research and innovation emerged with the exploration of aggregating distributed resources into a unified virtual power entity, laying the groundwork for the conceptualization and development of VPPs [ 13 ]. Since then, VPPs have evolved from theoretical notions to real-world applications owing to technical developments, and breakthroughs in communication technology. The adoption of VPPs has been hastened by the spread of smart grid technologies and the rise of renewable energy resources (RERs), making them a crucial component of contemporary power systems [ 12 , 16 ].

It is impossible to overstate the importance of VPPs as significant participants in contemporary power systems. VPPs are essential for facilitating the seamless integration of intermittent renewable resources into power grids as they shift from fossil fuel-based generation to renewable-dominated systems [ 3 , 17 , 18 ]. In addition, VPPs can control electricity consumption patterns to correspond with variations in renewable generation. Demand-side management improves grid reliability and efficiency by lowering peak demand and reducing grid congestion [ 19 , 20 ]. VPPs also significantly contribute to the optimization of the energy market. VPPs are crucial actors in the developing electricity market because of their involvement in energy trading and the provision of ancillary services, which help to stabilize prices and maintain system resilience [ 11 , 21 ]. A typical architecture of a VPP is shown in Fig.  2 . With the aid of technology like cloud computing, a VPP aggregates various power consumers, ESS, and power generators to provide flexible adjustments. A communication protocol is used by the components of a VPP to transfer data to the VPP communication system. This communication protocol enables efficient coordination for the VPP to adjust energy production which allows supply to the grid with dependable cost-effective electricity via the electricity market [ 22 ]. The data acquisition platform aids in gathering information about the generation, consumption, and state of charge of the portfolio of DERs for optimal decision-making.

Architecture of a VPP

From the above discussion, it is clear that VPPs have become an important player in modern power systems, providing a dynamic and revolutionary method of managing energy. The idea of VPPs has recently received a lot of interest in energy systems. Studies have provided insightful information by highlighting their potential to transform the way we produce, distribute, and use power. It is critical to understand that this dynamic and developing discipline poses several notable issues, gaps, and areas that require added research.

In the review presented in [ 23 ], an overview of VPP operations, including the integration of DERs, controlled loads, and EVs for resource aggregation and cooperative optimization as well as market and grid operations, is the goal. The evaluation did however not discuss regulatory and policy issues that might affect how widely VPPs are used and implemented in the power market.

Also, the difficulties, solutions, and prospects related to the conceptual review of the conversion of a microgrid to a VPP have also been covered by [ 24 ]. The overview examines RERs integration, opportunities for VPPs in the field of smart distribution systems, and effective management mechanisms. The management mechanism, however, did not discuss the optimization of the DERs for optimal operation. Authors in [ 25 ] gave a thorough overview of the VPP concept and its potential advantages in integrating DERs to assist grid security and stability. Resource optimization, as a main part of the VPP operation, is not covered in this study. Also, Ref. [ 11 ] provided an overview of VPP models and how they interacted with various energy markets. Finding the most profitable VPP scheme to be implemented in each regulatory environment is the focus. DER integration challenges, which affect the operation of VPPs in the energy markets, are not considered in this study. In [ 26 ], the idea of VPPs to participate in various energy markets is proposed. The model evaluates the VPP's technical and commercial prospects. Engaging in various energy markets revolves around sharing of data between the VPP and operators of the markets. The issue of data privacy and cybersecurity was not included in this study. Authors in [ 27 ] provided a review with a focus on integrating DERs into the electricity grid. The assessment gave a summary of the development and use of VPP for carbon reduction in the Chinese power system. The study, however, did not cover technologies that can improve the management and operation of VPPs, notably in addressing the intermittent and volatile nature of DERs. In the domain of energy management, authors of [ 28 ] provided a summary of resource scheduling in VPPs and addressed questions on scheduling procedures. However, despite concentrating on both technical and economic elements of scheduling in VPPs, this analysis did not address potential influences like the state of the energy markets that could have an impact on the scheduling issue. The case of a multi-energy coupled VPP has been presented in [ 29 ]. The purpose of this study was to address the advantages of multi-energy linked VPPs engaging in various energy markets. The issue of enhanced communication technology, data privacy and cybersecurity are some of the challenges which were not featured in this study.

The idea and structure of VPPs are concisely described in [ 30 ] with regard to its two main goals—energy management and power markets. Solutions are suggested to alleviate the problems with DER uncertainties that were highlighted. In order to create future sustainable power grids, authors of [ 3 ] have presented a comprehensive overview of the cutting-edge VPP technology. The study discusses recent technological advancements as well as the significant economic benefits of VPPs. However, this study did not cover the legislation that specifies how VPPs can access and participate in the energy markets. Below are some of the gaps found in existing literature:

Analysis of cybersecurity and data privacy as crucial elements in the VPP development.

Environmental and sustainability focus. The SDGs that VPPs could support, and how the support can be achieved.

Rigor analysis of legislation or regulations which will dictate the operation of the VPP.

Considering the above research gaps in literature, this review article advances the knowledge of energy systems by providing a thorough analysis of VPPs, their historical development, and their crucial roles as essential stakeholders in modern power systems. There will be focus on technical and market operations, real-world case studies, the identification of challenges and prospects, the emphasis on technical and market operations highlight the relevance and transformative potential of VPPs in creating sustainable and effective energy ecosystems. The contributions of this paper can be summarized as follows:

Comprehensive understanding of VPPs to provide readers with a concise definition, key traits, and core values of VPPs.

Tracing historical developments of VPPs from their theoretical roots to their current popularity.

Emphasis on VPPs as key stakeholders in modern power systems. This emphasis highlights the vital role that VPPs play in ensuring grid stability, fostering the integration of RES, and promoting sustainability.

Integration of technical and market aspects by providing a comprehensive analysis of VPP operation. This integration is crucial as it shows that VPPs actively participate in energy markets and actively optimize energy resources, which facilitates effective electricity trading and grid balancing.

Application of cybersecurity and data privacy techniques that protect the VPP from cyber threats, assuring grid stability, data integrity, and consumer trust in the ever-changing energy sector.

Real-world case studies of VPP deployments to offer insights and experiences.

Discussion of the regulatory frameworks that control how VPP operates.

Identification of challenges, providing recommendations, and prospects.

VPP advancements

The traditional centralized power generation model is being replaced by a decentralized, adaptable, and sustainable system thanks to VPP, which represents a revolutionary paradigm in the energy sector. Early theoretical ideas from the late twentieth century established the foundation for the development of VPPs and their eventual prominence in modern power systems [ 31 , 32 ]. This part of the paper will focus on the evolutionary journey of VPPs, highlighting the early concepts, key milestones, and technological advancements that shaped their development into critical enablers of modern energy ecosystems.

The embryonic stage (1990s–2000s)

Although the idea of VPPs was initially put forth in the 1997 [ 13 ] by Dr. Shimon Awerbuch, it did not really take off until the early 2000s. Early academic publications proposed the idea of coordinating and optimizing a portfolio of distributed energy resources to increase operational effectiveness and grid reliability. However, due to limited technological capabilities and a lack of enabling legal frameworks, the practical deployment of VPPs remained primarily theoretical at this point. Also, the absence of developed distributed generating technology, the high cost of communication and control systems, and the regulatory uncertainties surrounding VPPs were some of the causes of lack of practical deployment. References [ 33 , 34 , 35 , 36 , 37 , 38 ] provides a description of the early years concept of the VPP, its difficulties, including consumer resistance to participating, economic viability in infrastructure setup, investors' perceptions of risk, and grid operators' reluctance to adopt the unique strategy.

The breakthrough stage (2010s–2020)

The growth years presented milestones and key turning points in VPP deployment from the early years. At this point, the VPP has encountered rapid growth as a result of increasing interest in adoption of distributed generation technology, decreasing communication and control system costs, and expanding regulatory backing for VPPs. In a declaration on the future of the European electricity market that was issued in 2011, the European Commission emphasized the potential of VPPs to increase grid flexibility and integrate renewable energy. This communication aided in increasing policymakers’ and stakeholders’ understanding of VPPs [ 39 , 40 , 41 , 42 , 43 , 44 , 45 ]. Later, in March 2023, it was amended in Strasbourg, France, by recommending an expansion of the EU electricity market structure to further integrate RESs, improve customer protection and industrial competitiveness [ 46 ]. Notable milestones of the growth years include grid integration [ 47 ], market participation [ 48 ], technological advancement, and demand response programs[ 49 ], allowing aggregated DERs to respond to grid signals and enhance grid stability [ 50 ]. This marked the initial practical application of VPPs, showcasing their ability to support grid operations.

The future (2021 and beyond)

The demand for flexible grids and the incorporation of RESs is anticipated to drive further growth of VPP. VPPs are viewed as one of the techniques to lower carbon emissions and increase energy efficiency [ 51 ]. The key drivers for this growth are the increasing deployment of distributed generation technologies (DGT), falling cost of communication and control systems, growing regulatory support for VPPs, and also prosumers who want to receive incentives for their surplus generation [ 45 ].

In summary, it is evident that early theoretical insights were followed by practical and revolutionary applications in modern power systems as VPPs evolved. The development of VPPs into essential enablers of decentralized, flexible, and sustainable energy ecosystems has been shaped by significant turning points and milestones, as well as technological development and innovations. A thorough summary is provided in Table  1 for further reading.

VPP planning, roles, and sustainability

VPP planning is a crucial and multifaceted process that entails strategic design, coordination, and optimization to provide effective and dependable energy management. The main goal of VPP planning is to maximize the advantages for both grid operators and consumers while optimizing the potential of varied DERs and guaranteeing their seamless integration with the power grid. The planning approach necessitates a thorough comprehension of the energy landscape, individual DER capabilities, market dynamics, and regulatory frameworks.

To ensure that VPPs perform as planned and expected, their technological constraints must be recognized and measured [ 55 ]. Before interacting with external and internal elements, the VPP schedules and plans its operations. It is also a good performance criterion for the VPP to keep accurate data to engage the electricity market and reap favorable effects by analyzing the uncertainties resulting from elements like weather and producing forecasts with a high level of assertiveness [ 56 ]. The issue of forecasting will be discussed later in the section dedicated to the roles of VPPs. The VPP operations may be constrained by infrastructure, technological, and technical limits [ 57 ]. The model shown in [ 26 ] emphasizes the importance of effectively measuring and managing controllable loads in heating, ventilation, and air conditioning (HVAC) systems. Also, it emphasizes the significance of photovoltaic (PV) and BESS influences in determining the viability and adaptability of a VPP. VPPs can improve their coordination with all stakeholders by developing a methodical technique for evaluating and controlling power availability at time intervals. Surely, this enhances the performance of the VPP and enables a more seamless interaction with the power grid.

VPP planning also includes economic and legal factors in addition to the technical ones. The aspects of technical and economic frameworks of the VPP will be delved deeper in the sections dedicated to the technical and economic aspects of VPPs. It is important to note that good operational planning directly affects good economic outcomes [ 55 ]. The economic viability of the VPP and its prospective revenue streams, including energy trading [ 58 ], demand response participation [ 59 ], and the supply of ancillary services [ 21 ], are assessed using financial models and cost–benefit analysis [ 60 ]. Collaboration with grid operators, legislators, and other stakeholders is also necessary for successful VPP planning to overcome regulatory obstacles and build an environment that facilitates VPP integration. To ensure effective planning, the VPP should be continuously monitored and improved to respond to shifting grid conditions and market dynamics [ 61 ]:

VPP planning opens the way for a more resilient, and sustainable energy future by integrating technological, economic, and regulatory factors. It has enormous potential to optimize resource use, improve grid stability, and contribute to the global quest for a reduction in carbon emissions produced by energy systems. It is therefore imperative that stakeholders comprehend the complexities of VPP planning to influence the energy industry’s future and advance the cause for greener and a more sustainable and effective energy future. This planning phase can be summarized as: aggregating existing and new energy resources.

Ownership structure: The internal ownership structure of VPPs can vary depending on the specific implementation and stakeholders involved. It may involve collaboration between multiple stakeholders including energy producers, consumers, and aggregators.

Regulating and market considerations governing energy markets and grid operations.

Implementation of an energy management system to provide functionalities such as real-time monitoring, forecasting, dispatching, and scheduling energy resources to meet grid requirements and maximize economic benefits.

Agreement formulation such as power purchase agreements.

Profit sharing mechanisms taking into consideration factors such as investment contributions, operational cost, risk allocation, etc.

Compensation structures for various stakeholders involved in the VPP including incentives for demand response participations from consumers.

The way electricity is produced, controlled, and used has been revolutionized by VPPs as explained in the previous sections. VPPs are flexible and dynamic entities that perform a variety of roles in modern power systems. Because of the variety and importance of their tasks, they are key players in creating an energy ecosystem that is sustainable, effective, and resilient. The following are the main responsibilities of VPPs in power systems.

Aggregation of DERs: Various DERs, such as solar panels, wind turbines, ESS, EVs, and demand response loads are gathered by VPPs. VPPs construct an adaptable and manageable portfolio of assets by combining these decentralized resources into a single virtual entity. Through this aggregation, grid management is improved, enabling the VPP to maximize DER usage in response to grid signals. The DERs’ activity within the VPP is managed and coordinated by the VPP operators. The main responsibility is resource optimization and involvement in energy markets.

The authors of [ 62 ] described the aggregator concept as a central control node that collects information from both the power grid and controlled loads. A load aggregator can also serve as a conduit between the controllable loads and the grid operator, allowing the regulated management to consider user and grid benefits simultaneously. When interfacing with the power market, aggregators are employed in power charging models for EVs to help optimize the batteries’ charging as well as the modeling of driving patterns and price estimates [ 63 ]. As DERs are dispatched depending on compensation rates and power levels, an aggregator can stand in for them to maximize profits [ 64 ]. Furthermore, in [ 65 ], for a power market with bilateral contracts, the aggregator has the facility to select between various power plants based on power-cost-based offers.

Grid stabilization and reliability: VPPs make a major contribution to the reliability and stability of the grid. VPPs maintain a stable and steady supply of electricity while minimizing the possibility of blackouts and voltage variations by balancing energy generation and consumption from various DERs [ 66 ]. They are able to provide ancillary services like frequency regulation and voltage management, which are essential for preserving grid stability [ 67 ]. The general stability and dependability of the electrical system are the responsibility of grid operators. In accordance with grid norms and standards, the grid operators work with VPP operators to incorporate DERs.

Renewable energy integration: In 2016, in Paris, an emission reduction plan was enacted which has made the use of DERs very essential [ 68 , 69 , 70 ]. This integration is the VPP operator’s responsibility. This is accomplished by coordinating the operation of diverse RERs, such as solar panels, wind turbines, and such that they work as a unified system. However, due to their erratic nature, integrating RESs into the power systems presents its own challenges [ 71 , 72 ]. These challenges come about because of generation fluctuations due to weather conditions and time of the day. The variability adds complexity to power system operations. For instance, rapid changes in wind speed or cloud cover can result in fluctuations in generation, requiring grid operators to make quick adjustments to maintain system stability. VPPs take on this problem by combining several RESs and using intelligent management processes, they make it easier for the integration of the RESs effectively. They ensure the integration of these RESs to provide a steady supply of electricity while lowering reliance on conventional fossil fuel-based power plants.

Authors in [ 72 ] proposed a solution for integration of RESs into the grid to maintain power quality. This is important because RESs are becoming increasingly popular due to their environmental benefits, but they can also introduce power quality issues. This is a challenge that a VPP is sought to address. Large scale penetration of RESs means a hike in capital and operational cost. Authors in [ 73 ] discussed a mechanism that could aid in lowering the high cost of RESs integration and bringing electricity prices into affordable band. Spreading the benefits of renewable integration into the spheres of agriculture, where in [ 74 ], authors have created a mechanism to encourage energy-efficient agriculture by minimizing dependency on fossil fuels for water-table pumping. Through the aggregation and optimization of DERs, VPPs enable farmers to reduce their dependency on fossil fuels while enhancing energy efficiency and resilience in agricultural practices. This synergy not only fosters economic sustainability for farmers, but also contributes to the broader goal of renewable energy integration, paving the way for a greener energy future.

Successful integration depends on several important aspects. Forecasting methods that accurately estimate the patterns of RESs generation must be put in place [ 75 , 76 ]. This allows better grid management and optimization of the DERs. The VPP employs such tools to better manage the generation of DERs. A summary of various forecasting techniques provided in the literature is listed in Table  2 . Analysis of forecasting models to aid in the integration of RESs in the context of VPPs has been provided in [ 77 ].

Moreover, for optimal integration of RESs, the power grid must be modernized with smart technologies. Real-time monitoring, control, and communication between DERs and grid infrastructure are made possible using smart approaches like the VPP [ 16 , 78 , 79 ]. This improves the reliability and effectiveness of the grid. Additionally, VPPs provide beneficial grid functions, such as frequency regulation [ 67 ] and voltage control [ 80 ] in addition to balancing energy supply and demand [ 81 ]. These services boost the grid’s dependability and resilience even more, promoting a stronger energy infrastructure that can handle the rising proportion of RESs.

The VPP approach to integrating RESs into the power grid is a cutting-edge strategy that is revolutionizing the way energy is produced, distributed, and consumed. VPPs offer an effective response to the problems caused by intermittent renewables by utilizing the combined potential of DERs and modern technology. VPPs will unquestionably be essential in advancing the transition to a cleaner, more dependable, and efficient energy system as the world progresses toward a sustainable energy future.

DER technologies applied in VPPs

In VPPs, various DERs are used, including solar panels, wind turbines, ESS, EVs, and demand response loads. These DERs are aggregated and optimized within the VPPs, allowing for efficient management and coordination [ 55 ]. By harnessing the collective capacity of diverse DERs, VPPs enhance grid stability, enable renewable energy integration, and support demand response strategies, contributing to a more sustainable and flexible energy ecosystem. A VPP should ensure that DER integration keeps the system operating properly by ensuring the stakeholders’ continual consumption requirements [ 92 ]. Various DER technologies applied in VPPs in the reviewed literature are summarized in Table  3 .

Out of the 15 References evaluated regarding DER technologies used in VPPs, it is evident from Table  3 that wind turbines and solar panels hold the largest share, as shown in Fig.  3 . It proves how easily the technology of wind turbines and solar panels have been embraced. However, more renewables should be added to the energy mix to hasten the shift to a less carbon-oriented energy landscape.

DER application in literature

VPP sustainability focus

One of the viable ways to address numerous Sustainable Development Goals (SDGs) of the United Nations (UN) and contribute to a more sustainable energy future is through VPPs. By encouraging the integration of RESs and boosting energy efficiency, VPPs, as a fundamental enabler of the energy transition, contribute significantly to achieving SDG 7 (Affordable and Clean Energy). VPPs promote the integration of sustainable energy into the power grid by aggregating and optimizing DERs thereby lowering greenhouse gas emissions and addressing climate change (SDG 13—Climate Action).

Additionally, through promoting technological advancements and innovation in the energy industry, VPPs provide a substantial contribution to SDG 9 (Industry, Innovation, and Infrastructure). VPPs promote grid modernization and improve overall energy infrastructure by integrating smart grid technologies, advanced analytics, and artificial intelligence. These developments result in more effective and adaptable energy systems, advancing the objectives of SDG 9 to develop robust infrastructure and encourage sustainable industrialization.

However, while VPPs offer considerable potential for achieving various SDGs, several challenges must be addressed to ensure their long-term sustainability. Access to VPP technologies must be equally available, as this can influence SDG 1 (No Poverty) and SDG 10 (Reduced Inequalities). For VPPs to be deployed in a way that supports SDG goals for eradicating poverty and minimizing inequality, marginalized people and neglected areas must be able to benefit from them. In simple terms, it is essential to make sure that everyone has an equal opportunity to profit from VPPs to realize SDG 1 and SDG 10. This calls for figuring out ways to make technology more accessible and inexpensive for everyone, especially those living in rural or underdeveloped areas. By doing this, VPPs may contribute to the development of a more just and sustainable energy future in which everyone, regardless of financial situation, has access to safe and dependable energy.

Furthermore, the environmental impact of VPPs [ 108 ] and their associated technologies require careful consideration to achieve SDG 12 (Responsible Consumption and Production). Lithium-ion batteries, which are used in ESS, are one example of a crucial mineral and material whose demand is on the rise, prompting questions regarding responsible sourcing, recycling, and end-of-life management. It is not a surprise that there has been extensive literature on ways to increase the lifespan of lithium-ion batteries [ 109 ]. Authors in [ 110 ] proposed a precise lifespan model for the battery cells used in VPP applications. To reduce the negative environmental and social effects of VPP deployment, sustainable methods must be implemented in material sourcing and VPP operation.

Moreover, numerous steps can be taken to guarantee the sustainability of a VPP itself. Stakeholders must work together to build supporting regulatory frameworks and financial incentives for VPP development. VPPs will become more widely available and long-lasting if investments are encouraged in their research, development, and implementation. This will also encourage technological breakthroughs and cost reductions. Also, a successful integration of VPPs into the energy economy depends on raising consumer awareness and engagement. The acceptance of VPP technology can be increased by educating consumers about the advantages of VPP participation, such as lower energy costs and increased grid reliability [ 111 , 112 ].

To sum up, VPPs have a significant potential to help achieve several SDGs pertaining to renewable energy, tackling climate change, and sustainable infrastructure. They support SDGs 7 and 9 by fostering the integration of RESs and improving energy efficiency. To achieve more general sustainability goals, it is necessary to address issues with fair access to VPP advantages and responsible use and production. VPPs are critical enablers of a greener, more inclusive, and resilient energy future and can help accomplish specific SDGs by establishing supportive policies, encouraging innovation and consumer engagement. Using VPP's revolutionary potential in promoting the UN’s sustainability agenda [ 113 ] requires advocating for and making contributions to their sustainable deployment and optimization.

Cybersecurity and data privacy

The protection of the grid’s stability and dependability is one of the main justifications for prioritizing cybersecurity in VPP application. As crucial nodes in the grid, VPPs coordinate the functioning of DERs and provide a constant and reliable supply of electricity. A cyber-attack on a VPP has the potential to impair energy production, distribution, and grid management, resulting in power outages [ 114 ] and large financial losses.

The efficient operation of VPPs depends on data integrity [ 115 ]. For making decisions about the generation, distribution, and use of energy, VPPs depend on accurate data. Cybersecurity measures guard against data alteration or manipulation, ensuring that VPP operators have reliable data for maximizing energy resources and delivering crucial grid services. In order to increase consumer and prosumer confidence in VPP services, data privacy procedures on data collection and usage are essential [ 116 ].

VPPs are desirable targets for cybercriminals because of their crucial functions in grid management and their strength in the marketplace. VPPs are shielded by cybersecurity from a variety of dangers, such as malware and hacker attempts [ 117 ]. To address the cybersecurity issues, various approaches have been suggested and has been categorized by [ 118 ] as human and non-human approaches. Human approaches like updates and incremental patches installation aids in robust security posture, addressing vulnerabilities in software, but also require reboots causing downtime to regular operations. Engaging in customer interactions also creates awareness to recognize and respond to potential threats. However, allocating time and resources may be challenging for organizations with limited budgets and manpower.

Non-human approaches like the adoption of blockchain technology reduce the risk of single point failure as the technology operates on a decentralized network. This enhances resilience, making it more challenging for attackers to compromise the entire system. Another non-human approach is cloud computing which typically encrypts data during transmission and storage. This safeguards sensitive information from interception or unauthorized access.

Data privacy and cybersecurity are essential elements of VPP operations. They protect against cyberthreats, guarantee data integrity, enhance grid stability [ 119 ], promote consumer trust, enable regulatory compliance, and support the viability of VPPs financially. To ensure a secure, dependable, and sustainable energy future, cybersecurity and data privacy must be prioritized as VPPs continue to develop and broaden their role in contemporary energy systems [ 120 ].

Regulation and compliance

The operation of VPPs is greatly influenced by legislative or regulatory activities. This section will cover the regulatory structure that governs VPPs, emphasizing significant importance and their effects on the energy industry.

In the domain of grid integration standard and requirements, regulating bodies establish grid codes and integration standards that the VPP must adhere to when connecting to the electrical grid. The safe and dependable grid integration of DERs is ensured by these standards. The basis for secure VPP functioning is grid codes and standards. A manual for connecting DERs to the utility grid is provided by the IEC 62786. DER planning, operation, protection, and connectivity to distribution networks are the key applications. A global agreement on the use of DER in electrical power systems is being sought through the IEEE 1547 set of standards. This standard has received widespread acceptance on a global scale in outlining the requirements for the design, implementation, testing, and security of all sorts of DERs. Due to the increased penetration of DERs and the need to maintain system stability, the IEEE 1547 has recently been updated to IEEE 1547-2018 and IEEE 1547.1-2020 [ 121 ]. A crucial series of standards released to control the grid’s interconnection and operability is the IEEE 2030. It is modified to implement cutting-edge communication and information technologies that provide interoperability solutions for the promotion of DER connectivity.

The European Committee for Electrotechnical Standardization (CENLEC), which is made up of 34 European Nations, oversees standardization efforts to increase commercial viability and foster technological growth. The CENLEC released the EN 50549-1 and EN 50549-2 DER integration standards with the goal of addressing all DER capabilities that are necessary for operation in tandem with distribution networks [ 121 ].

Also, there may be regional variations in regulations governing the integration of DERs with the grid [ 121 ]. For example, Canadian standards C22.3 No. 9 and C22.2 No. 257 offer technical advice for DER integration with the grid at medium and low voltage under 50 kV and low voltage systems under 0.6 kV, respectively. The British standard BS EN50438:2007 also offers technical advice for DER interconnection. The VDE-AR-N 4105 standard in Germany also offers technical recommendations for connecting DERs and low voltage systems. The JEAG.9701-2001 standard in Japan offers technical recommendations for distributed generating grid-connection. The standard permits DER owners to sell surplus energy to utility grids and mandates that power grids supply DER owners with backup power.

Various environmental and sustainability regulations may pertain to different jurisdictions [ 122 ], and they may provide incentives or requirements for VPPs to assist the integration of RERs and the reduction of emissions. In certain regions, these rules may have an impact on how VPPs function. The level of support for VPPs that use RERs may vary depending on the targets and incentives that jurisdictions set for renewable energy [ 123 ].

VPP operators and stakeholders must negotiate a complicated regulatory environment that is unique to their locations. It is essential for the implementation and operation of VPPs to comprehend and follow local legislations. Furthermore, as VPPs become more crucial to the world’s energy landscape, regulators and industry participants must cooperate to unify rules and encourage uniformity in grid integration techniques across various jurisdictions.

Technical aspects of VPPs

The technical operations of a VPP involve a series of complex and coordinated processes to efficiently manage and optimize the aggregated DERs within the VPP. According to Ref. [ 124 ], the technical features of VPPs provide dynamic interaction for the integration of power distribution based on auxiliary services. These technical operations can vary depending on the specific architecture and goals of the respective VPP. This section of the paper delves into the technical intricacies of VPPs and explores their roles as key enablers in the transition toward a sustainable and resilient energy future. Some of these technical aspects of the VPPs are emphasized below:

Resource optimization and scheduling: In a VPP, resource optimization and scheduling of various DERS are essential to achieve efficient and reliable energy management [ 28 , 125 ]. It is also important to note that advanced algorithms and real-time data analytics [ 76 ] as summarized earlier in Table  2 are employed to forecast energy generation and demand profiles, ensuring dynamic resource optimization. The VPP intelligently dispatches DERs based on grid conditions and market signals, balancing supply and demand to enhance grid stability and maximize revenue generation [ 126 ]. By coordinating diverse DERs, VPPs optimize energy use, contribute to renewable integration, and support grid flexibility, making them crucial enablers in the transition to a sustainable resilient energy ecosystem.

A summary of the relevant literature in accordance with resource optimization and scheduling is provided in Table 4 .

Load balancing and grid support/ancillary service: The load balancing and grid support functions of a VPP are very crucial [ 135 ]. The VPP dynamically modifies energy generation and consumption to fit grid demands by aggregating and optimizing various DERs. While storing excess energy during times of low demand, the VPP can supply additional power from DERs during times of peak demand to balance out high demand. This load-balancing ability makes VPPs essential for guaranteeing a dependable and resilient electricity supply since it improves grid stability, lowers grid stress, and adds to overall grid support.

In addition to its role of aggregating and optimizing DERs, a VPP offers a range of essential ancillary services. These services include frequency regulation. This is achieved by maintaining grid frequency within acceptable bounds through rapid power adjustment [ 136 , 137 , 138 , 139 ]. VPPs also provide voltage support by injecting or absorbing reactive power to stabilize voltage levels [ 80 , 140 , 141 ].

Moreover, VPPs contribute to peak regulation, managing demand during high load periods to alleviate grid stress [ 142 , 143 , 144 ]. The comprehensive suite of ancillary services offered by VPPs ensures grid stability, enhances reliability, and facilitates the integration of RESs, making them vital assets in modern power systems.

Demand response and load management: A VPP inherent components of demand response and load control enable effective energy usage. By actively communicating with connected consumers to alter electricity consumption in response to grid circumstances and price signals, VPPs participate in demand response. In order to avoid peak demand times and lessen grid load, VPPs optimize the scheduling of operations and equipment that consume a lot of electricity [ 59 , 81 , 96 ]. This demand-side flexibility not only supports grid stability, but also empowers consumers to actively participate in energy conservation, contributing to a sustainable energy ecosystem [ 66 , 145 ]. The VPP’s ability to efficiently balance energy supply and demand through demand response and load management strategies makes it a pivotal stakeholder in modern power systems.

The technical aspects of VPPs represent a dynamic and transformative force in the energy sector. VPPs provide effective renewable energy integration, grid stability, and demand response capabilities by aggregating and optimizing various DERs.

Market/economical aspect of VPP

VPPs provide an appealing scenario for the future of energy systems in terms of their commercial and financial prepositions. VPPs can completely alter the economics of electricity generation and consumption as they are dynamic aggregators of various DERs. VPPs maximize the use of DERs, optimize income generation, and improve participation in the energy market [ 11 ]. The VPP does this via real-time data analytics, complex forecasting algorithms, and clever energy trading methods. As a result of their capacity to offer a versatile and dispatchable portfolio of assets (DERs), VPPs are better equipped to meet swiftly to dynamic market conditions, such as energy pricing and demand patterns. VPPs deliver a strong economic case for sustainability, affordability, and resilience in the energy ecosystem by making it possible to efficiently deploy renewable sources of energy, support demand response programs, and provide ancillary services to the grid. VPPs technology’s commercial implications hold significant promise for developing a more effective, competitive, and customer-focused energy landscape as it continues to advance.

Currently, the majority of jurisdictions have already started deregulation or liberalization and competition-opening process in their individual power markets [ 11 ]. In order to finance new infrastructure investments, increase the economic efficiency of power company operations, and particularly lower the ultimate prices of electricity delivery, deregulation or privatization has been advocated [ 146 ]. A vertical structure as stipulated by [ 146 ], where all activities were merged, was replaced with an organization where generation, transmission, distribution, and commerce work separately as a result of this reform in the energy sector.

Additionally, the large integration of renewables into the power grid that characterizes the contemporary energy landscape suggests a greater need for the system’s balancing mechanism due to the random nature of the RESs generation schedule. One significant benefit of VPPs is that they boost their shared profit by selling energy on behalf of the DER owners to improve the balancing mechanism when they access the wholesale electricity markets. The participation of VPPs in various electricity markets is covered in this section.

Day-ahead market: Day-ahead market refers to the buying and selling of electricity on the day before the actual production and delivery. VPPs actively participate in the day-ahead market by supplying their aggregated portfolio of DERs for electricity trading. VPPs forecast energy generation trends for the next day using advanced forecasting and data analytics. Based on these insights and market prices, VPPs strategically bid these aggregated resources to optimize revenue generation [ 84 , 147 , 148 , 149 , 150 , 151 ].

Ancillary service market: VPPs actively participate in the ancillary services market by providing critical assistance to the electric grid. The VPP does this by dynamically altering the output of their aggregated DERs. VPPs respond in real-time to grid signals to maintain stability, assure a continuous power supply, and improve grid reliability. With this, VPPs play an important role in supporting grid operations and optimizing grid performance. Several studies have incorporated the ability to engage in ancillary services markets into VPP modeling in order to enable regulation that ensures the security of electricity supply [ 26 , 143 , 150 , 152 , 153 , 154 , 155 , 156 ].

Reserve market: In the reserve market, VPPs actively participate by offering their combined output of DER as a reserve capacity to support the grid’s reliability. VPPs reserve a portion of their generated power from the DERs, ready to be dispatched within short notice to address sudden changes in electricity demand and supply or even an outage of grid operator’s outage of generators. By participating in the reserve market, VPPs offer a valuable and flexible solution for grid operators to maintain grid reliability. As VPP technology advances, their involvement in the reserve market will become ever more vital in contributing to the efficient and secure operation of the electric grid. Various strategies to make ideal or optimal reserve market decisions have been studied in several papers. According to the findings of these studies, the reserve market is more significant at times of peak demand since a contingency can have a higher impact [ 26 , 127 , 157 , 158 , 159 , 160 ].

Intra-day/real-time market: The VPP actively participates in the intra-day market by precisely adjusting the energy traded in the day-ahead market. The VPP strategically optimizes its DER dispatch and offers flexible resources in response to dynamic market prices and grid needs [ 11 ].

Although intraday markets enable VPPs to adjust scheduled energy after the day-ahead market, an exchange power imbalance may still emerge as the dispatch time approaches. VPPs can thus participate in real-time balancing markets to avoid penalties. The goal of the real-time market is to reduce the imbalance errors and their associated cost. The various electricity markets in which the VPP participates are provided in Table 5 to outline the key characteristics. Figure 4 also gives a graphical analysis of the key characteristics of the electricity market that the VPP operates in.

Electricity markets characteristics

Real-world implementation of VPPs

VPPs in the real world provide fascinating insights on their revolutionary impact on contemporary power systems. VPP implementations around the world demonstrate their adaptability in maximizing DERs. These examples elaborate on the value of VPPs in grid stability, renewable generation, and demand response. VPP projects are becoming more common, proving their potential to revolutionize energy systems. The VPP market is expected to grow from $1.3billion in 2019 to $5.9billion in 2027, with a compound annual growth rate of 21.3% from 2020 to 2027 [ 25 ]. In Norway, Statkraf is the world’s largest VPP with a capacity of 10GW from over 1000 aggregated assets. Recently, Tesla announced to scale up the south Australia VPP which connects assets from 4000 to 50,000 homes, which will make it the world’s largest VPP [ 172 ]. Storing and distributing power from residential and commercial customers, Tesla’s Powerpacks and Powerwall promote grid dependability and the integration of renewable energy. These real-world examples demonstrate how important VPPs are in creating a global energy ecosystem that is robust, efficient, and sustainable. Selected real-world applications [ 124 , 172 ] are summarized in Table  6 .

Applications of VPPs in the real world have offered an important lesson that will guide their development, deployment, and scalability. Key insights from these applications include the following but not limited to:

Flexibility and scalability: The significance of developing flexible and scalable systems has been shown by the successful VPP deployments. VPPs support a variety of DERs and adjust to shifting market dynamics and grid conditions.

Integration of DERs: For the VPP to operate at its best, several DERs must be integrated into a single, coordinated system. Advanced data analytics and control algorithms are essential for managing DERs efficiently and maximizing their contributions, as demonstrated by real-world applications.

Interoperability and interconnection: VPPs generally operate in sophisticated energy ecosystems with a variety of stakeholders. Smooth VPP integration and operation require interoperability and seamless interconnection with grid operators, and other market participants.

Market participation: The significance of active market participation has been emphasized by real-world VPP applications. Using effective energy trading techniques and intelligent bidding in electricity markets. VPPs can maximize income production and assist the integration of RESs at a fair price.

The ongoing development and deployment of VPPs can be improved by taking lessons from these practical applications, ensuring that they continue to contribute to a sustainable, effective, and decentralized energy future.

However, despite the successes chalked up by these projects, there are still challenges that must be addressed. Cybersecurity threats, consumer engagement, data management and analytics, achieving a positive return on investment and profitability are some of the model challenges that these projects face. Collaboration between stakeholders is necessary to overcome these obstacles.

Conclusions

VPPs have become transformative solutions revolutionizing the modern energy landscape. Applications in the real world have sounded their importance and have also demonstrated the adaptability and advantages of VPPs. VPPs have shown that they can promote the integration of renewable energy sources, aggregate and optimize a variety of DERs, and facilitate effective demand response.

Flexibility and scalability, which enable seamless adaptability to shifting grid conditions and market dynamics, have been shown to be essential for successful VPP adoption. VPPs have been able to improve cost-effective renewable energy integration and optimize revenue generation through active market participation and smart bidding tactics. Additionally, for VPPs including residential or commercial participants, consumer engagement and education are crucial for assuring buy-in and demand response programs.

Embracing the lessons learnt in the referenced literature, a VPP stands as a pivotal enabler in our journey towards a sustainable, decentralized, and resilient energy future. There can be an effective and customer-focused energy ecosystem that leads the path for a greener and more sustainable society by fully utilizing VPPs and maximizing their important contributions.

The ability of VPPs to maximize DERs, boost renewable energy integration, and improve grid stability makes them a crucial element in reaching a sustainable energy future. A VPP has the undisputed potential to change the energy landscape. The successful operation of VPPs in the modern era depends on a judicious blend of cutting-edge technology, supportive regulatory frameworks, and seamless connectivity with the existing electricity infrastructure. The aggregation and control of various DERs can be optimized by using real-time data analytics, artificial intelligence, and smart grid technologies. However, VPPs must overcome several obstacles, such as data security, grid interconnection, and scalability to realize their full potential. In a dynamic energy environment, taking care of these issues is essential to ensure the proper operation of VPPs.

Also, the development of flexible regulatory frameworks that support VPP implementation and market involvement is essential for the efficient operation of VPPs. The seamless integration of VPPs into current energy markets and the promotion of novel business models are made possible by clear regulations on market access, price structures, and grid services. Overall, an effective operation of VPPs in this era and beyond will depend on the following:

Advanced technological integration such as data analytics, smart grid technologies which are vital real-time data processing, accurate forecasting, and efficient optimization.

Regulatory support to encourage supportive and accommodative regulatory frameworks that will promote VPP deployment, and market participation.

Implementation of robust data security measures to protect sensitive information, guarantee consumer privacy, and safeguard against potential cyberattacks.

Implementing these recommendations will help shape and harness the potential of VPPs to transform the energy industry. With correct planning, VPPs will significantly contribute to the modern era’s goals of energy resource optimization, grid stability enhancement, and improved integration of RESs.

Availability of data and materials

Data sharing is not applicable to this article as no datasets were generated or analyzed during this work.

Abbreviations

Artificial Neural Network

Biogas/biomass power

Battery energy storage system

Combined heat and power

Convolutional Neural Network

Controlled load

• Distributed energy resources

Distributed generation

Distribution system operator

Energy storage system

European Union

Electric vehicles

Gas turbine

Heat pump power

Heating, ventilation, and air conditioning

Internet of Things

Long short-term memory

Mixed Integer Linear Programming

Model predictive control

Nuclear power

Pumped hydro storage

Programmable logic control

Power System Network

Particle Swarm Optimization

Photovoltaic

Renewable energy resources

• Renewable energy sources

Sustainable Development Goals

Thermal power

Transmission system operator

United Nations

Virtual power plant

Wind turbine

Zhao J, Patwary AK, Qayyum A et al (2022) The determinants of renewable energy sources for the fueling of green and sustainable economy. Energy 238:122029. https://doi.org/10.1016/j.energy.2021.122029

Article   Google Scholar  

Rapier R Global Energy Trends: insights from the 2023 Statistical Review of World Energy. In: Forbes. https://www.forbes.com/sites/rrapier/2023/08/06/global-energy-trends-insights-from-the-2023-statistical-review-of-world-energy/ . Accessed 12 Oct 2023

Liu J, Hu H, Yu SS, Trinh H (2023) Virtual power plant with renewable energy sources and energy storage systems for sustainable power grid-formation. Control Tech Demand Response Energies 16:3705. https://doi.org/10.3390/en16093705

Electricity demand globally 2040. In: Statista. https://www.statista.com/statistics/1118777/electricity-demand-worldwide/ . Accessed 2 Oct 2023

The Impact of Distributed Generation on Power Distribution. In: Util. One. https://utilitiesone.com/the-impact-of-distributed-generation-on-power-distribution . Accessed 26 Feb 2024

Strezoski L, Padullaparti H, Ding F, Baggu M (2022) Integration of utility distributed energy resource management system and aggregators for evolving distribution system operators. J Mod Power Syst Clean Energy 10:277–285. https://doi.org/10.35833/MPCE.2021.000667

Abedrabboh K, Karaki A, Al-Fagih L (2023) A combinatorial double auction for community shared distributed energy resources. IEEE Access 11:28355–28369. https://doi.org/10.1109/ACCESS.2023.3260022

Poudel S, Keene SJ, Kini RL et al (2022) Modeling environment for testing a distributed energy resource management system (DERMS) using GridAPPS-D platform. IEEE Access 10:77383–77395. https://doi.org/10.1109/ACCESS.2022.3192845

Tan Z, Zhong H, Xia Q et al (2020) Estimating the robust P-Q capability of a technical virtual power plant under uncertainties. IEEE Trans Power Syst 35:4285–4296. https://doi.org/10.1109/TPWRS.2020.2988069

Babatunde OM, Munda JL, Hamam Y (2020) Power system flexibility: a review. Energy Rep 6:101–106. https://doi.org/10.1016/j.egyr.2019.11.048

Naval N, Yusta JM (2021) Virtual power plant models and electricity markets—a review. Renew Sustain Energy Rev 149:111393. https://doi.org/10.1016/j.rser.2021.111393

Khan R, Islam N, Das SK et al (2021) Energy sustainability-survey on technology and control of microgrid, smart grid and virtual power plant. IEEE Access 9:104663–104694. https://doi.org/10.1109/ACCESS.2021.3099941

Sarmiento-Vintimilla JC, Torres E, Larruskain DM, Pérez-Molina MJ (2022) Applications, operational architectures and development of virtual power plants as a strategy to facilitate the integration of distributed energy resources. Energies 15:775. https://doi.org/10.3390/en15030775

Johansson P, Vendel M, Nuur C (2020) Integrating distributed energy resources in electricity distribution systems: an explorative study of challenges facing DSOs in Sweden. Util Policy 67:101117. https://doi.org/10.1016/j.jup.2020.101117

Lopes JAP, Hatziargyriou N, Mutale J et al (2007) Integrating distributed generation into electric power systems: a review of drivers, challenges, and opportunities. Electr Power Syst Res 77:1189–1203. https://doi.org/10.1016/j.epsr.2006.08.016

Nadeem F, Aftab MA, Hussain SMS et al (2019) Virtual power plant management in smart grids with XMPP based IEC 61850 communication. Energies 12:2398. https://doi.org/10.3390/en12122398

Marinescu B, Gomis-Bellmunt O, Dorfler F et al (2022) Dynamic virtual power plant: a new concept for grid integration of renewable energy sources. IEEE Access 10:104980–104995. https://doi.org/10.1109/ACCESS.2022.3205731

Gough M, Santos SF, Lotfi M et al (2022) Operation of a technical virtual power plant considering diverse distributed energy resources. IEEE Trans Ind Appl 58:2547–2558. https://doi.org/10.1109/TIA.2022.3143479

Palensky P, Dietrich D (2011) Demand side management: demand response, intelligent energy systems, and smart loads. IEEE Trans Ind Inform 7:381–388. https://doi.org/10.1109/TII.2011.2158841

Pasetti M, Rinaldi S, Manerba D (2018) A virtual power plant architecture for the demand-side management of smart prosumers. Appl Sci 8:432. https://doi.org/10.3390/app8030432

Lan G, Zhang Z, Guo M, et al (2022) Research on Virtual Power Plants Participating in Ancillary Service Market. In: 2022 2nd international conference on electrical engineering and control science (IC2ECS). P. 979–985

Nosratabadi SM, Hooshmand R-A, Gholipour E (2017) A comprehensive review on microgrid and virtual power plant concepts employed for distributed energy resources scheduling in power systems. Renew Sustain Energy Rev 67:341–363. https://doi.org/10.1016/j.rser.2016.09.025

Goia B, Cioara T, Anghel I (2022) Virtual power plant optimization in smart grids: a narrative review. Future Internet 14:128. https://doi.org/10.3390/fi14050128

Panda S, Mohanty S, Rout PK, Sahu BK (2022) A conceptual review on transformation of micro-grid to virtual power plant: issues, modeling, solutions, and future prospects. Int J Energy Res 46:7021–7054. https://doi.org/10.1002/er.7671

Zhang J (2022) The concept, project, and current status of virtual power plant: a review. J Phys Conf Ser 2152:012059. https://doi.org/10.1088/1742-6596/2152/1/012059

Wang H, Riaz S, Mancarella P (2020) Integrated techno-economic modeling, flexibility analysis, and business case assessment of an urban virtual power plant with multi-market co-optimization. Appl Energy 259:114142. https://doi.org/10.1016/j.apenergy.2019.114142

Yan P, Wang L, Yang S, et al (2022) Review on the development and application of virtual power plant under the background of dual carbon. In: 2nd international conference on mechanical, electronics, and electrical and automation control (METMS 2022). SPIE, pp 654–665

Rouzbahani HM, Karimipour H, Lei L (2021) A review on virtual power plant for energy management. Sustain Energy Technol Assess 47:101370. https://doi.org/10.1016/j.seta.2021.101370

Feng Y, Jia H, Wang X et al (2023) Review of operations for multi-energy coupled virtual power plants participating in electricity market. Energy Rep 9:992–999. https://doi.org/10.1016/j.egyr.2023.04.149

Wang L, Guo Z, Zhang Y, et al (2021) A review of virtual power plant: concepts and essential issues. In: 2021 IEEE sustainable power and energy conference (iSPEC). pp 655–660

Yang Z, Liu J, Baskaran S et al (2010) Enabling renewable energy—and the future grid—with advanced electricity storage. JOM 62:14–23. https://doi.org/10.1007/s11837-010-0129-0

Romero-Cadaval E, Francois B, Malinowski M, Zhong Q-C (2015) Grid-connected photovoltaic plants: an alternative energy source, replacing conventional sources. IEEE Ind Electron Mag 9:18–32. https://doi.org/10.1109/MIE.2014.2362211

Awerbuch S, Preston A (2012) The virtual utility: accounting, technology & competitive aspects of the emerging industry. Springer Science & Business Media

Google Scholar  

Dielmann K, van der Velden A (2003) Virtual power plants (VPP)—a new perspective for energy generation? In: Proceedings of the 9th international scientific and practical conference of students, post-graduates modern techniques and technologies; 2003. MTT 2003. pp 18–20

Caldon R, Patria AR, Turri R (2004) Optimisation algorithm for a virtual power plant operation. In: 39th international universities power engineering conference, vol. 2; 2004. UPEC 2004. pp 1058–1062

Pudjianto D, Ramsay C, Strbac G (2007) Virtual power plant and system integration of distributed energy resources. IET Renew Power Gener 1:10–16. https://doi.org/10.1049/iet-rpg:20060023

Pudjianto D, Ramsay C, Strbac G (2008) Microgrids and virtual power plants: Concepts to support the integration of distributed energy resources. Proc Inst Mech Eng Part J Power Energy 222:731–741. https://doi.org/10.1243/09576509JPE556

Ruiz N, Iñ C, Oyarzabal J (2009) A direct load control model for virtual power plant management. IEEE Trans Power Syst 24:959–966. https://doi.org/10.1109/TPWRS.2009.2016607

Asmus P (2010) Microgrids, virtual power plants and our distributed energy future. Electr J 23:72–82. https://doi.org/10.1016/j.tej.2010.11.001

Mashhour E, Moghaddas-Tafreshi SM (2011) Bidding strategy of virtual power plant for participating in energy and spinning reserve markets—part II: numerical analysis. IEEE Trans Power Syst 26:957–964. https://doi.org/10.1109/TPWRS.2010.2070883

Bremer J, Sonnenschein M (2014) Parallel tempering for constrained many criteria optimization in dynamic virtual power plants. In: 2014 IEEE symposium on computational intelligence applications in smart grid (CIASG). pp 1–8

Al-Awami AT, Amleh NA, Muqbel AM (2017) Optimal demand response bidding and pricing mechanism with fuzzy optimization: application for a virtual power plant. IEEE Trans Ind Appl 53:5051–5061. https://doi.org/10.1109/TIA.2017.2723338

Koraki D, Strunz K (2018) Wind and solar power integration in electricity markets and distribution networks through service-centric virtual power plants. IEEE Trans Power Syst 33:473–485. https://doi.org/10.1109/TPWRS.2017.2710481

Yu S, Fang F, Liu Y, Liu J (2019) Uncertainties of virtual power plant: Problems and countermeasures. Appl Energy 239:454–470. https://doi.org/10.1016/j.apenergy.2019.01.224

Mahmud K, Khan B, Ravishankar J et al (2020) An internet of energy framework with distributed energy resources, prosumers, and small-scale virtual power plants: an overview. Renew Sustain Energy Rev 127:109840. https://doi.org/10.1016/j.rser.2020.109840

Peng D, Poudineh R (2019) Electricity market design under increasing renewable energy penetration: Misalignments observed in the European Union. Util Policy 61:100970. https://doi.org/10.1016/j.jup.2019.100970

Liu R, Liu Y, Jing Z (2020) Impact of industrial virtual power plant on renewable energy integration. In: 2020 IEEE/IAS industrial and commercial power system Asia (I&CPS Asia). pp 1198–1202

Jin X, Wang J, Shen X, et al (2018) An overview of virtual power plant development from the perspective of market participation. In: 2018 2nd IEEE conference on energy internet and energy system integration (EI2). pp 1–6

Wang Z, Yang P, Liu S et al (2017) Coordination and optimization strategy of VPP considering demand response and multi-energy coordination. Electr Power Constr 38:60–66

Othman M, Hegazy YG, Abdelaziz A (2015) A review of virtual power plant definitions, components, framework and optimization. Int Electr Eng J IEEJ 6:2010–2024

Tong Y, Meng Z, Qiu P et al (2023) The carbon trading operation optimization for virtual power plants of industrial parks considering wind power. J Phys Conf Ser 2474:012032. https://doi.org/10.1088/1742-6596/2474/1/012032

Venkatachary SK, Alagappan A, Andrews LJB (2021) Cybersecurity challenges in energy sector (virtual power plants)—can edge computing principles be applied to enhance security? Energy Inform 4:5. https://doi.org/10.1186/s42162-021-00139-7

Hongliang W, Benjie L, Daoxin P, Ling W (2021) Virtual power plant participates in the two-level decision-making optimization of internal purchase and sale of electricity and external multi-market. IEEE Access 9:133625–133640. https://doi.org/10.1109/ACCESS.2021.3112549

Wang S, Jia R, Shi X et al (2022) Research on capacity allocation optimization of commercial virtual power plant (CVPP). Energies 15:1303. https://doi.org/10.3390/en15041303

Domingo-Mondejar ID (2022) A review of the evolution and main roles of virtual power plants as key stakeholders in power systems. IEEE Access 10:47937–47964. https://doi.org/10.1109/ACCESS.2022.3171823

Luo J, Gao Y, Yang W et al (2018) Optimal operation modes of virtual power plants based on typical scenarios considering output evaluation criteria. Energies 11:2634. https://doi.org/10.3390/en11102634

Zhang Y, Pan W, Lou X, et al (2021) Operation characteristics of virtual power plant and function design of operation management platform under emerging power system. In: 2021 international conference on power system technology (POWERCON). pp 194–196

Fan S, Xiao J, Li Z, He G (2022) Characterization and trading of energy level and energy shift considering virtual power plant. J Mod Power Syst Clean Energy 10:1784–1789. https://doi.org/10.35833/MPCE.2021.000192

Chantzis G, Papadopoulos AM, Giama E, Nizetic S (2023) The potential of demand response as a tool for decarbonization in the energy transition. Energy Build. https://doi.org/10.1016/j.enbuild.2023.113255

Behi B, Baniasadi A, Arefi A et al (2020) Cost-benefit analysis of a virtual power plant including solar PV, flow battery, heat pump, and demand management: a western Australian case study. Energies 13:2614. https://doi.org/10.3390/en13102614

Behi B, Arefi A, Jennings P et al (2021) Advanced monitoring and control system for virtual power plants for enabling customer engagement and market participation. Energies 14:1113. https://doi.org/10.3390/en14041113

Shen J, Jiang C, Li B (2015) Controllable load management approaches in smart grids. Energies 8:11187–11202. https://doi.org/10.3390/en81011187

Abbasi MH, Taki M, Rajabi A et al (2019) Coordinated operation of electric vehicle charging and wind power generation as a virtual power plant: a multi-stage risk constrained approach. Appl Energy 239:1294–1307. https://doi.org/10.1016/j.apenergy.2019.01.238

Hatziargyriou ND, Asimakopoulou GE (2020) DER integration through a monopoly DER aggregator. Energy Policy 137:111124. https://doi.org/10.1016/j.enpol.2019.111124

Tascikaraoglu A, Erdinc O, Uzunoglu M, Karakas A (2014) An adaptive load dispatching and forecasting strategy for a virtual power plant including renewable energy conversion units. Appl Energy 119:445–453. https://doi.org/10.1016/j.apenergy.2014.01.020

Oladimeji O, Ortega Á, Sigrist L, et al (2022) Modeling demand flexibility of res-based virtual power plants. In: 2022 IEEE power & energy society general meeting (PESGM). pp 1–5

Chen J, Liu M, Milano F (2021) Aggregated model of virtual power plants for transient frequency and voltage stability analysis. IEEE Trans Power Syst 36:4366–4375. https://doi.org/10.1109/TPWRS.2021.3063280

Prabatha T, Hager J, Carneiro B et al (2020) Analyzing energy options for small-scale off-grid communities: a Canadian case study. J Clean Prod 249:119320. https://doi.org/10.1016/j.jclepro.2019.119320

Lima MA, Mendes LFR, Mothé GA et al (2020) Renewable energy in reducing greenhouse gas emissions: reaching the goals of the Paris agreement in Brazil. Environ Dev 33:100504. https://doi.org/10.1016/j.envdev.2020.100504

Akaev AA, Davydova OI (2020) The Paris agreement on climate is coming into force: will the great energy transition take place? Her Russ Acad Sci 90:588–599. https://doi.org/10.1134/S1019331620050111

Muruganantham B, Gnanadass R, Padhy NP (2017) Challenges with renewable energy sources and storage in practical distribution systems. Renew Sustain Energy Rev 73:125–134. https://doi.org/10.1016/j.rser.2017.01.089

Goud BS, Reddy CR, Kalyan CN et al (2021) Grid integration of renewable energy sources using GA technique for improving power quality. Int J Renew Energy Res IJRER 11:1390–1402

Nekrasov SA (2021) Reducing costs for integration of renewable energy sources: a way to making renewable energy more accessible. Therm Eng 68:593–603. https://doi.org/10.1134/S0040601521070077

Bouali E-T, Abid MR, Boufounas E-M et al (2022) Renewable energy integration into cloud & IoT-based smart agriculture. IEEE Access 10:1175–1191. https://doi.org/10.1109/ACCESS.2021.3138160

Panda S, Dhaka RK, Panda B, et al (2022) A review on application of machine learning in solar energy & photovoltaic generation prediction. In: 2022 international conference on electronics and renewable systems (ICEARS). pp 1180–1184

Prema V, Bhaskar MS, Almakhles D et al (2022) Critical review of data, models and performance metrics for wind and solar power forecast. IEEE Access 10:667–688. https://doi.org/10.1109/ACCESS.2021.3137419

Wang H, Lei Z, Zhang X et al (2019) A review of deep learning for renewable energy forecasting. Energy Convers Manag 198:111799. https://doi.org/10.1016/j.enconman.2019.111799

Suo S, Kuang X, Cheng R, et al (2022) Research of real-time monitoring and control technology for distributed energy storage based on 5G. In: 2022 IEEE/IAS industrial and commercial power system Asia (I&CPS Asia). pp 1496–1500

Pal P, Parvathy AK, Devabalaji KR et al (2021) IoT-based real time energy management of virtual power plant using PLC for transactive energy framework. IEEE Access 9:97643–97660. https://doi.org/10.1109/ACCESS.2021.3093111

Moutis P, Georgilakis PS, Hatziargyriou ND (2018) Voltage regulation support along a distribution line by a virtual power plant based on a center of mass load modeling. IEEE Trans Smart Grid 9:3029–3038. https://doi.org/10.1109/TSG.2016.2624633

Hu D, Liu H, Zhu Y et al (2023) Demand response-oriented virtual power plant evaluation based on AdaBoost and BP neural network. Energy Rep 9:922–931. https://doi.org/10.1016/j.egyr.2023.05.012

Löschenbrand M (2021) A temporal neural network model for probabilistic multi-period forecasting of distributed energy resources. IEEE Access 9:147029–147041. https://doi.org/10.1109/ACCESS.2021.3121988

Amparore EG, Cinus F, Maestri C, et al (2021) Forecast of distributed energy generation and consumption in a partially observable electrical grid: a machine learning approach. In: 2021 IEEE Madrid PowerTech. pp 1–6

Vardhan BVS, Khedkar M, Srivastava I (2021) Cost effective day-ahead scheduling with stochastic load and intermittency forecasting for distribution system considering distributed energy resources. Energy Sour Part Recov Util Environ Eff. https://doi.org/10.1080/15567036.2021.1983669

Tripathy DS, Prusty BR (2021) Chapter 10—forecasting of renewable generation for applications in smart grid power systems. In: Tomar A, Kandari R (eds) Advances in smart grid power system. Academic Press, pp 265–298

Chapter   Google Scholar  

Zhou B, Meng Y, Huang W et al (2021) multi-energy net load forecasting for integrated local energy systems with heterogeneous prosumers. Int J Electr Power Energy Syst 126:106542. https://doi.org/10.1016/j.ijepes.2020.106542

Akhtar I, Kirmani S, Ahmad M, Ahmad S (2021) Average monthly wind power forecasting using fuzzy approach. IEEE Access 9:30426–30440. https://doi.org/10.1109/ACCESS.2021.3056562

Inteha A, Nahid-Al-Masood HF, Khan IA (2022) A data driven approach for day ahead short-term load forecasting. IEEE Access 10:84227–84243. https://doi.org/10.1109/ACCESS.2022.3197609

Abbasipour M, Igder MA, Liang X (2021) A novel hybrid neural network-based day-ahead wind speed forecasting technique. IEEE Access 9:151142–151154. https://doi.org/10.1109/ACCESS.2021.3126747

Nespoli A, Leva S, Mussetta M, Ogliari EGC (2022) A selective ensemble approach for accuracy improvement and computational load reduction in ANN-based PV power forecasting. IEEE Access 10:32900–32911. https://doi.org/10.1109/ACCESS.2022.3158364

Zheng J, Du J, Wang B et al (2023) A hybrid framework for forecasting power generation of multiple renewable energy sources. Renew Sustain Energy Rev 172:113046. https://doi.org/10.1016/j.rser.2022.113046

Maanavi M, Najafi A, Godina R et al (2019) Energy management of virtual power plant considering distributed generation sizing and pricing. Appl Sci 9:2817. https://doi.org/10.3390/app9142817

Raab AF, Ferdowsi M, Karfopoulos E, et al (2011) Virtual power plant control concepts with electric vehicles. In: 2011 16th international conference on intelligent system applications to power systems. pp 1–6

Béguin A, Nicolet C, Kawkabani B, Avellan F (2014) Virtual power plant with pumped storage power plant for renewable energy integration. In: 2014 international conference on electrical machines (ICEM). pp 1736–1742

Zafred K, Nieto-Martin J, Butans E (2016) Electric vehicles—effects on domestic low voltage networks. In: 2016 IEEE international energy conference (ENERGYCON). pp 1–6

Liu Z, Zheng W, Qi F et al (2018) Optimal dispatch of a virtual power plant considering demand response and carbon trading. Energies 11:1488. https://doi.org/10.3390/en11061488

Shropshire D, Purvins A, Papaioannou I, Maschio I (2012) Benefits and cost implications from integrating small flexible nuclear reactors with offshore wind farms in a virtual power plant. Energy Policy 46:558–573. https://doi.org/10.1016/j.enpol.2012.04.037

Rappaport RD, Miles J (2017) Cloud energy storage for grid scale applications in the UK. Energy Policy 109:609–622. https://doi.org/10.1016/j.enpol.2017.07.044

Mazzi N, Trivella A, Morales JM (2019) Enabling active/passive electricity trading in dual-price balancing markets. IEEE Trans Power Syst 34:1980–1990. https://doi.org/10.1109/TPWRS.2018.2888937

Sun G, Qian W, Huang W et al (2019) Stochastic adaptive robust dispatch for virtual power plants using the binding scenario identification approach. Energies 12:1918. https://doi.org/10.3390/en12101918

Liu J, Li J, Xiang Y et al (2019) Optimal sizing of cascade hydropower and distributed photovoltaic included virtual power plant considering investments and complementary benefits in electricity markets. Energies 12:952. https://doi.org/10.3390/en12050952

Pandžić H, Morales JM, Conejo AJ, Kuzle I (2013) Offering model for a virtual power plant based on stochastic programming. Appl Energy 105:282–292. https://doi.org/10.1016/j.apenergy.2012.12.077

Handschin E, Neise F, Neumann H, Schultz R (2006) Optimal operation of dispersed generation under uncertainty using mathematical programming. Int J Electr Power Energy Syst 28:618–626. https://doi.org/10.1016/j.ijepes.2006.03.003

Schulz C, Roder G, Kurrat M (2005) Virtual power plants with combined heat and power micro-units. In: 2005 international conference on future power systems. pp 5

Hany Elgamal A, Kocher-Oberlehner G, Robu V, Andoni M (2019) Optimization of a multiple-scale renewable energy-based virtual power plant in the UK. Appl Energy 256:113973. https://doi.org/10.1016/j.apenergy.2019.113973

Zhang J, Xu Z, Xu W et al (2019) Bi-objective dispatch of multi-energy virtual power plant: deep-learning-based prediction and particle swarm optimization. Appl Sci 9:292. https://doi.org/10.3390/app9020292

Fang F, Yu S, Liu M (2020) An improved Shapley value-based profit allocation method for CHP-VPP. Energy 213:118805. https://doi.org/10.1016/j.energy.2020.118805

Liu C, Yang RJ, Yu X et al (2021) Virtual power plants for a sustainable urban future. Sustain Cities Soc 65:102640. https://doi.org/10.1016/j.scs.2020.102640

Collath N, Tepe B, Englberger S et al (2022) Aging aware operation of lithium-ion battery energy storage systems: a review. J Energy Storage 55:105634. https://doi.org/10.1016/j.est.2022.105634

Stroe DI (2014) Lifetime models for lithium-ion batteries used in virtual power plant applications. Aalborg University, Department of Energy Technology

Behi B, Arefi A, Jennings P, et al (2020) Consumer engagement in virtual power plants through gamification. In: 2020 5th international conference on power and renewable energy (ICPRE). pp 131–137

Zurborg A (2010) Unlocking customer value: the virtual power plant. PowerWorld. https://www.energy.gov/oe/articles/unlocking-customer-value-virtual-power-plant

Henderson K, Loreau M (2023) A model of sustainable development goals: challenges and opportunities in promoting human well-being and environmental sustainability. Ecol Model 475:110164. https://doi.org/10.1016/j.ecolmodel.2022.110164

Onu Fergus U, Akpan Abasiam G (2017) Leveraging ICT for power delivery and electrification in Africa: cyber security, privacy and data protection. Int J Res 4:912–915

Sharghivand N, Derakhshan F (2021) Data security and privacy in industrial IoT. In: Karimipour H, Derakhshan F (eds) AI-enabled threat detection and security analysis for industrial IoT. Springer International Publishing, Cham, pp 21–39

Yang Q, Wang H, Wang T et al (2021) Blockchain-based decentralized energy management platform for residential distributed energy resources in a virtual power plant. Appl Energy 294:117026. https://doi.org/10.1016/j.apenergy.2021.117026

Maglaras LA, Ferrag MA, Janicke H et al (2022) Reliability, security, and privacy in power grids. Computer 55:85–88. https://doi.org/10.1109/MC.2022.3184425

Tufail S, Parvez I, Batool S, Sarwat A (2021) A survey on cybersecurity challenges, detection, and mitigation techniques for the smart grid. Energies 14:5894. https://doi.org/10.3390/en14185894

Alfiah F, Prastiwi NR (2022) Cyber security in smart grid technology: a systematic review. Int J Cyber IT Serv Manag 2:48–54

Khare U, Malviya A, Kumar Gawre S, Arya A (2023) Cyber physical security of a smart grid: a review. In: 2023 IEEE international students’ conference on electrical, electronics and computer science (SCEECS). pp 1–6

Shi J, Ma L, Li C et al (2022) A comprehensive review of standards for distributed energy resource grid-integration and microgrid. Renew Sustain Energy Rev 170:112957. https://doi.org/10.1016/j.rser.2022.112957

Saqib N, Duran IA, Sharif I (2022) Influence of energy structure, environmental regulations, and human capital on ecological sustainability in EKC framework; evidence from MINT countries. Front Environ Sci 10:968405

Patel S, Parkins JR (2023) Assessing motivations and barriers to renewable energy development: insights from a survey of municipal decision-makers in Alberta, Canada. Energy Rep 9:5788–5798. https://doi.org/10.1016/j.egyr.2023.05.027

Sikorski T, Jasiński M, Ropuszyńska-Surma E et al (2020) A case study on distributed energy resources and energy-storage systems in a virtual power plant concept: technical aspects. Energies 13:3086. https://doi.org/10.3390/en13123086

Lin J, Zhang S, Yang B, et al (2021) Customer-side energy management considering the availability of renewable virtual power plants. In: Retracted on September 15, 2021the sixth international conference on information management and technology. association for computing machinery, New York, NY, USA, pp 1–5

Liu X (2022) Research on optimal dispatch method of virtual power plant considering various energy complementary and energy low carbonization. Int J Electr Power Energy Syst 136:107670. https://doi.org/10.1016/j.ijepes.2021.107670

Iraklis C, Smend J, Almarzooqi A, Ghaoud T (2021) Optimal scheduling of synthetic reserves provided by virtual power plants. In: 2021 international conference on electrical, computer, communications and mechatronics engineering (ICECCME). pp 1–7

Rahimi M, Ardakani FJ, Ardakani AJ (2021) Optimal stochastic scheduling of electrical and thermal renewable and non-renewable resources in virtual power plant. Int J Electr Power Energy Syst 127:106658. https://doi.org/10.1016/j.ijepes.2020.106658

Dunnan L, Yuan G, Weiye W, Jiahao L (2021) Optimal scheduling method of virtual power plant based on bi level programming. IOP Conf Ser Earth Environ Sci 687:012141. https://doi.org/10.1088/1755-1315/687/1/012141

Zhang Y, Liu F, Wang Z et al (2022) Robust scheduling of virtual power plant under exogenous and endogenous uncertainties. IEEE Trans Power Syst 37:1311–1325. https://doi.org/10.1109/TPWRS.2021.3105418

Abdolrasol GMM, Hannan MA, Hussain SMS et al (2021) Energy management scheduling for microgrids in the virtual power plant system using artificial neural networks. Energies 14:6507. https://doi.org/10.3390/en14206507

Heydarian-Forushani E, Elghali SB, Zerrougui M, et al (2021) A centralized-stochastic solution for smart energy management in a virtual power plant. In: 2021 IEEE international conference on environment and electrical engineering and 2021 IEEE industrial and commercial power systems Europe (EEEIC/I&CPS Europe). pp 1–5

Ul Ali Binte Wasif J, Kazmi SAA, Altamimi A et al (2022) Smart energy management in virtual power plant paradigm with a new improved multilevel optimization based approach. IEEE Access 10:50062–50077. https://doi.org/10.1109/ACCESS.2022.3169707

Yin S, Ai Q, Li Z et al (2020) Energy management for aggregate prosumers in a virtual power plant: a robust Stackelberg game approach. Int J Electr Power Energy Syst 117:105605. https://doi.org/10.1016/j.ijepes.2019.105605

Oshnoei A, Kheradmandi M, Blaabjerg F et al (2022) Coordinated control scheme for provision of frequency regulation service by virtual power plants. Appl Energy 325:119734. https://doi.org/10.1016/j.apenergy.2022.119734

Hu Q, Han R, Quan X et al (2022) Grid-forming inverter enabled virtual power plants with inertia support capability. IEEE Trans Smart Grid 13:4134–4143. https://doi.org/10.1109/TSG.2022.3141414

Chen W, Qiu J, Zhao J et al (2021) Bargaining game-based profit allocation of virtual power plant in frequency regulation market considering battery cycle life. IEEE Trans Smart Grid 12:2913–2928. https://doi.org/10.1109/TSG.2021.3053000

Yang J, Zheng Q, Zhao J, et al (2017) Control strategy of virtual power plant participating in the system frequency regulation service. In: 2017 4th international conference on systems and informatics (ICSAI). pp 324–328

Jiazhen Wang, Xinwei Shen, Yinliang Xu, et al (2018) Ancillary service for frequency regulation based on multi-energy virtual power plant aggregating factory load. In: 11th IET international conference on advances in power system control, operation and management (APSCOM 2018). Institution of Engineering and Technology, Hong Kong, China, p 51 (7 pp.)-51 (7 pp.)

Zhong W, Tzounas G, Milano F (2022) Improving the power system dynamic response through a combined voltage-frequency control of distributed energy resources. IEEE Trans Power Syst 37:4375–4384. https://doi.org/10.1109/TPWRS.2022.3148243

Liu Q, Wang Y, Wang S et al (2022) Voltage regulation strategy for DC distribution networks based on coordination of centralized control and adaptive droop control. IEEE Trans Power Deliv 37:3730–3739. https://doi.org/10.1109/TPWRD.2021.3135884

Guili Y, Sixuan C, Xiaoxuan D (2021) Research on two-stage game strategy of virtual power plant in deep peak regulation auxiliary service market. E3S Web Conf 256:01026. https://doi.org/10.1051/e3sconf/202125601026

Li Y, Deng Y, Wang Y et al (2023) Robust bidding strategy for multi-energy virtual power plant in peak-regulation ancillary service market considering uncertainties. Int J Electr Power Energy Syst 151:109101. https://doi.org/10.1016/j.ijepes.2023.109101

Ya L, Deliang Z, Xuanyuan W (2019) A peak regulation ancillary service optimal dispatch method of virtual power plant based on reinforcement learning. In: 2019 IEEE innovative smart grid technologies—Asia (ISGT Asia). pp 4356–4361

Rashidizadeh-Kermani H, Vahedipour-Dahraie M, Shafie-khah M, et al (2020) Optimal scheduling of a virtual power plant with demand response in short-term electricity market. In: 2020 IEEE 20th Mediterranean electrotechnical conference (MELECON). pp 599–604

Liu H, Khan AR, Aslam S et al (2022) Financial impact of energy efficiency and energy policies aimed at power sector reforms: mediating role of financing in the power sector. Environ Sci Pollut Res 29:18891–18904. https://doi.org/10.1007/s11356-021-16882-z

Michael NE, Hasan S, Al-Durra A, Mishra M (2023) Economic scheduling of virtual power plant in day-ahead and real-time markets considering uncertainties in electrical parameters. Energy Rep 9:3837–3850. https://doi.org/10.1016/j.egyr.2023.02.092

Hadayeghparast S, SoltaniNejad Farsangi A, Shayanfar H (2019) Day-ahead stochastic multi-objective economic/emission operational scheduling of a large scale virtual power plant. Energy 172:630–646. https://doi.org/10.1016/j.energy.2019.01.143

Yang D, He S, Wang M, Pandžić H (2020) Bidding strategy for virtual power plant considering the large-scale integrations of electric vehicles. IEEE Trans Ind Appl 56:5890–5900. https://doi.org/10.1109/TIA.2020.2993532

Shayegan-Rad A, Badri A, Zangeneh A (2017) Day-ahead scheduling of virtual power plant in joint energy and regulation reserve markets under uncertainties. Energy 121:114–125. https://doi.org/10.1016/j.energy.2017.01.006

Dabhi D, Pandya K (2020) Metaheuristic optimization algorithm for day-ahead energy resource management (ERM) in microgrid environment of power system. In: Mehta A, Rawat A, Chauhan P (eds) Recent advances in communication infrastructure. Springer, Singapore, pp 115–125

Tajeddini MA, Rahimi-Kian A, Soroudi A (2014) Risk averse optimal operation of a virtual power plant using two stage stochastic programming. Energy 73:958–967. https://doi.org/10.1016/j.energy.2014.06.110

Zhou B, Liu X, Cao Y et al (2016) Optimal scheduling of virtual power plant with battery degradation cost. IET Gener Transm Distrib 10:712–725. https://doi.org/10.1049/iet-gtd.2015.0103

Heredia F-J, Cuadrado MD, Corchero C (2018) On optimal participation in the electricity markets of wind power plants with battery energy storage systems. Comput Oper Res 96:316–329. https://doi.org/10.1016/j.cor.2018.03.004

Fusco A, Gioffrè D, Francesco Castelli A et al (2023) A multi-stage stochastic programming model for the unit commitment of conventional and virtual power plants bidding in the day-ahead and ancillary services markets. Appl Energy 336:120739. https://doi.org/10.1016/j.apenergy.2023.120739

Jia H, Wang X, Zhang X, Liu D (2023) Optimal operation of virtual power plants participating in auxiliary service market coordinating with energy storage allocation. In: Jia H, Wang X, Zhang X, Liu D (eds) Business models and reliable operation of virtual power plants. Singapore, Springer Nature, pp 69–90

Baringo A, Baringo L, Arroyo JM (2019) Day-ahead self-scheduling of a virtual power plant in energy and reserve electricity markets under uncertainty. IEEE Trans Power Syst 34:1881–1894. https://doi.org/10.1109/TPWRS.2018.2883753

Alahyari A, Ehsan M, Mousavizadeh M (2019) A hybrid storage-wind virtual power plant (VPP) participation in the electricity markets: a self-scheduling optimization considering price, renewable generation, and electric vehicles uncertainties. J Energy Storage 25:100812. https://doi.org/10.1016/j.est.2019.100812

Ju L, Tan Q, Lu Y et al (2019) A CVaR-robust-based multi-objective optimization model and three-stage solution algorithm for a virtual power plant considering uncertainties and carbon emission allowances. Int J Electr Power Energy Syst 107:628–643. https://doi.org/10.1016/j.ijepes.2018.12.012

Zhou Y, Wei Z, Sun G et al (2019) Four-level robust model for a virtual power plant in energy and reserve markets. IET Gener Transm Distrib 13:2036–2043. https://doi.org/10.1049/iet-gtd.2018.5197

(2021) Optimal scheduling of synthetic reserves provided by virtual power plants. https://doi.org/10.1109/ICECCME52200.2021.9591118

Ziegler C, Richter A, Hauer I, Wolter M (2018) Technical integration of virtual power plants enhanced by energy storages into German system operation with regard to following the schedule in intra-day. In: 2018 53rd international universities power engineering conference (UPEC). pp 1–6

Wozabal D, Rameseder G (2020) Optimal bidding of a virtual power plant on the Spanish day-ahead and intraday market for electricity. Eur J Oper Res 280:639–655. https://doi.org/10.1016/j.ejor.2019.07.022

Article   MathSciNet   Google Scholar  

Toubeau J-F, De Grève Z, Vallée F (2018) Medium-term multimarket optimization for virtual power plants: a stochastic-based decision environment. IEEE Trans Power Syst 33:1399–1410. https://doi.org/10.1109/TPWRS.2017.2718246

Ko R, Kang D, Joo S-K (2019) Mixed integer quadratic programming based scheduling methods for day-ahead bidding and intra-day operation of virtual power plant. Energies 12:1410. https://doi.org/10.3390/en12081410

Kong X, Xiao J, Wang C et al (2019) Bi-level multi-time scale scheduling method based on bidding for multi-operator virtual power plant. Appl Energy 249:178–189. https://doi.org/10.1016/j.apenergy.2019.04.130

Rahimiyan M, Baringo L (2019) Real-time energy management of a smart virtual power plant. IET Gener Transm Distrib 13:2015–2023. https://doi.org/10.1049/iet-gtd.2018.5637

Tang W, Yang H-T (2019) Optimal operation and bidding strategy of a virtual power plant integrated with energy storage systems and elasticity demand response. IEEE Access 7:79798–79809. https://doi.org/10.1109/ACCESS.2019.2922700

Gao R, Guo H, Zhang R et al (2019) A two-stage dispatch mechanism for virtual power plant utilizing the CVaR Theory in the electricity spot market. Energies 12:3402. https://doi.org/10.3390/en12173402

Hu J, Jiang C, Liu Y (2019) Short-term bidding strategy for a price-maker virtual power plant based on interval optimization. Energies 12:3662. https://doi.org/10.3390/en12193662

Wu H, Liu X, Ye B, Xu B (2020) Optimal dispatch and bidding strategy of a virtual power plant based on a Stackelberg game. IET Gener Transm Distrib 14:552–563. https://doi.org/10.1049/iet-gtd.2019.0493

Virtual Power Plants: The Future of Renewable Power? https://www.linkedin.com/pulse/virtual-power-plants-future-renewable-ashik-kalam . Accessed 19 Sept 2023

Download references

Acknowledgements

Authors express their sincere gratitude to Professor Farrag of School of Computing, Engineering and the Built Environment at Glasgow Caledonian University for proofreading and providing valuable insights in enhancing the accuracy, clarity, and readability of this manuscript.

Open access funding provided by The Science, Technology & Innovation Funding Authority (STDF) in cooperation with The Egyptian Knowledge Bank (EKB).

Author information

Authors and affiliations.

Department of Electrical Power Engineering, Egypt Japan University of Science and Technology, New Borg El-Arab City, Egypt

Sobhy Abdelkader, Jeremiah Amissah & Omar Abdel-Rahim

Electrical Engineering Department, Mansoura University, Mansoura, 35516, Egypt

Sobhy Abdelkader

Electrical Engineering Department, Aswan University, Aswan, Egypt

Omar Abdel-Rahim

You can also search for this author in PubMed   Google Scholar

Contributions

SA set the main topic of the paper. JA searched for and collected most of the references. All authors contributed in analysis and writing. OA and SA worked on the review comments and carried out the required amendments. All the authors reviewed and approved the final version before submission.

Corresponding author

Correspondence to Sobhy Abdelkader .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Abdelkader, S., Amissah, J. & Abdel-Rahim, O. Virtual power plants: an in-depth analysis of their advancements and importance as crucial players in modern power systems. Energ Sustain Soc 14 , 52 (2024). https://doi.org/10.1186/s13705-024-00483-y

Download citation

Received : 14 August 2023

Accepted : 07 August 2024

Published : 27 August 2024

DOI : https://doi.org/10.1186/s13705-024-00483-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Contemporary power systems
• Virtual power plants

Energy, Sustainability and Society

ISSN: 2192-0567

• General enquiries: [email protected]