action research discussion of results

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

7 Let it Be Known! Sharing your Results

ESSENTIAL QUESTIONS

• What are the best ways to share my findings?
• Why should I share my work at a conference?
• What are the key components of a report detailing your findings?

Many action researchers are not expected to share their findings or produce written reports, yet it is a useful endeavor for not only the educator-researcher, but also for colleagues in their related fields. For those who are compelled or required to share their findings, Hopkins (2003, 140) provides some guidance, asserting that all action researchers need to share their data and share it in a way that:

• the study could be replicated in another context;
• the evidence used to generate claims or action is clearly documented;
• the action taken as a result of the research is tracked;
• the findings are accessible to the consumer and relatable to their practice.

I personally believe it is important to formally share your work one way or another, or at least prepare it to be shared. This process helps you think deeply and concisely about what you have researched, what your findings were, and what the significance is for you and your colleagues. When you prepare your work for public consumption, you add another layer of scrutiny and validity to your thinking and editing process.

Action researchers can share their findings in several ways that colleagues and other consumers of research will be able to engage with their work. The three following ways are the most common paths for educator-researchers to share their work:

• Develop a report for personal documentation or to be shared with colleagues.
• Write an article summarizing your research and its significance to the field.
• Local, State, Regional, National, or International Conference
• District or School-Wide Professional Learning Session or Workshop
• Research Symposium
• Personal Website

Writing a Report or Article

Regardless of the purposes for writing your report or article, there are a few factors to consider as you begin to write. Remember, the purpose of action research is to improve your practice and/or implement change, based on the findings of your research, as part of professional learning and development as an educator. As mentioned in previous chapters, your goal as an action researcher is not to make generalizable claims, but to share your research with other educators who want to learn from it, develop a similar study, or use your findings to improve their own teaching in a similar context. Whether it is a report that you share on your own, or an article accepted, edited, and published by a journal or magazine, the important part is to share your findings and contribute to the knowledge base.

Before you begin, the primary task is to consider the audience you are addressing and the requirements and the purpose of your report or article. An article usually has a specific audience and purpose. For example, if I submit an article to the Elementary Social Studies Journal , then I am trying to inform elementary teachers about my findings in social studies and I am providing pedagogical insights to them. However, reports can have several purposes depending on the intent and audience. Reports can be for the purpose of:

• Reporting to Grant or External Funding Agency;
• Completing a thesis or dissertation;
• Contributing to a Pedagogical or Educational Database;
• Documenting for Personal, Administrative, or District-Level Record.

Regardless of the purpose, it is important to demonstrate a clear and consistent understanding of the issues you have researched. With the exception of reports to some grant or external funding agencies (as they may require formal writing or templates), when reporting on your action research, the quality of your writing can be enhanced by writing in an authentic and personal style. I have always felt that reporting action research is often powerful for one’s own professional learning and development because of the personal nature of the writing. It may be useful to think about it as you are reporting your own story, based on your experiences and collaborations with other people.

When writing a report or article you will want to have representations of the following sections:

• Problematization of your Topic (Why is your topic important to you or the field?)
• Literature Review and Underlying Theories (What do we know and not know?)
• Methodology (How was your study structured, what data was collected, and how was data analyzed?)
• Summary of Findings (What were the predominant themes, codes, patterns, or meaningful consequences of the study?)
• Discussion of Findings’ Significance (How do your findings compare to the literature?)
• Implications of Findings for Practice (How will your findings impact your practice?)

These sections will help you think about the important aspects of your study, as well as the aspects that will be of interest to potential readers.

Imagine the Reader

As an educator-researcher you can imagine many of your colleagues as potential readers of your work. Imagining potential readers is a useful strategy to utilize as you write your report. In this vein, and as you think about the aforementioned sections, the following considerations provide further guidance in the writing process:

• Always provide the background to your study, your context, and your positionality as an educator-researcher. Readers will potentially relate to your study and more easily apply the findings to their own context.
• Clearly present your aims, intentions, and purposes to situate your study and present your findings within the context of what you have set out to achieve.
• Do not be afraid to describe the process, success and challenges, as readers appreciate realism and honesty.
• Write clearly and concisely so others may be able to replicate the study.
• Write in first person if it feels more natural and accurate to the study.
• Readers may not be knowledgeable about your topic. Be concise and explain all aspects of your study in clear, simple language, and explain any educational jargon to be clear about its meaning.
• It is easier to read text with subheadings. Use subheadings when possible.

Since your study will likely be an inquiry into your own practice, remember our discussions from other chapters related to subjectivity:

• Acknowledge your own beliefs, prior assumptions, and values as part of your positionality or bias statement.
• Acknowledge any experiences that will relate directly to the study and your interpretations of the data.
• Discuss any ethical issues and how you addressed them.

Presentations of Action Research

There are many ways for educator researchers to present their findings. Some educator researchers present their research findings to colleagues and others at discipline-specific conferences before writing their final reports, as they believed that the preparation for the presentation helped to bring their thoughts together. Many others present their research findings after they have written out their reports, and still, many other researchers do not write a formal report, but instead disseminate their research through various presentations in other ways. These different methods of presentations all serve the purpose of bringing their ideas together and reflecting on them before sharing their work with colleagues and others. Here are some examples of presentations.

Conference presentations

A primary way for academic researchers to disseminate their research is through conference presentations at either the local, state, regional, national, or international level. I encourage educator-researchers to do the same, as these are some of the best ways to share your research with engaged and captivated audiences who attended the conference specifically to find out about new research. Similar to writing an article for a specific journal, many conferences will have a disciplinary or developmental level focus that will allow you to present your work to the most interested audience.

District or school-wide professional learning session or workshop

As an educator in a school context, your districts and schools will undoubtedly offer professional learning opportunities or workshops. Educators in the district or school are often encouraged to present at these events, especially if you are researching a new initiative implemented by the district or school.

Research symposium

If you and some other colleagues have all done action research studies, or maybe a group of colleagues researched the same topic, it would be appropriate to create a research symposium to share your work. These can be formal or informal, but they are a way to have a conference-like setting focused on a specific topic and for specific audience.

Web-Based Contributions

Many educator researchers are simply and effectively sharing their research online. There are many ways to share your research online, including some ways that would be in combination with writing an article, report, or sharing at a conference on an organization’s website. However, the most common ways for individual educator researchers to share their work is through providing a webinar, contributing to a blog, or uploading to a personal website. These online formats all provide a way for educator researchers to present their work and reflect on it with the potential to receive feedback from others. Below are some journals specifically focused on publishing education-based action research:

Action Research Publications

• Action Research – a print-based, international, interdisciplinary, peer reviewed, quarterly published refereed journal which is a forum for the development of the theory and practice of action research https://journals.sagepub.com/home/arj
• Educational Action Research – Supported by Collaborative Action Research Network (CARN) a print-based peer reviewed journal. https://www.tandfonline.com/toc/reac20/current
• Journal of Teacher Action Research – an open-access, online, international journal that publishes peer-reviewed articles and lesson plans written by teachers and researchers to inform classroom practice. http://www.practicalteacherresearch.com/
• Inquiry in Education – a online, peer reviewed international journal of action research in education and related fields. https://digitalcommons.nl.edu/ie/
• Networks: An Online Journal for Teacher Research – offers a place for sharing reports of action research, in which teachers at all levels, kindergarten to postgraduate, are reflecting on classroom practice through research ventures. https://newprairiepress.org/networks/

Concluding Thoughts

As we discussed in Chapter 1 of this book, Action Research is a cycle—the process is ongoing, and for many teachers, once you engage in Action Research, it becomes difficult to stop pursuing new and interesting questions in your classroom. As you answer one question, new ideas and issues emerge, prompting a new modification, and so on. Action Research, as such, is not finite. For teacher action researchers, disseminating your work is an important step in this cycle, as it offers you an opportunity to contribute your new knowledge to the field at-large, and it can open the door to new learning opportunities for both you and your colleagues. Please do not get stressed out about the dissemination portion of this cycle. Simply find the best way for you to share your hard work and accomplish your intended goals. The important part is to share your work and share in a way that allows you to deeply reflect, celebrate your progress, get feedback, and contemplate your next steps or project. The best teachers are lifelong learners, and Action Research allows you the space to continue the deep learning that is necessary in education. Hopefully, this book has provided a vehicle to engage in a cycle of research in your classroom.

The following supplemental chapter contains a full-length vignette from a high school English teacher. The vignette details the steps to an action research project using a real-life example from her classroom. While every project will look different, the vignette serves as an outline for how action research can develop from your classroom wonderings, and it includes the detailed steps the teacher took to fulfill all the parts of action research as outlined in this book.

Action Research Copyright © by J. Spencer Clark; Suzanne Porath; Julie Thiele; and Morgan Jobe is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License , except where otherwise noted.

Share This Book

Action Research: Steps, Benefits, and Tips

Introduction

History of action research, what is the definition of action research, types of action research, conducting action research.

Action research is an approach to qualitative inquiry in social science research that involves the search for practical solutions to everyday issues. Rooted in real-world problems, it seeks not just to understand but also to act, bringing about positive change in specific contexts. Often distinguished by its collaborative nature, the action research process goes beyond traditional research paradigms by emphasizing the involvement of those being studied in resolving social conflicts and effecting positive change.

The value of action research lies not just in its outcomes, but also in the process itself, where stakeholders become active participants rather than mere subjects. In this article, we'll examine action research in depth, shedding light on its history, principles, and types of action research.

Tracing its roots back to the mid-20th century, Kurt Lewin developed classical action research as a response to traditional research methods in the social sciences that often sidelined the very communities they studied. Proponents of action research championed the idea that research should not just be an observational exercise but an actionable one that involves devising practical solutions. Advocates believed in the idea of research leading to immediate social action, emphasizing the importance of involving the community in the process.

Applications for action research

Over the years, action research has evolved and diversified. From its early applications in social psychology and organizational development, it has branched out into various fields such as education, healthcare, and community development, informing questions around improving schools, minority problems, and more. This growth wasn't just in application, but also in its methodologies.

How is action research different?

Like all research methodologies, effective action research generates knowledge. However, action research stands apart in its commitment to instigate tangible change. Traditional research often places emphasis on passive observation , employing data collection methods primarily to contribute to broader theoretical frameworks . In contrast, action research is inherently proactive, intertwining the acts of observing and acting.

The primary goal isn't just to understand a problem but to solve or alleviate it. Action researchers partner closely with communities, ensuring that the research process directly benefits those involved. This collaboration often leads to immediate interventions, tweaks, or solutions applied in real-time, marking a departure from other forms of research that might wait until the end of a study to make recommendations.

This proactive, change-driven nature makes action research particularly impactful in settings where immediate change is not just beneficial but essential.

Action research is best understood as a systematic approach to cooperative inquiry. Unlike traditional research methodologies that might primarily focus on generating knowledge, action research emphasizes producing actionable solutions for pressing real-world challenges.

This form of research undertakes a cyclic and reflective journey, typically cycling through stages of planning , acting, observing, and reflecting. A defining characteristic of action research is the collaborative spirit it embodies, often dissolving the rigid distinction between the researcher and the researched, leading to mutual learning and shared outcomes.

Advantages of action research

One of the foremost benefits of action research is the immediacy of its application. Since the research is embedded within real-world issues, any findings or solutions derived can often be integrated straightaway, catalyzing prompt improvements within the concerned community or organization. This immediacy is coupled with the empowering nature of the methodology. Participants aren't mere subjects; they actively shape the research process, giving them a tangible sense of ownership over both the research journey and its eventual outcomes.

Moreover, the inherent adaptability of action research allows researchers to tweak their approaches responsively based on live feedback. This ensures the research remains rooted in the evolving context, capturing the nuances of the situation and making any necessary adjustments. Lastly, this form of research tends to offer a comprehensive understanding of the issue at hand, harmonizing socially constructed theoretical knowledge with hands-on insights, leading to a richer, more textured understanding.

Disadvantages of action research

Like any methodology, action research isn't devoid of challenges. Its iterative nature, while beneficial, can extend timelines. Researchers might find themselves engaged in multiple cycles of observation, reflection, and action before arriving at a satisfactory conclusion. The intimate involvement of the researcher with the research participants , although crucial for collaboration, opens doors to potential conflicts. Through collaborative problem solving, disagreements can lead to richer and more nuanced solutions, but it can take considerable time and effort.

Another limitation stems from its focus on a specific context: results derived from a particular action research project might not always resonate or be applicable in a different context or with a different group. Lastly, the depth of collaboration this methodology demands means all stakeholders need to be deeply invested, and such a level of commitment might not always be feasible.

Examples of action research

To illustrate, let's consider a few scenarios. Imagine a classroom where a teacher observes dwindling student participation. Instead of sticking to conventional methods, the teacher experiments with introducing group-based activities. As the outcomes unfold, the teacher continually refines the approach based on student feedback, eventually leading to a teaching strategy that rejuvenates student engagement.

In a healthcare context, hospital staff who recognize growing patient anxiety related to certain procedures might innovate by introducing a new patient-informing protocol. As they study the effects of this change, they could, through iterations, sculpt a procedure that diminishes patient anxiety.

Similarly, in the realm of community development, a community grappling with the absence of child-friendly public spaces might collaborate with local authorities to conceptualize a park. As they monitor its utilization and societal impact, continual feedback could refine the park's infrastructure and design.

Contemporary action research, while grounded in the core principles of collaboration, reflection, and change, has seen various adaptations tailored to the specific needs of different contexts and fields. These adaptations have led to the emergence of distinct types of action research, each with its unique emphasis and approach.

Collaborative action research

Collaborative action research emphasizes the joint efforts of professionals, often from the same field, working together to address common concerns or challenges. In this approach, there's a strong emphasis on shared responsibility, mutual respect, and co-learning. For example, a group of classroom teachers might collaboratively investigate methods to improve student literacy, pooling their expertise and resources to devise, implement, and refine strategies for improving teaching.

Participatory action research

Participatory action research (PAR) goes a step further in dissolving the barriers between the researcher and the researched. It actively involves community members or stakeholders not just as participants, but as equal partners in the entire research process. PAR is deeply democratic and seeks to empower participants, fostering a sense of agency and ownership. For instance, a participatory research project might involve local residents in studying and addressing community health concerns, ensuring that the research process and outcomes are both informed by and beneficial to the community itself.

Educational action research

Educational action research is tailored specifically to practical educational contexts. Here, educators take on the dual role of teacher and researcher, seeking to improve teaching practices, curricula, classroom dynamics, or educational evaluation. This type of research is cyclical, with educators implementing changes, observing outcomes, and reflecting on results to continually enhance the educational experience. An example might be a teacher studying the impact of technology integration in her classroom, adjusting strategies based on student feedback and learning outcomes.

Community-based action research

Another noteworthy type is community-based action research, which focuses primarily on community development and well-being. Rooted in the principles of social justice, this approach emphasizes the collective power of community members to identify, study, and address their challenges. It's particularly powerful in grassroots movements and local development projects where community insights and collaboration drive meaningful, sustainable change.

Key insights and critical reflection through research with ATLAS.ti

Organize all your data analysis and insights with our powerful interface. Download a free trial today.

Engaging in action research is both an enlightening and transformative journey, rooted in practicality yet deeply connected to theory. For those embarking on this path, understanding the essentials of an action research study and the significance of a research cycle is paramount.

Understanding the action research cycle

At the heart of action research is its cycle, a structured yet adaptable framework guiding the research. This cycle embodies the iterative nature of action research, emphasizing that learning and change evolve through repetition and reflection.

The typical stages include:

• Identifying a problem : This is the starting point where the action researcher pinpoints a pressing issue or challenge that demands attention.
• Planning : Here, the researcher devises an action research strategy aimed at addressing the identified problem. In action research, network resources, participant consultation, and the literature review are core components in planning.
• Action : The planned strategies are then implemented in this stage. This 'action' phase is where theoretical knowledge meets practical application.
• Observation : Post-implementation, the researcher observes the outcomes and effects of the action. This stage ensures that the research remains grounded in the real-world context.
• Critical reflection : This part of the cycle involves analyzing the observed results to draw conclusions about their effectiveness and identify areas for improvement.
• Revision : Based on the insights from reflection, the initial plan is revised, marking the beginning of another cycle.

Rigorous research and iteration

It's essential to understand that while action research is deeply practical, it doesn't sacrifice rigor . The cyclical process ensures that the research remains thorough and robust. Each iteration of the cycle in an action research project refines the approach, drawing it closer to an effective solution.

The role of the action researcher

The action researcher stands at the nexus of theory and practice. Not just an observer, the researcher actively engages with the study's participants, collaboratively navigating through the research cycle by conducting interviews, participant observations, and member checking . This close involvement ensures that the study remains relevant, timely, and responsive.

Drawing conclusions and informing theory

As the research progresses through multiple iterations of data collection and data analysis , drawing conclusions becomes an integral aspect. These conclusions, while immediately beneficial in addressing the practical issue at hand, also serve a broader purpose. They inform theory, enriching the academic discourse and providing valuable insights for future research.

Identifying actionable insights

Keep in mind that action research should facilitate implications for professional practice as well as space for systematic inquiry. As you draw conclusions about the knowledge generated from action research, consider how this knowledge can create new forms of solutions to the pressing concern you set out to address.

Collecting data and analyzing data starts with ATLAS.ti

Download a free trial of our intuitive software to make the most of your research.

the encyclopaedia of pedagogy and informal education

What is action research and how do we do it?

In this article, we explore the development of some different traditions of action research and provide an introductory guide to the literature., contents : what is action research ·  origins · the decline and rediscovery of action research · undertaking action research · conclusion · further reading · how to cite this article . see, also: research for practice ..

In the literature, discussion of action research tends to fall into two distinctive camps. The British tradition – especially that linked to education – tends to view action research as research-oriented toward the enhancement of direct practice. For example, Carr and Kemmis provide a classic definition:

Action research is simply a form of self-reflective enquiry undertaken by participants in social situations in order to improve the rationality and justice of their own practices, their understanding of these practices, and the situations in which the practices are carried out (Carr and Kemmis 1986: 162).

Many people are drawn to this understanding of action research because it is firmly located in the realm of the practitioner – it is tied to self-reflection. As a way of working it is very close to the notion of reflective practice coined by Donald Schön (1983).

The second tradition, perhaps more widely approached within the social welfare field – and most certainly the broader understanding in the USA is of action research as ‘the systematic collection of information that is designed to bring about social change’ (Bogdan and Biklen 1992: 223). Bogdan and Biklen continue by saying that its practitioners marshal evidence or data to expose unjust practices or environmental dangers and recommend actions for change. In many respects, for them, it is linked into traditions of citizen’s action and community organizing. The practitioner is actively involved in the cause for which the research is conducted. For others, it is such commitment is a necessary part of being a practitioner or member of a community of practice. Thus, various projects designed to enhance practice within youth work, for example, such as the detached work reported on by Goetschius and Tash (1967) could be talked of as action research.

Kurt Lewin is generally credited as the person who coined the term ‘action research’:

The research needed for social practice can best be characterized as research for social management or social engineering. It is a type of action-research, a comparative research on the conditions and effects of various forms of social action, and research leading to social action. Research that produces nothing but books will not suffice (Lewin 1946, reproduced in Lewin 1948: 202-3)

His approach involves a spiral of steps, ‘each of which is composed of a circle of planning, action and fact-finding about the result of the action’ ( ibid. : 206). The basic cycle involves the following:

This is how Lewin describes the initial cycle:

The first step then is to examine the idea carefully in the light of the means available. Frequently more fact-finding about the situation is required. If this first period of planning is successful, two items emerge: namely, “an overall plan” of how to reach the objective and secondly, a decision in regard to the first step of action. Usually this planning has also somewhat modified the original idea. ( ibid. : 205)

The next step is ‘composed of a circle of planning, executing, and reconnaissance or fact-finding for the purpose of evaluating the results of the second step, and preparing the rational basis for planning the third step, and for perhaps modifying again the overall plan’ ( ibid. : 206). What we can see here is an approach to research that is oriented to problem-solving in social and organizational settings, and that has a form that parallels Dewey’s conception of learning from experience.

The approach, as presented, does take a fairly sequential form – and it is open to a literal interpretation. Following it can lead to practice that is ‘correct’ rather than ‘good’ – as we will see. It can also be argued that the model itself places insufficient emphasis on analysis at key points. Elliott (1991: 70), for example, believed that the basic model allows those who use it to assume that the ‘general idea’ can be fixed in advance, ‘that “reconnaissance” is merely fact-finding, and that “implementation” is a fairly straightforward process’. As might be expected there was some questioning as to whether this was ‘real’ research. There were questions around action research’s partisan nature – the fact that it served particular causes.

The decline and rediscovery of action research

Action research did suffer a decline in favour during the 1960s because of its association with radical political activism (Stringer 2007: 9). There were, and are, questions concerning its rigour, and the training of those undertaking it. However, as Bogdan and Biklen (1992: 223) point out, research is a frame of mind – ‘a perspective that people take toward objects and activities’. Once we have satisfied ourselves that the collection of information is systematic and that any interpretations made have a proper regard for satisfying truth claims, then much of the critique aimed at action research disappears. In some of Lewin’s earlier work on action research (e.g. Lewin and Grabbe 1945), there was a tension between providing a rational basis for change through research, and the recognition that individuals are constrained in their ability to change by their cultural and social perceptions, and the systems of which they are a part. Having ‘correct knowledge’ does not of itself lead to change, attention also needs to be paid to the ‘matrix of cultural and psychic forces’ through which the subject is constituted (Winter 1987: 48).

Subsequently, action research has gained a significant foothold both within the realm of community-based, and participatory action research; and as a form of practice-oriented to the improvement of educative encounters (e.g. Carr and Kemmis 1986).

Exhibit 1: Stringer on community-based action research

A fundamental premise of community-based action research is that it commences with an interest in the problems of a group, a community, or an organization. Its purpose is to assist people in extending their understanding of their situation and thus resolving problems that confront them….

Community-based action research is always enacted through an explicit set of social values. In modern, democratic social contexts, it is seen as a process of inquiry that has the following characteristics:

• It is democratic , enabling the participation of all people.

• It is equitable , acknowledging people’s equality of worth.

• It is liberating , providing freedom from oppressive, debilitating conditions.

• It is life enhancing , enabling the expression of people’s full human potential.

(Stringer 1999: 9-10)

Undertaking action research

As Thomas (2017: 154) put it, the central aim is change, ‘and the emphasis is on problem-solving in whatever way is appropriate’. It can be seen as a conversation rather more than a technique (McNiff et. al. ). It is about people ‘thinking for themselves and making their own choices, asking themselves what they should do and accepting the consequences of their own actions’ (Thomas 2009: 113).

The action research process works through three basic phases:

Look -building a picture and gathering information. When evaluating we define and describe the problem to be investigated and the context in which it is set. We also describe what all the participants (educators, group members, managers etc.) have been doing.

Think – interpreting and explaining. When evaluating we analyse and interpret the situation. We reflect on what participants have been doing. We look at areas of success and any deficiencies, issues or problems.

Act – resolving issues and problems. In evaluation we judge the worth, effectiveness, appropriateness, and outcomes of those activities. We act to formulate solutions to any problems. (Stringer 1999: 18; 43-44;160)

The use of action research to deepen and develop classroom practice has grown into a strong tradition of practice (one of the first examples being the work of Stephen Corey in 1949). For some, there is an insistence that action research must be collaborative and entail groupwork.

Action research is a form of collective self-reflective enquiry undertaken by participants in social situations in order to improve the rationality and justice of their own social or educational practices, as well as their understanding of those practices and the situations in which the practices are carried out… The approach is only action research when it is collaborative, though it is important to realise that action research of the group is achieved through the critically examined action of individual group members. (Kemmis and McTaggart 1988: 5-6)

Just why it must be collective is open to some question and debate (Webb 1996), but there is an important point here concerning the commitments and orientations of those involved in action research.

One of the legacies Kurt Lewin left us is the ‘action research spiral’ – and with it there is the danger that action research becomes little more than a procedure. It is a mistake, according to McTaggart (1996: 248) to think that following the action research spiral constitutes ‘doing action research’. He continues, ‘Action research is not a ‘method’ or a ‘procedure’ for research but a series of commitments to observe and problematize through practice a series of principles for conducting social enquiry’. It is his argument that Lewin has been misunderstood or, rather, misused. When set in historical context, while Lewin does talk about action research as a method, he is stressing a contrast between this form of interpretative practice and more traditional empirical-analytic research. The notion of a spiral may be a useful teaching device – but it is all too easy to slip into using it as the template for practice (McTaggart 1996: 249).

Further reading

This select, annotated bibliography has been designed to give a flavour of the possibilities of action research and includes some useful guides to practice. As ever, if you have suggestions about areas or specific texts for inclusion, I’d like to hear from you.

Explorations of action research

Atweh, B., Kemmis, S. and Weeks, P. (eds.) (1998) Action Research in Practice: Partnership for Social Justice in Education, London: Routledge. Presents a collection of stories from action research projects in schools and a university. The book begins with theme chapters discussing action research, social justice and partnerships in research. The case study chapters cover topics such as: school environment – how to make a school a healthier place to be; parents – how to involve them more in decision-making; students as action researchers; gender – how to promote gender equity in schools; writing up action research projects.

Carr, W. and Kemmis, S. (1986) Becoming Critical. Education, knowledge and action research , Lewes: Falmer. Influential book that provides a good account of ‘action research’ in education. Chapters on teachers, researchers and curriculum; the natural scientific view of educational theory and practice; the interpretative view of educational theory and practice; theory and practice – redefining the problem; a critical approach to theory and practice; towards a critical educational science; action research as critical education science; educational research, educational reform and the role of the profession.

Carson, T. R. and Sumara, D. J. (ed.) (1997) Action Research as a Living Practice , New York: Peter Lang. 140 pages. Book draws on a wide range of sources to develop an understanding of action research. Explores action research as a lived practice, ‘that asks the researcher to not only investigate the subject at hand but, as well, to provide some account of the way in which the investigation both shapes and is shaped by the investigator.

Dadds, M. (1995) Passionate Enquiry and School Development. A story about action research , London: Falmer. 192 + ix pages. Examines three action research studies undertaken by a teacher and how they related to work in school – how she did the research, the problems she experienced, her feelings, the impact on her feelings and ideas, and some of the outcomes. In his introduction, John Elliot comments that the book is ‘the most readable, thoughtful, and detailed study of the potential of action-research in professional education that I have read’.

Ghaye, T. and Wakefield, P. (eds.) CARN Critical Conversations. Book one: the role of the self in action , Bournemouth: Hyde Publications. 146 + xiii pages. Collection of five pieces from the Classroom Action Research Network. Chapters on: dialectical forms; graduate medical education – research’s outer limits; democratic education; managing action research; writing up.

McNiff, J. (1993) Teaching as Learning: An Action Research Approach , London: Routledge. Argues that educational knowledge is created by individual teachers as they attempt to express their own values in their professional lives. Sets out familiar action research model: identifying a problem, devising, implementing and evaluating a solution and modifying practice. Includes advice on how working in this way can aid the professional development of action researcher and practitioner.

Quigley, B. A. and Kuhne, G. W. (eds.) (1997) Creating Practical Knowledge Through Action Research, San Fransisco: Jossey Bass. Guide to action research that outlines the action research process, provides a project planner, and presents examples to show how action research can yield improvements in six different settings, including a hospital, a university and a literacy education program.

Plummer, G. and Edwards, G. (eds.) CARN Critical Conversations. Book two: dimensions of action research – people, practice and power , Bournemouth: Hyde Publications. 142 + xvii pages. Collection of five pieces from the Classroom Action Research Network. Chapters on: exchanging letters and collaborative research; diary writing; personal and professional learning – on teaching and self-knowledge; anti-racist approaches; psychodynamic group theory in action research.

Whyte, W. F. (ed.) (1991) Participatory Action Research , Newbury Park: Sage. 247 pages. Chapters explore the development of participatory action research and its relation with action science and examine its usages in various agricultural and industrial settings

Zuber-Skerritt, O. (ed.) (1996) New Directions in Action Research , London; Falmer Press. 266 + xii pages. A useful collection that explores principles and procedures for critical action research; problems and suggested solutions; and postmodernism and critical action research.

Action research guides

Coghlan, D. and Brannick, D. (2000) Doing Action Research in your own Organization, London: Sage. 128 pages. Popular introduction. Part one covers the basics of action research including the action research cycle, the role of the ‘insider’ action researcher and the complexities of undertaking action research within your own organisation. Part two looks at the implementation of the action research project (including managing internal politics and the ethics and politics of action research). New edition due late 2004.

Elliot, J. (1991) Action Research for Educational Change , Buckingham: Open University Press. 163 + x pages Collection of various articles written by Elliot in which he develops his own particular interpretation of action research as a form of teacher professional development. In some ways close to a form of ‘reflective practice’. Chapter 6, ‘A practical guide to action research’ – builds a staged model on Lewin’s work and on developments by writers such as Kemmis.

Johnson, A. P. (2007) A short guide to action research 3e. Allyn and Bacon. Popular step by step guide for master’s work.

Macintyre, C. (2002) The Art of the Action Research in the Classroom , London: David Fulton. 138 pages. Includes sections on action research, the role of literature, formulating a research question, gathering data, analysing data and writing a dissertation. Useful and readable guide for students.

McNiff, J., Whitehead, J., Lomax, P. (2003) You and Your Action Research Project , London: Routledge. Practical guidance on doing an action research project.Takes the practitioner-researcher through the various stages of a project. Each section of the book is supported by case studies

Stringer, E. T. (2007) Action Research: A handbook for practitioners 3e , Newbury Park, ca.: Sage. 304 pages. Sets community-based action research in context and develops a model. Chapters on information gathering, interpretation, resolving issues; legitimacy etc. See, also Stringer’s (2003) Action Research in Education , Prentice-Hall.

Winter, R. (1989) Learning From Experience. Principles and practice in action research , Lewes: Falmer Press. 200 + 10 pages. Introduces the idea of action research; the basic process; theoretical issues; and provides six principles for the conduct of action research. Includes examples of action research. Further chapters on from principles to practice; the learner’s experience; and research topics and personal interests.

Action research in informal education

Usher, R., Bryant, I. and Johnston, R. (1997) Adult Education and the Postmodern Challenge. Learning beyond the limits , London: Routledge. 248 + xvi pages. Has some interesting chapters that relate to action research: on reflective practice; changing paradigms and traditions of research; new approaches to research; writing and learning about research.

Other references

Bogdan, R. and Biklen, S. K. (1992) Qualitative Research For Education , Boston: Allyn and Bacon.

Goetschius, G. and Tash, J. (1967) Working with the Unattached , London: Routledge and Kegan Paul.

McTaggart, R. (1996) ‘Issues for participatory action researchers’ in O. Zuber-Skerritt (ed.) New Directions in Action Research , London: Falmer Press.

McNiff, J., Lomax, P. and Whitehead, J. (2003) You and Your Action Research Project 2e. London: Routledge.

Thomas, G. (2017). How to do your Research Project. A guide for students in education and applied social sciences . 3e. London: Sage.

Acknowledgements : spiral by Michèle C. | flickr ccbyncnd2 licence

How to cite this article : Smith, M. K. (1996; 2001, 2007, 2017) What is action research and how do we do it?’, The encyclopedia of pedagogy and informal education. [ https://infed.org/mobi/action-research/ . Retrieved: insert date] .

© Mark K. Smith 1996; 2001, 2007, 2017

Our websites may use cookies to personalize and enhance your experience. By continuing without changing your cookie settings, you agree to this collection. For more information, please see our University Websites Privacy Notice .

Neag School of Education

Educational Research Basics by Del Siegle

Action research.

An Introduction to Action Research Jeanne H. Purcell, Ph.D.

 Your Options

• Review Related Literature
• Examine the Impact of an Experimental Treatment
• Monitor Change
• Identify Present Practices
• Describe Beliefs and Attitudes

Action Research Is…

• Action research is a three-step spiral process of (1) planning which involves fact-finding, (2) taking action, and (3) fact-finding about the results of the action. (Lewin, 1947)
• Action research is a process by which practitioners attempt to study their problems scientifically in order to guide, correct, and evaluate their decisions and action. (Corey, 1953).
• Action research in education is study conducted by colleagues in a school setting of the results of their activities to improve instruction. (Glickman, 1990)
• Action research is a fancy way of saying Let’s study what s happening at our school and decide how to make it a better place. (Calhoun,1994)

Conditions That Support Action Research

• A faculty where a majority of teachers wish to improve some aspect (s) of education in their school.
• Common agreement about how collective decisions will be made and implemented.
• A team that is willing to lead the initiative.
• Study groups that meet regularly.
• A basic knowledge of the action research cycle and the rationale for its use.
• Someone to provide technical assistance and/or support.

The Action Research Cycle

• Identify an area of interest/problem.
• Identify data to be collected, the format for the results, and a timeline.
• Collect and organize the data.
• Analyze and interpret the data.
• Decide upon the action to be taken.
• Evaluate the success of the action.

Collecting Data: Sources

Existing Sources

• Attendance at PTO meetings
• + and – parent communications
• Office referrals
• Special program enrollment
• Standardized scores

Inventive Sources

• Interviews with parents
• Library use, by grade, class
• Minutes of meetings
• Nature and amount of in-school assistance related to the innovation
• Number of books read
• Observation journals
• Record of peer observations
• Student journals
• Teacher journals
• Videotapes of students: whole class instruction
• Videotapes of students: Differentiated instruction
• Writing samples

Collecting Data: From Whom?

• From everyone when we are concerned about each student’s performance.
• From a sample when we need to increase our understanding while limiting our expenditure of time and energy; more in-depth interviews or observations may follow.

Collecting Data: How Often?

• At regular intervals
• At critical points

Collecting Data: Guidelines

• Use both existing and inventive data sources.
• Use multiple data sources.
• Collect data regularly.
• Seek help, if necessary.

Organizing Data

• Keep it simple.
• Disaggregate numbers from interviews and other qualitative types of data.
• Plan plenty of time to look over and organize the data.
• Seek technical assistance if needed.

Analyzing Data

• What important points do they data reveal?
• What patterns/trends do you note? What might be some possible explanations?
• Do the data vary by sources? Why might the variations exist?
• Are there any results that are different from what you expected? What might be some hypotheses to explain the difference (s)?
• What actions appear to be indicated?

Taking Action

• Do the data warrant action?
• What might se some short-term actions?
• What might be some long-term actions?
• How will we know if our actions have been effective?
• What benchmarks might we expect to see along the way to effectiveness ?

Action Plans

• Target date
• Responsibility
• Evidence of Effectiveness

Action Research Handout

Bibliography

Brubacher, J. W., Case, C. W., & Reagan, T. G. (1994). Becoming a reflective educator . Thousand Oaks: CA: Corwin Press.

Burnaford, G., Fischer, J., & Hobson, D. (1996). Teachers doing research . Mahwah, NJ: Lawrence Erlbaum.

Calhoun, Emily (1994). How to use action research in the self-renewing school . Alexandria, VA: ASCD.

Corey, S. M. (1953). Action research to improve school practices . New York: Teachers College Press.

Glickman, C. D. (1990). Supervision of instruction: A developmental approach . Boston: Allyn and Bacon.

Hubbard, R. S. & Power, B. M. (1993). The art of classroom inquiry . Portsmouth, NH: Heineman.

Lewin, K. (1947). Group decisions and social change. In Readings in social psychology . (Eds. T M. Newcomb and E. L. Hartley). New York: Henry Holt.

• Our Mission

How Teachers Can Learn Through Action Research

A look at one school’s action research project provides a blueprint for using this model of collaborative teacher learning.

When teachers redesign learning experiences to make school more relevant to students’ lives, they can’t ignore assessment. For many teachers, the most vexing question about real-world learning experiences such as project-based learning is: How will we know what students know and can do by the end of this project?

Teachers at the Siena School in Silver Spring, Maryland, decided to figure out the assessment question by investigating their classroom practices. As a result of their action research, they now have a much deeper understanding of authentic assessment and a renewed appreciation for the power of learning together.

Their research process offers a replicable model for other schools interested in designing their own immersive professional learning. The process began with a real-world challenge and an open-ended question, involved a deep dive into research, and ended with a public showcase of findings.

Start With an Authentic Need to Know

Siena School serves about 130 students in grades 4–12 who have mild to moderate language-based learning differences, including dyslexia. Most students are one to three grade levels behind in reading.

Teachers have introduced a variety of instructional strategies, including project-based learning, to better meet students’ learning needs and also help them develop skills like collaboration and creativity. Instead of taking tests and quizzes, students demonstrate what they know in a PBL unit by making products or generating solutions.

“We were already teaching this way,” explained Simon Kanter, Siena’s director of technology. “We needed a way to measure, was authentic assessment actually effective? Does it provide meaningful feedback? Can teachers grade it fairly?”

Focus the Research Question

Across grade levels and departments, teachers considered what they wanted to learn about authentic assessment, which the late Grant Wiggins described as engaging, multisensory, feedback-oriented, and grounded in real-world tasks. That’s a contrast to traditional tests and quizzes, which tend to focus on recall rather than application and have little in common with how experts go about their work in disciplines like math or history.

The teachers generated a big research question: Is using authentic assessment an effective and engaging way to provide meaningful feedback for teachers and students about growth and proficiency in a variety of learning objectives, including 21st-century skills?

Take Time to Plan

Next, teachers planned authentic assessments that would generate data for their study. For example, middle school science students created prototypes of genetically modified seeds and pitched their designs to a panel of potential investors. They had to not only understand the science of germination but also apply their knowledge and defend their thinking.

In other classes, teachers planned everything from mock trials to environmental stewardship projects to assess student learning and skill development. A shared rubric helped the teachers plan high-quality assessments.

Make Sense of Data

During the data-gathering phase, students were surveyed after each project about the value of authentic assessments versus more traditional tools like tests and quizzes. Teachers also reflected after each assessment.

“We collated the data, looked for trends, and presented them back to the faculty,” Kanter said.

Among the takeaways:

• Authentic assessment generates more meaningful feedback and more opportunities for students to apply it.
• Students consider authentic assessment more engaging, with increased opportunities to be creative, make choices, and collaborate.
• Teachers are thinking more critically about creating assessments that allow for differentiation and that are applicable to students’ everyday lives.

To make their learning public, Siena hosted a colloquium on authentic assessment for other schools in the region. The school also submitted its research as part of an accreditation process with the Middle States Association.

Strategies to Share

For other schools interested in conducting action research, Kanter highlighted three key strategies.

• Focus on areas of growth, not deficiency:  “This would have been less successful if we had said, ‘Our math scores are down. We need a new program to get scores up,’ Kanter said. “That puts the onus on teachers. Data collection could seem punitive. Instead, we focused on the way we already teach and thought about, how can we get more accurate feedback about how students are doing?”
• Foster a culture of inquiry:  Encourage teachers to ask questions, conduct individual research, and share what they learn with colleagues. “Sometimes, one person attends a summer workshop and then shares the highlights in a short presentation. That might just be a conversation, or it might be the start of a school-wide initiative,” Kanter explained. In fact, that’s exactly how the focus on authentic assessment began.
• Build structures for teacher collaboration:  Using staff meetings for shared planning and problem-solving fosters a collaborative culture. That was already in place when Siena embarked on its action research, along with informal brainstorming to support students.

For both students and staff, the deep dive into authentic assessment yielded “dramatic impact on the classroom,” Kanter added. “That’s the great part of this.”

In the past, he said, most teachers gave traditional final exams. To alleviate students’ test anxiety, teachers would support them with time for content review and strategies for study skills and test-taking.

“This year looks and feels different,” Kanter said. A week before the end of fall term, students were working hard on final products, but they weren’t cramming for exams. Teachers had time to give individual feedback to help students improve their work. “The whole climate feels way better.”

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case AskWhy Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research Research Tools and Apps

Action Research: What it is, Stages & Examples

The best way to get things accomplished is to do it yourself. This statement is utilized in corporations, community projects, and national governments. These organizations are relying on action research to cope with their continuously changing and unstable environments as they function in a more interdependent world.

In practical educational contexts, this involves using systematic inquiry and reflective practice to address real-world challenges, improve teaching and learning, enhance student engagement, and drive positive changes within the educational system.

This post outlines the definition of action research, its stages, and some examples.

Content Index

What is action research?

Stages of action research, the steps to conducting action research, examples of action research, advantages and disadvantages of action research.

Action research is a strategy that tries to find realistic solutions to organizations’ difficulties and issues. It is similar to applied research.

Action research refers basically learning by doing. First, a problem is identified, then some actions are taken to address it, then how well the efforts worked are measured, and if the results are not satisfactory, the steps are applied again.

It can be put into three different groups:

• Positivist: This type of research is also called “classical action research.” It considers research a social experiment. This research is used to test theories in the actual world.
• Interpretive: This kind of research is called “contemporary action research.” It thinks that business reality is socially made, and when doing this research, it focuses on the details of local and organizational factors.
• Critical: This action research cycle takes a critical reflection approach to corporate systems and tries to enhance them.

All research is about learning new things. Collaborative action research contributes knowledge based on investigations in particular and frequently useful circumstances. It starts with identifying a problem. After that, the research process is followed by the below stages:

Stage 1: Plan

For an action research project to go well, the researcher needs to plan it well. After coming up with an educational research topic or question after a research study, the first step is to develop an action plan to guide the research process. The research design aims to address the study’s question. The research strategy outlines what to undertake, when, and how.

Stage 2: Act

The next step is implementing the plan and gathering data. At this point, the researcher must select how to collect and organize research data . The researcher also needs to examine all tools and equipment before collecting data to ensure they are relevant, valid, and comprehensive.

Stage 3: Observe

Data observation is vital to any investigation. The action researcher needs to review the project’s goals and expectations before data observation. This is the final step before drawing conclusions and taking action.

Different kinds of graphs, charts, and networks can be used to represent the data. It assists in making judgments or progressing to the next stage of observing.

Stage 4: Reflect

This step involves applying a prospective solution and observing the results. It’s essential to see if the possible solution found through research can really solve the problem being studied.

The researcher must explore alternative ideas when the action research project’s solutions fail to solve the problem.

Action research is a systematic approach researchers, educators, and practitioners use to identify and address problems or challenges within a specific context. It involves a cyclical process of planning, implementing, reflecting, and adjusting actions based on the data collected. Here are the general steps involved in conducting an action research process:

Identify the action research question or problem

Clearly define the issue or problem you want to address through your research. It should be specific, actionable, and relevant to your working context.

Review existing knowledge

Conduct a literature review to understand what research has already been done on the topic. This will help you gain insights, identify gaps, and inform your research design.

Plan the research

Develop a research plan outlining your study’s objectives, methods, data collection tools, and timeline. Determine the scope of your research and the participants or stakeholders involved.

Collect data

Implement your research plan by collecting relevant data. This can involve various methods such as surveys, interviews, observations, document analysis, or focus groups. Ensure that your data collection methods align with your research objectives and allow you to gather the necessary information.

Analyze the data

Once you have collected the data, analyze it using appropriate qualitative or quantitative techniques. Look for patterns, themes, or trends in the data that can help you understand the problem better.

Reflect on the findings

Reflect on the analyzed data and interpret the results in the context of your research question. Consider the implications and possible solutions that emerge from the data analysis. This reflection phase is crucial for generating insights and understanding the underlying factors contributing to the problem.

Develop an action plan

Based on your analysis and reflection, develop an action plan that outlines the steps you will take to address the identified problem. The plan should be specific, measurable, achievable, relevant, and time-bound (SMART goals). Consider involving relevant stakeholders in planning to ensure their buy-in and support.

Implement the action plan

Put your action plan into practice by implementing the identified strategies or interventions. This may involve making changes to existing practices, introducing new approaches, or testing alternative solutions. Document the implementation process and any modifications made along the way.

Evaluate and monitor progress

Continuously monitor and evaluate the impact of your actions. Collect additional data, assess the effectiveness of the interventions, and measure progress towards your goals. This evaluation will help you determine if your actions have the desired effects and inform any necessary adjustments.

Reflect and iterate

Reflect on the outcomes of your actions and the evaluation results. Consider what worked well, what did not, and why. Use this information to refine your approach, make necessary adjustments, and plan for the next cycle of action research if needed.

Remember that participatory action research is an iterative process, and multiple cycles may be required to achieve significant improvements or solutions to the identified problem. Each cycle builds on the insights gained from the previous one, fostering continuous learning and improvement.

Explore Insightfully Contextual Inquiry in Qualitative Research

Here are two real-life examples of action research.

Action research initiatives are frequently situation-specific. Still, other researchers can adapt the techniques. The example is from a researcher’s (Franklin, 1994) report about a project encouraging nature tourism in the Caribbean.

In 1991, this was launched to study how nature tourism may be implemented on the four Windward Islands in the Caribbean: St. Lucia, Grenada, Dominica, and St. Vincent.

For environmental protection, a government-led action study determined that the consultation process needs to involve numerous stakeholders, including commercial enterprises.

First, two researchers undertook the study and held search conferences on each island. The search conferences resulted in suggestions and action plans for local community nature tourism sub-projects.

Several islands formed advisory groups and launched national awareness and community projects. Regional project meetings were held to discuss experiences, self-evaluations, and strategies. Creating a documentary about a local initiative helped build community. And the study was a success, leading to a number of changes in the area.

Lau and Hayward (1997) employed action research to analyze Internet-based collaborative work groups.

Over two years, the researchers facilitated three action research problem -solving cycles with 15 teachers, project personnel, and 25 health practitioners from diverse areas. The goal was to see how Internet-based communications might affect their virtual workgroup.

First, expectations were defined, technology was provided, and a bespoke workgroup system was developed. Participants suggested shorter, more dispersed training sessions with project-specific instructions.

The second phase saw the system’s complete deployment. The final cycle witnessed system stability and virtual group formation. The key lesson was that the learning curve was poorly misjudged, with frustrations only marginally met by phone-based technical help. According to the researchers, the absence of high-quality online material about community healthcare was harmful.

Role clarity, connection building, knowledge sharing, resource assistance, and experiential learning are vital for virtual group growth. More study is required on how group support systems might assist groups in engaging with their external environment and boost group members’ learning. 

Action research has both good and bad points.

• It is very flexible, so researchers can change their analyses to fit their needs and make individual changes.
• It offers a quick and easy way to solve problems that have been going on for a long time instead of complicated, long-term solutions based on complex facts.
• If It is done right, it can be very powerful because it can lead to social change and give people the tools to make that change in ways that are important to their communities.

Disadvantages

• These studies have a hard time being generalized and are hard to repeat because they are so flexible. Because the researcher has the power to draw conclusions, they are often not thought to be theoretically sound.
• Setting up an action study in an ethical way can be hard. People may feel like they have to take part or take part in a certain way.
• It is prone to research errors like selection bias , social desirability bias, and other cognitive biases.

LEARN ABOUT: Self-Selection Bias

This post discusses how action research generates knowledge, its steps, and real-life examples. It is very applicable to the field of research and has a high level of relevance. We can only state that the purpose of this research is to comprehend an issue and find a solution to it.

At QuestionPro, we give researchers tools for collecting data, like our survey software, and a library of insights for any long-term study. Go to the Insight Hub if you want to see a demo or learn more about it.

LEARN MORE         FREE TRIAL

Frequently Asked Questions(FAQ’s)

Action research is a systematic approach to inquiry that involves identifying a problem or challenge in a practical context, implementing interventions or changes, collecting and analyzing data, and using the findings to inform decision-making and drive positive change.

Action research can be conducted by various individuals or groups, including teachers, administrators, researchers, and educational practitioners. It is often carried out by those directly involved in the educational setting where the research takes place.

The steps of action research typically include identifying a problem, reviewing relevant literature, designing interventions or changes, collecting and analyzing data, reflecting on findings, and implementing improvements based on the results.

MORE LIKE THIS

Net Trust Score: Tool for Measuring Trust in Organization

Sep 2, 2024

Why You Should Attend XDAY 2024

Aug 30, 2024

Alchemer vs Qualtrics: Find out which one you should choose

Target Population: What It Is + Strategies for Targeting

Aug 29, 2024

Other categories

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Tuesday CX Thoughts (TCXT)
• Uncategorized
• What’s Coming Up
• Workforce Intelligence

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

11.3 Action research

Learning objectives.

• Define and provide at least one example of action research
• Describe the role of stakeholders in action research

Action research is defined as research that is conducted for the purpose of creating social change. When conducting action research, scholars collaborate with community stakeholders at all stages of the research process with the aim of producing results that will be usable in the community and by scientists. Stakeholders are individuals or groups who have an interest in the outcome of your study. Social workers who engage in action research never just go it alone; instead, they collaborate with the people who are affected by the research at each stage in the process. In action research, stakeholders, particularly those with the least power, are consulted on the purpose of the research project, research questions, design, and reporting of results.

Action research also distinguishes itself from other research in that its purpose is to create change on an individual and community level. Kristin Esterberg (2002) puts it quite eloquently when she says, “At heart, all action researchers are concerned that research not simply contribute to knowledge but also lead to positive changes in people’s lives” (p. 137).  As you might imagine, action research is consistent with the assumptions of the critical paradigm, which focuses on liberating people from oppressive structures. Action research has multiple origins across the globe, including Kurt Lewin’s psychological experiments in the United States and Paulo Friere’s literacy and education programs (Adelman, 1993; Reason, 1994). Over the years, action research has become increasingly popular among scholars who wish for their work to have tangible outcomes that benefit the groups they study.

Action research does not bring any new methodological tricks or terms, but it uses the processes of science in a different way from traditional research. What topics are important to study in a neighborhood or with a target population? A traditional scientist might look at the literature or use their practice wisdom to formulate a research question. An action researcher, on the other hand, would consult with the target population itself to see what they thought were the most pressing issues and best solutions. In this way, action research flips traditional research on its head. Scientists are more like consultants who provide the tools and resources necessary for a target population to achieve their goals and address social problems.

According to Healy (2001), the assumptions of participatory-action research are that (a) oppression is caused by macro-level structures such as patriarchy and capitalism; (b) research should expose and confront the powerful; (c) researcher and participant relationships should be equal, with equitable distribution of research tasks and roles; and (d) research should result in consciousness-raising and collective action. Coherent with social work values, action research supports the self-determination of oppressed groups and privileges their voice and understanding through the conceptualization, design, data collection, data analysis, and dissemination processes of research.

There are many excellent examples of action research. Some of them focus solely on arriving at useful outcomes for the communities upon which and with whom research is conducted. Other action research projects result in some new knowledge that has a practical application and purpose in addition to the creation of knowledge for basic scientific purposes.

One example of action research can be seen in Fred Piercy and colleagues’ (Piercy, Franz, Donaldson, & Richard, 2011) work with farmers in Virginia, Tennessee, and Louisiana. Together with farmers in these states, the researchers conducted focus groups to understand how farmers learn new information about farming. Ultimately, the aim of this study was to “develop more meaningful ways to communicate information to farmers about sustainable agriculture” (p. 820). This improved communication, the researchers and farmers believed, would benefit not just researchers interested in the topic but also farmers and their communities. Farmers and researchers were both involved in all aspects of the research, from designing the project and determining focus group questions to conducting the focus groups and finally to analyzing data and disseminating findings.

Perhaps one of the most unique and rewarding aspects of action research is that it is often interdisciplinary. Action research projects might bring together researchers from any number of disciplines, from the social sciences, such as sociology, political science, and psychology; to an assortment of physical and natural sciences, such as biology and chemistry; to engineering, philosophy, and history (to name just a few).

Anyone interested in social change can benefit from having some understanding of social scientific research methods. The knowledge you’ve gained from your methods course can be put to good use even if you don’t have an interest in pursuing a career in research. As a member of a community, perhaps you will find that the opportunity to engage in action research presents itself to you one day. Your background in research methodology will no doubt assist you in making life better for yourself and those who share your interests, circumstances, or geographic region.

Spotlight on UTA School of Social Work

Dr. maxine davis shares experiences with action research.

There are various types of action research. Although the degree to which stakeholders are involved may vary across different stages of the research and dissemination process, each type is valuable and aims to accomplish shared decision-making, responsibility, and power between the researcher and the researched. I will share with you a few examples of recent research that I have had the pleasure of being involved in.

Case 1 (St. Louis, MO) Community based participatory research (CBPR)

As a community organizer, activist, and Missionary, Ms. Johnson is well connected to her community in North St. Louis city. She has worked in partnership with a number of clergy members throughout St. Louis on improving the overall well-being of African-Americans for a number of years. From education to political engagement, she has her pulse on the many issues of local residents and a wide network of clergy and ministers who trust her. In 2014, I partnered with Ms. Johnson to explore clergy perceptions on religious or spiritual (R/S) related abuse within intimate partner violence (IPV). Ms. Johnson conducted more than half of the interviews (many of which occurred only because of the trust clergy members had with her, not due to my recruitment efforts). We coded the data independently and analyzed it as a team. As a result, Ms. Johnson gained the skills to conduct basic qualitative data analysis that may be applicable to her other work. The study results revealed that R/S abuse in IPV was a serious issue that Black clergy often faced in ministry. Furthermore, they desired training to help them to better prepare in responding to this problem. The project did not end at manuscript development, rather the efforts to address this issue continue as we develop and plan to implement R/S specific IPV training for Black clergy in St. Louis.

Case 2 (Chicago, IL) Community-engaged research using a Community Collaborative/Advisory Board (CCB)

A colleague who knew of my interest in the intersection of religious faith and IPV connected me with a priest at St. Pius V parish who was looking for someone to evaluate a portion of the church’s domestic violence program.  The project combined evaluation research and action research. I sought and obtained funding tosupport the first step of a multi-phase project involving process evaluation in preparation for a longitudinal impact (i.e. outcome) evaluation. I convened a collaborative board of relevant stakeholders from different organizations and relocated to Chicago (Pilsen neighborhood) to embark upon the research. Over the course of one year, I lived in the community and collected various types of data from a variety of sources while the CCB and I developed an evaluation plan that would meet the organization’s needs. The primary research questions explored were: “What is The Men’s Group (TMG)?” and “Why do participants attend and remain engaged in TMG?” We discovered that TMG was a trauma-informed, culturally-tailored (to Latino men), spirituality and group based partner abuse intervention program (PAIP) aiming to stop violence perpetration and help participants become self-aware. Men remained engaged in the PAIP because they were met with respect by staff/facilitators, reported gaining benefits because of participation, and connected with other group members through a brotherhood. A quasi-experimental design using quantitative data is currently underway.

Case 3 (Grand Prairie, TX) Youth-led CBPR

The Grand Prairie Storm Track & Field Association (GP Storm) reached out to me after their founders saw me present on the potential of hip-hop music influencing public perceptions about IPV. Our shared interest on increasing Black/African-American representation in health-related research careers brought us together. I invited high school students who were affiliated with the program to join me in examining this area, but also encouraged them to develop a set of their own research questions that they were excited to explore. We met weekly over the course of 3 months in the summer of 2019 and analyzed the lyrics of 7 hip-hop songs. The youth-led research team consisted of six Black/Multiracial young women (5 high school; 1 middle school), the organization founder/director, a PhD student, and myself. The findings revealed that hip-hop music brings awareness to IPV/A by discussing Death, Denial, Freedom, and Physical violence/various types of consequences. Partnering with the GP Storm and affiliated students (the community researchers) allowed the research team to examine research questions that were of interest to a wider audience and do so by drawing on multiple perspectives, thereby improving the rigor of the study. The research did not end here; rather next steps involve hosting a listening party as an intervention to reduce violence and acceptability thereof amongst youth and adults.

Lessons learned

I have learned a few lessons through conducting community-engaged research that I think are worth sharing. It is imperative that you are comfortable openly discussing race and diversity if you plan on engaging in action research with communities of color. This applies, regardless of your own identity, but is especially relevant for those who are an “outsider” in terms of gender or race/ethnicity. The second lesson is that trust need not be earned once, rather you must continuously build and maintain trust in order to conduct sound research. You must also plan to nurture and intend to maintain these relationships in a humanistic manner, beyond that of “a research product.” If your intentions are genuine and you are honest with any trepidations, that plus meaningful project delivery will carry you far.

Refer to following articles for more exploration into this research:

Davis, M., ^Johnson, M., Bowland, S. (In Draft) “I hate it…but it’s real”: Black Clergy Perspectives on Intimate Partner Violence related Religious/Spiritual Abuse

Davis, M., ^Dahm, C., Jonson-Reid, M., Stoops, C., Sabri, B. (Revisions Submitted-Awaiting Final Decision). “The Men’s Group” at St. Pius V: A Case Study of a Parish-Based Voluntary Partner Abuse Intervention Program.

^denotes community partners

Key Takeaways

• Action research is conducted by researchers who wish to create some form of social change.
• Stakeholders are true collaborators in action research.
• Action research is often conducted by teams of interdisciplinary researchers.
• Action research- research that is conducted for the purpose of creating some form of social change in collaboration with stakeholders
• Stakeholders – individuals or groups who have an interest in the outcome of your study

Image attributions

protest by BruceEmmerling CC-0

Maxine Johnson and Maxine Davis by Maxine Davis CC BY-NC-ND

Community Researchers in Partnership by Maxine Davis CC BY-NC-ND

GP Storm by Maxine Davis CC BY-NC-ND

Foundations of Social Work Research Copyright © 2020 by Rebecca L. Mauldin is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Reflections on 'reflection' in Action Research

• Conference: Uncertainty is a Great Opportunity.15th European Academy of Management Conference (EURAM) 17-20 June 2015 ,Warsaw

• Atlantic Technological University

• University of Galway

• National University of Ireland, Maynooth

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• Pauline Anne Logue

• Arielle Leitner

• Barbara Bell Angus

• T. Honderich
• I. L. P. Masters
• MANAGE LEARN

• Oscar Hauptman

• Alfred Schutz

• Walter Cohen
• R. O. Elyeton

• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Dissertation
• How to Write a Results Section | Tips & Examples

How to Write a Results Section | Tips & Examples

Published on August 30, 2022 by Tegan George . Revised on July 18, 2023.

A results section is where you report the main findings of the data collection and analysis you conducted for your thesis or dissertation . You should report all relevant results concisely and objectively, in a logical order. Don’t include subjective interpretations of why you found these results or what they mean—any evaluation should be saved for the discussion section .

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

How to write a results section, reporting quantitative research results, reporting qualitative research results, results vs. discussion vs. conclusion, checklist: research results, other interesting articles, frequently asked questions about results sections.

When conducting research, it’s important to report the results of your study prior to discussing your interpretations of it. This gives your reader a clear idea of exactly what you found and keeps the data itself separate from your subjective analysis.

Here are a few best practices:

• Your results should always be written in the past tense.
• While the length of this section depends on how much data you collected and analyzed, it should be written as concisely as possible.
• Only include results that are directly relevant to answering your research questions . Avoid speculative or interpretative words like “appears” or “implies.”
• If you have other results you’d like to include, consider adding them to an appendix or footnotes.
• Always start out with your broadest results first, and then flow into your more granular (but still relevant) ones. Think of it like a shoe store: first discuss the shoes as a whole, then the sneakers, boots, sandals, etc.

Don't submit your assignments before you do this

The academic proofreading tool has been trained on 1000s of academic texts. Making it the most accurate and reliable proofreading tool for students. Free citation check included.

Try for free

If you conducted quantitative research , you’ll likely be working with the results of some sort of statistical analysis .

Your results section should report the results of any statistical tests you used to compare groups or assess relationships between variables . It should also state whether or not each hypothesis was supported.

The most logical way to structure quantitative results is to frame them around your research questions or hypotheses. For each question or hypothesis, share:

• A reminder of the type of analysis you used (e.g., a two-sample t test or simple linear regression ). A more detailed description of your analysis should go in your methodology section.
• A concise summary of each relevant result, both positive and negative. This can include any relevant descriptive statistics (e.g., means and standard deviations ) as well as inferential statistics (e.g., t scores, degrees of freedom , and p values ). Remember, these numbers are often placed in parentheses.
• A brief statement of how each result relates to the question, or whether the hypothesis was supported. You can briefly mention any results that didn’t fit with your expectations and assumptions, but save any speculation on their meaning or consequences for your discussion  and conclusion.

A note on tables and figures

In quantitative research, it’s often helpful to include visual elements such as graphs, charts, and tables , but only if they are directly relevant to your results. Give these elements clear, descriptive titles and labels so that your reader can easily understand what is being shown. If you want to include any other visual elements that are more tangential in nature, consider adding a figure and table list .

As a rule of thumb:

• Tables are used to communicate exact values, giving a concise overview of various results
• Graphs and charts are used to visualize trends and relationships, giving an at-a-glance illustration of key findings

Don’t forget to also mention any tables and figures you used within the text of your results section. Summarize or elaborate on specific aspects you think your reader should know about rather than merely restating the same numbers already shown.

A two-sample t test was used to test the hypothesis that higher social distance from environmental problems would reduce the intent to donate to environmental organizations, with donation intention (recorded as a score from 1 to 10) as the outcome variable and social distance (categorized as either a low or high level of social distance) as the predictor variable.Social distance was found to be positively correlated with donation intention, t (98) = 12.19, p < .001, with the donation intention of the high social distance group 0.28 points higher, on average, than the low social distance group (see figure 1). This contradicts the initial hypothesis that social distance would decrease donation intention, and in fact suggests a small effect in the opposite direction.

Figure 1: Intention to donate to environmental organizations based on social distance from impact of environmental damage.

In qualitative research , your results might not all be directly related to specific hypotheses. In this case, you can structure your results section around key themes or topics that emerged from your analysis of the data.

For each theme, start with general observations about what the data showed. You can mention:

• Recurring points of agreement or disagreement
• Patterns and trends
• Particularly significant snippets from individual responses

Next, clarify and support these points with direct quotations. Be sure to report any relevant demographic information about participants. Further information (such as full transcripts , if appropriate) can be included in an appendix .

When asked about video games as a form of art, the respondents tended to believe that video games themselves are not an art form, but agreed that creativity is involved in their production. The criteria used to identify artistic video games included design, story, music, and creative teams.One respondent (male, 24) noted a difference in creativity between popular video game genres:

“I think that in role-playing games, there’s more attention to character design, to world design, because the whole story is important and more attention is paid to certain game elements […] so that perhaps you do need bigger teams of creative experts than in an average shooter or something.”

Responses suggest that video game consumers consider some types of games to have more artistic potential than others.

Your results section should objectively report your findings, presenting only brief observations in relation to each question, hypothesis, or theme.

It should not  speculate about the meaning of the results or attempt to answer your main research question . Detailed interpretation of your results is more suitable for your discussion section , while synthesis of your results into an overall answer to your main research question is best left for your conclusion .

I have completed my data collection and analyzed the results.

I have included all results that are relevant to my research questions.

I have concisely and objectively reported each result, including relevant descriptive statistics and inferential statistics .

I have stated whether each hypothesis was supported or refuted.

I have used tables and figures to illustrate my results where appropriate.

All tables and figures are correctly labelled and referred to in the text.

There is no subjective interpretation or speculation on the meaning of the results.

You've finished writing up your results! Use the other checklists to further improve your thesis.

If you want to know more about AI for academic writing, AI tools, or research bias, make sure to check out some of our other articles with explanations and examples or go directly to our tools!

Research bias

• Survivorship bias
• Self-serving bias
• Availability heuristic
• Halo effect
• Hindsight bias
• Deep learning
• Generative AI
• Machine learning
• Reinforcement learning
• Supervised vs. unsupervised learning

 (AI) Tools

• Grammar Checker
• Paraphrasing Tool
• Text Summarizer
• AI Detector
• Plagiarism Checker
• Citation Generator

The results chapter of a thesis or dissertation presents your research results concisely and objectively.

In quantitative research , for each question or hypothesis , state:

• The type of analysis used
• Relevant results in the form of descriptive and inferential statistics
• Whether or not the alternative hypothesis was supported

In qualitative research , for each question or theme, describe:

• Recurring patterns
• Significant or representative individual responses
• Relevant quotations from the data

Don’t interpret or speculate in the results chapter.

Results are usually written in the past tense , because they are describing the outcome of completed actions.

The results chapter or section simply and objectively reports what you found, without speculating on why you found these results. The discussion interprets the meaning of the results, puts them in context, and explains why they matter.

In qualitative research , results and discussion are sometimes combined. But in quantitative research , it’s considered important to separate the objective results from your interpretation of them.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

George, T. (2023, July 18). How to Write a Results Section | Tips & Examples. Scribbr. Retrieved September 3, 2024, from https://www.scribbr.com/dissertation/results/

Is this article helpful?

Tegan George

Other students also liked, what is a research methodology | steps & tips, how to write a discussion section | tips & examples, how to write a thesis or dissertation conclusion, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• Open access
• Published: 28 August 2024

The design, implementation, and evaluation of a blended (in-person and virtual) Clinical Competency Examination for final-year nursing students

• Rita Mojtahedzadeh 1 ,
• Tahereh Toulabi 2 , 3 &
• Aeen Mohammadi 1  

BMC Medical Education volume  24 , Article number:  936 ( 2024 ) Cite this article

7 Altmetric

Metrics details

Introduction

Studies have reported different results of evaluation methods of clinical competency tests. Therefore, this study aimed to design, implement, and evaluate a blended (in-person and virtual) Competency Examination for final-year Nursing Students.

This interventional study was conducted in two semesters of 2020–2021 using an educational action research method in the nursing and midwifery faculty. Thirteen faculty members and 84 final-year nursing students were included in the study using a census method. Eight programs and related activities were designed and conducted during the examination process. Students completed the Spielberger Anxiety Inventory before the examination, and both faculty members and students completed the Acceptance and Satisfaction questionnaire.

The results of the analysis of focused group discussions and reflections indicated that the virtual CCE was not capable of adequately assessing clinical skills. Therefore, it was decided that the CCE for final-year nursing students would be conducted using a blended method. The activities required for performing the examination were designed and implemented based on action plans. Anxiety and satisfaction were also evaluated as outcomes of the study. There was no statistically significant difference in overt, covert, and overall anxiety scores between the in-person and virtual sections of the examination ( p  > 0.05). The mean (SD) acceptance and satisfaction scores for students in virtual, in-person, and blended sections were 25.49 (4.73), 27.60 (4.70), and 25.57 (4.97), respectively, out of 30 points, in which there was a significant increase in the in-person section compared to the other sections. ( p  = 0.008). The mean acceptance and satisfaction scores for faculty members were 30.31 (4.47) in the virtual, 29.86 (3.94) in the in-person, and 30.00 (4.16) out of 33 in the blended, and there was no significant difference between the three sections ( p  = 0.864).

Evaluating nursing students’ clinical competency using a blended method was implemented and solved the problem of students’ graduation. Therefore, it is suggested that the blended method be used instead of traditional in-person or entirely virtual exams in epidemics or based on conditions, facilities, and human resources. Also, the use of patient simulation, virtual reality, and the development of necessary virtual and in-person training infrastructure for students is recommended for future research. Furthermore, considering that the acceptance of traditional in-person exams among students is higher, it is necessary to develop virtual teaching strategies.

Peer Review reports

The primary mission of the nursing profession is to educate competent, capable, and qualified nurses with the necessary knowledge and skills to provide quality nursing care to preserve and improve the community’s health [ 1 ]. Clinical education is one of the most essential and fundamental components of nursing education, in which students gain clinical experience by interacting with actual patients and addressing real problems. Therefore, assessing clinical skills is very challenging. The main goal of educational evaluation is to improve, ensure, and enhance the quality of the academic program. In this regard, evaluating learners’ performance is one of the critical and sensitive aspects of the teaching and learning process. It is considered one of the fundamental elements of the educational program [ 2 ]. The study area is educational evaluation.

Various methods are used to evaluate nursing students. The Objective Structured Clinical Examination (OSCE) is a valid and reliable method for assessing clinical competence [ 1 , 2 ]. In the last twenty years, the use of OSCE has increased significantly in evaluating medical and paramedical students to overcome the limitations of traditional practical evaluation systems [ 3 , 4 ]. The advantages of this method include providing rapid feedback, uniformity for all examinees, and providing conditions close to reality. However, the time-consuming nature and the need for a lot of personnel and equipment are some disadvantages of OSCE [ 5 , 6 ]. Additionally, some studies have shown that this method is anxiety-provoking for some students and, due to time constraints, being observed by the evaluator and other factors can cause dissatisfaction among students [ 7 , 8 ].

However, some studies have also reported that this method is not only not associated with high levels of stress among students [ 9 ] but also has higher satisfaction than traditional evaluation methods [ 4 ]. In addition, during the COVID-19 pandemic, problems such as overcrowding and student quarantine during the exam have arisen. Therefore, reducing time and costs, eliminating or reducing the tiring quarantine time, optimizing the exam, utilizing all facilities for simulating the clinical environment, using innovative methods for conducting the exam, reducing stress, increasing satisfaction, and ultimately preventing the transmission of COVID-19 are significant problems that need to be further investigated.

Studies show that using virtual space as an alternative solution is strongly felt [ 10 , 11 , 12 ]. In the fall of 2009, following the outbreak of H1N1, educational classes in the United States were held virtually [ 13 ]. Also, in 2005, during Hurricane Katrina, 27 universities in the Gulf of Texas used emergency virtual education and evaluation [ 14 ].

One of the challenges faced by healthcare providers in Iran, like most countries in the world, especially during the COVID-19 outbreak, was the shortage of nursing staff [ 15 , 16 ]. Also, in evaluating and conducting CCE for final-year students and subsequent job seekers in the Clinical Skills Center, problems such as student overcrowding and the need for quarantine during the implementation of OSCE existed. This problem has been reported not only for us but also in other countries [ 17 ]. The intelligent use of technology can solve many of these problems. Therefore, almost all educational institutions have quickly started changing their policies’ paradigms to introduce online teaching and evaluation methods [ 18 , 19 ].

During the COVID-19 pandemic, for the first time, this exam was held virtually in our school. However, feedback from professors and students and the experiences of researchers have shown that the virtual exam can only partially evaluate clinical and practical skills in some stations, such as basic skills, resuscitation, and pediatrics [ 20 ].

Additionally, using OSCE in skills assessment facilitates the evaluation of psychological-motor knowledge and attitudes and helps identify strengths and weaknesses [ 21 ]. Clinical competency is a combination of theoretical knowledge and clinical skills. Therefore, using an effective blended method focusing on the quality and safety of healthcare that measures students’ clinical skills and theoretical expertise more accurately in both in-person and virtual environments is essential. The participation of students, professors, managers, education and training staff, and the Clinical Skills Center was necessary to achieve this important and inevitable goal. Therefore, the Clinical Competency Examination (CCE) for nursing students in our nursing and midwifery school was held in the form of an educational action research process to design, implement, and evaluate a blended method. Implementing this process during the COVID-19 pandemic, when it was impossible to hold an utterly in-person exam, helped improve the quality of the exam and address its limitations and weaknesses while providing the necessary evaluation for students.

The innovation of this research lies in evaluating the clinical competency of final-year nursing students using a blended method that focuses on clinical and practical aspects. In the searches conducted, only a few studies have been done on virtual exams and simulations, and a similar study using a blended method was not found.

The research investigates the scientific and clinical abilities of nursing students through the clinical competency exam. This exam, traditionally administered in person, is a crucial milestone for final-year nursing students, marking their readiness for graduation. However, the unforeseen circumstances of the COVID-19 pandemic and the resulting restrictions rendered in-person exams impractical in 2020. This necessitated a swift and significant transition to an online format, a decision that has profound implications for the future of nursing education. While the adoption of online assessment was a necessary step to ensure student graduation and address the nursing workforce shortage during the pandemic, it was not without its challenges. The accurate assessment of clinical skills, such as dressing and CPR, proved to be a significant hurdle. This underscored the urgent need for a change in the exam format, prompting a deeper exploration of innovative solutions.

To address these problems, the research was conducted collaboratively with stakeholders, considering the context and necessity for change in exam administration. Employing an Action Research (AR) approach, a blend of online and in-person exam modalities was adopted. Necessary changes were implemented through a cyclic process involving problem identification, program design, implementation, reflection, and continuous evaluation.

The research began by posing the following questions:

What are the problems of conducting the CCE for final-year nursing students during COVID-19?

How can these problems be addressed?

What are the solutions and suggestions from the involved stakeholders?

How can the CCE be designed, implemented, and evaluated?

What is the impact of exam type on student anxiety and satisfaction?

These questions guided the research in exploring the complexities of administering the CCE amidst the COVID-19 pandemic and in devising practical solutions to ensure the validity and reliability of the assessment while meeting stakeholders’ needs.

Materials and methods

Research setting, expert panel members, job analysis, and role delineation.

This action research was conducted at the Nursing and Midwifery School of Lorestan University of Medical Sciences, with a history of approximately 40 years. The school accommodates 500 undergraduate and graduate nursing students across six specialized fields, with 84 students enrolled in their final year of undergraduate studies. Additionally, the school employs 26 full-time faculty members in nursing education departments.

An expert panel was assembled, consisting of faculty members specializing in various areas, including medical-surgical nursing, psychiatric nursing, community health nursing, pediatric nursing, and intensive care nursing. The panel also included educational department managers and the examination department supervisor. Through focused group discussions, the panel identified and examined issues regarding the exam format, and members proposed various solutions. Subsequently, after analyzing the proposed solutions and drawing upon the panel members’ experiences, specific roles for each member were delineated.

Sampling and participant selection

Given the nature of the research, purposive sampling was employed, ensuring that all individuals involved in the design, implementation, and evaluation of the exam participated in this study.

The participants in this study included final-year nursing students, faculty members, clinical skills center experts, the dean of the school, the educational deputy, group managers, and the exam department head. However, in the outcome evaluation phase, 13 faculty members participated in-person and virtually (26 times), and 84 final-year nursing students enrolled in the study using a census method in two semesters of 2020–2021 completed the questionnaires, including 37 females and 47 males. In addition, three male and ten female faculty members participated in this study; of this number, 2 were instructors, and 11 were assistant professors.

Data collection tools

In order to enhance the validity and credibility of the study and thoroughly examine the results, this study utilized a triangulation method consisting of demographic information, focus group discussions, the Spielberger Anxiety Scale questionnaire, and an Acceptance and Satisfaction Questionnaire.

Demographic information

A questionnaire was used to gather demographic information from both students and faculty members. For students, this included age, gender, and place of residence, while for faculty members, it included age, gender, field of study, and employment status.

Focus group discussion

Multiple focused group discussions were conducted with the participation of professors, administrators, experts, and students. These discussions were held through various platforms such as WhatsApp Skype, and in-person meetings while adhering to health protocols. The researcher guided the talks toward the research objectives and raised fundamental questions, such as describing the strengths and weaknesses of the previous exam, determining how to conduct the CCE considering the COVID-19 situation, deciding on virtual and in-person stations, specifying the evaluation checklists for stations, and explaining the weighting and scoring of each station.

Spielberger anxiety scale questionnaire

This study used the Spielberger Anxiety Questionnaire to measure students’ overt and covert anxiety levels. This questionnaire is an internationally standardized tool known as the STAI questionnaire that measures both overt (state) and covert (trait) anxiety [ 22 ]. The state anxiety scale (Form Y-1 of STAI) comprises twenty statements that assess the individual’s feelings at the moment of responding. The trait anxiety scale (Form Y-2 of STAI) also includes twenty statements that measure individuals’ general and typical feelings. The scores of each of the two scales ranged from 20 to 80 in the current study. The reliability coefficient of the test for the apparent and hidden anxiety scales, based on Cronbach’s alpha, was confirmed to be 0.9084 and 0.9025, respectively [ 23 , 24 ]. Furthermore, in the present study, Cronbach’s alpha value for the total anxiety questionnaire, overt anxiety, and covert anxiety scales were 0.935, 0.921, and 0.760, respectively.

Acceptance and satisfaction questionnaire

The Acceptability and Satisfaction Questionnaire for Clinical Competency Test was developed by Farajpour et al. (2012). The student questionnaire consists of ten questions, and the professor questionnaire consists of eleven questions, using a four-point Likert scale. Experts have confirmed the validity of these questionnaires, and their Cronbach’s alpha coefficients have been determined to be 0.85 and 0.87 for the professor and student questionnaires, respectively [ 6 ]. In the current study, ten medical education experts also confirmed the validity of the questionnaires. Regarding internal reliability, Cronbach’s alpha coefficients for the student satisfaction questionnaire for both virtual and in-person sections were 0.76 and 0.87, respectively. The professor satisfaction questionnaires were 0.84 and 0.87, respectively. An online platform was used to collect data for the virtual exam.

Data analysis and rigor of study

Qualitative data analysis was conducted using the method proposed by Graneheim and Lundman. Additionally, the criteria established by Lincoln and Guba (1985) were employed to confirm the rigor and validity of the data, including credibility, transferability, dependability, and confirmability [ 26 ].

In this research, data synthesis was performed by combining the collected data with various tools and methods. The findings of this study were reviewed and confirmed by participants, supervisors, mentors, and experts in qualitative research, reflecting their opinions on the alignment of findings with their experiences and perspectives on clinical competence examinations. Therefore, the member check method was used to validate credibility.

Moreover, efforts were made in this study to provide a comprehensive description of the research steps, create a suitable context for implementation, assess the views of others, and ensure the transferability of the results.

Furthermore, researchers’ interest in identifying and describing problems, reflecting, designing, implementing, and evaluating clinical competence examinations, along with the engagement of stakeholders in these examinations, was ensured by the researchers’ long-term engagement of over 25 years with the environment and stakeholders, seeking their opinions and considering their ideas and views. These factors contributed to ensuring confirmability.

In this research, by reflecting the results to the participants and making revisions by the researchers, problem clarification and solution presentation, design, implementation, and evaluation of operational programs with stakeholder participation and continuous presence were attempted to prevent biases, assumptions, and research hypotheses, and to confirm dependability.

Data analysis was performed using SPSS version 21, and descriptive statistical tests (absolute and relative frequency, mean, and standard deviation) and inferential tests (paired t-test, independent t-test, and analysis of variance) were used. The significance level was set at 0.05. Parametric tests were used based on the normality of the data according to the Kolmogorov-Smirnov statistical test.

Given that conducting the CCE for final-year nursing students required the active participation of managers, faculty members, staff, and students, and to answer the research question “How can the CCE for final-year nursing students be conducted?” and achieve the research objective of “designing, implementing, and evaluating the clinical competency exam,” the action research method was employed.

The present study was conducted based on the Dickens & Watkins model. There are four primary stages (Fig.  1 ) in the cyclical action research process: reflect, plan, act, observe, and then reflect to continue through the cycle [ 27 ].

The cyclical process of action research [ 27 ]

Stage 1: Reflection

Identification of the problem.

According to the educational regulations, final semester nursing students must complete the clinical competency exam. However, due to the COVID-19 pandemic and the critical situation in most provinces, inter-city travel restrictions, and insufficient dormitory space, conducting the CCE in-person was not feasible.

This exam was conducted virtually at our institution. However, based on the reflections from experts, researchers have found that virtual exams can only partially assess clinical and practical skills in certain stations, such as basic skills, resuscitation, and pediatrics. Furthermore, utilizing Objective Structured Clinical Examination (OSCE) in skills assessment facilitates the evaluation of psychomotor skills, knowledge, and attitudes, aiding in identifying strengths and weaknesses.

P3, “Due to the COVID-19 pandemic and the critical situation in most provinces, inter-city travel restrictions, and insufficient dormitory space, conducting the CCE in-person is not feasible.”

Stage 2: Planning

Based on the reflections gathered from the participants, the exam was designed using a blended approach (combining in-person and virtual components) as per the schedule outlined in Fig.  2 . All planned activities for the blended CCE for final-year nursing students were executed over two semesters.

P5, “Taking the exam virtually might seem easier for us and the students, but in my opinion, it’s not realistic. For instance, performing wound dressing or airway management is very practical, and it’s not possible to assess students with a virtual scenario. We need to see them in person.”

P6"I believe it’s better to conduct those activities that are highly practical in person, but for those involving communication skills like report writing, professional ethics, etc., we can opt for virtual assessment.”

Design and implementation of the blended CCE

Stage 3: Act

Cce implementation steps.

The CCE was conducted based on the flowchart in Fig.  3 and the following steps:

Steps for conducting the CCE for final-year nursing students using a blended method

Step 1: Designing the framework for conducting the blended Clinical Competency Examination

The panelists were guided to design the blended exam in focused group sessions and virtual panels based on the ADDIE (Analysis, Design, Development, Implementation, Evaluation) model [ 28 ]. Initially, needs assessment and opinion polling were conducted, followed by the operational planning of the exam, including the design of the blueprint table (Table  1 ), determination of station types (in-person or virtual), designing question stems in the form of scenarios, creating checklists and station procedure guides by expert panel groups based on participant analysis, and the development of exam implementation guidelines with participant input [ 27 ]. The design, execution, and evaluation were as follows:

In-person and virtual meetings with professors were held to determine the exam schedule, deadlines for submitting checklists, decision-making regarding the virtual or in-person nature of stations based on the type of skill (practical, communication), and presenting problems and solutions. Based on the decisions, primary skill stations, as well as cardiac and pediatric resuscitation stations, were held in person. In contrast, virtual stations for health, nursing ethics, nursing reports, nursing diagnosis, physical examinations, and psychiatric nursing were held.

News about the exam was communicated to students through the college website and text messages. Then, an online orientation session was held on Skype with students regarding the need assessment of pre-exam educational workshops, virtual and in-person exam standards, how to use exam software, how to conduct virtual exams, explaining the necessary infrastructure for participating in the exam by students, completing anxiety and satisfaction questionnaires, rules and regulations, how to deal with rejected individuals, and exam testing and Q&A. Additionally, a pre-exam in-person orientation session was held.

To inform students about the entire educational process, the resources and educational content recommended by the professors, including PDF files, photos and videos, instructions, and links, were shared through a virtual group on the social media messenger, and scientific information was also, questions were asked and answered through this platform.

Correspondence and necessary coordination were made with the university clinical skills center to conduct in-person workshops and exams.

Following the Test-centered approach, the Angoff Modified method [ 29 , 30 ] was used to determine the scoring criteria for each station by panelists tasked with assigning scores.

Additionally, in establishing standards for this blended CCE for fourth-year nursing students, for whom graduation was a prerequisite, the panelists, as experienced clinical educators familiar with the performance and future roles of these students and the assessment method of the blended exam, were involved [ 29 , 30 ](Table 1 ).

Step 2: Preparing the necessary infrastructure for conducting the exam

Software infrastructure.

The pre- and post-virtual exam questions, scenarios, and questionnaires were uploaded using online software.

The exam was conducted on a trial basis in multiple sessions with the participation of several faculty members, and any issues were addressed. Students were authenticated to enter the exam environment via email and personal information verification. The questions for each station were designed and entered into the software by the respective station instructors and the examination coordinator, who facilitated the exam. The questions were formatted as clinical scenarios, images, descriptive questions, and multiple-choice questions, emphasizing the clinical and practical aspects. This software had various features for administering different types of exams and various question formats, including multiple-choice, descriptive, scenario-based, image-based, video-based, matching, Excel output, and graphical and descriptive statistical analyses. It also had automatic questionnaire completion, notification emails, score addition to questionnaires, prevention of multiple answer submissions, and the ability to upload files up to 4 gigabytes. Student authentication was based on national identification numbers and student IDs, serving as user IDs and passwords. Students could enter the exam environment using their email and multi-level personal information verification. If the information did not match, individuals could not access the exam environment.

Checklists and questionnaires

A student list was prepared, and checklists for the in-person exam and anxiety and satisfaction questionnaires were reproduced.

Empowerment workshops for professors and education staff

Educational needs of faculty members and academic staff include conducting clinical competency exams using the OSCE method; simulating and evaluating OSCE exams; designing standardized questions, checklists, and scenarios; innovative approaches in clinical evaluations; designing physical spaces and setting up stations; and assessing ethics and professional commitment in clinical competency exams.

Student empowerment programs

According to the students’ needs assessment results, in-person workshops on cardiopulmonary resuscitation and airway management and online workshops were held on health, pediatrics, cardiopulmonary resuscitation, ethics, nursing diagnosis, and report writing through Skype messenger. In addition, vaccination notes, psychiatric nursing, and educational files on clinical examinations and basic skills were recorded by instructors and made available to students via virtual groups.

Step 3: CCE implementation

The CCE was held in two parts, in-person and virtual.

In-person exam

The OSCE method was used for this section of the exam. The basic skills station exam included dressing and injections, and the CPR and pediatrics stations were conducted in person. The students were divided into two groups of 21 each semester, and the exam was held in two shifts. While adhering to quarantine protocols, the students performed the procedures for seven minutes at each station, and instructors evaluated them using a checklist. An additional minute was allotted for transitioning to the next station.

Virtual exam

The professional ethics, nursing diagnosis, nursing report, health, psychiatric nursing, and physical examination stations were conducted virtually after the in-person exam. This exam was made available to students via a primary and a secondary link in a virtual space at the scheduled time. Students were first verified, and after the specified time elapsed, the ability to respond to inactive questions and submitted answers was sent. During the exam, full support was provided by the examination center.

The examination coordinator conducted the entire virtual exam process. The exam results were announced 48 h after the exam. A passing grade was considered to be a score higher than 60% in all stations. Students who failed in various stations were given the opportunity for remediation based on faculty feedback, either through additional study or participation in educational workshops. Subsequent exams were held one week apart from the initial exam. It was stipulated that students who failed in more than half of the stations would be evaluated in the following semester. If they failed in more than three sessions at a station, a decision would be made by the faculty’s educational council. However, no students met these situations.

Step 4: Evaluation

The evaluation of the exam was conducted by examiners using a checklist, and the results were announced as pass or fail.

Stage 4: Observation / evaluation

In this study, both process and outcome evaluations were conducted:

Process evaluation

All programs and activities implemented during the test design and administration process were evaluated in the process evaluation. This evaluation was based on operational program control and reflections received from participants through group discussion sessions and virtual groups.

Sample reflections received from faculty members, managers, experts, and students through group discussions and social messaging platforms after the changes:

P7: “The implementation of the blended virtual exam, in the conditions of the COVID-19 crisis where the possibility of holding in-person exams was not fully available, in my opinion, was able to improve the quality of exam administration and address the limitations and weaknesses of the exam entirely virtually.”

P5: “In my opinion, this blended method was able to better evaluate students in terms of clinical readiness for entering clinical practice.”

Outcomes evaluation

The study outcomes were student anxiety, student acceptance and satisfaction, and faculty acceptance and satisfaction. Before the start of the in-person and virtual exams, the Spielberger Anxiety Questionnaire was provided to students. Additionally, immediately after the exam, students and instructors completed the acceptance and satisfaction questionnaire for the relevant section. After the exam, students and instructors completed the acceptance and satisfaction questionnaire again for the entire exam process, including feasibility, satisfaction with its implementation, and educational impact.

Design framework and implementation for the blended Clinical Competency Examination

The exam was planned using a blended method (part in-person, part virtual) according to the Fig.  2 schedule, and all planned programs for the blended CCE for final-year nursing students were implemented in two semesters.

Evaluation results

In this study, 84 final-year nursing students participated, including 37 females (44.05%) and 47 males (55.95%). Among them, 28 (33.3%) were dormitory residents, and 56 (66.7%) were non-dormitory residents.

In this study, both process and outcome evaluations were conducted.

All programs and activities implemented during the test design and administration process were evaluated in the process evaluation (Table  2 ). This evaluation was based on operational program control and reflections received from participants through group discussion sessions and virtual groups on social media.

Anxiety and satisfaction were examined and evaluated as study outcomes, and the results are presented below.

The paired t-test results in Table  3 showed no statistically significant difference in overt anxiety ( p  = 0.56), covert anxiety ( p  = 0.13), and total anxiety scores ( p  = 0.167) between the in-person and virtual sections before the blended Clinical Competency Examination.

However, the mean (SD) of overt anxiety in persons in males and females was 49.27 (11.16) and 43.63 (13.60), respectively, and this difference was statistically significant ( p  = 0.03). Also, the mean (SD) of overt virtual anxiety in males and females was 45.70 (11.88) and 51.00 (9.51), respectively, and this difference was statistically significant ( p  = 0.03). However, there was no significant difference between males and females regarding covert anxiety in the person ( p  = 0.94) and virtual ( p  = 0.60) sections. In addition, the highest percentage of overt anxiety was apparent in the virtual section among women (15.40%) and the in-person section among men (21.28%) and was prevalent at a moderate to high level.

According to Table  4 , One-way analysis of variance showed a significant difference between the virtual, in-person, and blended sections in terms of acceptance and satisfaction scores.

The results of the One-way analysis of variance showed that the mean (SD) acceptance and satisfaction scores of nursing students of the CCE in virtual, in-person, and blended sections were 25.49 (4.73), 27.60 (4.70), and 25.57 (4.97) out of 30, respectively. There was a significant difference between the three sections ( p  = 0.008).

In addition, 3 (7.23%) male and 10 (76.3%) female faculty members participated in this study; of this number, 2 (15.38%) were instructors, and 11 (84.62%) were assistant professors. Moreover, they were between 29 and 50 years old, with a mean (SD) of 41.37 (6.27). Furthermore, they had 4 to 20 years of work experience with a mean and standard deviation of 13.22(4.43).

The results of the analysis of variance showed that the mean (SD) acceptance and satisfaction scores of faculty members of the CCE in virtual, in-person, and blended sections were 30.31 (4.47), 29.86 (3.94), and 30.00 (4.16) out of 33, respectively. There was no significant difference between the three sections ( p  = 0.864).

This action research study showed that the blended CCE for nursing students is feasible and, depending on the conditions and objectives, evaluation stations can be designed and implemented virtually or in person.

The blended exam, combining in-person and virtual elements, managed to address some of the weaknesses of entirely virtual exams conducted in previous terms due to the COVID-19 pandemic. Given the pandemic conditions, the possibility of performing all in-person stations was not feasible due to the risk of students and evaluators contracting the virus, as well as the need for prolonged quarantine. Additionally, to meet the staffing needs of hospitals, nursing students needed to graduate. By implementing the blended exam idea and conducting in-person evaluations at clinical stations, the assessment of nursing students’ clinical competence was brought closer to reality compared to the entirely virtual method.

Furthermore, the need for human resources, station setup costs, and time spent was less than the entirely in-person method. Therefore, in pandemics or conditions where sufficient financial resources and human resources are not available, the blended approach can be utilized.

Additionally, the evaluation results showed that students’ total and overt anxiety in both virtual and in-person sections of the blended CCE did not differ significantly. However, the overt anxiety of female students in the virtual section and male students in the in-person section was considerably higher. Nevertheless, students’ covert anxiety related to personal characteristics did not differ in virtual and in-person exam sections. However, students’ acceptance and satisfaction in the in-person section were higher than in the virtual and blended sections, with a significant difference. The acceptance and satisfaction of faculty members from the CCE in in-person, virtual, and blended sections were the same and relatively high.

A blended CCE nursing competency exam was not found in the literature review. However, recent studies, especially during the COVID-19 pandemic, have designed and implemented this exam using virtual OSCE. Previously, the CCE was held in-person or through traditional OSCE methods.

During the COVID-19 pandemic, nursing schools worldwide faced difficulties administering clinical competency exams for students. The virtual simulation was used to evaluate clinical competency and develop nursing students’ clinical skills in the United States, including standard videos, home videos, and clinical scenarios. Additionally, an online virtual simulation program was designed to assess the clinical competency of senior nursing students in Hong Kong as a potential alternative to traditional clinical training [ 31 ].

A traditional in-person OSCE was also redesigned and developed through a virtual conferencing platform for nursing students at the University of Texas Medical Branch in Galveston. Survey findings showed that most professors and students considered virtual OSCE a highly effective tool for evaluating communication skills, obtaining a medical history, making differential diagnoses, and managing patients. However, professors noted that evaluating examination techniques in a virtual environment is challenging [ 32 ].

However, Biranvand reported that less than half of the nursing students believed the in-person OSCE was stressful [ 33 ]. At the same time, the results of another study showed that 96.2% of nursing students perceived the exam as anxiety-provoking [ 1 ]. Students believe that the stress of this exam is primarily related to exam time, complexity, and the execution of techniques, as well as confusion about exam methods [ 7 ]. In contrast to previous research results, in a study conducted in Egypt, 75% of students reported that the OSCE method has less stress than other examination methods [ 9 ]. However, there has yet to be a consensus across studies on the causes and extent of anxiety-provoking in the OSCE exam. In a study, the researchers found that in addition to the factors mentioned above, the evaluator’s presence could also be a cause of stress [ 34 ]. Another survey study showed that students perceived the OSCE method as more stressful than the traditional method, mainly due to the large number of stations, exam items, and time constraints [ 7 ]. Another study in Egypt, which designed two stages of the OSCE exam for 75 nursing students, found that 65.6% of students reported that the second stage exam was stressful due to the problem-solving station. In contrast, only 38.9% of participants considered the first-stage exam stressful [ 35 ]. Given that various studies have reported anxiety as one of the disadvantages of the OSCE exam, in this study, one of the outcomes evaluated was the anxiety of final-year nursing students. There was no significant difference in total anxiety and overt anxiety between students in the in-person and virtual sections of the blended Clinical Competency Examination. The overt anxiety was higher in male students in the in-person part and female students in the virtual section, which may be due to their personality traits, but further research is needed to confirm this. Moreover, since students’ total and overt anxiety in the in-person and virtual sections of the exam are the same in resource and workforce shortages or pandemics, the blended CCE is suggested as a suitable alternative to the traditional OSCE test. However, for generalization of the results, it is recommended that future studies consider three intervention groups, where all OSCE stations are conducted virtually in the first group, in-person in the second group, and a blend of in-person and virtual in the third group. Furthermore, the results of the study by Rafati et al. showed that the use of the OSCE clinical competency exam using the OSCE method is acceptable, valid, and reliable for assessing nursing skills, as 50% of the students were delighted, and 34.6% were relatively satisfied with the OSCE clinical competency exam. Additionally, 57.7% of the students believed the exam revealed learning weaknesses [ 1 ]. Another survey study showed that despite higher anxiety about the OSCE exam, students thought that this exam provides equal opportunities for everyone, is less complicated than the traditional method, and encourages the active participation of students [ 7 ]. In another study on maternal and infant care, 95% of the students believed the traditional exam only evaluates memory or practical skills. In contrast, the OSCE exam assesses knowledge, understanding, cognitive and analytical skills, communication, and emotional skills. They believed that explicit evaluation goals, appropriate implementation guidelines, appropriate scheduling, wearing uniforms, equipping the workroom, evaluating many skills, and providing fast feedback are among the advantages of this exam [ 36 ]. Moreover, in a survey study, most students were satisfied with the clinical environment offered by the OSCE CCE using the OSCE method, which is close to reality and involves a hypothetical patient in necessary situations that increase work safety. On the other hand, factors such as the scheduling of stations and time constraints have led to dissatisfaction among students [ 37 ].

Furthermore, another study showed that virtual simulations effectively improve students’ skills in tracheostomy suctioning, triage concepts, evaluation, life-saving interventions, clinical reasoning skills, clinical judgment skills, intravenous catheterization skills, role-based nursing care, individual readiness, critical thinking, reducing anxiety levels, and increasing confidence in the laboratory, clinical nursing education, interactive communication, and health evaluation skills. In addition to knowledge and skills, new findings indicate that virtual simulations can increase confidence, change attitudes and behaviors, and be an innovative, flexible, and hopeful approach for new nurses and nursing students [ 38 ].

Various studies have evaluated the satisfaction of students and faculty members with the OSCE Clinical Competency Examination. In this study, one of the evaluated outcomes was the acceptability and satisfaction of students and faculty members with implementing the CCE in blended, virtual, and in-person sections, which was relatively high and consistent with other studies. One crucial factor that influenced the satisfaction of this study was the provision of virtual justification sessions for students and coordination sessions with faculty members. Social messaging groups were formed through virtual and in-person communication, instructions were explained, expectations and tasks were clarified, and questions were answered. Students and faculty members could access the required information with minimal presence in medical education centers and time and cost constraints. Moreover, with the blended evaluation, the researcher’s communication with participants was more accessible. The written guidelines and uploaded educational content of the workshops enabled students to save the desired topics and review them later if needed. Students had easy access to scientific and up-to-date information, and the application of social messengers and Skype allowed for sending photos and videos, conducting workshops, and questions and answering questions. However, the clinical workshops and examinations were held in-person to ensure accuracy. The virtual part of the examination was conducted through online software, and questions focused on each station’s clinical and practical aspects. Students answered various questions, including multiple-choice, descriptive, scenario, picture, and puzzle questions, within a specified time. The blended examination evaluated clinical competency and did not delay these individuals’ entry into the job market. Moreover, during the severe human resource shortage faced by the healthcare system, the examination allowed several nurses to enter the country’s healthcare system. The blended examination can substitute in-person examination in pandemic and non-pandemic situations, saving facilities, equipment, and human resources. The results of this study can also serve as a model to guide other nursing departments that require appropriate planning and arrangements for Conducting Clinical Competency Examinations in blended formats. This examination can also be developed to evaluate students’ clinical performance.

One of the practical limitations of the study was the possibility that participants might need to complete the questionnaires accurately or be concerned about losing marks. Therefore, in a virtual session before the in-person exam, the objectives and importance of the study were explained. Participants were assured that it would not affect their evaluation and that they should not worry about losing marks. Additionally, active participation from all nursing students, faculty members, and staff was necessary for implementing this plan, achieved through prior coordination, virtual meetings, virtual group formation, and continuous reflection of results, creating the motivation for continued collaboration and participation.

Among other limitations of this study included the use of the Spielberger Anxiety Questionnaire to measure students’ anxiety. It is suggested that future studies use a dedicated anxiety questionnaire designed explicitly for pre-exam anxiety measurement. Another limitation of the current research was its implementation in nursing and midwifery faculty. Therefore, it is recommended that similar studies be conducted in nursing and midwifery faculties of other universities, as well as in related fields, and over multiple consecutive semesters. Additionally, for more precise effectiveness assessment, intervention studies in three separate virtual, in-person, and hybrid groups using electronic checklists are proposed. Furthermore, it is recommended that students be evaluated in terms of other dimensions and variables such as awareness, clinical skill acquisition, self-confidence, and self-efficacy.

Conducting in-person Clinical Competency Examination (CCE) during critical situations, such as the COVID-19 pandemic, is challenging. Instead of virtual exams, blended evaluation is a feasible approach to overcome the shortages of virtual ones and closely mimic in-person scenarios. Using a blended method in pandemics or resource shortages, it is possible to design, implement, and evaluate stations that evaluate basic and advanced clinical skills in in-person section, as well as stations that focus on communication, reporting, nursing diagnosis, professional ethics, mental health, and community health based on scenarios in a virtual section, and replace traditional OSCE exams. Furthermore, the use of patient simulators, virtual reality, virtual practice, and the development of virtual and in-person training infrastructure to improve the quality of clinical education and evaluation and obtain the necessary clinical competencies for students is recommended. Also, since few studies have been conducted using the blended method, it is suggested that future research be conducted in three intervention groups, over longer semesters, based on clinical evaluation models and influential on other outcomes such as awareness and clinical skill acquisition self-efficacy, confidence, obtained grades, and estimation of material and human resources costs. This approach reduced the need for physical space for in-person exams, ensuring participant quarantine and health safety with higher quality. Additionally, a more accurate assessment of nursing students’ practical abilities was achieved compared to a solely virtual exam.

Data availability

The datasets generated and analyzed during the current study are available on request from the corresponding author.

Rafati F, Pilevarzade M, Kiani A. Designing, implementing and evaluating once to assess nursing students’ clinical competence in Jiroft faculty of nursing and midwifery. Nurs Midwifery J. 2020;18(2):118–28.

Google Scholar  

Sadeghi T, Ravari A, Shahabinejad M, Hallakoei M, Shafiee M, Khodadadi H. Performing of OSCE method in nursing students of Rafsanjan University of Medical science before entering the clinical field in the year 2010: a process for quality improvement. Community Health J. 2012;6(1):1–8.

Ali GA, Mehdi AY, Ali HA. Objective structured clinical examination (OSCE) as an assessment tool for clinical skills in Sohag University: nursing students’ perspective. J Environ Stud. 2012;8(1):59–69.

Article   Google Scholar  

Bolourchifard F, Neishabouri M, Ashktorab T, Nasrollahzadeh S. Satisfaction of nursing students with two clinical evaluation methods: objective structured clinical examination (OSCE) and practical examination of clinical competence. Adv Nurs Midwifery. 2010;19(66):38–42.

Noohi E, Motesadi M, Haghdoost A. Clinical teachers’ viewpoints towards Objective Structured Clinical examination in Kerman University of Medical Science. Iran J Med Educ. 2008;8(1):113–20.

Reza Masouleh S, Zare A, Chehrzad M, Atrkarruoshan Z. Comparing two methods of evaluation, objective structured practical examination and traditional examination, on the satisfaction of students in Shahid Beheshti faculty of nursing and midwifery. J Holist Nurs Midwifery. 2008;18(1):22–30.

Bagheri M, Sadeghineajad Forotagheh M, Shaghayee Fallah M. The comparison of stressors in the assessment of basic clinical skills with traditional method and OSCE in nursing students. Life Sci J. 2012;9(4):1748–52.

Eldarir SH, El Sebaae HA, El Feky HA, Hussein HA, El Fadil NA, El Shaeer IH. An introduction of OSCE versus the traditional method in nursing education: Faculty capacity building and students’ perspectives. J Am Sci. 2010;6(12):1002–14.

Al-Zeftawy AM, Khaton SE. Student evaluation of an OSCE in Community Health nursing clinical course at Faculty of nursing, Tanta University. J Nurs Health Sci. 2016;5(4):68–76.

Hayter M, Jackson D. Pre-registration undergraduate nurses and the COVID-19 pandemic: students or workers? J Clin Nurs. 2020;29(17–18):3115–6.

Bayham J, Fenichel EP. Impact of school closures for COVID-19 on the US health-care workforce and net mortality: a modeling study. Lancet Public Health. 2020;5(5):e271–8.

Murphy MPA. COVID-19 and emergency eLearning: consequences of the securitization of higher education for post-pandemic pedagogy. Contemp Secur Policy. 2020;41(3):492–505.

Allen IE, Seaman J. Learning on demand: Online education in the United States, 2009.

Meyer KA, Wilson JL. The role of Online Learning in the emergency plans of Flagship Institutions. Online J Distance Learn Adm. 2011;14(1):110–8.

Kursumovic E, Lennane S, Cook TM. Deaths in healthcare workers due to COVID-19: the need for robust data and analysis. Anaesthesia. 2020;75(8):989–92.

Malekshahi Beiranvand F, Hatami Varzaneh A. Health care workers challenges during coronavirus outbreak: the qualitative study. J Res Behav Sci. 2020;18(2):180–90.

Boursicot K, Kemp S, Ong TH, Wijaya L, Goh SH, Freeman K, Curran I. Conducting a high-stakes OSCE in a COVID-19 environment. MedEdPublish. 2020;9:285–89.

Atwa H, Shehata MH, Al-Ansari A, Kumar A, Jaradat A, Ahmed J, Deifalla A, Online. Face-to-Face, or blended learning? Faculty and Medical Students’ perceptions during the COVID-19 pandemic: a mixed-method study. Front Med. 2022;9:791352.

Chan MMK, Yu DS, Lam VS, Wong JY. Online clinical training in the COVID-19 pandemic. Clin Teach. 2020;17(4):445–6.

Toulabi T, Yarahmadi S. Conducting a clinical competency test for nursing students in a virtual method during the Covid-19 pandemic: a case study. J Nurs Educ. 2021;9(5):33–42.

Meskell P, Burke E, Kropmans TJB, Byrne E, Setyonugroho W, Kennedy KM. Back to the future: an online OSCE Management Information System for nursing OSCEs. Nurse Educ Today. 2015;35(11):1091–6.

Lichtenberg PA. (2010). Handbook of Assessment in Clinical Gerontology, 2nd Ed. Academic Press, https://doi.org/10.1016/B978-0-12-374961-1.10030-2

Gholami Booreng F, Mahram B, Kareshki H. Construction and validation of a scale of research anxiety for students. IJPCP. 2017;23(1):78–93.

Esmaili M. A survey of the influence of Murita therapy on reducing the rate of anxiety in clients of counseling centers. Res Clin Psychol Couns. 2011;1(1):15–30.

Farajpour A, Amini M, Pishbin E, Arshadi H, Sanjarmusavi N, Yousefi J, Sarafrazyazdi M. Teachers’ and students’ satisfaction with DOPS Examination in Islamic Azad University of Mashhad, a study in Year 2012. Iran J Med Educ. 2014;14(2):165–73.

StraussAC, Corbin JM. Basics of qualitative research: grounded theory procedures and technique. 2nd ed. London: Sage, Newbury Park; 1998.

Dickens L, Watkins K. Action research: rethinking Lewin. Manage Learn. 1999;30(2):127–40.

Rezaeerad M, Nadri Kh, Mohammadi Etergoleh R. The effect of ADDIE (analysis, design, development, implementation, evaluation) designing method with emphasizing on mobile learning on students’ self-conception, development motivation and academic development in English course. Educational Adm Res Q. 2013;4(15):15–32.

Ben-David MF. AMEE Guide 18: standard setting in student assessment. Med Teach. 2000;22(2):120–30.

McKinley DW, Norcini JJ. How to set standards on performance-based examinations: AMEE Guide 85. Med Teach. 2014;36(2):97–110.

Fung JTC, Zhang W, Yeung MN, Pang MTH, Lam VSF, Chan BKY, Wong JYH. Evaluation of students perceived clinical competence and learning needs following an online virtual simulation education programmed with debriefing during the COVID-19 pandemic. Nurs Open. 2021;8(6):3045–54.

Luke S, Petitt E, Tombrella J, McGoff E. Virtual evaluation of clinical competence in nurse practitioner students. Med Sci Educ. 2021;31:1267–71.

Beiranvand SH, Hosseinabadi R, Ghasemi F, Anbari KH. An Assessment of nursing and Midwifery Student Veiwwpoin, Performance, and feedback with an objective structured clinical examination. J Nurs Educ. 2017;6(1):63–7.

Sheikh Abumasoudi R, Moghimian M, Hashemi M, Kashani F, Karimi T, Atashi V. Comparison of the Effect of Objective Structured Clinical evaluation (OSCE) with Direct and Indirect Supervision on nursing student’s test anxiety. J Nurs Educ. 2015;4(2):1–8.

Zahran EM, Taha EE. Students’ feedback on Objective Structured Clinical examinations (OSCEs) experience in emergency nursing. J High Inst Public Health. 2009;39(2):370–87.

Na A-G. Assessment of Students’ knowledge, clinical performance and satisfaction with objective structured clinical exam. Med J Cairo Univ. 2009;77(4):287–93.

Adib-Hajbaghery M, Yazdani M. Effects of OSCE on learning, satisfaction and test anxiety of nursing students: a review study. Iran J Med Educ. 2018;18:70–83.

Purwanti LE, Sukartini T, Kurniawati ND, Nursalam N, Susilowati T. Virtual Simulation in clinical nursing education to improve knowledge and clinical skills: Literature Review. Open Access Maced J Med Sci. 2022;10(F):396–404.

Download references

Acknowledgements

We want to thank the Research and Technology deputy of Smart University of Medical Sciences, Tehran, Iran, the faculty members, staff, and officials of the School of Nursing and Midwifery, Lorestan University of Medical Sciences, Khorramabad, Iran, and all individuals who participated in this study.

All steps of the study, including study design and data collection, analysis, interpretation, and manuscript drafting, were supported by the Deputy of Research of Smart University of Medical Sciences.

Author information

Authors and affiliations.

Department of E-Learning in Medical Education, Center of Excellence for E-learning in Medical Education, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran

Rita Mojtahedzadeh & Aeen Mohammadi

Department of Medical Education, Smart University of Medical Sciences, Tehran, Iran

Tahereh Toulabi

Cardiovascular Research Center, School of Nursing and Midwifery, Lorestan University of Medical Sciences, Khorramabad, Iran

You can also search for this author in PubMed   Google Scholar

Contributions

RM. Participating in study design, accrual of study participants, review of the manuscript, and critical revisions for important intellectual content. TT : The investigator; participated in study design, data collection, accrual of study participants, and writing and reviewing the manuscript. AM: Participating in study design, data analysis, accrual of study participants, and reviewing the manuscript. All authors read and approved the final version of the manuscript.

Corresponding author

Correspondence to Tahereh Toulabi .

Ethics declarations

Ethics approval and consent to participate.

This action research was conducted following the participatory method. All methods were performed according to the relevant guidelines and regulations in the Declaration of Helsinki (ethics approval and consent to participate). The study’s aims and procedures were explained to all participants, and necessary assurance was given to them for the anonymity and confidentiality of their information. The results were continuously provided as feedback to the participants. Informed consent (explaining the goals and methods of the study) was obtained from participants. The Smart University of Medical Sciences Ethics Committee approved the study protocol (IR.VUMS.REC.1400.011).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Mojtahedzadeh, R., Toulabi, T. & Mohammadi, A. The design, implementation, and evaluation of a blended (in-person and virtual) Clinical Competency Examination for final-year nursing students. BMC Med Educ 24 , 936 (2024). https://doi.org/10.1186/s12909-024-05935-9

Download citation

Received : 21 July 2023

Accepted : 20 August 2024

Published : 28 August 2024

DOI : https://doi.org/10.1186/s12909-024-05935-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Clinical Competency Examination (CCE)
• Objective Structural Clinical Examination (OSCE)
• Blended method
• Satisfaction

BMC Medical Education

ISSN: 1472-6920

• General enquiries: [email protected]

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals

You are here

• Volume 78, Issue 10
• Implications of child poverty reduction targets for public health and health inequalities in England: a modelling study between 2024 and 2033
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0001-8649-6913 Ronan McCabe 1 ,
• http://orcid.org/0000-0001-7286-8106 Roxana Pollack 1 ,
• Philip Broadbent 1 ,
• http://orcid.org/0000-0002-3060-939X Rachel M Thomson 1 ,
• http://orcid.org/0000-0002-2863-4983 Erik Igelström 1 ,
• http://orcid.org/0000-0003-0085-5263 Anna Pearce 1 ,
• http://orcid.org/0000-0002-1294-6851 Clare Bambra 2 ,
• http://orcid.org/0000-0003-3480-6566 Davara Lee Bennett 3 ,
• http://orcid.org/0000-0003-3533-3238 Alexiou Alexandros 3 ,
• http://orcid.org/0000-0002-4573-4628 Konstantinos Daras 3 ,
• http://orcid.org/0000-0002-5828-7724 David Taylor-Robinson 3 ,
• http://orcid.org/0000-0002-4208-9475 Benjamin Barr 3 ,
• http://orcid.org/0000-0001-6593-9092 Srinivasa Vittal Katikireddi 1
• 1 MRC/CSO Social & Public Health Sciences Unit , University of Glasgow , Glasgow , UK
• 2 Population Health Sciences Institute , Newcastle University Institute for Health and Society , Newcastle upon Tyne , UK
• 3 Public Health, Policy & Systems , University of Liverpool , Liverpool , UK
• Correspondence to Dr Ronan McCabe, MRC/CSO Social & Public Health Sciences Unit, University of Glasgow, Glasgow G12 8TB, UK; ronan.mccabe{at}glasgow.ac.uk

Background We investigated the potential impacts of child poverty (CP) reduction scenarios on population health and health inequalities in England between 2024 and 2033.

Methods We combined aggregate local authority-level data with published and newly created estimates on the association between CP and the rate per 100 000 of infant mortality, children (aged <16) looked after, child (aged <16) hospitalisations for nutritional anaemia and child (aged <16) all-cause emergency hospital admissions. We modelled relative, absolute (per 100 000) and total (per total population) annual changes for these outcomes under three CP reduction scenarios between 2024 and 2033— low-ambition (15% reduction), medium-ambition (25% reduction) and high-ambition (35% reduction)—compared with a baseline CP scenario (15% increase). Annual changes were aggregated between 2024 and 2033 at national, regional and deprivation (IMD tertiles) levels to investigate inequalities.

Results All CP reduction scenarios would result in substantial improvements to child health. Meeting the high-ambition reduction would decrease total cases of infant mortality (293; 95% CI 118 to 461), children looked after (4696; 95% CI 1987 to 7593), nutritional anaemia (458, 95% CI 336 to 574) and emergency admissions (32 650; 95% CI 4022 to 61 126) between 2024 and 2033. Northern regions (eg, North East) exhibited the greatest relative and absolute benefit. The most deprived tertile would experience the largest relative, absolute and total benefit; under high-ambition reduction, total infant mortality cases were predicted to fall by 126 (95% CI 51 to 199) in the most deprived tertile compared with 71 (95% CI 29 to 112) in the least between 2024 and 2033.

Conclusions Achieving reductions in CP could substantially improve child health and reduce health inequalities in England.

• INEQUALITIES
• CHILD HEALTH

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information. Alternatively, the data is also available through the place-based longitudinal data resource: https://pldr.org/ .

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See:  https://creativecommons.org/licenses/by/4.0/ .

https://doi.org/10.1136/jech-2024-222313

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

Child poverty is a key determinant of population health and health inequalities.

WHAT THIS STUDY ADDS

Child poverty is responsive to policy. We are the first to explore the health impact of meeting hypothetical future child poverty targets in England between 2024 and 2033. We show that reducing child poverty across this period would substantially improve child health and reduce health inequalities.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

We demonstrate the importance of renewed policy efforts to reduce child poverty.

Child poverty is a key determinant of population health and health inequalities. 1 Experiencing child poverty is associated with worse outcomes across a wide range of early years health indicators, with evidence suggesting that these associations are often causal. 2–4 Child poverty also likely reinforces the clustering and accumulation of adverse exposures. 5 Government policy exerts a major influence over rates of child poverty. For example, higher levels of social spending were associated with lower levels of child poverty across European countries in the aftermath of the 2008 financial crisis, whereas countries such as the UK that have enacted high levels of austerity following the crisis—including retrenchment of social spending and local government budgets—have exhibited worse trends in child health outcomes. 6–9

In the UK, progress had been made in reducing child poverty with the ‘New Labour’ Government (1997–2010) introducing several policies under the aim of being “(…) the first generation to end child poverty (in the UK)”. 10 These included targeted measures to supplement income such as the Child Tax Credit and increases in Child Benefit, alongside other measures to improve early years services such as Sure Start programmes. 10 11 Consequently, relative child poverty (before housing costs, BHC) declined from 27% to 20% across this period (a 25.9% reduction) 12 ; corresponding declines in infant mortality rates were observed, particularly in the most deprived areas. 13 However, following the 2008 financial crash and the subsequent enactment of austerity measures by consecutive Conservative-led Governments since 2010, child poverty levels began rising from 17% in 2014 to 23% (BHC) in 2020. 12 This period coincided with a rise in infant mortality. 9 Child poverty is responsive to policy; levels fell to 19% in 2021 following a brief uplift in social spending which was withdrawn by the end of that same year, with levels rising back to 22% in 2023. 12 14 The UK also exhibits wide geographical variation in child poverty levels and its devolved governments have (although limited) powers to influence levels; for example, in 2021, the Scottish Government introduced the weekly Scottish Child Payment for low-income parents/carers, although the impact of this policy on child poverty has not been evaluated yet. 15 16 The societal effects of the COVID-19 pandemic and the ongoing ‘cost of living’ crisis have heightened concerns about the level of child poverty in the UK and its current and future impact on child health. 17–21 While some broad measures have been taken by the UK Government in response to this situation, there has been a lack of policy explicitly addressing rising child poverty—such as removing the ‘two-child limit’ and ‘benefit cap’ on financial support. 22 Similarly, the UK Government’s initiative to ‘level up’ regional inequalities makes no reference to child poverty, despite the wide regional variations in child poverty rates. 16 23 As such, it is important to understand how levels of child poverty could change under different hypothetical policy scenarios and the likely consequences these scenarios would have for child health.

We therefore aimed to investigate the potential impact of meeting different child poverty reduction scenarios on child health outcomes and inequalities in England over the next decade. We selected four child health outcomes which are associated with poverty and deprivation in childhood and for which there were local authority-level data available in England: (1) infant mortality; (2) children (<16 years old) entering local authority care; (3) child (<16 years old) hospital admissions for nutritional anaemia; and (4) child (<16 years old) all-cause emergency hospital admissions. 9 13 24–27 While children entering care is not a direct measure of health, it is associated with a range of short-term and long-term adverse health consequences. 24

Study setting and design

We created a dynamic policy simulation model using aggregated local authority-level data from England. This model allows for the exploration of ex-ante policy impacts under different scenarios between 2024 and 2033, drawing on existing data and published evidence of the relationship between child poverty and health outcomes. 28

This ecological study used data for 145 English upper-tier local authorities (UTLAs). We excluded four UTLAs due to either small population size or irreconcilable boundary changes over the study period (City of London, Isles of Scilly, Bournemouth, Christchurch, and Poole and Dorset) 24 and two further UTLAs due to a lack of published outcome data (Buckinghamshire and Northamptonshire). Exposure data on relative child poverty were acquired from the children in low-income families (CiLIF) statistics, compiled by the Department of Work and Pensions and His Majesty’s Revenue and Customs. 29 Outcome data for infant mortality were derived from the Office for National Statistics (ONS). 30 Data for looked-after children were obtained from the UK Government’s Department of Education, 31 and local authority-level data on the number of hospitalisations for nutritional anaemia and all-cause emergency admissions were derived from NHS Hospital Episode Statistics data and supplied by the University of Liverpool’s Place-Based Longitudinal Data Resource (PLDR). 32 Data on local authority-level income deprivation were derived from the 2019 Index of Multiple Deprivation (IMD), using the local authority average rank. 33

We used the prevalence of relative child poverty BHC, captured in the CiLIF statistics, as our study exposure. This was defined as the proportion of children <16 years old living in families with an income of <60% of the contemporary national median income BHC. We used the 2020 CiLIF estimate to project annual values forward until the study end date in 2033 for each UTLA (see ‘modelled scenarios’ below); while estimates have subsequently been published until 2023, these are at present provisional.

We examined four outcome measures at UTLA level: infant mortality, defined as the total number of deaths under the age of one per 100 000 live births per year; children looked after, defined as the total number of children (<16 years old) entering local authority care (whose care had been with local authorities for >24 hours period) per 100 000 of the <16 population per year; total child (<16 years old) hospitalisations for nutritional anaemia per 100 000 of the <16 population per year; and total child (<16 years old) all-cause emergency admissions per 100 000 of the <16 population per year. The final available values (numerator and denominator) for each outcome—2021 for infant mortality and children looked after, and 2019 for nutritional anaemias and emergency admissions—were held constant until start of the intervention period in 2024 (see online supplemental data ).

Supplemental material

Data analysis, effect estimates.

We calculated additional cases attributable to changes in child poverty for each scenario using separate effect estimates for each outcome. For infant mortality and looked-after children, we used published estimates. For the former, we used an estimate from a time trends analysis of local authority-level data in England between 2000 and 2017, where a one-point change in the prevalence of child poverty was associated with a change in infant mortality of 5.8 (95% CI 2.4 to 8.9) deaths per 100 000 live births. 9 For the latter, we used an estimate from a longitudinal ecological analysis of local authority-level data in England between 2015 and 2020, where a one-point change in the prevalence of child poverty was associated with a change in children looked after of 5.2 (95% CI 2.2 to 8.3) children per 100 000 children <16 years old. 24

For nutritional anaemia and emergency admissions, we did not find relevant estimates in the published literature. Instead, we derived estimates for each outcome from new analysis of annual local authority-level data from the PLDR 32 between 2015 and 2019. Estimates were derived using linear within-between regression analysis, in line with similar studies. 24 This approach uses the strengths of both fixed and random effects models, integrating information on differences between and across areas. We found that a one-point change in the prevalence of child poverty was associated with 0.53 (95% CI 0.39 to 0.67) and 37.7 (95% CI 3.8 to 72.1) additional cases per 100 000 children <16 years old for nutritional anaemia and emergency admissions, respectively.

Modelled policy scenarios

We modelled a baseline child poverty scenario as a logarithmic annual increase (ie, curvilinear with a falling rate of change over time) from the 2020 prevalence of child poverty for each UTLA, resulting in a total cumulative increase of 15% from 2020 to 2033. This formed the baseline scenario to which the effects of other scenarios were compared (see below); that is, we were interested in modelling the potential effects of successful action to reduce child poverty versus unsuccessful or no action. Using the 2023 baseline prevalence of child poverty, we then modelled three scenarios at UTLA level over a 10-year period from 2024 until 2033 (see table 1 ): (1) low ambition reduction, a cumulative exponential decrease (ie, increasing rate of change over time) in child poverty of 15% on 2023 levels between 2027 and 2033 (3-year delay); (2) medium ambition reduction, a cumulative exponential decrease of 25% on 2023 levels between 2026 and 2033 (2-year delay); and (3) high ambition reduction, a cumulative exponential decrease of 35% on 2023 levels between 2025 and 2033 (1-year delay). We understood these scenarios to be realistic in light of the 26% fall in prevalence previously observed in the UK between 1997 and 2010 under previous governments. 34 All scenarios were created using MS Excel (see online supplemental data ).

• View inline

Descriptive statistics for baseline exposure and outcomes, derived from modelled projections

Modelling approach

We calculated the annual number of attributable (avoided or added) cases at UTLA level for each outcome under each scenario: the annual relative change in child poverty (%) multiplied by the effect size per number exposed in that same year. We used a Monte Carlo approach to randomly sample (1000 iterations) from the distribution of the effect size of child poverty for each health outcome based on its mean and SE, taking the median of the sample to determine the point estimate of attributable cases, and the 2.5th and 97.5th percentiles for the upper and lower CIs. For each scenario compared with baseline, we report the change in cases for each outcome as the (1) total change per individuals exposed, (2) absolute change, as the risk difference (RD) per 100 000 exposed, and (3) relative change, as risk ratio (RR) at local authority level, regional level, national level and by IMD tertiles across the whole intervention period (2024–2033). Both RR and RD account for differences in population size and are thus suitable for comparison, but only compare extreme categories of the distribution. To quantify effects on inequalities in outcomes taking account for the whole distribution of deprivation, we estimated absolute and relative changes, respectively, as the difference in slope index of inequality (SII) and ratio of relative index of inequality (RII) under each scenario compared with baseline (see online supplemental appendix 1 for details). 35 The SII can be interpreted as the difference in the rate of outcomes between the hypothetically most and least deprived local authorities, whereas the RII can be interpreted as the ratio between those local authorities.

Across the 145 UTLAs included in analysis, the population-weighted mean prevalence of child poverty in 2023 projected under the baseline scenario was 20.7% ( table 1 ). At regional level, the prevalence of child poverty was typically higher in northern regions compared with southern, with the North East having the highest median prevalence at 27.6% (IQR=4.2) and the South East and South West both had the lowest at 15.4% (IQR=8.5–6.7, respectively) in 2023. Across IMD tertiles, the median prevalence was 27.8% (IQR=10.1) in the most deprived tertile and 13.9% (IQR=4.6) in the least deprived tertile. Cases per 100 000 in 2023 are given for each outcome in table 2 , with emergency admission being the most frequent and hospitalisations for nutritional anaemia being the least. For each outcome, cases tended to be highest in regions with high child poverty. Outcome trends (cases per 100 000 exposed) at national level over the period for which official data were available (2015–2019) are presented in online supplemental appendix 2 figure A : admissions fluctuated across this period although were rising 2017–2019, hospitalisations for nutritional anaemia continued rising, and infant mortality and children looked after both fell from 2017 onwards.

Modelled relative and absolute changes (95% CI) under three child poverty reduction scenarios between 2024 and 2033, relative to a baseline scenario of increasing child poverty

Modelled changes

Increasingly ambitious scenarios corresponded to greater relative and absolute beneficial effects, with effect sizes in the high-ambition policy target around twice that of the low-ambition target across all outcome measures at all levels of aggregation ( tables 2 and 3 , online supplemental appendix 2 tables A,B ).

Modelled relative and absolute changes by Index of Multiple Deprivation (IMD) tertile and change in Slope Index of Inequality (SII) under three child poverty reduction scenarios between 2024 and 2033, relative to a baseline scenario of increasing child poverty

Between 2024 and 2033 across England, compared with baseline, we anticipate a reduction in: infant mortality of 1.6% (293 avoided cases, 95% CI 118 to 461) under the high-ambition scenario versus 0.9% (155 avoided cases, 95% CI 62 to 244) under the low-ambition scenario; children looked after of 2% (4696 avoided cases, 95% CI 1987 to 7593) versus 1% (2483 avoided cases, 95% CI 1051 to 4015); hospitalisations for nutritional anaemia of 4.1% (458 avoided cases, 95% CI 336 to 574) versus 2.2% (242 avoided cases, 95% CI 177 to 304); and emergency admissions of 0.4% (32 650 avoided cases, 95% CI 4022 to 34 126) versus 0.2% (17 266 avoided cases, 95% CI 2127 to 32 324) ( table 2 and online supplemental appendix 2 table A ).

At regional level, estimated absolute reductions were typically higher in the north and west of England (eg, North East, West Midlands and Yorkshire and The Humber) compared with the south (see table 2 ); this pattern is highlighted in figures 1 and 2 for cases of emergency admissions avoided per 100 000 compared with baseline under the high-ambition scenario. Between 2024 and 2033, for all child poverty reduction scenarios, we anticipate cases avoided (compared with baseline) per 100 000 would be largest in the North East for all outcomes and smallest in the South East ( table 2 ). Under the high-ambition scenario, estimated total avoided cases in the North East would be 18 (95% CI 7 to 28) for infant mortality, 298 (95% CI 126 to 482) for children looked after, 29 (95% CI 21 to 36) for nutritional anaemias, and 2070 (95% CI 255 to 3876) for emergency admissions ( online supplemental appendix 2 table A ). Regional patterns of relative change were less uniform ( table 2 ), while total cases avoided were typically highest in regions with greater population size (eg, London) ( online supplemental appendix 2 table A ). At local authority level across reduction scenarios, absolute changes per 100 000 were highest in Middlesborough, Oldham, Bradford and Birmingham for all outcome measures (see online supplemental data ); this is visually displayed for emergency admissions in figures 1 and 2 .

• Download figure
• Open in new tab
• Download powerpoint

Absolute changes in avoided cases of emergency admissions (per 100 000) for the high ambition scenario (compared to baseline) at local authority level. Grey areas represent excluded local authorities.

Absolute changes in avoided cases of emergency admissions (per 100 000) for the high ambition scenario (compared to baseline) at region level.

Considering deprivation level, anticipated reductions on the difference scale (per 100 000) compared with baseline were highest in the most deprived tertile of UTLAs for all outcome measures (see table 3 ). Under the high-ambition scenario, this equated to a total avoided cases of 126 (95% CI 51 to 199) in the most deprived versus 71 (95% CI 29 to 112) in the least for infant mortality, 1907 (95% CI 807 to 3083) versus 1199 (95% CI 507 to 1939) for children looked after, 189 (95% CI 137 to 234) versus 117 (95% CI 86 to 146) for nutritional anaemias and 13 302 (95% CI 1639 to 24 903) versus 8 322 (95% CI 1025 to 15 581) for emergency admissions ( online supplemental appendix 2 table B ); total avoided cases under each scenario for each outcome measure are shown in figure 3 . Changes on the ratio scale followed a broadly similar pattern ( table 3 ). Greater reductions in child poverty were associated with greater reductions in absolute (SII difference) and relative (RII ratio) inequalities ( table 3 and online supplemental appendix 2 table C , respectively).

Estimated total avoided cases of four health outcomes under low, medium and high poverty reduction scenarios by Index of Multiple Deprivation (IMD tertile), 2024-2033.

Reducing child poverty will likely improve a range of child health outcomes and reduce health inequalities if similar or larger declines to those observed between 1997 and 2010 were achieved. We estimated relative, absolute and total changes in infant mortality, children looked after, nutritional anaemias and all-cause emergency admissions using local authority-level data in England under three different child poverty reduction scenarios between 2024 and 2033 compared with a baseline scenario of increasing child poverty. Achieving an ambitious but realistic reduction of 35% on 2023 levels would be expected to result in avoiding a total of 293 infant deaths, 4696 children entering care, 458 childhood admissions with nutritional anaemias and 32 650 childhood emergency admissions. These reductions would likely translate into significant savings for, and relieve pressure on, local authorities (in relation to children looked after) and health services. Benefits are likely to be greatest in the most disadvantaged areas, helping efforts to ‘level up’. Other health impacts that we have not been able to quantify are also likely.

We used administrative data from trusted sources and outcome estimates from previous empirical studies where available. Our modelling approach was simple and transparent, relying on a limited set of assumptions and a realistic baseline scenario (eg, we predicted mean relative child poverty BHC at 20.7%, whereas the provisional CiLif estimate for 2023 gives 20.1%). 29 However, there are limitations to this work. We focused here on a limited set of outcomes which capture different dimensions of child health and for which there were data readily available. However, future work could extend this analysis to look at other common child health outcomes such as obesity and mental health which are both associated with child poverty. 36 37 Relatedly, we used emergency admissions as a health outcome but acknowledge that they can be affected by health service access (changes in admission practice, transport, etc). Nonetheless, our analyses to parameterise the model excluded the COVID-19 pandemic when changes in practice were most likely to be problematic. We adopted the exposure of relative child poverty rate BHC. However, findings may have differed with alternative measures of child poverty such as absolute rates and rates after housing costs. Additionally, our analyses are predicated on the associations between child poverty and health outcomes accurately reflecting causal effects. While our analyses of changes within local authorities account for time-invariant confounding, risks of residual confounding remain. It is also possible that the effect estimates we observed for each outcome could differ as a consequence of the differing time periods for which data were available. Shorter time periods may lead to underestimated effect sizes within panel data analyses. 38 This might imply our estimates of the impacts on emergency admissions and nutritional anaemia are underestimated. Relatedly, it is possible that the relationship between child poverty and outcomes does not exhibit the linear dose–response relationship that we have assumed here. A few local authorities were excluded due to small numbers, with possible consequences for overall estimates. Finally, our analyses are based on aggregate (ecological) data which could be subject to the ecological fallacy; although, while individual-level data analyses are of interest, these may be subject to the atomistic fallacy (ie, addressing child poverty could have positive impacts for communities beyond the individual). 39 Aggregate data meant that we were also unable to account for variation within and between local authorities in the mechanisms influencing child poverty—for example, the depth of child poverty might differ and the health effects of addressing severe child poverty might differ from addressing less severe poverty. Furthermore, different policies to reduce child poverty (such as minimum wages, tax credits, welfare benefits) might have quite heterogenous effects that we do not distinguish. We would anticipate the impacts of the above factors to result in our estimates being conservative.

To our knowledge, this study is the first to explore the potential impacts of future child poverty reductions on a range of child health outcomes in England. It builds on previous empirical work that has highlighted the consequences of child poverty on outcomes such as infant mortality and children looked after in England. 9 13 24 For example, this research found that reductions in child poverty in the UK between 1997 and 2010 led to a reduction in infant mortality, while subsequent increases in child poverty led to increases in infant mortality. 9 13 Tying into factors influencing child poverty, previous studies have also found associations between increased local authority spending in England and reductions in hospital admissions for nutritional anaemia, although this association lacked precision among those <14 years old (rate ratio=0.97, 95% CI 0.90 to 1.05). 40 Similarly, a study using local authority data by the Nuffield Trust showed that, in 2015/2016, the number of emergency admissions was higher with increasing deprivation among those <14 years old. 26

We highlight that if policy-makers were to set and achieve child poverty targets for England—for example, through suggested measures such as removing the two-child limit and benefit cap 22 —this would likely improve child health, particularly among the most socioeconomically disadvantaged and ‘level up’ regional inequalities.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

Acknowledgments.

For the purpose of open access, the author(s) have applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising from this submission.

• Whitehead M , et al
• Wickham S ,
• Barr B , et al
• Kelly-Irving M , et al
• Spencer N , et al
• Broadbent P ,
• Pearce A , et al
• Taylor-Robinson D ,
• Wickham S , et al
• Piachaud D ,
• Sutherland H
• UK Government Department for Work and Pensions
• Robinson T ,
• Norman PD , et al
• Ray-Chaudhuri S ,
• Social Security Scotland
• Lee AC , et al
• Hefferon C ,
• Bennett D , et al
• Uthayakumar-Cumarasamy A
• Thomson R ,
• Kopasker D , et al
• Taylor-Robinson DC ,
• Riordan R ,
• Ford J , et al
• University of York Cost of Living Research Group
• UK Government
• Bennett DL ,
• Schlüter DK ,
• Melis G , et al
• Arghavani H ,
• Issah A , et al
• Kossarova L ,
• Hargreaves D , et al
• Murray ET ,
• Maughan B , et al
• Zucchelli E ,
• Office for National Statistics (ONS)
• UK Government Department for Education
• ↵ Place-based logitudinal data resource . 2024 . Available : https://pldr.org/
• National statistics
• Moreno-Betancur M ,
• Latouche A ,
• Menvielle G , et al
• National statistics (UK)
• Roos JM , et al
• Diez-Roux AV
• Jenkins R ,
• Mason KE , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1
• Data supplement 2
• Data supplement 3

X @roxana_pollack, @Rachel_Thomson, @igelstorm, @ProfBambra, @benj_barr2, @vkatikireddi

BB and SVK contributed equally.

RM and RP contributed equally.

Contributors RM serves as guarantor for this study. SVK and BB conceptualised the study. RM, RP, DLB, AA and KD were involved in data curation. RP, EI, RMT and PB contributed to analysis code. RM finalised analysis code, conducted formal analyses and visualised findings. RM, RP, AP and SVK wrote the original draft. All authors were involved in the review and editing of the original draft.

Funding RM, RP, EI, RMT, AP and SVK declare funding from the Medical Research Council (MC_UU_00022/2) and the Scottish Government’s Chief Scientists Office (SPHSU17). CB declares funding from the Wellcome Trust (221266/Z/20/Z). AP declares funding from the Wellcome Trust (205412/Z/16/Z). SVK acknowledges funding from the European Research Council (949582). This work also received support from Population Health Improvement UK (PHI-UK), a national research network that seeks to transform health and reduce inequalities through change at the population level. UK Research and Innovation (UKRI) funding for the PHI-UK Policy Modelling for Health theme(s) is gratefully acknowledged [grant reference MR/Y030656/1].

Map disclaimer The depiction of boundaries on this map does not imply the expression of any opinion whatsoever on the part of BMJ (or any member of its group) concerning the legal status of any country, territory, jurisdiction or area or of its authorities. This map is provided without any warranty of any kind, either express or implied.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

• Open access
• Published: 26 August 2024

The impact of surge capacity enhancement training for nursing managers on hospital disaster preparedness and response: an action research study

• Alireza Shafiei 1 ,
• Narges Arsalani 2 ,
• Mehdi Beyrami Jam 3 &
• Hamid Reza Khankeh   ORCID: orcid.org/0000-0002-9532-5646 4  

BMC Emergency Medicine volume  24 , Article number:  153 ( 2024 ) Cite this article

Introduction

Hospitals as the main providers of healthcare services play an essential role in the management of disasters and emergencies. Nurses are one of the important and influential elements in increasing the surge capacity of hospitals. Accordingly, the present study aimed to assess the effect of surge capacity enhancement training for nursing managers on hospital disaster preparedness and response.

All nursing managers employed at Motahari Hospital in Tehran took part in this interventional pre- and post-test action research study. Ultimately, a total of 20 nursing managers were chosen through a census method and underwent training in hospital capacity fluctuations. The Iranian version of the “Hospital Emergency Response Checklist” was used to measure hospital disaster preparedness and response before and after the intervention.

The overall hospital disaster preparedness and response score was 184 (medium level) before the intervention and 216 (high level) after the intervention. The intervention was effective in improving the dimensions of hospital disaster preparedness, including “command and control”, “triage”, “human resources”, “communication”, “surge capacity”, “logistics and supply”, “safety and security”, and “recovery”, but had not much impact on the “continuity of essential services” component.

The research demonstrated that enhancing the disaster preparedness of hospitals can be achieved by training nursing managers using an action research approach. Encouraging their active participation in identifying deficiencies, problems, and weaknesses related to surge capacity, and promoting the adoption and implementation of suitable strategies, can enhance overall hospital disaster preparedness.

Peer Review reports

Hospitals, as the main providers of healthcare services, play an essential role in managing and reducing the suffering of injured people in emergencies and disasters [ 26 ]. Most of the definitive, life-saving and emergency care for injured people are carried out in hospitals. Therefore, the preparedness of hospitals is essential in moderating and decreasing the negative health consequences of disasters [ 29] . From an international perspective, the Sendai Framework for Disaster Risk Reduction 2015–2030 and World Health Organization (WHO), highlights the need for disaster preparedness and risk reduction measures in hospitals [ 30 , 31 ]. Based on WHO, the preparedness and well-trained hospital personnel is the main factor in minimizing the casualties and damages resulting from disasters. Therefore, assessing and improving hospitals’ capacity and preparedness for disasters is a crucial first step toward effective disaster response and achieving the objectives outlined in the Sendai Framework 2015–2030 [ 30 , 32 ].

In Iran, efforts to enhance hospitals’ disaster preparedness began in the winter of 2009 with the creation of the National Hospital Disaster Preparedness Plan (NHDPP) by the Health Research Center on Disasters at the University of Social Welfare and Rehabilitation Sciences. This initiative, serving as a national guideline, received backing from the Secretariat of the Disaster Health Working Group in the Ministry of Health and was communicated to all hospitals across the country [ 1 ]. Furthermore, in the third phase of Iran’s hospital accreditation program, criteria for disaster risk management were added in the form of seven standards and thirty-seven measurements, directly addressing the hospital’s preparedness and response to emergencies and disasters [ 2 ].

To effectively address disasters, a hospital needs a thorough preparedness strategy, necessary tools, equipment, sufficient space, skilled staff, and, in essence, enough surge capacity [ 33 ]. Surge capacity refers to the ability to acquire additional resources during a disaster or emergency. It is the ability to provide quickly the usual functions beyond the increased demand for experienced staff, medical care, and social health services. Surge capacity has three core components including staff, stuff, and structures [ 3 ].

Nurses are one of the major groups of healthcare providers in hospitals(staff) [ 4 ]. They have the most contact with patients and provide the most care [ 5 ]. Along with other disaster management teams, they also play crucial roles in planning, education and training, response, and recovery for hospital disaster preparedness [ 6 , 7 ].

Experiences have shown that training and exercises before the occurrence of disasters can significantly increase the ability of people to face critical situations such as natural disasters [ 4 , 6 ]. Therefore, providing effective disaster training for nurses has a crucial role in increasing hospital preparedness and capacity for response to disasters. Previous studies have demonstrated inadequate training for nurses on preparedness and response to emergencies and disasters [ 2 , 4 , 5 , 6 ]. Moreover, despite numerous investigations assessing the preparedness of Iranian hospitals for disasters [ 8 , 9 , 10 ], to the best of our knowledge, only a limited number of interventional studies have explored the impact of disaster training for nurses on enhancing hospital disaster preparedness in Iran. Hence, recognizing the crucial contributions of nurses to the development of hospital capacity, this research aimed to examine the effects of training of surge capacity enhancement for the nursing managers on the emergency and disaster preparedness of Motahari Hospital in Iran.

Study design and settings

The current investigation utilized a pretest-posttest interventional design, conducted at Shahid Motahari Burn Hospital, affiliated with Iran University of Medical Sciences in Tehran, Iran. This hospital is the first and only main and specialized center providing medical services to burn patients in the center of the country and plays an essential role in the management of the injured during disasters and emergencies, especially fires.

Population and sampling

Aligned with the study’s goals, we employed a census sampling method to select all nursing managers at Shahid Motahari Hospital in Tehran. The eligibility criteria encompassed individuals within the nursing profession, such as nursing managers, supervisors, and head nurses, who held a minimum of a bachelor’s degree and possessed a minimum of one year of managerial experience. Those who expressed unwillingness to participate in the study were excluded.

The data was collected using the Persian version of the Hospital Emergency Response Checklist developed by Khankeh et al. (2013) [ 34 ]. The checklist was used to estimate the current state of preparedness of hospitals and healthcare centers. The original version of this tool was formulated by the World Health Organization [ 35 ]. The checklist measures 9 key components including command and control (7 items), triage (10 items), human resources (15 items), communication (9 items), surge capacity (13 items), logistics and supply management (10 items), safety and security (10 items), continuity of essential services (8 items) and post-disaster recovery (8 items). The reliability and validity of the Persian version of the tool have been confirmed by Karimian et al. (2013) [ 14 ]. They confirmed the validity of the tool (CVI = 0.86) and its reliability with Cronbach’s alpha of 0.83. The items in the checklist are rated on a 3-point scale (1 = due for review, 2 = in progress, and 3 = completed).

Moreover, the hospital surge capacity guideline was used to examine the current situation, weaknesses, problems, and target actions and develop a hospital surge capacity training program. This guidance was formulated by the Health in Emergency and Disaster Research Center at the University of Social Welfare and Rehabilitation Sciences and approved and disseminated by the Iranian Ministry of Health [ 34 ].

Intervention

This intervention study adopted a participatory action research approach as the participants were involved in problem identification and intervention to improve the process. Research in action is a type of study used by people to change unfavorable situations into relatively favorable situations and finally improve procedures in their workplace [ 11 ]. Action research is a type of study that attempts to learn and understand purposeful interventions meant to bring about desired changes in the organizational environment [ 12 ]. Action research simultaneously promotes problem-solving and expands scientific knowledge, as well as strengthens the skills of research participants [ 13 ].

In general, in action research, participants are involved in all stages of the research, from identifying the problem and collecting the data to planning, implementation, and evaluation. The engagement of participants in all stages of the research will encourage their participation in the research procedure and make them interested in the research topic [ 7 ].

This study adopted Streubert Speziale and Carpenter’s five-step action research method [ 7 ]. These steps include (1) defining the problem (explaining the current situation), (2) collecting, analyzing, and interpreting data, (3) planning, (4) implementing, and (5) evaluating. In this research, nurses actively engaged in elucidating the issue, gathering and analyzing data related to hospital surge capacity, devising and executing capacity-enhancing strategies based on their training, and assessing these measures to enhance hospital disaster preparedness and response.

To collect the data, the required permits were obtained from the hospital managers and officials. Besides, some instructions about the research procedure and data gathering were provided in a briefing session for the participants. The researcher and the participants made the required arrangements and plans for conducting the training intervention. In the next step, the items on the instruments (the Hospital Emergency Response Checklist) were completed by the participants(pre-test). When completing the checklist, the officials and managers of the hospital were also interviewed to better identify the problems and challenges related to the surge capacity. After that, topics and concepts related to increasing surge capacity and hospital disaster preparedness were taught to the participants during a two-day workshop, and they did round table exercises. Following the National Hospital Emergency Preparedness and Response Instructions [ 1 ], the content of the workshop included hospital risk and hazard assessment, incident command system, early warning system, response plan, and enhancing hospital capacity in response to emergencies and disasters with emphasis on solving problems and weaknesses identified in the pre-intervention stage. After completing the training workshop, the participants were given a six-month opportunity to carry out interventions and transfer the training to other staff and nurses. During this period, the participants and other members of the disaster risk management committee attended meetings held every two weeks. In these meetings, the necessary actions for the next two weeks were set, and the officials to manage each action were specified. In addition, in each meeting, the extent to which the goals of the previous meeting were achieved and the reasons for not fulfilling them were discussed. Finally, the items in the Hospital Emergency Response Checklist were completed for the second time (post-test) and the collected data was analyzed.

Ethical considerations

To comply with ethical protocols, this research project was approved with the code of ethics of the Ethics Committee of the University of Rehabilitation Sciences and Social Health. Moreover, informed consent was obtained from all the participants. The participants completed the checklists anonymously and, they were assured that their participation was voluntary and had no impact on their evaluation procedure.

The participants in this study were 20 nursing managers and supervisors at Motahari Burn Hospital in Iran. The study participants had an average age of 38 years (30 to 52 years old) and an average work experience of 16 years (4 to 25 years). Most of the participants were female (15 persons), married (18 persons), had a bachelor’s degree (12 persons), and had served in managerial positions (9 persons). Table No. 1 Shows other demographic characteristics of the participants. The surge capacity enhancement strategies that were recognized and put into practice by the participants throughout the study(6 months) included: 1- Executing a memorandum with retired personnel and reactivating them when necessary, Executing a memorandum with the Iran University of Medical Sciences to hire students if needed, drafting instructions for requesting staff from the relevant authorities such as the Emergency Operations Center (EOC) of the Ministry of Health, in the realm of enhancing “staff” capacity. 2- Preparing and reserving medications and essential equipment for a minimum duration of 72 h, signing a memorandum with other hospitals and nearby health centers to provide equipment in emergencies, and also creating more water storage volume to be used in emergencies and disasters, in the realm of enhancing “stuff” capacity. 3- Identifying suitable non-clinical and clinical spaces in the Motahhari Hospital to place beds and admit patients during disasters and emergencies, concluding an agreement with a school near the hospital to provide physical space for the hospital, creating a new rehabilitation department in the hospital, enlarging the space of the emergency department in the realm of increasing “space” capacity. And, 4- developing plans and instructions necessary to manage the risk of emergencies and disasters, doing training and practice in the hospital, in the realm of enhancing “system” capacity. The data showed that hospital disaster preparedness was at an average level (184) before the intervention and reached the optimal level (216) after the intervention. Also, the results also demonstrated that, except for “continuity of essential services”, the intervention improved the hospital’s disaster preparedness score across all dimensions. Most notably, the intervention enhanced “surge capacity” by 10 units and “staff” by 6 units. For detailed information on the intervention’s effects on hospital preparedness dimensions, please refer to Table No. 2 .

This study aimed to examine how providing action research training to nursing managers enhances surge capacity and contributes to improving hospital disaster preparedness. Many hospitals may face numerous challenges due to inadequate preparedness in the face of disasters and the increased demand for healthcare services [ 36 , 37 ]. The results of this study indicated that implementing the surge capacity enhancement intervention for nursing managers and officials led to a 32-unit improvement in disaster preparedness at Motahari Hospital. This improvement was expected because surge capacity is one of the most important components of hospital disaster preparedness and response.

Regarding the impact of the intervention on enhancing hospital disaster preparedness, various studies have been conducted in Iran, each employing distinct approaches to bolster preparedness.

In a study conducted by Karimiyan et al. (2013), it was found that hospital preparedness training aligned with the national plan significantly enhanced the hospital’s preparedness to address emergencies and disasters [ 14 ]. Delshad et al. (2015) showed early warning system training improved the preparedness of Motahari Hospital in emergencies and disasters [ 15 ]. Also, Salawati et al. (2014) in another study, examined the effect of teaching and applying non-structural hospital safety principles for nurses on the preparedness of medical departments of several private and public hospitals in Tehran during disasters [ 16 ]. The findings indicated that the safety score of two non-structural and functional parts of the hospital safety index increased after the intervention. The authors concluded that teaching and applying non-structural safety principles to nurses improves hospital safety and preparedness [ 16 ].

Like numerous other hospitals in Iran [ 17 , 18 , 19 ], Motahari Hospital’s disaster preparedness status was assessed as moderate before the intervention. Nevertheless, some studies have indicated inadequacies in the preparedness level of the examined hospital. For example, both the investigation conducted by Hekmatkhah et al. [ 20 ] and that of Ojaghi et al. [ 21 ] revealed insufficient preparedness in the hospitals under examination.

The current study demonstrated that enhancing the hospital’s response capacity and hospital’s disaster preparedness across various components can be achieved through capacity-building training for nursing managers through action research. The greatest effect of the intervention in this study was on “surge capacity” and the “human resource” dimension(staff). This outcome can be primarily attributed to instructing the hospital surge capacity-building principles for participants in the training workshop. Additionally, due to steps were taken to augment capacity in terms of “human resources”, “medication, and equipment”. Two studies conducted in Iran have identified a shortage of human resources and equipment as a primary factor contributing to the limited preparedness of hospitals in dealing with disasters [ 22 , 23 ]. In this research, the re-employment of retired employees and the use of university students were among the most important strategies that were adopted to increase the hospital capacity and preparedness in the human resource dimension. Similarly, Dowlati et al. (2021) reported that the preparation of a list of employers from other hospitals and medical centers, including clinics and health students, is one of the most important strategies to increase the capacity of hospital staff to respond to chemical, biological, and nuclear hazards and disasters [ 38 ].

The results of this study show that the intervention improved the hospital preparedness scores in the “triage” and “command and control” dimensions. In this context, the educational intervention on triage by Rahmati and colleagues enhances the preparedness of the emergency department, as highlighted in their study [ 24 ]. Also, Delshad et al. conducted a study where actions such as designating an external location for triage and formulating a strategy for the postponement of elective surgeries contributed to an improvement in the hospital preparedness score [ 15 ].

The results of this study emphasize that enhancing hospital preparedness can be achieved through conducting a needs assessment, recognizing gaps within the organization as identified by study participants, and effectively communicating and raising awareness among hospital managers. In this context, Karimian et al. (2013) underscored the importance of providing additional training for officials, managers, and hospital staff concerning emergency preparedness and response in hospitals [ 14 ].

The data in the present study indicated the intervention had a smaller impact on the components of “continuity of essential services”, “logistics and supply”, and “safety and security” compared to other components of hospital preparedness. Perhaps one of the main reasons was the restricted timeframe of the study and limited financial resources to carry out capacity-building and preparedness measures in these dimensions. As stated earlier, measures to increase the surge capacity and improve preparedness were formulated and followed up during the meetings of the emergency and disaster risk committees. Since these meetings were held every two weeks, the 6-month timeframe of the study did not leave an opportunity to carry out measures to improve the mentioned components. Furthermore, the limited financial resources can be considered one of the main reasons for not carrying out the actions planned by the committee. The findings of the “logistics” and “essential services” are consistent with the findings of the study by Ingrassia et al. (2016). This study showed that hospital preparedness in these dimensions was poor [ 25 ]. The findings concerning the " logistics and supply” as well as the “countiniuty of essential services “dimensions in this research align with the outcomes observed in Ingrassia et al.‘s (2016) study, highlighting the inadequate preparedness of the hospital in these aspects [ 25 ].

Limitations

The study was constrained by a limited duration of 6 months and insufficient financial resources, restricting the ability to implement further measures to enhance hospital preparedness. Future investigations could overcome these limitations by extending the study period to at least one year and ensuring adequate financial resources. Furthermore, as this study solely assessed the impact of the intervention on the disaster preparedness level of a single hospital, statistical analysis could not be conducted due to the absence of mean and standard deviation data. The alterations were solely presented descriptively.

This study examined the effect of surge capacity training using an action research plan on disaster preparedness and response at Shahid Motahari Hospital in Tehran. The results showed that surge capacity enhancement training for nursing managers and officials increased their sensitivity to the importance of hospital emergency preparedness and response. Furthermore, their proactive involvement in recognizing capacities, deficiencies, problems, and weaknesses with appropriate tools and taking measures to address them can improve hospital emergency preparedness and response. The findings indicated that senior managers within the hospital can instigate changes through the provision of financial backing and the implementation of mandatory protocols.

Data availability

The datasets that were used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

HamidReza Khankeh. Hospital Disaster preparedness: National Program. Iran: Publications of University of Welfare and Rehabilitation Sciences; 2013.

Google Scholar  

Abbasabadi M, Khankeh H, Mosadegh Rad AM. Designing of disaster risk management accreditation standards in iranians hospitals. Int J Qual Health Care. 2018;30(suppl2):45–6.

Article   Google Scholar  

Kaji A, Koenig KL, Bey T. Surge capacity for healthcare systems: a conceptual framework. Acad Emerg Med. 2006;13(11):1157–9.

Article   PubMed   Google Scholar  

Al Harthi M, Al Thobaity A, Al Ahmari W, Almalki M. Challenges for nurses in disaster management: a scoping review. Risk Manage Healthc Policy. 2020:2627–34.

Imani Eea. Awareness of nurses about crisis management and related factors. J Health Care. 2011;13(4):10–8.

Al Thobaity A, Plummer V, Williams B. What are the most common domains of the core competencies of disaster nursing? A scoping review. Int Emerg Nurs. 2017;31:64–71.

Al Thobaity A, Alamri S, Plummer V, et al. Exploring the necessary disaster plan components in Saudi Arabian hospitals. Int J Disaster Risk Reduct. 2019;41:101–316.

Janati A, Sadeghi-Bazargani H, Hasanpoor E, Sokhanvar M, HaghGoshyie E, Salehi A. Emergency response of Iranian hospitals against disasters: a practical framework for improvement. Disaster Med Public Health Prep. 2018;12:166–71.

Hosseini S, Izadfar A, Rajabzadeh R, Sadeghi M, Sadeghi A. Assessing the levels of preparedness against disaster of the hospitals in North Khorasan; Iran, in 2016. J Biol Todays World. 2017;6:191–5.

Naser WN, Ingrassia PL, Aladhrae S, Abdulraheem WA. A study of hospital disaster preparedness in South Yemen. Prehosp Disaster Med. 2018;33:133–8.

UNDERSTANDING PARTICIPATORY ACTION RESEARCH: CM. A QUALITATIVE RESEARCH METHODOLOGY OPTION. Can J Action Res. 2012;13(2):34–50.

Braa K, Vidgen RT. Interpretation, intervention and reduction in the organizational laboratory: a framework for in-context information systems research. Inf Organ. 1999;9(1):25–47.

Hult M, Lennung S. Towards a definition of action research: a note and bibliography. J Manage Stud. 1980;17:241–50.

Karimiyan A, Khankeh H, Dalvandi A. Effect of hospital preparedness principles based on a national plan on readiness to respond to disasters in Shahid Motahari Tehran, 2013. Tehran: university of social welfare and rehabilitation sciences; 2013.

Delshad Ve. Early warning system and disaster preparedness in Motahari Hospital. J Res Dev Nurs Midwifery. 2015;12(2):51–8.

Salevaty J, Khankeh HR, Dalvandi A, Delshad V. The impact of nurses training and applying functional and nonstructural hospital safety in preparedness of Razi and day hospitals in disasters based on Hospital Safety Index. 2015.

Bazyar J, Pourvakhshoori N, Safarpour H, Farrokhi M, Khankeh HR, Daliri S, et al. Hospital disaster preparedness in Iran: a systematic review and meta-analysis. Iran J Public Health. 2020;49(5):837.

PubMed   PubMed Central   Google Scholar  

Kazemzadeh M, Shafiei E, Jahangiri K, Yousefi K, Sahebi A. The preparedness of hospital emergency departments for responding to disasters in Iran; a systematic review and meta-analysis. Archives of Academic Emergency Medicine. 2019;7(1).

Amiri Me. Preparedness of Affiliated hospitals of universities in North of Iran to confront disasters. Hosp J. 2013;12(1):19–28.

Hekmatkhah A, Rahimi H, Kamali Aghdam M, Taghavi Shahri M, Sadeghifar J, Hamouzadeh P. Assessing the preparedness rate againest earthquake risk in hospitals affiliated to urmia university of medical sciences 2011. J Urmia Nurs Midwifery Fac. 2012;10(2):0.

Ojaghi S, Nourizadeh S, MAHBOUBI M, Khazaei A, NAJAFI GA. Disaster crisis handling preparedness level of hospitals in Kermanshah. 2009.

Mahdaviazad H, Abdolahifar GR. Assessing hospital disaster preparedness in Shiraz, Iran 2011: teaching versus private hospitals. Am J Disaster Med. 2013;8(1):65–73.

Djalali A, Castren M, Khankeh H, Gryth D, Radestad M, Öhlen G, et al. Hospital disaster preparedness as measured by functional capacity: a comparison between Iran and Sweden. Prehosp Disaster Med. 2013;28(5):454–61.

Rahmati H, Azmoon M, Kalantari Meibodi M, Zare N. Effects of triage education on knowledge, practice and qualitative index of emergency room staff: a quasi-interventional study. Bull Emerg Trauma. 2013;1:69–75.

Ingrassia PL, Mangini M, Azzaretto M, Ciaramitaro I, Costa L, Burkle JF, et al. Hospital Disaster preparedness in Italy: a preliminary study utilizing the World Health Organization Hospital Emergency Response Evaluation Toolkit. Minerva Anestesiol. 2016;82(12):1259–66.

PubMed   Google Scholar  

Ardalan A, Kandi M, Talebian MT, Khankeh H, Masoumi G, Mohammadi R, Maleknia S, Miadfar J, Mobini A, Mehranamin S. Hospitals safety from disasters in IR iran: the results from assessment of 224 hospitals. PLoS currents. 2014 Feb 28;6.

Asefzadeh S, Rajaee R, Ghamari F, Kalhor R, Gholami S. Preparedness of Iranian hospitals against disasters. Biotechnology and Health Sciences. 2016;3(3):1-6.

Khankeh He. Management of health care in the event of natural disasters. Quarterly rehabilitation. 2006;7(2).

Krajewski MJ, Sztajnkrycer M, Baez AA. Hospital disaster preparedness in the United States: new issues, new challenges. The internet journal of Rescue and Disaster Medicine. 2005;4(2):22-5.

UNISDR U, editor Sendai framework for disaster risk reduction 2015–2030. Proceedings of the 3rd United Nations World Conference on DRR, Sendai, Japan; 2015.

Hospital Safety Index: Guide for Evaluators. Geneva, Switzerland: World Health Organization; 2008. Available at: http://www.paho.org/english/dd/ ped/SafeHosEvaluatorGuideEng.pdf, Accessed April 2014.

Djalali A, Castren M, Khankeh H, Gryth D, Radestad M, Öhlen G, et al. Hospital disaster preparedness as measured by functional capacity: a comparison between Iran and Sweden. Prehospital and disaster medicine. 2013;28(5):454-61.

Vahedparast H, Ravanipour M, Hajinezhad F, Kamali F, Gharibi T, Bagherzadeh R. Assessing Hospital Disaster Preparedness of Bushehr province. Iranian South Medical Journal. 2013 Mar 1;16(1).

Khankeh H. Risk assessment tools and health capabilities indicators of specialized in disaster. University of Welfare and Rehabilitation Sciences: Publications of University of Welfare and Rehabilitation Sciences; 2013.

Sorensen B, Zane R, Wante B, Rao M, Bortolin M, Rockenschaub G. Hospital emergency response checklist: an all-hazards tool for hospital administrators and emergency managers. 2011. World Health Organ. 2013.

Shabanikiya H, Jafari M, Gorgi HA, Seyedin H, Rahimi A. Developing a practical toolkit for evaluating hospital preparedness for surge capacity in disasters. International journal of disaster risk reduction. 2019; 34:423-8.

Pourhosseini SS, Ardalan A, Mehrolhassani MH. Key aspects of providing healthcare services in the disaster response stage. Iranian journal of public health. 2015;44(1):111.

Dowlati M, Seyedin H, Moslehi S. Hospital preparedness measures for biological hazards: a systematic review and meta-synthesis. Disaster medicine and public health preparedness. 2021;15(6):790-803.

Download references

Acknowledgements

The authors would like to express their acknowledgments to the staff at the Department of Postgraduate Studies in the University of Social Welfare and Rehabilitation Sciences and appreciate the sincere cooperation of hospital managers, officials, and staff of Shahid Motahhari Hospital for their contributions to conducting this research project.

This study was conducted as part of a master’s thesis at the University of Social Welfare and Rehabilitation Sciences.

Author information

Authors and affiliations.

Department of Nursing, University of Welfare and Rehabilitation Sciences, Tehran, Iran

Alireza Shafiei

Iranian Research Center on Aging, Department of Nursing, University of Social Welfare and Rehabilitation Sciences, Tehran, Iran

Narges Arsalani

Department of Pre-Hospital Medical Emergencies, School of Paramedical, Qazvin University of Medical Sciences, Qazvin, Iran

Mehdi Beyrami Jam

Health in Emergency and Disaster Research Center, University of Social Welfare and Rehabilitation Sciences, Tehran, Iran

Hamid Reza Khankeh

You can also search for this author in PubMed   Google Scholar

Contributions

ASH, HKH design of the study, MB, ASH and NA collect and analysed the data and ASH, MB, HKH preparation of the manuscript.

Corresponding author

Correspondence to Hamid Reza Khankeh .

Ethics declarations

Ethics approval and consent to participate.

This study was approved by the University of Social Welfare and Rehabilitation Sciences(USWRS) Research Ethics Committees with the Code of Ethics USWR.REC.1392.93. Also, the institutional review board of USWRS approved all the methods and steps for this study. Hence, all procedures were conducted in compliance with the appropriate guidelines and regulations, and written informed consent was obtained from study participants. They were informed that their involvement in the research was entirely voluntary, and they had the freedom to withdraw from the study at any point if they chose to do so.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Shafiei, A., Arsalani, N., Beyrami Jam, M. et al. The impact of surge capacity enhancement training for nursing managers on hospital disaster preparedness and response: an action research study. BMC Emerg Med 24 , 153 (2024). https://doi.org/10.1186/s12873-024-00930-1

Download citation

Received : 07 October 2023

Accepted : 03 January 2024

Published : 26 August 2024

DOI : https://doi.org/10.1186/s12873-024-00930-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Disaster preparedness
• Surge capacity
• Action research

BMC Emergency Medicine

ISSN: 1471-227X

• General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 27 August 2024

Charting sustainable urban development through a systematic review of SDG11 research

• Abdulaziz I. Almulhim   ORCID: orcid.org/0000-0002-5384-7219 1 ,
• Ayyoob Sharifi   ORCID: orcid.org/0000-0002-8983-8613 2 ,
• Yusuf A. Aina   ORCID: orcid.org/0000-0002-0763-9865 3 ,
• Shakil Ahmad 4 ,
• Luca Mora 5 , 6 ,
• Walter Leal Filho 7 , 8 &
• Ismaila Rimi Abubakar   ORCID: orcid.org/0000-0002-7994-2302 9  

Nature Cities ( 2024 ) Cite this article

1716 Accesses

2 Altmetric

Metrics details

• Environmental studies
• Social policy

The Sustainable Development Goal (SDG) 11 underscores the imperative of creating inclusive, safe, resilient and sustainable cities and communities by 2030. Here we employ bibliometric techniques to assess the evolving landscape of SDG11 research. Using a comprehensive dataset of over 21,000 scholarly publications, we investigate publication trends, thematic focus areas, authorship patterns, keyword co-occurrences and citation networks related to SDG11 research. The results reveal a consistent increase in research output, reflecting the growing global interest in urban sustainability studies. We identify influential authors, organizations and countries shaping the research landscape, highlighting existing global collaborative networks and emerging research hubs. Core thematic areas emphasize critical topics and interdisciplinary connections. Citation networks underscore the impacts of disseminating research outputs, including seminal works. This study offers insights for policymakers, academics and practitioners to align their collective efforts toward sustainable, inclusive and climate-resilient urban development. Moreover, it advances SDG11 by noting opportunities for further research, knowledge dissemination and international collaboration.

Similar content being viewed by others

A big data approach to assess progress towards Sustainable Development Goals for cities of varying sizes

Examining the unsustainable relationship between SDG performance, ecological footprint and international spillovers

Sustainable Development Goals (SDGs): Are we successful in turning trade-offs into synergies?

The growing interest in sustainable urban development is driven by challenges posed by urbanization, socioeconomic activities and environmental issues 1 . Urban areas contribute 80% of the world’s gross domestic product 2 , but also account for around 75% of global resource consumption, 65% of energy use and over 70% of carbon emissions 3 . The ecological footprint of urban environments, which measures the resources required to sustain socioeconomic activities, has been increasing 4 , 5 , and the global urban extent is projected to double by 2030 6 . Similarly, the global urban population is projected to reach 68% by 2050 7 , which could surpass the capacity of most urban areas 8 . Africa and Asia will host most of the future urban populations despite housing and infrastructure inadequacies 7 . Rapid urbanization, poverty and climate change (CC) further intensify the vulnerability of urban dwellers 9 .

Sustainable urban development aims to balance economic production, environmental protection and social inclusiveness. It emerged as a response to the critique of modernist views that prioritized physical appearance and order in cities over context, equity and inclusion 6 . Due to the limited progress in achieving the Millennium Development Goals, the Sustainable Development Goals (SDGs) were established in 2015 to ensure that no country is left behind in achieving sustainable development by 2030 10 . Many of the SDGs are closely related to urban settings, where sustainability challenges are complex and interwoven 11 . SDG11 specifically focuses on urban challenges and aims to make ‘cities and human settlements inclusive, safe resilient and sustainable’ by reducing the negative effects of urban development while improving socioeconomic development 10 .

The importance of SDG11 stems from the principles of inclusive, safe and resilient city. An inclusive city is characterized by the idea that all individuals, irrespective of their economic status, gender, race, ethnicity or religion, have the ability and empowerment to actively engage in the social, economic and political opportunities available within urban environments 6 . It seeks to address environmental racism and promote inclusive and fair urban development through social justice and equitable distribution of environmental benefits and burdens. In such a city, everyone is afforded equal access and participation in the diverse aspects that cities provide. On the other hand, a safe city refers to a city that possesses the capacity to provide protection and security against potential dangers, harm or risks, while a resilient city denotes a city’s ability to recover and restore its fundamental functions and structures following natural disasters and crises caused by human activities 6 , 8 . SDG11 is significant because it aims to ensure that cities develop sustainably.

However, SDG11 has been criticized for its limited emphasis on urban inequalities, decentralization and funding for local authorities 6 . Other challenges include localizing the universal indicators 12 , governance issues 13 , data accessibility and comparability 14 and smart city development 12 , 15 . Nevertheless, SDG11 serves as a platform for directing and monitoring urban development, fostering socioeconomic development and ensuring equity, inclusion and environmental protection 16 . Therefore, it is crucial to assess the literature on progress toward SDG11 targets 10 , especially at the halfway point to the target year, to inform interventions necessary for their achievement 17 .

While SDG11 has attracted significant global research attention 18 , comprehensive reviews of SDG11 literature are limited. Existing studies have primarily focused on assessing all the SDGs 19 , 20 , which obscures specific challenges and makes it difficult to track progress or design targeted interventions for individual goals. Recent work has highlighted the insufficient achievement of the SDGs and the need for transformative governance and participatory approaches 21 . Other studies have underscored the gap between research and policies, the underutilization of cities as pivotal arenas for achieving SDGs 22 and the lack of indicators to measure progress toward implementing SDGs 15 . Some studies have assessed SDGs’ implementation in specific region 17 , their impacts on addressing risks 23 and crises 1 , and their implications for health and well-being 24 , environmental research 25 and private sector involvement 26 . Most of the SDG research emanates from developed countries, showing a gap in the coverage of developing countries 27 . The few SDG11 studies in the Global South have narrow focus. While one paper investigated the impact of SDG11 on forest-based livelihoods 28 , another study researched the challenges of SDG11 implementation using a single-country experience 6 . Therefore, an in-depth and broad review of SDG11 literature is necessary to bridge this knowledge gap and identify key challenges and opportunities as well as potential pathways for achieving the targets set in SDG11.

Therefore, this research aims to assess the SDG11 research trends and themes using a bibliometric technique. It is the first global and comprehensive scientometric study on the SDG11 domain. By focusing on research conducted since the formulation of the SDGs, the study addresses the following research questions: (1) what are the global trends in SDG11 research? (2) How has the thematic focus of SDG11 research evolved over time? (3) What are the challenges and priority areas for SDG11 research? The contributions of the study to theory and practice are to:

Identify significant thematic areas and trends in SDG11 research since the promulgation of the SDGs, which can inform researchers, policymakers and practitioners about the current state of knowledge within the field and highlight priority areas for SDG11 research.

Map research clusters, knowledge sharing and collaboration patterns, thereby providing insights into the dynamics of research networks and facilitating the formulation of strategies to foster research excellence, interdisciplinary and international collaborations and the effective allocating of research resources.

Underscore the knowledge gaps, emerging topics and challenges within SDG11 research, offering evidence-based insights to align urban development initiatives with SDG11 research frontiers, enhance the efficacy of interventions and contribute to the development of inclusive, safe, resilient and sustainable cities.

SDG11 research trends

Research on SDG11 has significantly grown in terms of annual publications and citations since 2016, indicating a rising interest in this field (Fig. 1 ). The number of publications has increased by 1.3-fold, and this upward trajectory is expected to continue. Notable emerging research areas include the institutionalization of SDGs within local and global settings 18 and the impact of smart cities on advancing the SDGs 12 , 15 . Previously, studies on the epistemology and challenges of urban population growth were prevalent 29 . However, SDG11 research has now evolved into multidisciplinary fields, driven by heightened attention to urban challenges such as CC, urbanization and population growth.

A total of 21,153 articles were published, receiving 229,182 citations. The number of publications rose from 9,238 in period 1 (2016–2019) to 11,915 in period 2 (2020–2022).

Source data

The increasing trend in SDG11 publications can be attributed to several factors, including the desire to improve institutional rankings, a supportive research environment, investments and endowments, faculty promotion requirements and advancements in information and communication technology. There are also socioeconomic factors, such as increasing urbanization rates and gross domestic product, urban expansion and transformation, a deeper understanding of urban dynamics and challenges. Additionally, the policy environments in different countries can influence academic interests and research in urban studies, shaping research priorities and collaborations. Other contributing factors include research challenges faced by low-income countries and research support by governments, the private sector, international development agencies and scholars, all focusing on sustainable urban development.

SDG11 research is further propelled by recent international summits and collaborations that highlight the urgency of protecting the ecosystem and ensuring human safety 1 . Since 2015, CC issues have received greater attention due to key factors. The adoption of the Paris Agreement raised awareness and urgency for action on CC, resulting in a greater focus on related issues in various sectors, including urban planning and policy 13 . Scientific consensus on CC impacts and the role of human activities has also strengthened over the years, with Intergovernmental Panel on Climate Change assessments emphasizing the significance of cities in addressing CC 23 . As a result, CC considerations are increasingly integrated into research, policy and planning processes.

Urban planning and development strategies have prioritized climate mitigation and adaptation measures, such as reducing greenhouse gas emissions, promoting renewable energy, enhancing resilience to extreme weather events and incorporating green infrastructure. The focus on CC has accelerated the transition toward low-carbon and resilient cities, with efforts directed toward sustainable transportation, energy-efficient buildings, green spaces and climate-responsive infrastructure 6 . Collaboration and international cooperation are essential in addressing climate change, with cities and countries sharing best practices, knowledge and resources to develop and implement climate action plans 24 . Initiatives such as the C40 Cities Climate Leadership Group facilitate knowledge exchange and collective action among cities 30 . The increased attention to CC signifies a shift toward more sustainable and resilient urban development, emphasizing the need for proactive measures to mitigate greenhouse gas emissions, adapt to climate risks and promote equitable and sustainable urban environments.

Thematic focus of SDG11 research

There is an imbalance in the attention given to research themes within SDG11 as revealed by co-occurrence map (Supplementary Fig. 1 ). The dominant themes are affordable housing (SDG11.1), urban transport (SDG11.2), policy and governance (SDG11.3) and access to public spaces (SDG11.7). Housing affordability issues have consistently remained a focal point in SDG11 research, with urban studies, policy development and community-driven efforts for finding solutions to these complex challenges 30 , 31 . These issues were highlighted in Habitat I (Vancouver, 1976), which emphasized the importance of shifting governance and planning paradigms to develop policies and strategies to address rapid urbanization challenges, including shelter shortages and urban inequalities, and promote affordable housing options 30 , 32 . Habitat I has laid the foundation for subsequent global efforts and policy frameworks, such as Habitat II (Istanbul, 1996) and the New Urban Agenda, which continue to prioritize housing as a pivotal component of sustainable urban development. The persistent focus on affordable housing shows that cities still face many challenges in providing adequate housing for all 30 .

Urban policy and governance are other significant terms, indicating scholarly focus on strategies for promoting inclusive and sustainable urban development, enhancing participatory, integrated and sustainable urban planning and management. However, many cities lack the capacity to address urban inequalities, provide adequate housing 31 , public spaces and other urban services, which disproportionately affect women and racial minorities 30 . Moreover, urban redevelopment practices that lead to gentrification exacerbate existing inequalities 32 . Governance-based approaches seek to improve collaboration between public agencies and civil society to prioritize the implementation of urban planning strategies that enhance livability standards while addressing challenges such as CC and sustainability 30 .

Urban transport, which is related to SDG11.2 aiming to ensure safe, affordable, accessible and sustainable transport systems for all, has emerged as a key research theme. Important issues related to mobility, transportation and urban form include increased automobile dependence amid growing urbanization and suburbanization, challenges faced by public transit systems, growing awareness of environmental concerns, shift toward sustainable and multimodal transportation, transit-oriented development, integration of technology in transportation systems and the relationship between transportation and urban densification, compact development, CC adaptation and resilience, equity and social inclusion, and shifts in policy and governance approaches 1 , 6 , 11 . This theme also emphasizes the importance of walkability, public transit infrastructure and their role in enhancing transportation accessibility and influencing mode choice 33 . The transportation cluster also suggests that improving accessibility through urban form and built environment interventions can impact the travel behavior of urban residents and offer cobenefits for human health and environmental sustainability 24 . Incorporating such cobenefits in SDG11.2 could provide more incentives for access to safe efficient, equitable and sustainable transport infrastructure and systems in cities.

The implications of urbanization and land-use changes for sustainability, resilience and CC adaptation and mitigation in cities are also major themes. SDG11.6 aims to reduce the environmental impacts of cities, particularly in relation to air pollution and waste. The literature suggests that regulating urban growth 6 , controlling land-use changes, conserving biodiversity 27 and promoting green infrastructure are essential for achieving this target 34 . These actions, when implemented within integrated planning frameworks, can also reduce vulnerability, enhance resilience and contribute to progress in CC adaptation and mitigation, as emphasized in SDG11.5 (ref. 6 ). Such integrated frameworks should recognize the interconnections between various urban systems, including water, food, energy, waste and transportation, to promote sustainable and resilient urban development 35 . Cities are adopting strategies to reduce their carbon footprint, enhance energy efficiency and prepare for climate risks.

Smart cities and innovation enabled by information and communication technologies have increasingly been utilized to tackle urban development challenges and facilitate innovative and transformative urban governance mechanisms that contribute to the SDGs 15 . The rapid development and integration of digital technologies, such as the Internet of Things, artificial intelligence, big data analytics and sensor networks, have opened new possibilities for improving urban services, infrastructure and quality of life 33 . Smart cities leverage these technologies to enhance efficiency, connectivity and sustainability. The interest in smart cities stems from the recognition that technology can play a transformative role in addressing urban challenges, improving quality of life, promoting sustainability and fostering economic growth 12 , 36 . However, it is important to ensure that smart city initiatives are inclusive, equitable and responsive to the needs and aspirations of all residents.

Comparing the co-occurrence maps of period 1 and period 2 reveals limited changes in key thematic areas, despite the emergence of the coronavirus disease 2019 (COVID-19) pandemic during period 2 (Fig. 2 ). The key thematic areas in period 2, including urban governance and policy, transportation, urban sustainability and resilience, and urbanization and urban growth, remain consistent with period 1, indicating the continued relevance of these topics in research, albeit with potential expansions. However, a closer analysis of the clusters reveals that COVID-19 has emerged as a new area of SDG11 research in period 2, as attention has shifted toward adapting to the pandemic’s detrimental effects on cities. The pandemic has triggered paradigm shifts in various SDG11 domains, including public health, remote work, digitalization, vulnerabilities, inequalities, resilience, sustainability, urban spaces, proximity-based planning approaches such as the 15-minute city and global cooperation 9 . These shifts have influenced work, health, social equity, environmental stewardship 2 and urban planning, shaping innovative approaches and priorities in the postpandemic world. Urban inequality terms, such as slums and informality, inadequate housing and poverty, are brought to the forefront by the pandemic. Controlling the pandemic and addressing the citizen demand in slums and informal settlements has received significant attention 37 , 38 , 39 , 40 . Mobility restrictions and lockdowns to curb the virus’s transmission have presented challenges for service accessibility, particularly in disadvantaged neighborhoods where vulnerable groups reside. Lastly, the connection between sustainability and resilience has strengthened in the postpandemic period. The pandemic has offered new insights into the susceptibility of cities to various stressors and highlighted the inseparable connections between urban resilience and SDG11 (ref. 28 ).

a , b , The key thematic areas in period 1 (2016–2019) ( a ) are urban governance and policy (red), transportation (blue), urban sustainability and resilience (green), and urbanization and urban growth (yellow), while period 2 (2020–2022) ( b ) primarily focuses on urban governance and policies (red), urban studies (red), transportation (blue) and urbanization (green), particularly after the pandemic.

However, three SDG11 targets are not well-represented in both periods. One such target is SDG11.4, which aims to enhance efforts in preserving and conserving natural heritage, vital for improving urban sustainability 41 . Another target, SDG11.a, which focuses on strengthening urban–rural linkages, is also not adequately reflected in Fig. 2 . The intrinsic connection between cities and their surrounding rural areas necessitates the incorporation and strengthening of ties between urban and rural regions to achieve SDG11 (ref. 6 ). Gaps related to rural–urban linkages include limited understanding of interdependencies, inadequate infrastructure and services in rural areas, weak governance and coordination mechanisms, and social and cultural disconnect 13 . These gaps hinder the development of integrated strategies, contribute to economic disparities, limit access to services, impact agricultural productivity and food security, and create environmental and social challenges. Lastly, there is a lack of research on SDG11.c, which aims to support least-developed nations in developing safe and resilient urban areas, which is not surprising as these countries are often underrepresented in urban studies research 30 .

Major contributors to SDG11 research

Various countries, institutions, journals and authors have contributed to SDG11 research between 2016 and 2022. China leads in terms of the number of publications and citations generated, followed by the United States and the United Kingdom (Supplementary Fig. 2 and Supplementary Table 1 ). Among the top 20 productive countries, 14 are from the Global North countries, with South Africa and Brazil as the sole representative of Africa and Latin America and the Caribbean, respectively (Supplementary Fig. 3 and Supplementary Table 2 ). Increasing research collaboration among the top countries (Fig. 3 ), research infrastructure and facilities, manpower and financial support significantly contribute to their high SDG11 research output.

China followed by the United States and the United Kingdom dominates SDG11 research collaborations. There are significant connections among European, North American and Asian institutions, while Africa is less connected with Asia and Latin America and the Caribbean. Freq, frequently.

A co-citation analysis (Supplementary Table 3 ) reveals that Chinese institutions, such as the Chinese Academy of Sciences, have the highest number of articles and citation counts, followed by University College London and the University of Melbourne. The leading affiliations have changed over time, highlighting the strengthening of research institutes and the correlation between research collaboration and societal impacts (Supplementary Table 4 ). In terms of influential journals for SDG11 research, ‘land’ followed by ‘cities and land use’ policy tops the list (Supplementary Tables 5 and 6 ), with a growing interest in fields related to smart and sustainable cities, transport policies, regional planning and environmentally conscious building practices (Supplementary Fig. 4 ). These journals also address multiple issues related to environmental concerns, technological advancements, economic benefits, quality of life, justice and public awareness, driving the development of smart and sustainable cities.

The 15 most published authors in both periods focused on urbanization and urban growth, and the implementation, challenges and achievements of SDG11 (Supplementary Fig 5 ). This indicates an increased recognition of the SDG11 targets and their implementation over time, with the contributions of these authors significantly increasing from 2002 to 2016. Supplementary Table 7 shows that Chinese authors dominate the SDG11 publications, which correlates with China’s lead in institutions, affiliations and collaborations related to SDG11 research. The most cited SDG11 articles are revealed in Supplementary Table 8 , while the prominent authors that influenced SDG11 research are reported in Supplementary Table 9 . The top cited papers by SDG11 research are presented in Supplementary Tables 10 and 11 .

Key facts from the bibliometric analysis

The research on SDG11 has gained significant prominence across various fields, including urban studies, environmental sciences, geography, transportation and urban governance (Supplementary Table 12 ). The increasing environmental concerns, urbanization and global economic growth have spurred academic interest in SDG11 research from disciplines such as human geography, transportation, forestry, CC and sustainability science (Supplementary Table 13 ). Key thematic areas within SDG11 research encompass urban governance, affordable housing, transportation, urban sustainability and resilience, smart cities, urbanization and urban growth, which align closely with SDG11 targets 18 , 20 , 42 , 43 . However, research focus on SDG11 has remained relatively stable, with limited attention given to urban inequalities, safeguarding cultural and natural heritage 41 and specific impacts of the COVID-19 pandemic on urban sustainability.

This study reveals a notable increase in the total SDG11 research output from 2016 to 2022, reflecting the growing emphasis on SDG11 research in recent years compared with earlier periods. China emerges as the leaders in terms of research outputs, citations, authors, institutions and collaborations, closely followed by the United States and the United Kingdom. These three countries contribute 47.71% of SDG11 research productivity within this period, which is higher than 31% reported in a previous similar study 28 .

The dominance of Global North countries in the top 20 countries with the highest number of publications and citations related to SDG11 research is expected given their strong institutional capacity, research funding, highly ranked universities and collaborations. China’s surge in publications on SDG11 can be attributed to rapid urbanization, economic growth, government support and active international collaborations 2 , 11 . Generally, the landscape of research on SDG11 demonstrates an Anglo–American hegemony, which may reinforce power asymmetries and have significant implications for sustainability and resilience 30 . It is concerning that while projections indicate that 90% of future urban population growth will occur in cities of the Global South, particularly Africa and Asia, there is limited research on urban development challenges in these regions 7 .

The debate about the politics of knowledge production in SDG11 research often revolves around the controls of knowledge production processes. Large, well-funded institutions in developed countries tend to dominate research agendas, focusing on themes and solutions relevant to their own contexts, overlooking the unique needs and challenges of the Global South, which perpetuate existing inequalities and privileging certain types of knowledge. Also, knowledge production involves recognizing and integrating diverse ways of knowing. While Western scientific paradigms have traditionally dominated SDG11 research, there is an increasing recognition of the importance of indigenous and non-Western knowledge systems. Integrating these diverse epistemologies enriches understanding and leads to more effective and culturally relevant solutions.

Additionally, SDG11 research is inherently interdisciplinary, involving fields such as urban planning, sociology, environmental science and public policy. However, interdisciplinary collaboration can be challenging due to differing terminologies, methodologies and research priorities. Navigating these differences becomes crucial in the politics of knowledge production to create cohesive and comprehensive research outputs. Finally, bridging the gap between knowledge production and its implementation faces political, economic and social barriers. Researchers and practitioners are increasingly considering how knowledge on urban sustainability can effectively influence policymaking and practice in diverse urban contexts. Mobilizing knowledge to address these barriers becomes a key consideration in the politics of knowledge production.

Challenges to achieving SDG11

There are several challenges to achieving SDG11 targets, including inadequate provision of affordable housing 31 , essential services 24 , green spaces 2 , 34 , efficient transportation 33 and conservation of cultural and natural assets 25 . Rapid urbanization 1 , 7 , CC impacts 44 , insufficient investment in public infrastructure 30 , poor governance 13 and widening livelihood, land and resources inequalities 43 further exacerbate these challenges. For example, rapid urbanization puts immense pressure on housing, infrastructure, services and resources, making it challenging to effectively manage urban growth and ensure sustainable urban development 11 . Inadequate urban planning and land-use policies lead to inefficient land utilization, urban sprawl and inadequate provision of basic services 7 , 21 . The existence of slums and informal settlements where a large portion of the urban dwellers live in substandard housing conditions without tenure security 14 and limited access to electricity, water, sanitation, education, healthcare and employment opportunities 23 , 37 , and marginalized and vulnerable populations facing social exclusion, add to the complexity.

Moreover, competing priorities and trade-offs, lack of integration among various urban sectors and agencies 35 , inadequate human, technical and material resources at local government levels 45 , and insufficient local indicators and methods for implementation and monitoring 46 often hamper the implementation of SDG11 targets. Additionally, limited awareness of SDG-related challenges for policy formulation and implementation hinders context-depended decision-making and targeted interventions 21 , 27 . Addressing social inequalities, ensuring inclusivity in urban development and synergy among multiple fields, including social, technical, environmental, policy and management are crucial for achieving SDG11 (refs. 14 , 26 , 46 ). A valuable lesson can be learned from the success of the framework for assessing the implementation of SDG11 targets at the local level in Japan 42 .

Conclusions

This study aims to enhance our understanding of urban sustainability and provide insights for future research, policies and actions needed to achieve SDG11 targets. By conducting a comprehensive bibliometric assessment of over 21,000 publications from 2016 to 2022, it significantly contributes to the existing body of knowledge, highlighting trends, thematic areas and knowledge gaps related to SDG11 research across countries, institutions, authors and journals. SDG11 research has evolved into a multidisciplinary field, encompassing diverse themes, such as transportation, housing, urban sustainability, smart cities, urbanization and urban governance and policy. However, there is a need to address the gaps in research on urban safety and inclusion, which are critical dimensions often overlooked in favor of environmental and economic aspects of sustainability. This imbalance in research thematic areas risks perpetuation of already existing disparities within SDG11 research and its goals.

China, the United States and the United Kingdom emerge as the top contributors to SDG11 research and collaboration. To foster more SDG11 research in low-income economies, it is essential to provide increased funding support, capacity building and training for scholars, promote collaboration and knowledge exchange, and improve research infrastructure and data collection. Despite global challenges such as armed conflicts, CC and the COVID-19 pandemic, progress toward achieving the SDGs will become apparent by 2030. However, there are still opportunities for further research, knowledge dissemination and international collaboration toward developing safe, sustainable and inclusive urban development. The following are priority areas for SDG11 research:

Urban policy and governance: reforms should focus on providing equitable access to basic services such as water, sanitation, electricity, healthcare and education; upgrading and formalizing informal settlements; and improving living conditions of over one billion people residing in slums 37 . Participatory governance, community engagement and empowerment can enhance social inclusion by considering the voices and needs of marginalized groups 13 , 23 . Urban policy should also prioritize preserving historic and natural resources, protecting vulnerable areas and implementing sustainable urban design principles 47 . Future studies can help understand the dynamics, challenges and opportunities and monitor progress toward SDG11 targets 15 .

Localizing SDG11 targets: spatial planning and land-use strategies should consider the needs of diverse urban populations, promote inclusive zoning and engage local communities and stakeholders in decision-making processes, crucial for fostering ownership, empowerment and social cohesion, leading to more sustainable and inclusive urban development 3 . However, enhancing the capacity for localizing SDG11 targets requires building the knowledge and skills of local governments, policymakers and practitioners. Capacity-building initiatives, such as training programs, workshops and knowledge exchange, can promote interdisciplinary understanding and sharing of best practices.

Concerted and collaborative efforts: the international community, academics, policymakers and stakeholders can work together to create inclusive, safe, resilient and sustainable communities. Collaborative efforts can facilitate a comprehensive understanding of urban challenges and potential solutions by integrating diverse perspectives, data and methodologies. Disseminating research findings contributes to evidence-based policy development and informed decision-making, enabling the learning of lessons and replication of successful interventions.

Breaking down silos: integrated and cross-sectoral approaches help narrow the gaps between sectors, local governments, policymakers and stakeholders, leveraging local resources and capacities while fostering communication, knowledge sharing and collaboration 31 . Cross-sectoral working groups, joint planning processes and integrated policy frameworks promote holistic and coordinated decision-making among various sectors, including urban planning, housing, transportation, health, education, environment and social welfare 47 .

Digitalization and smart city development: maximizing the benefits of digitalization and smart city solutions requires addressing challenges such as bridging digital divides and ensuring data access, privacy and security. Prioritizing citizen-centric approaches and public accessibility to technology 36 are essential for leveraging expertise and resources 15 . Interoperability, scalability, data-driven decision-making and inclusivity contribute to evidence-based planning and equitable access to smart city technologies 12 , 48 , 49 , 50 , 51 .

This study comprehensively assessed SDG11 research, emphasizing significant thematic areas, trends, challenges and suggestions for prioritizing SDG11, including effective urban policy and governance, localizing SDG11 targets, concerted and collaborative efforts, and digitalization and smart city development. To broaden the scope of SDG11 research, future bibliometric reviews should encompass non-Web of Science databases and gray literature, including publications from government and nongovernmental agencies. Despite its limitations, this study’s findings provide valuable references for further research on SDG11.

The present study utilized a bibliometric technique to analyze academic publication on SDG11, tracing the research trend, the evolving key themes and identifying contributing authors, institutions and countries. Bibliometrics is a quantitative technique that allows for the analysis of trends in scholarly publications, such as research articles, conference papers and books, and visualizes scholarly publication patterns 52 . This technique is instrumental in analyzing extensive literature sets by relying on statistical observations and text-mining capabilities, which qualitative review methods such as systematic reviews cannot accomplish 53 . Additionally, it presents a scientific landscape of authors, countries, organizations and collaborations that contribute to worldwide scientific literature.

Bibliometric analysis requires interpretation, introducing an element of subjectivity 54 . Therefore, a sensemaking approach was adopted to transition from describing the bibliometric results to interpreting them. Sensemaking helps derive insightful information from bibliometric analysis and can be integrated into systematic literature reviews 55 , 56 . It applies to various international indexing, abstracting and citation databases, such as Scopus, Web of Science, Dimensions, PubMed and Education Resources Information Center, which cover journals, books, reviews and conference proceedings from around the world and different regions. For this study, Web of Science was chosen as the database to obtain bibliographic data due to its wide range of topics in various fields of study such as natural sciences, health sciences, engineering, social science, computer science and materials sciences. It is one of the world’s largest peer-reviewed scientific literature databases, with 87 million indexed items.

Specialized bibliometrics software were employed, including VOSviewer (version 1.6.19) 52 , Biblioshiny (version 4.1.3) 55 and BibExcel (version 2017) 57 . VOSviewer, known for its user-friendly interface, was used to understand the thematic focus and evolution of research on SDG11. It generates networks of nodes and links, with node size representing the frequency of the studied item, and link width indicating the strength of connections between items. Clusters of intricately linked nodes are shown in distinct colors. The thematic focus was examined for two periods: period 1 (2016–2019) and period 2 (2020–2022), considering the time since the SDGs were introduced to the time of data collection in this study. Another reason for this categorization is that evidence shows that the pandemic has significantly affected progress toward achieving SDGs 58 . VOSviewer allows for various types of analysis, including term co-occurrence, co-citation, citation and bibliographic coupling 53 . A term co-occurrence analysis was used in this study to highlight key thematic areas. To ensure accuracy and avoid separate counting of synonyms, a thesaurus file was developed and added to the software before the analysis. A summary of the data, including the number of authors and journals, used in the analysis is presented in Table 1 and will be further explained below.

A comprehensive search query was formulated to retrieve relevant data on SDG11, and it was executed in the title, abstract and keywords fields (TS) in Web of Science on 5 July 2023. The initial query shown the following box resulted in a total of 334,224 documents. Co-citation analysis was employed to identify the most influential journals contributing to SDG11 research. Two works are considered co-cited when they are both mentioned in the works cited section of a subsequent publication 59 (Zhao, 2006).

TS = ((‘city’ OR ‘cities’ OR ‘human settlement*’ OR ‘urban’ OR ‘metropoli*’ OR ‘town*’ OR ‘municipal*’ OR ‘peri-urban*’ OR ‘urban-rural’ OR ‘rural-urban’) AND (‘gentrification’ OR ‘congestion’ OR ‘transport*’ OR ‘housing’ OR ‘slum*’ OR ‘informal settlement*’ OR ‘sendai framework’ OR ‘Disaster Risk Reduction’ OR ‘disaster’ OR ‘DRR’ OR ‘smart cit*’ OR ‘resilient building*’ OR ‘sustainable building*’ OR ‘building design’ OR ‘buildings design’ OR ‘urbani?ation’ OR ‘zero energy’ OR ‘zero-energy’ OR ‘basic service*’ OR ‘governance’ OR ‘citizen participation’ OR ‘collaborative planning’ OR ‘participatory planning’ OR ‘inclusiveness’ OR ‘cultural heritage’ OR ‘natural heritage’ OR ‘UNESCO’ OR ‘ecological footprint’ OR ‘environmental footprint’ OR ‘waste’ OR ‘pollution’ OR ‘pollutant*’ OR ‘waste water’ OR wastewater* OR waste-water* OR ‘recycling’ OR ‘circular economy’ OR ‘air quality’ OR ‘green space’ OR ‘green spaces’ OR ‘nature inclusive’ OR ‘nature inclusive building’ OR ‘nature inclusive buildings’ OR ‘resilient’ OR ‘resilience’ OR ‘healthy cit*’ OR ‘sustainable’ OR ‘sustainability’ OR ‘green’ OR ‘nature*’ OR ‘Green infrastructure*’ OR ‘nature-based solution*’ OR ‘nature based solution*’ OR ‘child*’ OR ‘wom?n’ OR ‘elderl*’ OR ‘disabl*’ OR ‘disabilit*’ OR ‘disabled’)) AND PY = (2016–2022) NOT PY = (2023)

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework was used to report document search and filtration process. The PRISMA framework is designed to help scholars transparently report why their review study is conducted, what activities are performed and what discoveries are made, ideal for both systematic reviews and bibliometric studies 60 . PRISMA presents the four stages of the above query’s overall searching and filtration process (Fig. 4 ). The identification stage yielded 334,224 records, which were then screened to select only article-type documents ( n  = 277,165). Subsequently, documents were further screened based on language, selecting only English documents ( n  = 257,374). In the final stage, documents were screened based on specific categories closely related to cities and SDG11, resulting in a selection of six major categories: urban studies, environmental studies, geography, urban and regional planning, architecture, transportation and physical geography ( n  = 21,168). Finally, 15 duplicated documents were removed, resulting in a final dataset of 21,153 documents.

A four-phase flow diagram of the data extraction and filtration process of SDG11 literature, adapted from Priyadarshini 57 . WoS, Web of Science.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Data availability

The data that support the findings of this study are available as supplementary information. The steps for curating the data from the Web of Science have been provided in the text. If there is a further need, data are available on figshare at https://doi.org/10.6084/m9.figshare.26360125 . Source data are provided with this paper.

Almulhim, A. I. & Cobbinah, P. B. Can rapid urbanization be sustainable? The case of Saudi Arabian cities. Habitat Int. 139 , 102884 (2023).

Article   Google Scholar  

Huang, W. et al. Evaluating green city development in China using an integrated analytical toolbox. J. Clean. Prod. 400 , 136703 (2023).

Feng, T. & Zhou, B. Impact of urban spatial structure elements on carbon emissions efficiency in growing megacities: the case of Chengdu. Sci. Rep. 13 , 9939 (2023).

Javeed, S., Siddique, H. M. A. & Javed, F. Ecological footprint, globalization, and economic growth: evidence from Asia. Environ. Sci. Pollut. Res. 30 , 77006–77021 (2023).

Rees, W. E. Cities, energy, and the uncertain future of urban civilization. Oxf. Dev. Stud. 51 , 11–17 (2023).

Abubakar, I. R. & Aina, Y. A. The prospects and challenges of developing more inclusive, safe, resilient, and sustainable cities in Nigeria. Land Use Policy 87 , 104105 (2019).

UN-Habitat World Cities Report 2022: Envisaging the Future of Cities. (UN-Habitat, 2022).

Almulhim, A. I. & Cobbinah, P. B. Framing resilience in Saudi Arabian cities: on climate change and urban policy. Sustain. Cities Soc. 101 , 105172 (2024).

Barbier, E. B. & Burgess, J. C. Sustainability and development after COVID-19. World Dev. 135 , 105082 (2020).

The Sustainable Development Goals Report 2018 (United Nations, 2018).

Chen, M., Chen, L., Cheng, J. & Yu, J. Identifying interlinkages between urbanization and Sustainable Development Goals. Geogr. Sustain. 3 , 339–346 (2022).

Clement, J., Ruysschaert, B. & Crutzen, N. Smart city strategies—a driver for localizing sustainable development goals? Ecol. Econ. 213 , 107941 (2023).

Hansson, S., Arfvidsson, H. & Simon, D. Governance for sustainable urban development: the double function of SDG indicators. Area Dev. Policy 4 , 217–235 (2019).

Leal Filho, W. et al. Using the sustainable development goals towards a better understanding of sustainability challenges. Int. J. Sustain. Dev. World Ecol. 26 , 179–190 (2019).

Parra-Dominguez, J., Gil-Egido, A. & Rodríguez-González, S. SDGs as one of the drivers of smart city development: the indicator selection process. Smart Cities 5 , 1025–1038 (2022).

Grainger-Brown, J., Malekpour, S., Raven, R. & Taylor, E. Exploring urban transformation to inform the implementation of the Sustainable Development Goals. Cities 131 , 103928 (2022).

Xu, Z. et al. Assessing progress towards sustainable development over space and time. Nature 577 , 74–78 (2020).

Salvia, A. L., Leal Filho, W., Brandli, L. L. & Griebeler, J. S. Assessing research trends related to Sustainable Development Goals: local and global issues. J. Clean. Prod. 208 , 841–849 (2019).

Indana, F. & Pahlevi, R. W. A bibliometric approach to Sustainable Development Goals (SDGs) systematic analysis. Cogent Bus. Manag. 10 , 2224174 (2023).

Yamaguchi, N. U. et al. Sustainable development goals: a bibliometric analysis of literature reviews. Environ. Sci. Pollut. Res. 30 , 5502–5515 (2023).

Pattberg, P. & Bäckstrand, K. Enhancing the achievement of the SDGs: lessons learned at the half-way point of the 2030 Agenda. Int. Environ. Agree. 23 , 107–114 (2023).

Keith, M. et al. A new urban narrative for sustainable development. Nat. Sustain. 6 , 115–117 (2023).

Pedersen, A. B. et al. SDGs at the halfway point: how the 17 global goals address risks and wicked problems. Ambio 52 , 679–682 (2023).

Sweileh, W. M. Bibliometric analysis of scientific publications on ‘sustainable development goals’ with emphasis on ‘good health and well-being’ goal (2015–2019). Global. Health 16 , 68 (2020).

Mihelcic, J. R. et al. Environmental research addressing Sustainable Development Goals. Environ. Sci. Technol. 57 , 3457–3460 (2023).

Palau-Pinyana, E., Llach, J. & Bagur-Femenías, L. Mapping enablers for SDG implementation in the private sector: a systematic literature review and research agenda. Manag. Rev. Q. 26 , 1–30 (2023).

Google Scholar  

Mishra, M. et al. A bibliometric analysis of sustainable development goals (SDGs): a review of progress, challenges, and opportunities. Environ. Dev. Sustain. 26 , 11101–11143 (2023).

Devisscher, T. et al. in Sustainable Development Goals: Their Impacts on Forests and People (eds. Katila, P. et al.) (Cambridge Univ. Press, 2019).

Brenner, N. & Theodore, N. Cities and the geographies of ‘actually existing neoliberalism’. Antipode 34 , 349–379 (2002).

Sharifi, A., Khavarian-Garmsir, A. R., Allam, Z. & Asadzadeh, A. Progress and prospects in planning: a bibliometric review of literature in urban studies and regional and urban planning, 1956–2022. Prog. Plann. 173 , 100740 (2023).

Olanrewaju, A., Tan, S. Y. & Abdul-Aziz, A.-R. Housing providers’ insights on the benefits of sustainable, affordable housing. Sustain. Dev. 26 , 847–858 (2018).

Rice, J. L., Cohen, D. A., Long, J. & Jurjevich, J. R. Contradictions of the climate-friendly city: new perspectives on eco-gentrification and housing justice. Int. J. Urban Reg. Res. 44 , 145–165 (2020).

Park, S., Choi, K. & Lee, J. S. To walk or not to walk: testing the effect of path walkability on transit users’ access mode choices to the station. Int. J. Sustain. Transp. 9 , 529–541 (2015).

Jayasooriya, V. M., Ng, A. W. M., Muthukumaran, S. & Perera, B. J. C. Green infrastructure practices for the improvement of urban air quality. Urban For. Urban Green. 21 , 34–47 (2017).

Hachaichi, M. & Egieya, J. Water–food–energy nexus in global cities: addressing complex urban interdependencies. Water Resour. Manag. 37 , 1811–1825 (2023).

Mora, L., Deakin, M. & Reid, A. Combining co-citation clustering and text-based analysis to reveal the main development paths of smart cities. Technol. Forecast. Soc. Change 142 , 56–69 (2019).

McFarlane, C. Rethinking informality: politics, crisis, and the city. Plan. Theory Pract. 13 , 89–108 (2012).

Harvey, D. From managerialism to entrepreneurialism: the transformation in urban governance in late capitalism. Geogr. Ann. B 71 , 3–17 (1989).

Peck, J. Political economies of scale: fast policy, interscalar relations, and neoliberal workfare. Econ. Geogr. 78 , 331–360 (2002).

Cervero, R. & Landis, J. Twenty years of the Bay Area rapid transit system: land use and development impacts. Transport. Res. A 31 , 309–333 (1997).

Guzman, P. C., Roders, A. R. P. & Colenbrander, B. J. F. Measuring links between cultural heritage management and sustainable urban development: an overview of global monitoring tools. Cities 60 , 192–201 (2017).

Yamasaki, K. & Yamada, T. A framework to assess the local implementation of Sustainable Development Goal 11. Sustain. Cities Soc. 84 , 104002 (2022).

van Zanten, J. A. & van Tulder, R. Towards nexus-based governance: defining interactions between economic activities and Sustainable Development Goals (SDGs). Int. J. Sustain. Dev. World Ecol. 28 , 210–226 (2021).

Londono-Pineda, A. A. & Cano, J. A. Assessments under the United Nations sustainable development goals: a bibliometric analysis. Environ. Clim. Technol. 26 , 166–181 (2022).

Biggeri, M. A. ‘Decade for Action’ on SDG localization. J. Hum. Dev. Capabil. 22 , 706–712 (2021).

Benedek, J., Ivan, K., Török, I., Temerdek, A. & Holobâcă, I. H. Indicator-based assessment of local and regional progress toward the Sustainable Development Goals (SDGs): an integrated approach from Romania. Sustain. Dev. 29 , 860–875 (2021).

Abubakar, I. R. & Alshammari, M. S. Urban planning schemes for developing low-carbon cities in the Gulf Cooperation Council region. Habitat Int. 138 , 102881 (2023).

Batty, M. Smart cities, big data. Environ. Plann. B 39 , 191–193 (2012).

Smith, N. Toward a theory of gentrification, a back to the city movement by capital, not people. J. Am. Plann. Assoc. 45 , 538–548 (1979).

Jacobs, J. The Death and Life of Great American Cities (Random House, 1961).

Smith, N. The New Urban Frontier: Gentrification and the Revanchist City (Routledge, 1996).

Donthu, N., Kumar, S., Mukherjee, D., Pandey, N. & Lim, W. M. How to conduct a bibliometric analysis: an overview and guidelines. J. Bus. Res. 133 , 285–296 (2021).

van Eck, N. J. & Waltman, L. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics 84 , 523–538 (2010).

Hajek, P., Youssef, A. & Hajkova, V. Recent developments in smart city assessment: a bibliometric and content analysis-based literature review. Cities 126 , 103709 (2022).

Lim, W. M. & Kumar, S. Guidelines for interpreting the results of bibliometrics analysis: a sensemaking approach. Glob. Bus. Organ. Excell. 43 , 17–26 (2023).

Aria, M. & Cuccurullo, C. bibliometrix: an R-tool for comprehensive science mapping analysis. J. Informetr. 11 , 959–975 (2017).

Åström, F., Danell, R., Larsen, B. & Schneider, J. (eds) Celebrating Scholarly Communication Studies: A Festschrift for Olle Persson at His 60th Birthday Vol. 05-S (International Society for Scientometrics and Informetrics, 2009).

Priyadarshini, P. The COVID-19 pandemic has derailed the progress of Sustainable Development Goals. Anthr. Sci. 1 , 410–412 (2022).

Zhao, D. Towards all-author co-citation analysis. Inf. Process. Manag. 42 , 1578–1591 (2006).

Shamseer, L. et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 349 , g7647 (2015).

Download references

Acknowledgements

A.I.A. acknowledges Imam Abdulrahman Bin Faisal University in Dammam, Saudi Arabia, for their support in conducting this study. A.S. acknowledges the support of the Japan Society for the Promotion of Science KAKENHI grant number 22K04493. We appreciate Hiroshima University for supporting the open-access publication of this article.

Author information

Authors and affiliations.

Department of Urban and Regional Planning, College of Architecture and Planning, Imam Abdulrahman Bin Faisal University, Dammam, Saudi Arabia

Abdulaziz I. Almulhim

The IDEC Institute and Network for Education and Research on Peace and Sustainability, Hiroshima University, Hiroshima, Japan

Ayyoob Sharifi

Department of Geomatics Engineering Technology, Yanbu Industrial College, Yanbu, Saudi Arabia

Yusuf A. Aina

Directorate of Library Affairs, Imam Abdulrahman Bin Faisal University, Dammam, Saudi Arabia

Shakil Ahmad

The Business School, Edinburgh Napier University, Edinburgh, UK

Academy of Architecture and Urban Studies, Tallinn University of Technology, Tallinn, Estonia

Department of Natural Sciences, Manchester Metropolitan University, Manchester, UK

Walter Leal Filho

Research and Transfer Centre ‘Sustainable Development and Climate Change Management’, Hamburg University of Applied Sciences, Hamburg, Germany

College of Architecture and Planning, Imam Abdulrahman Bin Faisal University, Dammam, Saudi Arabia

Ismaila Rimi Abubakar

You can also search for this author in PubMed   Google Scholar

Contributions

A.I.A.: conceptualization, methodology, formal analysis, data curation, writing—original draft, writing—review and editing, investigation and project administration. A.S.: methodology, software, formal analysis, visualization and writing—original draft. Y.A.A.: conceptualization, writing—original draft, investigation and validation. S.A.: methodology, software, formal analysis, visualization and data curation. L.M.: writing—review and editing, and investigation. W.L.F.: writing—review and editing, and investigation. I.R.A.: writing—review and editing, investigation, supervision, validation and resources.

Corresponding authors

Correspondence to Abdulaziz I. Almulhim or Ayyoob Sharifi .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature Cities thanks Aliyu Salisu Barau, Oluwatobi Owojori, and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information.

Supplementary Figs. 1–5 and Tables 1–13.

Reporting Summary

Source data fig. 1.

Statistical source data in Excel format.

Source Data Fig. 2

Source data fig. 3, source data fig. 4, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Almulhim, A.I., Sharifi, A., Aina, Y.A. et al. Charting sustainable urban development through a systematic review of SDG11 research. Nat Cities (2024). https://doi.org/10.1038/s44284-024-00117-6

Download citation

Received : 22 September 2023

Accepted : 30 July 2024

Published : 27 August 2024

DOI : https://doi.org/10.1038/s44284-024-00117-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• Article Information

Data Sharing Statement

• As Ozempic’s Popularity Soars, Here’s What to Know About Semaglutide and Weight Loss JAMA Medical News & Perspectives May 16, 2023 This Medical News article discusses chronic weight management with semaglutide, sold under the brand names Ozempic and Wegovy. Melissa Suran, PhD, MSJ
• Patents and Regulatory Exclusivities on GLP-1 Receptor Agonists JAMA Special Communication August 15, 2023 This Special Communication used data from the US Food and Drug Administration to analyze how manufacturers of brand-name glucagon-like peptide 1 (GLP-1) receptor agonists have used patent and regulatory systems to extend periods of market exclusivity. Rasha Alhiary, PharmD; Aaron S. Kesselheim, MD, JD, MPH; Sarah Gabriele, LLM, MBE; Reed F. Beall, PhD; S. Sean Tu, JD, PhD; William B. Feldman, MD, DPhil, MPH
• What to Know About Wegovy’s Rare but Serious Adverse Effects JAMA Medical News & Perspectives December 12, 2023 This Medical News article discusses Wegovy, Ozempic, and other GLP-1 receptor agonists used for weight management and type 2 diabetes. Kate Ruder, MSJ
• GLP-1 Receptor Agonists and Gastrointestinal Adverse Events—Reply JAMA Comment & Response March 12, 2024 Ramin Rezaeianzadeh, BSc; Mohit Sodhi, MSc; Mahyar Etminan, PharmD, MSc
• GLP-1 Receptor Agonists and Gastrointestinal Adverse Events JAMA Comment & Response March 12, 2024 Karine Suissa, PhD; Sara J. Cromer, MD; Elisabetta Patorno, MD, DrPH
• GLP-1 Receptor Agonist Use and Risk of Postoperative Complications JAMA Research Letter May 21, 2024 This cohort study evaluates the risk of postoperative respiratory complications among patients with diabetes undergoing surgery who had vs those who had not a prescription fill for glucagon-like peptide 1 receptor agonists. Anjali A. Dixit, MD, MPH; Brian T. Bateman, MD, MS; Mary T. Hawn, MD, MPH; Michelle C. Odden, PhD; Eric C. Sun, MD, PhD
• Glucagon-Like Peptide-1 Receptor Agonist Use and Risk of Gallbladder and Biliary Diseases JAMA Internal Medicine Original Investigation May 1, 2022 This systematic review and meta-analysis of 76 randomized clinical trials examines the effects of glucagon-like peptide-1 receptor agonist use on the risk of gallbladder and biliary diseases. Liyun He, MM; Jialu Wang, MM; Fan Ping, MD; Na Yang, MM; Jingyue Huang, MM; Yuxiu Li, MD; Lingling Xu, MD; Wei Li, MD; Huabing Zhang, MD
• Cholecystitis Associated With the Use of Glucagon-Like Peptide-1 Receptor Agonists JAMA Internal Medicine Research Letter October 1, 2022 This case series identifies cases reported in the US Food and Drug Administration Adverse Event Reporting System of acute cholecystitis associated with use of glucagon-like peptide-1 receptor agonists that did not have gallbladder disease warnings in their labeling. Daniel Woronow, MD; Christine Chamberlain, PharmD; Ali Niak, MD; Mark Avigan, MDCM; Monika Houstoun, PharmD, MPH; Cindy Kortepeter, PharmD

See More About

Select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing

Others Also Liked

• Download PDF
• X Facebook More LinkedIn

Sodhi M , Rezaeianzadeh R , Kezouh A , Etminan M. Risk of Gastrointestinal Adverse Events Associated With Glucagon-Like Peptide-1 Receptor Agonists for Weight Loss. JAMA. 2023;330(18):1795–1797. doi:10.1001/jama.2023.19574

Manage citations:

© 2024

• Permissions

Risk of Gastrointestinal Adverse Events Associated With Glucagon-Like Peptide-1 Receptor Agonists for Weight Loss

• 1 Faculty of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
• 2 StatExpert Ltd, Laval, Quebec, Canada
• 3 Department of Ophthalmology and Visual Sciences and Medicine, University of British Columbia, Vancouver, Canada
• Medical News & Perspectives As Ozempic’s Popularity Soars, Here’s What to Know About Semaglutide and Weight Loss Melissa Suran, PhD, MSJ JAMA
• Special Communication Patents and Regulatory Exclusivities on GLP-1 Receptor Agonists Rasha Alhiary, PharmD; Aaron S. Kesselheim, MD, JD, MPH; Sarah Gabriele, LLM, MBE; Reed F. Beall, PhD; S. Sean Tu, JD, PhD; William B. Feldman, MD, DPhil, MPH JAMA
• Medical News & Perspectives What to Know About Wegovy’s Rare but Serious Adverse Effects Kate Ruder, MSJ JAMA
• Comment & Response GLP-1 Receptor Agonists and Gastrointestinal Adverse Events—Reply Ramin Rezaeianzadeh, BSc; Mohit Sodhi, MSc; Mahyar Etminan, PharmD, MSc JAMA
• Comment & Response GLP-1 Receptor Agonists and Gastrointestinal Adverse Events Karine Suissa, PhD; Sara J. Cromer, MD; Elisabetta Patorno, MD, DrPH JAMA
• Research Letter GLP-1 Receptor Agonist Use and Risk of Postoperative Complications Anjali A. Dixit, MD, MPH; Brian T. Bateman, MD, MS; Mary T. Hawn, MD, MPH; Michelle C. Odden, PhD; Eric C. Sun, MD, PhD JAMA
• Original Investigation Glucagon-Like Peptide-1 Receptor Agonist Use and Risk of Gallbladder and Biliary Diseases Liyun He, MM; Jialu Wang, MM; Fan Ping, MD; Na Yang, MM; Jingyue Huang, MM; Yuxiu Li, MD; Lingling Xu, MD; Wei Li, MD; Huabing Zhang, MD JAMA Internal Medicine
• Research Letter Cholecystitis Associated With the Use of Glucagon-Like Peptide-1 Receptor Agonists Daniel Woronow, MD; Christine Chamberlain, PharmD; Ali Niak, MD; Mark Avigan, MDCM; Monika Houstoun, PharmD, MPH; Cindy Kortepeter, PharmD JAMA Internal Medicine

Glucagon-like peptide 1 (GLP-1) agonists are medications approved for treatment of diabetes that recently have also been used off label for weight loss. 1 Studies have found increased risks of gastrointestinal adverse events (biliary disease, 2 pancreatitis, 3 bowel obstruction, 4 and gastroparesis 5 ) in patients with diabetes. 2 - 5 Because such patients have higher baseline risk for gastrointestinal adverse events, risk in patients taking these drugs for other indications may differ. Randomized trials examining efficacy of GLP-1 agonists for weight loss were not designed to capture these events 2 due to small sample sizes and short follow-up. We examined gastrointestinal adverse events associated with GLP-1 agonists used for weight loss in a clinical setting.

We used a random sample of 16 million patients (2006-2020) from the PharMetrics Plus for Academics database (IQVIA), a large health claims database that captures 93% of all outpatient prescriptions and physician diagnoses in the US through the International Classification of Diseases, Ninth Revision (ICD-9) or ICD-10. In our cohort study, we included new users of semaglutide or liraglutide, 2 main GLP-1 agonists, and the active comparator bupropion-naltrexone, a weight loss agent unrelated to GLP-1 agonists. Because semaglutide was marketed for weight loss after the study period (2021), we ensured all GLP-1 agonist and bupropion-naltrexone users had an obesity code in the 90 days prior or up to 30 days after cohort entry, excluding those with a diabetes or antidiabetic drug code.

Patients were observed from first prescription of a study drug to first mutually exclusive incidence (defined as first ICD-9 or ICD-10 code) of biliary disease (including cholecystitis, cholelithiasis, and choledocholithiasis), pancreatitis (including gallstone pancreatitis), bowel obstruction, or gastroparesis (defined as use of a code or a promotility agent). They were followed up to the end of the study period (June 2020) or censored during a switch. Hazard ratios (HRs) from a Cox model were adjusted for age, sex, alcohol use, smoking, hyperlipidemia, abdominal surgery in the previous 30 days, and geographic location, which were identified as common cause variables or risk factors. 6 Two sensitivity analyses were undertaken, one excluding hyperlipidemia (because more semaglutide users had hyperlipidemia) and another including patients without diabetes regardless of having an obesity code. Due to absence of data on body mass index (BMI), the E-value was used to examine how strong unmeasured confounding would need to be to negate observed results, with E-value HRs of at least 2 indicating BMI is unlikely to change study results. Statistical significance was defined as 2-sided 95% CI that did not cross 1. Analyses were performed using SAS version 9.4. Ethics approval was obtained by the University of British Columbia’s clinical research ethics board with a waiver of informed consent.

Our cohort included 4144 liraglutide, 613 semaglutide, and 654 bupropion-naltrexone users. Incidence rates for the 4 outcomes were elevated among GLP-1 agonists compared with bupropion-naltrexone users ( Table 1 ). For example, incidence of biliary disease (per 1000 person-years) was 11.7 for semaglutide, 18.6 for liraglutide, and 12.6 for bupropion-naltrexone and 4.6, 7.9, and 1.0, respectively, for pancreatitis.

Use of GLP-1 agonists compared with bupropion-naltrexone was associated with increased risk of pancreatitis (adjusted HR, 9.09 [95% CI, 1.25-66.00]), bowel obstruction (HR, 4.22 [95% CI, 1.02-17.40]), and gastroparesis (HR, 3.67 [95% CI, 1.15-11.90) but not biliary disease (HR, 1.50 [95% CI, 0.89-2.53]). Exclusion of hyperlipidemia from the analysis did not change the results ( Table 2 ). Inclusion of GLP-1 agonists regardless of history of obesity reduced HRs and narrowed CIs but did not change the significance of the results ( Table 2 ). E-value HRs did not suggest potential confounding by BMI.

This study found that use of GLP-1 agonists for weight loss compared with use of bupropion-naltrexone was associated with increased risk of pancreatitis, gastroparesis, and bowel obstruction but not biliary disease.

Given the wide use of these drugs, these adverse events, although rare, must be considered by patients who are contemplating using the drugs for weight loss because the risk-benefit calculus for this group might differ from that of those who use them for diabetes. Limitations include that although all GLP-1 agonist users had a record for obesity without diabetes, whether GLP-1 agonists were all used for weight loss is uncertain.

Accepted for Publication: September 11, 2023.

Published Online: October 5, 2023. doi:10.1001/jama.2023.19574

Correction: This article was corrected on December 21, 2023, to update the full name of the database used.

Corresponding Author: Mahyar Etminan, PharmD, MSc, Faculty of Medicine, Departments of Ophthalmology and Visual Sciences and Medicine, The Eye Care Center, University of British Columbia, 2550 Willow St, Room 323, Vancouver, BC V5Z 3N9, Canada ( [email protected] ).

Author Contributions: Dr Etminan had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Sodhi, Rezaeianzadeh, Etminan.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Sodhi, Rezaeianzadeh, Etminan.

Critical review of the manuscript for important intellectual content: All authors.

Statistical analysis: Kezouh.

Obtained funding: Etminan.

Administrative, technical, or material support: Sodhi.

Supervision: Etminan.

Conflict of Interest Disclosures: None reported.

Funding/Support: This study was funded by internal research funds from the Department of Ophthalmology and Visual Sciences, University of British Columbia.

Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Data Sharing Statement: See Supplement .

• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts

• Open access
• Published: 30 August 2024

Enhanced “learning to learn” through a hierarchical dual-learning system: the case of action video game players

• Yu-Yan Gao   ORCID: orcid.org/0000-0002-9410-8922 1 , 2 , 3 , 4 ,
• Zeming Fang   ORCID: orcid.org/0000-0002-8091-4413 2 ,
• Qiang Zhou   ORCID: orcid.org/0000-0002-3045-0198 4   na1 &
• Ru-Yuan Zhang   ORCID: orcid.org/0000-0002-0654-715X 1 , 2   na1  

BMC Psychology volume  12 , Article number:  460 ( 2024 ) Cite this article

1 Altmetric

Metrics details

In contrast to conventional cognitive training paradigms, where learning effects are specific to trained parameters, playing action video games has been shown to produce broad enhancements in many cognitive functions. These remarkable generalizations challenge the conventional theory of generalization that learned knowledge can be immediately applied to novel situations (i.e., immediate generalization). Instead, a new “learning to learn” theory has recently been proposed, suggesting that these broad generalizations are attained because action video game players (AVGPs) can quickly acquire the statistical regularities of novel tasks in order to increase the learning rate and ultimately achieve better performance. Although enhanced learning rate has been found for several tasks, it remains unclear whether AVGPs efficiently learn task statistics and use learned task knowledge to guide learning. To address this question, we tested 34 AVGPs and 36 non-video game players (NVGPs) on a cue-response associative learning task. Importantly, unlike conventional cognitive tasks with fixed task statistics, in this task, cue-response associations either remain stable or change rapidly (i.e., are volatile) in different blocks. To complete the task, participants should not only learn the lower-level cue-response associations through explicit feedback but also actively estimate the high-level task statistics (i.e., volatility) to dynamically guide lower-level learning. Such a dual learning system is modelled using a hierarchical Bayesian learning framework, and we found that AVGPs indeed quickly extract the volatility information and use the estimated higher volatility to accelerate learning of the cue-response associations. These results provide strong evidence for the “learning to learn” theory of generalization in AVGPs. Taken together, our work highlights enhanced hierarchical learning of both task statistics and cognitive abilities as a mechanism underlying the broad enhancements associated with action video game play.

Peer Review reports

Introduction

Humans possess impressive and adaptable learning abilities, as evidenced by the rapid learning of diverse cognitive tasks and the flexible application of learned knowledge to unfamiliar scenarios. Optimizing learning and facilitating generalization has been a fundamental challenge in cognitive science. Traditional cognitive training often exhibits specificity to the training settings (tasks or parameters)–-the improvement in learning are greatly reduced in previously unseen situations [ 1 , 2 ]. If the benefits of cognitive training cannot efficiently generalize across different application situations, its real-world applicability is significantly diminished. Action video game training has been shown a unique training regime that can overcome such limitations. A large body of cognitive science research have shown that playing action video games can directly enhance a wide range of seemingly unrelated cognitive functions, such as attention [ 2 , 3 ], memory [ 4 , 5 , 6 ], perception [ 2 , 7 , 8 ], and reasoning [ 9 ]. Importantly, players are not directly trained on these specific cognitive tasks when playing action video games. Because of these astonishingly broad generalizations, action video games have also been suggested as a useful paradigm for cognitive training [ 2 ] and even for therapeutic purposes [ 10 ]. As generalization is the key for observers to learn infinite knowledge based on finite learning samples, it is of paramount importance to understand the neurocomputational mechanisms of broad generalization induced by action video game play.

Why can action video game play lead to broad generalization in stark contrast to conventional training approaches? Classic theories of learning generalization postulate that an observer generalizes learned knowledge to novel cases by inferring the common constructs between training and application situations [ 11 , 12 , 13 ]. This view assumes that, once common constructs are identified, improvement on novel tasks is immediately achievable. This classic view is often referred to as “immediate generalization” [ 14 , 15 ]. More recently, a new mechanism of generalization has been proposed, which suggests that action video game play induces broad generalizations by enabling observers to “learning to learn” [ 10 ]. In contrast to the “immediate generalization” theory, the “learning to learn” theory predicts that avid action video game players (AVGPs) can quickly capture the underlying structural knowledge of new tasks and thus accelerate learning. Faster learning (i.e., taking less time to achieve good performance) on new tasks, as a hallmark of “learning to learn”, has been found in several recent studies of action video games [ 16 ] and classical perceptual learning [ 7 , 8 , 9 ].

Although “learning to learn” is an elegant theory that can potentially explain the remarkable generalization afforded by action video game play, two issues remain unresolved. First, in addition to predicting faster learning of novel tasks, the “learning to learn” theory has two other key predictions — (1) action video game players (AVGPs) can estimate and understand task statistics more quickly and accurately, and (2) the learned task statistics can in turn guide faster learning of a task. However, the enhanced ability of AVGPs to learn the statistical structure of tasks has not been directly investigated. Second, the “learning to learn” theory also implicitly assumes that, even in an apparently simple task, a hierarchical dual learning system operates: a high-level system for learning task statistics and a lower-level system for learning appropriate responses. Previous studies only assessed observers’ learning behavior as a result of the low-level learning system. It remains unclear whether a high-level learning system exists and how it supports the lower-level response learning. To address these two questions directly, two factors should be considered. First, to demonstrate the superior ability of AVGPs to extract task statistics, we need a task with systematic variation in stimulus statistical regularities and test whether AVGPs are indeed sensitive to such variation. Second, the “learning to learn” ability should be explicitly formulated. In other words, a computational framework is needed to explicitly specify how the correct decisions emerge according to the interactions within the dual-learning system in an online fashion.

In this study, we aim to directly test the “learning to learn” theory using a volatile reversal learning task [ 17 , 18 ]. In this task, participants learn the associations between a visual cue and its corresponding response through trial-by-trial feedback. Importantly, such cue-response associations either remain stable over several trials (i.e., stable block) or change rapidly on other trials (i.e., volatile block, see Methods for details). This volatility variation allows us to assess participants’ ability to learn such task statistics, and, unlike classic learning tasks [ 19 , 20 , 21 ], such an associative learning task also allows us to explicitly estimate participants’ learning rate at both levels. Furthermore, we used the Hierarchical Gaussian Filter (HGF, [ 22 ]) to formulate the “learning to learn” process. In particular, unlike classical reinforcement learning models that only formulate the learning of cue-response associations [ 23 , 24 ], the HGF also specifies a high-level learning process of task statistics (i.e., association volatility). Importantly, changes in the lower-level cue-response associations lead to trial-by-trial updates in the high-level belief of association volatility, and the high-level estimates of association volatility in turn adjust the rate of the lower-level association learning. These bidirectional interactions between a hierarchical dual-learning systems exactly corresponds to the “learning to learn” hypothesis.

Our results show that AVGPs display higher learning rates in the volatile reversal learning task, consistent with previous studies. Most importantly, this higher learning rate is a result of an efficient representation of the association volatility, as evidenced by a higher estimate of association volatility in the AVGPs. All these results are consistent with the “learning to learn” theory of action video game play.

Materials and methods

Ethics and participants.

All experimental protocols were approved by the institutional review board of Shanghai Jiao Tong University. All research was conducted in accordance with relevant guidelines and regulations. Informed written consent was obtained from all participants.

We firstly administered the Chinese version of the Video-Game-Expertise Classification Scheme [ 25 ] to screen for action video game players (AVGPs) and non-video game players (NVGPs). Both English and Chinese versions of the video game questionnaire can be downloaded from https://www.unige.ch/fapse/brainlearning/vgq/ . The basic inclusion criteria require participants to have Chinese as a first or second language; normal or corrected-to-normal vision; no history of mental disorders; not taking significant psychiatric medications; and an age range of 18 to 40 years old. NVGPs need to meet the following criteria:(1) play first/third-person shooter, action/sports, real-time strategy/ Multiplayer Online Battle Arena (MOBA) games, or simulation games for no more than 1 h/week in the past year and the year before; (2) play any other type of games for no more than 3 h/week in the past year; (3) play any other type of games for no more than 5 h/week a year ago. AVGPs need to meet any of the following criteria:(1) play other games for no more than 3 h/week, but play action games for at least 5 h/week in the past year; (2) play action games for at least 3 h/week in the past year, with other games not exceeding 3 h/week, and play action games for at least 5 h/week a year ago; (3) play other games for no more than 3 h/week in the past year, with at least 3 h/week for action games and at least 5 h/week for sports/driving games; (4) play other games for no more than 3 h/week in the past year, with at least 3 h/week for action games and at least 5 h/week for real-time strategy/MOBA games. There exist other inclusion criteria for both groups. More detailed screening criteria can be found in the questionnaire above.

Previous studies have documented several important ingredients of AVGs that enable generalization effect, including (i) decision-making under time constraints, (ii) maintaining divided attention, and (iii) the necessity for prompt transitions between two distinct attentional states (focused and divided) [ 1 , 26 , 27 ]. These factors have also been incorporated into a number of other game genres, including sports and driving games, as well as real-time strategy and MOBA games. We thus also include these genres in the screening of AVGPs.

Based on the filtering criteria, 34 AVGPs (24 males and 10 females) and 36 NVGPs (12 males and 24 females) were recruited to participate in the formal experiment after obtaining their consents. All participants were right-handed and had normal or corrected-to-normal vision. After excluding the subjects who exhibited extreme performance (see data analysis below), data from 33 AVGPs (23 males and 10 females) and 34 NVGPs (12 males and 22 females) were included for further analysis.

Stimulus and task

This experiment was hosted on the Naodao platform ( https://www.naodao.com/ ). Participants accessed the task remotely and completed it online. They received the corresponding participant compensation after the experiment.

Both AVGPs and NVGPs performed the same volatile reversal learning task (Fig.  1 A). Each trial began with a 500 ms fixation period. A cue stimulus (i.e., a yellow or a blue window) was presented. The cue stimulus disappeared after the participant made a keypress response to predict which outcome stimulus (i.e., a cat or a dog) was more likely to appear after the cue stimulus. After the keypress response, an outcome stimulus was presented for 1000 ms. The whole experiment consists of four blocks (80 trials per block) with a total of 320 trials. In each block, the association settings between the cue and outcome stimuli were changed (Fig.  1 B).

Task design and model. A Each trial started with a fixation cross in the center of the screen. After a delay of 500 ms, a stimulus was presented on the screen. Participants were instructed to predict the animal behind the window based on the current yellow or blue window and press the ‘F’ key for a cat or the ‘J’ key for a dog. Immediate feedback and outcome stimuli were provided after each response, lasting for 1000 ms before proceeding to the next trial. B The experiment was divided into four blocks based on the probability of cue-response association: stable (trials 1–80, p  = 0.75)—volatile (trials 81–160, with a switching sequence of p values: 0.2–0.8–0.2–0.8)—stable (trials 161–240, p  = 0.25)—volatile (trials 241–320 with a switching sequence of p values: 0.8–0.2–0.8–0.2). The yellow line parallel to the x-axis represents trials in the stable blocks, the green line represents trials in the volatile blocks. In the stable blocks. the association probability remained constant within 80 trials, while in the volatile blocks, the probability changed every 20 trials. C Generative process of the HGF. \(A\) represents action; \(R\) indicates the estimated association probability between the given window cue and the corresponding animal response; \(V\) represents the estimated association volatility. \(t\) denotes each time point. \({A}_{t}\) depends on \({R}_{t-1}\) , \({V}_{t-1}\) , and parameters \(\theta\) , \({\kappa }_{2}\) , \(\omega\) . The interconnection between levels is achieved through uncertainty

Here, association is defined as the probability of a cue-response pair. For example, in the first 80 trials, the outcome stimulus cat (or dog) appeared after the cue stimulus yellow window with a probability of 0.75 (or 0.25, respectively). Similarly, the association “blue window-dog” is 0.75. The association settings changed in each block (Fig.  1 B). The key point here is that the association setting is stable (i.e., stable condition) in Block 1 (i.e., trials 1–80) and Block 3 (i.e., trials 161–240) but switches rapidly between 0.8 and 0.2 (i.e., volatile condition) in Block 2 (i.e., trials 81–160) and Block 4 (i.e., trials 241–320).

The stimulus materials for this task were created using Photoshop, and each stimulus material has a resolution of 1080 × 720. The presentation order of the stimuli was pseudorandomized and generated in MATLAB 2020a according to the number of trials in each experimental block and the four cue-response association probabilities. The presentation order of the cues within the experimental block was fixed by a predetermined shuffled order. Thus, each participant received the same stimulus sequence, allowing for a comparable learning process and model parameter estimation. The experimental procedure was developed using jsPsych-6.3.0 ( https://www.jspsych.org/6.3/ ). Participants were informed that these probabilities would change, but were not given with specific information about the four blocks and the exact values of the probabilities.

Computational modeling

The HGF [ 22 ] model is used to analyze the participants’ behavior. We plotted and compared the trial-by-trial generated data from two groups of participants. At the same time, we used t-tests to compare the parameters of the two groups of participants.

Generative model

The HGF can be understood via two distinct components: prediction and update. Briefly, this model formulates the prediction and update process in a two-level hierarchy (Fig.  2 ). The prediction (i.e., generative) process can be seen in Fig.  1 C and the left part of Fig.  2 . Specifically, the higher level of the model represents the estimated association volatility ( \(V\) ) (i.e., how quickly the cue-response associations switch), which is updated by

where \(\theta\) is a constant parameter which determines the variance of estimated association volatility (the high-level, \(V\) ). Estimated association volatility \(V\) determines the magnitude for updating the lower-level cue-response association ( \(R\) , the estimated association probability between the given window cue and the corresponding behavioral choices in the logarithmic domain).

where \({\kappa }_{2}\) is the top-down influence factor that determines the coupling strength between the association probability (the low-level, \(R\) ) and the estimated association volatility (the high-level, \(V\) ); \(\omega\) is a constant component of the association variance \(\left({\kappa }_{2}*{V}_{t}+\omega \right)\) , independent of the state of the estimated association volatility (the high-level, \(V\) ). The behavioral action \(A\) is generated by the association probability ( \(R\) ), and \(\mu\) (i.e., correct or incorrect) is the actual outcome the participant received.

where the function \(s(\cdot )\) is the sigmoid function with \({\kappa }_{1}\) as the inverse temperature. To simplify our modeling, we fixed the coupling factor controlling the influence of association probability (the low-level, \(R\) ) on action (i.e., \({\kappa }_{1}\) ) to 1 .

Overview of the HGF model. The probability at each level is determined by the previous level and parameters. Throughout the paper, we analyzed several key variables of this model. We color labeled the variables of interest and illustrate the figure number where the group differences in the variables are compared to facilitate reading

This model has three free parameters \(:\) \(\theta\) , \({\kappa }_{2}\) , and \(\omega\) .

Trial-by-trial update rule of model parameters

The detailed trial-by-trial update rule of model parameters in HGF has been documented in Mathys, et al. [ 22 ]. Furthermore, this update process is illustrated in the right part of Fig.  2 . Here we provide a short overview and an introduction of the variables and free parameters.

On the t -th trial, the action ( \({A}_{t}\) ) is determined by the actual outcome the subject received ( \({\mu }_{t}\) ), where \({\mu }_{t}\in \{\text{0,1}\}\) indicates the correct/incorrect feedback.

The update of estimated association probability ( \({R}_{t}\) ) depends on the association learning rate ( \({\alpha }_{t}^{R}\) ) and the association prediction errors ( \({PE}_{t}^{R}\) ).

Note that the association learning rate ( \({\alpha }_{t}^{R}\) ) varies trial-by-trial and is determined by association expectation ( \({\widehat{\alpha }}_{t}^{R}\) ) and action expectation ( \({\widehat{\alpha }}_{t}^{A}\) ). The superscript \(R\) denotes the variables as the ones operating at the low-level association learning.

The association expectation ( \({\widehat{\alpha }}_{t}^{R}\) ) per se also varies trial-by-trial and is determined by the learning rate of the last trial ( \({\alpha }_{t-1}^{R}\) ) and the upper-level estimated association volatility ( \({V}_{t-1}\) ), where \({\kappa }_{2}\) and \(\omega\) are free parameters.

The action expectation ( \({\widehat{\alpha }}_{t}^{A}\) ) per se also varies trial-by-trial and is determined by the action of the last trial ( \({A}_{t-1}\) ).

the association prediction errors ( \({PE}_{t}^{R}\) ) is given by:

The update of the estimated volatility ( \({V}_{t}\) ) depends on the volatility learning rate ( \({\alpha }_{t}^{V}\) ) and the volatility prediction errors ( \({PE}_{t}^{V}\) ) The superscript \(V\) denotes the variables as the ones operating at the high-level volatility learning.

where the volatility learning rate ( \({\alpha }_{t}^{V}\) ) consists of three components:

Here, \({\overline{\alpha }}_{t}^{V}\) represents unweighted volatility learning rate of \(V\) and varies trial-by-trial:

where \(\theta\) is a free parameter. \({w}_{t}^{V}\) denotes a precision weighting factor.

the volatility prediction errors ( \({PE}_{t}^{V}\) ) is given by:

In summary, the estimated free parameters for each participant are \({\kappa }_{2}\) , \(\omega\) , and \(\theta\) . The variables with subscript “ t ” change from trial to trial, and the three free parameters without subscript “ t ” are fixed values that hold for all trials.

The analysis was performed using the HGF toolbox in MATLAB ( https://translationalneuromodeling.github.io/tapas ). The tapas_fitModel function was used to iteratively fit the model 100 times for each participant, using the Maximum A Posteriori (MAP) method for parameter estimation. Configuration settings, facilitated by functions such as tapas_hgf_binary_config , tapas_unitsq_sgm_config ,and tapas_quasinewton_optim_config , were used to set prior ranges for the parameters. The ranges of priors for the parameters to be fitted are as follows: top-down factor \(\text{log}\left({\kappa }_{2}\right)\sim \mathcal{N}\left(\text{log}\left(1\right), 4\right)\) ; association constant uncertainty \(\omega \sim \mathcal{N}\left(-3, 16\right)\) ; volatility constant uncertainty \(\text{log}(\theta )\sim \mathcal{N}\left(-6, 16\right)\) . All other parameters involved in the code, including their ranges and initial values, follow the default settings in the toolbox.

Statistical analysis

Linear mixed model analysis was performed in JASP 0.18.1.0 ( https://jasp-stats.org/ ), and all multiple comparisons were corrected using the Holm correction in JASP. All t-tests were performed using the Pingouin package in Python and were all two-tailed. In this experiment, participants with an average association learning rate exceeding (or fall below) the mean plus (or minus) two standard deviations of the overall sample were excluded. A total of 4 participants met these criteria. 33 AVGPs and 34 NVGPs were included in the reported results.

Superior low-level learning rate of cue-response associations in AVGPs

Participants performed a volatile reversal learning task (Fig.  1 A). On each trial, a fixation was shown for 500 ms and followed by a cue stimulus (i.e., a yellow window or a blue window). Participants were asked to predict the subsequent outcome stimulus (i.e., a cat or a dog) associated with the cue. Following a keypress response, an outcome stimulus was presented for 1000 ms as feedback. The two cue stimuli and the two outcome stimuli were paired. For example, within a stable block, the cat (or dog) appeared after the yellow window (or blue) window in 75% (or 25%, respectively) of the trials. Such cue-response associations varied across blocks. Importantly, the task statistic is defined as the changing rate of such cue-response associations (i.e., volatility). In particular, in the two stable blocks (Block 1, trials 1–80; Block 3, trials 161–240), the cue-response association settings remained constant. In contrast, in the two volatile blocks (Block 2, trials 81–160; Block 4, trials 241–320), the cue-response associations switched between 0.8 and 0.2 every 20 trials. The key question here is whether participants can learn the stability and volatility of the associations and use this information to guide their learning. Followed by the conventional approach [ 17 , 28 ], we directly fitted computational models (see below) to represent participants’ learning process in this task.

We first asked whether we could replicate the finding that AVGPs learn a novel task faster than NVGPs [ 7 , 10 , 16 , 29 ]. Unlike the conventional reinforcement learning approach that estimates a single learning rate parameter throughout the task [ 30 , 31 ], HGF assumes that participants’ learning rate also varies from trial to trial based on updated beliefs about the task statistics (i.e., volatility). In this task, participants learned the cue-response associations. The trial-by-trial association learning rate ( \({\alpha }_{t}^{R}\) , Eqs. 6 – 8 ) in both groups is plotted as a function of trials in Fig.  3 A.

Comparison of association learning rate between two groups. A The log association learning rate ( \({\alpha }_{t}^{R}\) , Eqs. 7 , 8 , 9 ) required for updating the estimated association probability for each participant. The x-axis represents the trial sequence ( t ), and the y-axis illustrates participants’ log association learning rate ( \({\alpha }_{t}^{R}\) ). The red line represents AVGPs, and the blue line represents NVGPs. The shaded area represents S.E.M across all participants within each group (33 AVGPs, 34 NVGPs). Significance symbol conventions is **: p  < 0.01. B Two groups’ association prediction errors ( \({PE}_{t}^{R}\) , Eqs. 6 & 10 ) across trials. The x-axis represents the trial sequence ( t ), the y-axis illustrates association prediction errors ( \({PE}_{t}^{R}\) ). Significance symbol convention is n.s.: non-significant

A linear mixed model (LMM) was built in JASP with Trial as a random effect factor, Group (AVGPs/NVGPs) as a fixed effect factor, Log Association Learning Rate ( \({\alpha }_{t}^{R}\) , Eqs.7–9) in each trial as the dependent variable. We found that the effect of Group is significant, indicating the overall higher learning rate of the AVGPs than that of the NVGPs ( t (21119 )  = 2.852, p  = 0.004, Estimate  = 0.055, SE  = 0.019, CI  = [0.017, 0.093]). In summary, Fig.  3 A shows that the AVGPs indeed had a generally higher learning rate than the NVGPs, although the learning rate in both groups varied from trial to trial in both groups.

Because the trial-by-trial update of the association probability ( \({\Delta R}_{t}\) , Eq.  6 ) is determined by both association learning rate ( \({\alpha }_{t}^{R}\) ) and association prediction errors ( \({PE}_{t}^{R}\) , Eqs. 6 & 10 ), we also analyzed the association prediction errors ( \({PE}_{t}^{R}\) ) in both groups and plotted them as a function of trials in Fig.  3 B. A LMM was performed with the Association Prediction Errors ( \({PE}_{t}^{R}\) ) as the dependent variable; Group (AVGPs/NVGPs) as a fixed effect factor and Trial as a random effect factor. We found no significant effect of Group ( t (21119)  = -0.036, p  = 0.971, Estimate  = -0.001, SE  = 0.002, CI  = [-0.003, 0.003]), suggesting the superior learning in AVGPs is mostly due to the association learning rate rather than association prediction errors.

Higher low-level learning rate in AVGPs is due to high-level association volatility

We have confirmed the overall higher association learning rate in AVGPs. A higher association learning rate ( \({\alpha }_{t}^{R}\) ) leads to a larger update ( \({\Delta R}_{t}\) ) of the estimated association probability. But how did AVGPs develop an overall higher association learning rate in the volatile reversal task? The key aspect of the HGF is that association learning rate is determined by association variance in the last trial ( \({\kappa }_{2}*{V}_{t-1}+\omega\) ), which is further controlled by high-level volatility \({V}_{t-1}\) in the last trial (Eqs. 7 – 9 ). Here, we examine whether higher association variance leading to an increased association learning rate in the AVGPs.

A LMM was performed with Association variance ( \({\kappa }_{2}*{V}_{t}+\omega\) ) as the dependent variable; Group (AVGPs/NVGPs) as a fixed effect factor and Trial as a random effect factor. We found that the effect of the Group was significant, indicating overall greater association variance of AVGPs compared to NVGPs ( t (21119)  = 2.516, p  = 0.012, Estimate  = 0.100, SE  = 0.040, CI  = [0.022, 0.179], Fig.  4 A). For completeness, in addition to the association variance ( \({\kappa }_{2}*{V}_{t}+\omega\) ) and the association learning rate from the previous trial ( \({\alpha }_{t-1}^{R}\) ), we also compared action expectation ( \({\widehat{\alpha }}_{t}^{A}\) , Eqs. 6 & 7 ) that contribute to the update of association learning rate (Eq.  7 ). We found no significant effect of Group ( t (21119)  = -0.071, p  = 0.944, Estimate  = -0.001, SE  = 0.005, CI  = [-0.010, 0.009]) . This suggests that the higher association learning rate ( \({\alpha }_{t}^{R}\) ) observed in AVGPs is likely due to their overall higher association variance ( \({\kappa }_{2}*{V}_{t}+\omega\) ).

Association variance and estimated association volatility in two groups. A Participants’ association variance ( \({\kappa }_{2}*{V}_{t}+\omega\) ) across trials. The x-axis represents the trial sequence ( t ), and the y-axis illustrates participants’ association variance ( \({\kappa }_{2}*{V}_{t}+\omega\) ). The red line represents AVGPs, and the blue line represents NVGPs. The shaded area represents S.E.M across all participants within each group (33 AVGPs, 34 NVGPs). Significance symbol conventions is *: p  < 0.05. B Participants’ estimated log association volatility ( \({V}_{t}\) ) across trials. The x-axis represents the trial sequence ( t ), and the y-axis illustrates participants’ estimated association volatility ( \({V}_{t}\) ). The y-axis is plotted on a logarithmic scale. The red line represents AVGPs, and the blue line represents NVGPs. Significance symbol conventions is ***: p  < 0.001

The association variance ( \({\kappa }_{2}*{V}_{t}+\omega\) ) is determined by the linear addition of two components: a top-down component ( \({\kappa }_{2}*{V}_{t}\) ) and a constant component ( \(\omega\) ). The top-down component indicates that a higher estimated association volatility ( \({V}_{t}\) ) leads to a larger update of the association learning rate, where \({\kappa }_{2}\) is the top-down coupling factor. The constant step indicates the default magnitude of the update in the subject. Note that the top-down factor \({\kappa }_{2}\) and the association constant step \(\omega\) are considered as traits of each subject and are fixed across trials, while the high-level estimated association volatility \({V}_{t}\) varied across trials.

Next, we sought to understand which factor of association variance contributed most to the increased association learning rate. There were no significant differences in both \({\kappa }_{2}\) ( t (58.569)  = -0.236, p  = 0.814, Cohen’s d  = 0.058, CI  = [-0.320, 0.250]) and \(\omega\) ( t (64.677)  = -0.597, p  = 0.552, Cohen’s d  = 0.146, CI  = [-1.740, 0.940]). A LMM was performed with Estimated Association Volatility ( \({V}_{t}\) ) in each trial as the dependent variable, Group (AVGPs/NVGPs), Block Type (stable/volatile), and their interaction as the fixed effect factors, and Trial as a random effect factor. The “learning to learn” theory predicts that AVGPs should be more sensitive to task statistics (i.e., volatility). Indeed, we found that AVGPs estimated higher association volatility than NVGPs ( t (21116)  = 8.453, p  < 0.001, Estimate  = 0.073, SE  = 0.009, CI  = [0.056, 0.090]). Post-hoc pairwise comparisons revealed that AVGPs had significantly higher estimated association volatility ( \({V}_{t}\) ) than NVGPs in the second stable block(stable block 2, t (211116)  = 3.737, p  < 0.001, Estimate  = 0.016, SE  = 0.004, CI  = [0.008, 0.025]) and two volatile blocks (volatile block 1, t (21116)  = 2.378, p  = 0.017, Estimate  = 0.010, SE  = 0.004, CI  = [0.002, 0.019]; volatile block 2, ( t (21116)  = 11.1778, p  < 0.001, Estimate  = 0.048, SE  = 0.004, CI  = [0.040, 0.057]) but not in the first stable block (stable block 1, t (21116)  = -0.387, p  = 0.698, Estimate  = -0.002, SE  = 0.004, CI  = [-0.011, 0.007], Fig.  4 B). This may be because the first block was a stable block. These results show that the AVGPs can detect relatively higher association volatility ( \({V}_{t}\) ) as the task proceeds and then produce a greater trial-by-trial update of the association learning rate, resulting in faster learning of low-level associations. This process is consistent with the “learning to learn” theory that AVGPs can quickly adapt to ever-changing task environments.

Furthermore, we found that the estimated association volatility \({V}_{t}\) during the volatile blocks was significantly higher than that during the stable blocks in both groups ( t (316.125)  = 19.862, p  < 0.001, Estimate  = 0.183, SE  = 0.009, CI  = [0.164, 0.201]). This result indicates that both groups can indeed recognize the different levels of volatility of the task. This is also consistent with the well-established theory in reinforcement learning that an agent should relatively increase learning rate in a volatile reward environment [ 32 ].

Superior high-level learning rate of tasks statistics in AVGPs

The above results suggest that AVGPs subjectively experience a higher high-level association volatility ( \({V}_{t}\) ) and use this information to increase the low-level association learning rate ( \({\alpha }_{t}^{R}\) ). Here, we further asked how AVGPs learn the task statistics and obtain the higher association volatility. Again, we examined the volatility learning rate ( \({\alpha }_{t}^{V}\) , Eq.  12 ), which indicates how quickly the association volatility ( \({V}_{t}\) ) evolves across trials. The volatility learning rate is plotted as a function of trials in Fig.  5 A. A LMM was performed with Log Volatility Learning Rate as the dependent variable; Group (AVGPs/NVGPs) as a fixed effect factor and Trial as a random effect factor. We found that the volatility learning rate of AVGPs consistently exceeded that of NVGPs’ ( t (211119)  = 3.995, p  < 0.001, Estimate  = 0.081, SE  = 0.020, CI  = [0.041, 0.120]).

Volatility learning in two groups. A The log volatility learning rate ( \({\alpha }_{t}^{V}\) ) over all trials of the two groups. The x-axis represents the trial sequence, and the y-axis reflects the volatility learning rate. The red line represents the AVGPs, and the blue line represents the NVGPs. The shaded area represents S.E.M across all participants within each group (33 AVGPs, 34 NVGPs). Significance symbol conventions is ***: p  < 0.001. B The unweighted volatility learning rate ( \({\overline{\alpha }}_{t}^{V}\) ) of the two groups across trials. The y-axis is plotted on a logarithmic scale. C The precision weighting factor ( \({w}_{t}^{V}\) ) of the association prediction errors of the two groups across trials. Significance symbol conventions is *: p  < 0.005. D the volatility prediction errors ( \({PE}_{t}^{V}\) ) of the two groups across trials. Significance symbol convention is n.s.: non-significant

It was mentioned earlier that an advantage of the HGF model over traditional reinforcement learning models is that the precision-weighted learning rates (including the association learning rate and the volatility learning rate) in HGF can vary from trial to trial, allowing more flexible adaptation of individual beliefs to volatilities. According to the HGF model (Eq.  12 , \({\alpha }_{t}^{V}={\overline{\alpha }}_{t}^{V}*\frac{{\kappa }_{2}}{2}*{w}_{t}^{V}\) ), the volatility learning rate ( \({\alpha }_{t}^{V}\) ) is determined by three factors: the unweighted volatility learning rate \({\overline{\alpha }}_{t}^{V}\) (see Eq.  13 ), the top-down factor \({\kappa }_{2}\) introduced above, and the precision weighting factor ( \({w}_{t}^{V}\) , Eq.  14 ) of the volatility prediction errors ( \({PE}_{t}^{V}\) Eq.  16 ). Note that \({\overline{\alpha }}_{t}^{V}\) and \({w}_{t}^{V}\) varied from trial to trial but \({\kappa }_{2}\) is a fixed value in each subject.

The trial-by-trial unweighted volatility learning rate, precision weighting factor, and volatility prediction errors are plotted as function of trials in Fig.  5 B-D. Three LMMs were performed with Unweighted Volatility Learning Rate ( \({\overline{\alpha }}_{t}^{V}\) ), Precision Weighting Factor ( \({w}_{t}^{V}\) ), and Volatility Prediction Errors ( \({PE}_{t}^{V}\) ) as the dependent variables; Group (AVGPs/NVGPs) as the fixed effect factor and Trial as a random effect factor. We found that AVGPs had an overall higher unweighted learning rate ( t (21119)  = 5.142, p  < 0.001, Estimate  = 0.219, SE  = 0.043, CI  = [0.136, 0.303]) and an overall higher precision weighting ( t (21119)  = 2.459, p  = 0.014, Estimate  = 0.048, SE  = 0.020, CI  = [0.010, 0.087]) than NVGPs. However, there was no group difference on the volatility prediction errors ( t (21119)  = -0.767, p  = 0.443, Estimate  = -0.003, SE  = 0.004, CI  = [-0.010, 0.005]).

Taken together, we found that AVGPs can perceive higher association volatility because they can learn volatility per se faster (i.e., higher volatility learning rate) rather than because of higher volatility prediction errors. This higher volatility learning rate is augmented by more optimal uncertainty processing (i.e., higher precision weighting factor).

The theory of “learning to learn” has recently been proposed as a novel mechanism of learning generalization [ 10 ], in particular the broad cross-task generalizations found in avid AVGPs. In this study, we proposed that enhanced “learning to learn” in AVGPs is achieved by an improved hierarchical dual learning system that takes into account both low-level cue-response associations and high-level task statistics (i.e., volatility). 34 AVGPs and 36 NVGPs completed a volatile reversal learning task in which participants should learn both cue-response associations and the temporal volatility of these associations (i.e., task statistics). We used Hierarchical Gaussian Filter (HGF) to quantify both low-level association learning and high-level volatility learning in the two groups and made three main observations. First, consistent with “learning to learn” and previous results, we found that AVGPs indeed exhibit a higher low-level learning rate of cue-response associations. Second, the higher low-level learning rate of associations is primarily driven by a higher high-level volatility on a trial-by-trial basis. Third, we further investigated the evolution of estimated volatility and found that the high-level learning rate of volatility per se is also higher in the AVGP group. These results strongly support the “learning to learn” theory of action video game play and show that AVGPs can quickly learn the intrinsic statistics of novel tasks and use the learned task knowledge to guide low-level learning of correct responses. Our work sheds new light on generalization in action video games and, more broadly, on cognitive training in general.

Two aspects of “learning to learn”

“Learning to learn” has two key components—enhanced learning rate and multi-level hierarchical learning.

Within the framework of “learning to learn”, enhanced learning rate in novel tasks is a new form of learning generalization. The classical theory of learning generalization posits that observers immediately and directly generalize what they have learned by inferring the shared constructs of the trained and generalization task contexts. This classical view is often referred to as immediate generalization [ 14 , 15 ]. However, immediate generalization highly depends on the recognition of shared constructs between training and generalization. This means that learned experience may be limited to some specific task components. In contrast, the “learning to learn” theory emphasizes the general ability to quickly acquire task statistics and facilitate learning in real time [ 7 , 10 , 16 ]. Most importantly, this “learning to learn” ability should not be specific to a particular task component and thus has the potential to produce broad generalizations across different types of tasks. This new form of generalization has recently been discovered in sequential perceptual learning [ 33 ] and has also been proposed to underlie broad generalization associated with action video game play [ 10 , 34 ]. Both cross-sectional and intervention studies have identified the increased learning rate, as a hallmark of “learning to learn”, associated with action video game play in perceptual [ 7 , 16 ], cognitive [ 16 ], and motor learning tasks [ 35 ].

“Learning to learn” also proposes that high-level statistical learning of task structure is the underlying mechanism for increasing learning rate. Hierarchical learning allows individuals to flexibly adjust their learning rates in response to changing environments. The environments we face are often filled with different types of uncertainty [ 17 , 36 ], such as uncertainty about how an reward is obtained and uncertainty about how tasks may evolve. A lack of flexibility in responding to environmental changes is likely to be associated with psychiatric disorders, such as social anxiety disorder and major depressive disorder [ 37 , 38 ]. Traditional reinforcement learning often assumes that the learning rate is a fixed property of an agent [ 39 ]. This means that an agent has the same learning rate across throughout the task, which is obviously suboptimal and inflexible [ 40 , 41 ]. A better approach is to adjust the learning rate according to task statistics. For example, if the task statistics (e.g., the probabilistic mapping between action and reward) change rapidly, an agent needs to increase the learning rate to adapt quickly to the changes. However, if this task statistics are stable, individuals should decrease the learning rate to avoid overfitting to noise [ 36 , 42 , 43 ]. In other words, the hierarchical form of “learning to learn” allows an agent to flexibly adjust learning speed accordingly in different tasks.

The underlying mechanisms associated with enhanced “learning to learn” in AVGPs

We speculate that several unique characteristics of action video games may be the reasons.

First, the fast pace of action video games may lead to superior cognitive functions. Fast-paced games require players to switch quickly between different scenarios or tasks [ 10 , 26 ]. Several studies have shown that AVGPs have greater task switching abilities [ 34 , 44 , 45 , 46 ]. Given limited cognitive resources [ 47 , 48 ], the reduced cognitive cost of task switching allows AVGPs to allocate more cognitive resources to hierarchical learning, leading to better “learning to learn”. The fast pace of action video games also requires players to simultaneously track and store multiple rapid processes and predict future game events in real time. For example, in a first-person shooting game (i.e., Overwatch), a player must quickly determine where other players have previously attacked and predict their possible current and next locations. Training to track and store information is associated with improved working memory in AVGPs [ 49 , 50 ]. Improved working memory allows players to retain task statistics during sequential tasks and respond more quickly and accurately.

Second, the complex spatial environments of action video games promote perceptual sensitivity. Action video games tend to contain highly complex and realistic spatial environments, and this is associated with increased perceptual sensitivity to external sensory events [ 51 ]. Enhanced perceptual sensitivity allows AVGPs to quickly and accurately detect real-time fluctuations or changes in new tasks, thereby improving “learning to learn”.

However, this is a cross-sectional study, and we cannot exclude the possibility that the people with enhanced “learning to learn’ are more attracted by action video games such that they are related. Researchers [ 52 , 53 ] postulated that the capacity to make multilevel predictions and to learn from uncertainties that emerge during gameplay will facilitate the expeditious and efficacious reduction of prediction errors in game scenarios. This will enable players to “feel good” and, as a result, select and persist with such games.

Neural mechanisms underlying enhanced “learning to learn”

What are the neural mechanisms underlying enhanced “learning to learn”? Previous studies have shown that hierarchical learning exist in the human brain. Existing studies have focused on the neural mechanisms associated with different levels of learning rates and prediction errors (PEs). A study combining HGF modeling with electroencephalogram (EEG) found that beta power in the sensorimotor cortex is negatively correlated with volatility learning rate before action execution and positively correlated with association learning rate after action execution [ 54 ]. Another EEG study found that the P300 response in the frontal and central scalp regions is positively correlated with the absolute values of low-level PEs and negatively correlated with high-level PEs [ 43 ]. In other words, beta power in sensorimotor cortex and P300 responses in the frontal and central scalp may serve as neural markers of hierarchical learning. In this study, we found both increased volatile and association learning rate. Our results predict that enhanced “learning to learn” may produce a weaker and stronger beta wave in sensorimotor cortex before and after action execution. Interestingly, these predictions are consistent with two recent EEG studies of AVGPs. In the two EEG studies, the researchers did not find the changes in beta wave power in the frontal lobes before and after movement but found that the variation of beta-wave power is greater before and after action execution in AVGPs [ 55 ]. In addition, beta wave power has been shown to increase significantly during high-intensity action video game activities [ 56 ]. Our findings also predict a stronger P300 response in the frontal and central scalp regions associated with enhanced “learning to learn”. This prediction is consistent with a recent EEG study that identified a greater amplitude of the task-evoked P300 component in AVGPs [ 57 ].

Corresponding brain regions for learning rate-weighted prediction errors at different levels demonstrated in the previous studies [ 58 , 59 ]

The studies combining HGF modeling with functional magnetic resonance imaging (fMRI) have shown that low-level PEs are encoded in dopamine-related regions of the midbrain, including the ventral tegmental area (VTA) and substantia nigra (SN). These regions have been shown to regulate dopamine release [ 60 , 61 , 62 ]. In contrast, high-level PEs are encoded in the basal forebrain, which regulates acetylcholine release [ 58 , 59 ]. These results predict stronger activities in the midbrain VTA and SN (Fig. 6 ). These predictions are consistent with several recent fMRI studies of AVGPs. One fMRI study found stronger activation of reward-related midbrain structures in AVGPs [ 63 ]. Another longitudinal fMRI study showed that action video games can increase functional connectivity within the basal ganglia [ 64 ]. Similarly, some fMRI studies have found elevated activity in the striatum, as part of the basal forebrain, of AVGPs [ 65 , 66 ]. All of these studies suggest that enhanced “learning to learn” is likely to be associated with stronger activation or inhibition in the midbrain and basal forebrain.

In conclusion, this study employed a Hierarchical Gaussian Filter (HGF) model to test 34 AVGPs and 36 NVGPs in a volatile reversal learning task. The results of the study demonstrate that AVGPs indeed rapidly extract volatility information and utilize the estimated higher volatility to accelerate learning of cue-response associations. These findings provide strong evidence for the “learning to learn” theory of generalization in AVGPs.

Availability of data and materials

The source code of Hierarchical Gaussian Filter (HGF) can be downloaded from https://translationalneuromodeling.github.io/tapas . The HGF task and data for each group of subjects, as well as the code used for analysis and plotting, can be downloaded from https://osf.io/sk82r/ .

Bavelier D, Green CS, Pouget A, Schrater P. Brain plasticity through the life span: learning to learn and action video games. Annu Rev Neurosci. 2012;35:391–416.

Article   PubMed   Google Scholar  

Green CS, Gorman T, Bavelier D. Action video-game training and its effects on perception and attentional control. Cognitive training: An overview of features and applications. 2016;107–116.

Dye MW, Green CS, Bavelier D. The development of attention skills in action video game players. Neuropsychologia. 2009;47:1780–9.

Article   PubMed   PubMed Central   Google Scholar  

Blacker KJ, Curby KM, Klobusicky E, Chein JM. Effects of action video game training on visual working memory. J Exp Psychol Hum Percept Perform. 2014;40:1992–2004.

McDermott AF, Bavelier D, Green CS. Memory abilities in action video game players. Comput Hum Behav. 2014;34:69–78.

Article   Google Scholar  

Gonçalves, E. d. S. & Castilho, G. M. d. Effects of action video game engagement on attention and working memory. Psychology & Neuroscience (2024).

Bejjanki VR, et al. Action video game play facilitates the development of better perceptual templates. Proc Natl Acad Sci U S A. 2014;111:16961–6.

Green CS, Li R, Bavelier D. Perceptual learning during action video game playing. Top Cogn Sci. 2010;2:202–16.

Green CS, Pouget A, Bavelier D. Improved probabilistic inference as a general learning mechanism with action video games. Curr Biol. 2010;20:1573–9.

Poggio T, Bizzi E. Generalization in vision and motor control. Nature. 2004;431:768–74.

Shepard RN. Toward a universal law of generalization for psychological science. Science. 1987;237:1317–23.

Schulz, E. Towards a unifying theory of generalization. UCL (University College London). (2017).

Johnson BP, et al. Generalization of procedural motor sequence learning after a single practice trial. NPJ Sci Learn. 2023;8:45.

Liu Z, Weinshall D. Mechanisms of generalization in perceptual learning. Vision Res. 2000;40:97–109.

Zhang R-Y, et al. Action video game play facilitates “learning to learn.” Communications biology. 2021;4:1154.

Behrens TE, Woolrich MW, Walton ME, Rushworth MF. Learning the value of information in an uncertain world. Nat Neurosci. 2007;10:1214–21.

Browning M, Behrens TE, Jocham G, O’Reilly JX, Bishop SJ. Anxious individuals have difficulty learning the causal statistics of aversive environments. Nat Neurosci. 2015;18:590–6.

Balleine BW, O’Doherty JP. Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology. 2010;35:48–69.

Owen AM, McMillan KM, Laird AR, Bullmore E. N-back working memory paradigm: A meta-analysis of normative functional neuroimaging studies. Hum Brain Mapp. 2005;25:46–59.

Baddeley A. Working memory: looking back and looking forward. Nat Rev Neurosci. 2003;4:829–39.

Mathys C, Daunizeau J, Friston KJ, Stephan KE. A bayesian foundation for individual learning under uncertainty. Front Hum Neurosci. 2011;5:39.

Glascher J, Daw N, Dayan P, O’Doherty JP. States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron. 2010;66:585–95.

Dayan P, Daw ND. Decision theory, reinforcement learning, and the brain. Cogn Affect Behav Neurosci. 2008;8:429–53.

Green CS, et al. Playing some video games but not others is related to cognitive abilities: A critique of Unsworth et al.(2015). Psychol Sci. 2017;28:679–82.

Novak, E. & Tassell, J. in 2017 IEEE 17th International Conference on Advanced Learning Technologies (ICALT). 142–144 (IEEE).

Dale G, Kattner F, Bavelier D, Green CS. Cognitive abilities of action video game and role-playing video game players: Data from a massive open online course. Psychology of Popular Media. 2020;9:347.

Gagne C, Zika O, Dayan P, Bishop SJ. Impaired adaptation of learning to contingency volatility in internalizing psychopathology. Elife. 2020;9: e61387.

Bejjanki VR, Sims CR, Green CS, Bavelier D. Evidence for action video game induced ‘learning to learn’in a perceptual decision-making task. J Vis. 2012;12:287–287.

Schonberg T, Daw ND, Joel D, O’Doherty JP. Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J Neurosci. 2007;27:12860–7.

Li J, Schiller D, Schoenbaum G, Phelps EA, Daw ND. Differential roles of human striatum and amygdala in associative learning. Nat Neurosci. 2011;14:1250–2.

Hein TP, et al. Anterior cingulate and medial prefrontal cortex oscillations underlie learning alterations in trait anxiety in humans. Commun Biol. 2023;6:271.

Kattner F, Cochrane A, Cox CR, Gorman TE, Green CS. Perceptual learning generalization from sequential perceptual training as a change in learning rate. Curr Biol. 2017;27:840–6.

Green CS, Bavelier D. Action video game modifies visual selective attention. Nature. 2003;423:534–7.

Gozli DG, Bavelier D, Pratt J. The effect of action video game playing on sensorimotor learning: Evidence from a movement tracking task. Hum Mov Sci. 2014;38:152–62.

Nassar MR, et al. Age differences in learning emerge from an insufficient representation of uncertainty in older adults. Nat Commun. 2016;7:11609.

Sohail A, Zhang L. Informing the treatment of social anxiety disorder with computational and neuroimaging data. Psychoradiol. 2024;kkae010.

Li H, et al. Altered cortical morphology in major depression disorder patients with suicidality. Psychoradiology. 2021;1:13–22.

Simoens J, Verguts T, Braem S. Learning environment-specific learning rates. PLoS Comput Biol. 2024;20: e1011978.

Leimar, O., Quiñones, A. E. & Bshary, R. Flexibility of learning in complex worlds. bioRxiv , 2023.2006. 2012.544544 (2023).

Jiang J, Beck J, Heller K, Egner T. An insula-frontostriatal network mediates flexible cognitive control by adaptively predicting changing control demands. Nat Commun. 2015;6:8165.

Jepma M, et al. Catecholaminergic Regulation of Learning Rate in a Dynamic Environment. PLoS Comput Biol. 2016;12: e1005171.

Liu M, Dong W, Qin S, Verguts T, Chen Q. Electrophysiological Signatures of Hierarchical Learning. Cereb Cortex. 2022;32:626–39.

Green CS, Sugarman MA, Medford K, Klobusicky E, Bavelier D. The effect of action video game experience on task-switching. Comput Hum Behav. 2012;28:984–94.

Cain MS, Landau AN, Shimamura AP. Action video game experience reduces the cost of switching tasks. Atten Percept Psychophys. 2012;74:641–7.

Karle JW, Watter S, Shedden JM. Task switching in video game players: Benefits of selective attention but not resistance to proactive interference. Acta Physiol (Oxf). 2010;134:70–8.

Google Scholar  

Bjorklund DF, Harnishfeger KK. The resources construct in cognitive development: Diverse sources of evidence and a theory of inefficient inhibition. Dev Rev. 1990;10:48–71.

Bruder G, Lubos P, Steinicke F. Cognitive resource demands of redirected walking. IEEE Trans Visual Comput Graphics. 2015;21:539–44.

Green CS, Bavelier D. Learning, attentional control, and action video games. Curr Biol. 2012;22:R197–206.

Colzato LS, van den Wildenberg WP, Zmigrod S, Hommel B. Action video gaming and cognitive control: playing first person shooter games is associated with improvement in working memory but not action inhibition. Psychol Res. 2013;77:234–9.

Chopin A, Bediou B, Bavelier D. Altering perception: the case of action video gaming. Curr Opin Psychol. 2019;29:168–73.

Deterding S, Andersen MM, Kiverstein J, Miller M. Mastering uncertainty: A predictive processing account of enjoying uncertain success in video game play. Front Psychol. 2022;13: 924953.

Andersen MM, Kiverstein J, Miller M, Roepstorff A. Play in predictive minds: A cognitive theory of play. Psychol Rev. 2023;130:462.

Palmer CE, Auksztulewicz R, Ondobaka S, Kilner JM. Sensorimotor beta power reflects the precision-weighting afforded to sensory prediction errors. Neuroimage. 2019;200:59–71.

Salminen M, Ravaja N. Oscillatory brain responses evoked by video game events: The case of Super Monkey Ball 2. Cyberpsychol Behav. 2007;10:330–8.

McMahan T, Parberry I, Parsons TD. Modality specific assessment of video game player’s experience using the Emotiv. Entertainment Computing. 2015;7:1–6.

Mishra J, Zinni M, Bavelier D, Hillyard SA. Neural basis of superior performance of action videogame players in an attention-demanding task. J Neurosci. 2011;31:992–8.

Iglesias S, et al. Cholinergic and dopaminergic effects on prediction error and uncertainty responses during sensory associative learning. Neuroimage. 2021;226: 117590.

Iglesias S, et al. Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning. Neuron. 2019;101:1196–201.

Beier KT, et al. Circuit architecture of VTA dopamine neurons revealed by systematic input-output mapping. Cell. 2015;162:622–34.

Engelhard B, et al. Specialized coding of sensory, motor and cognitive variables in VTA dopamine neurons. Nature. 2019;570:509–13.

Wise RA. Roles for nigrostriatal—not just mesocorticolimbic—dopamine in reward and addiction. Trends Neurosci. 2009;32:517–24.

Klasen M, Weber R, Kircher TT, Mathiak KA, Mathiak K. Neural contributions to flow experience during video game playing. Soc Cogn Affect Neurosci. 2012;7:485–95.

Pujol J, et al. Video gaming in school children: How much is enough? Ann. Neurol. 2016;80:424–33.

Benady-Chorney J, et al. Action video game experience is associated with increased resting state functional connectivity in the caudate nucleus and decreased functional connectivity in the hippocampus. Comput Hum Behav. 2020;106: 106200.

Kühn S, Gleich T, Lorenz RC, Lindenberger U, Gallinat J. Playing Super Mario induces structural brain plasticity: gray matter changes resulting from training with a commercial video game. Mol Psychiatry. 2014;19:265–71.

Download references

Acknowledgements

The authors thank the participants for their support to this study.

This works was supported by the National Natural Science Foundation of China (32100901) and Natural Science Foundation of Shanghai (21ZR1434700) to R-Y.Z.

Author information

Qiang Zhou and Ru-Yuan Zhang co-senior authors.

Authors and Affiliations

School of Psychology, Shanghai Jiao Tong University, Shanghai, 200030, China

Yu-Yan Gao & Ru-Yuan Zhang

Shanghai Mental Health Center, School of Medicine, Shanghai Jiao Tong University, Shanghai, 200030, China

Yu-Yan Gao, Zeming Fang & Ru-Yuan Zhang

Cixi Biomedical Research Institute, Wenzhou Medical University, Zhejiang, 315302, China

Department of Psychology, Wenzhou Medical University, Wenzhou, 325035, China

Yu-Yan Gao & Qiang Zhou

You can also search for this author in PubMed   Google Scholar

Contributions

R-Y.Z. and Y.G. conceived and designed the study. Y.G. prepared the computer program for the Behavioral task and collected the data. Y.G. and Z.F. analyzed the data. Y.G. wrote the first draft of the manuscript. R-Y.Z., Y.G., Q.Z., and Z.F. revised the manuscript.

Corresponding authors

Correspondence to Qiang Zhou or Ru-Yuan Zhang .

Ethics declarations

Ethics approval and consent to participate.

All experimental protocols were approved by the institutional review board of Shanghai Jiao Tong University (I2021115I). All research was conducted in accordance with relevant guidelines and regulations. Informed written consent was obtained from all participants.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Gao, YY., Fang, Z., Zhou, Q. et al. Enhanced “learning to learn” through a hierarchical dual-learning system: the case of action video game players. BMC Psychol 12 , 460 (2024). https://doi.org/10.1186/s40359-024-01952-x

Download citation

Received : 23 April 2024

Accepted : 14 August 2024

Published : 30 August 2024

DOI : https://doi.org/10.1186/s40359-024-01952-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Action video game
• Learning to learn
• Generalization
• Hierarchical Gaussian Filter

BMC Psychology

ISSN: 2050-7283

• General enquiries: [email protected]

Kia ora - welcome to our new website. We've tried to ensure old links will redirect to the correct pages, but some links may no longer work. Find out more .

About us Mō mātou

About the Ministry of Health and the New Zealand health system. 

Quick links

• Consultations

New Zealand’s health system

• Government Policy Statement
• Health system roles and organisations
• Overview and statutory framework
• Vote Health

Organisation and leadership

• Chief clinical officers
• Executive Governance Team
• Ministry directorates
• Public Health Agency
• Statutory roles

Corporate publications

• Annual reports
• Briefings to incoming Ministers
• Health and Independence reports
• Mental health annual reports
• Strategic intentions
• Current vacancies
• Employee benefits
• Information for job applicants

Regulation & legislation Ngā here me ngā ture

Health providers and products we regulate, and laws we administer.

• Legislation we administer
• Rest home audit reports
• Assisted dying

Certification of health care services

Drug checking, health practitioners competence assurance act, medicinal cannabis.

• Medicines control

Mental health and addiction

• Notifiable diseases

Pharmacy licensing

Psychoactive substances, smoked tobacco, vaping, herbal smoking and smokeless tobacco, strategies & initiatives he rautaki, he tūmahi hou.

How we’re working to improve health outcomes for all New Zealanders.

Health strategies

• New Zealand Health Strategy
• Health of Disabled People Strategy
• Pae Tū: Hauora Māori Strategy
• Rural Health Strategy
• Te Mana Ola: The Pacific Health Strategy
• Women’s Health Strategy

Programmes and initiatives

• Gambling harm
• Health workforce
• Mental health, addiction and suicide prevention
• Pacific health
• Smokefree 2025

Māori health Hauora Māori

Increasing access to health services, achieving equity and improving outcomes for Māori.

• Subscribe for Māori health updates

Ao Mai te Rā anti-racism initiative

Māori health models, te tiriti o waitangi framework, wai 2575 health services and outcomes inquiry, whakamaua: māori health action plan, statistics & research he tatauranga, he rangahau.

Data and insights from our health surveys, research and monitoring.

• New Zealand Health Survey
• Health targets

System monitoring

• Mental health and addiction targets
• Planning and performance data
• New Zealand Gambling Survey
• Past surveys

Statistics and data sets

• Māori health data and statistics
• Mental health data and statistics
• Obesity data and statistics
• Suicide data and statistics
• Tobacco data and statistics
• COVID-19 and immunisation research projects
• Gambling harm research and evaluation
• Our role in health research

News Ngā karere

Find all the latest news and updates from the Ministry of Health - Manatū Hauora.

• Media centre

Research into health and wellbeing impacts of adverse weather conditions

Key findings and recommendations on the health and wellbeing impacts of Cyclone Gabrielle have been published today.

• News article
• Environmental health
• Emergency management

Three additional ‘Safe Areas’ announced

New ‘Safe Areas’ will be in place around three abortion service providers from 27 September 2024, to protect the safety, wellbeing, privacy and dignity of people who are accessing or providing abortion services. 

Ministry prosecutes vape retailer

A Christchurch shopkeeper has been fined $56,000 for the illegal sale of vapes and cigarettes, including selling vapes to minors.  

2023 Health and Independence Report published

The Ministry of Health – Manatū Hauora has published the 2023 Health and Independence Report. 

Redesigned Ministry of Health website now live

The redesigned Ministry of Health website has been launched today.

• About the Ministry

Consultation for Draft Strategy to Prevent and Minimise Gambling Harm 2025/26 to 2027/28 opens

Today the Ministry of Health – Mānatu Hauora has opened consultation on the proposed strategy to prevent and minimise gambling harm for the next three years.

Consultation to update the New Zealand clinical trial regulatory guidelines

A plan aiming to streamline clinical trial reporting procedures in some circumstances is now out for consultation. Medsafe is seeking public feedback on a series of proposed updates to the regulatory guidelines for people conducting clinical trials for medicines and medical devices.

Pause on Surgical mesh update

It’s been one year since the Director-General of Health, Dr Diana Sarfati, supported a pause in the use of surgical mesh to treat stress urinary incontinence (SUI) on the recommendation of the Surgical Mesh Roundtable (MRT). 

• Elective services

Publication of the Registrar (assisted dying) Annual Report 2024

The Ministry of Health has published the Registrar (assisted dying) Annual Report to the Minister of Health, covering the period 1 April 2023 to 31 March 2024. 

Severe disease invasive Group A Strep now a ‘notifiable’ disease

Invasive group A streptococcal infection (iGAS) will become a ‘notifiable’ disease on 1 October 2024. In addition, Cabinet has now also removed COVID-19 from the list of quarantinable infectious diseases.

• Public health

New Zealand’s first health strategy for people with rare disorders

• Diseases & conditions
• Health system

End of Life Choice Act Review

The Ministry of Health will be inviting the public to share their views on the End of Life Choice Act as part of a review of the Act.

Ministry warns against the use of progesterone for ‘abortion reversal’

2023 drug harm index published, review of evidence behind suicide prevention initiatives released, filter by type.

• Media release (529)
• News article (645)
• Programme update (10)

Filter by topic

• Abortion (5)
• About the Ministry (9)
• Addiction (1)
• Alcohol (2)
• Allied health (4)
• Antimicrobial resistance (1)
• Assisted dying (5)
• Babies and toddlers (0–2 years) (3)
• Border health (1)
• Bowel cancer (2)
• Certification of health care services (3)
• Child health (1)
• COVID-19 (948)
• Crown entities & agencies (2)
• Diabetes (1)
• Disability (1)
• Diseases & conditions (13)
• Drinking-water (3)
• Elective services (3)
• Emergency management (190)
• Environmental health (5)
• Funding & payments (1)
• Gambling harm (2)
• Health IT (8)
• Health system (11)
• Health targets (1)
• Hospital events (1)
• Hospitals & specialist care (1)
• Immunisation (21)
• Inequalities (1)
• Influenza (16)
• Long-term conditions (2)
• Māori health (4)
• Maternity (3)
• Measles (5)
• Medicinal cannabis (3)
• Medicine issues (10)
• Medicines control (6)
• Mental health (17)
• Ministerial health committees (1)
• Mortality (1)
• Natural disasters (2)
• News & media (5)
• Notifiable diseases (7)
• Nursing (3)
• Nutrition (5)
• Oral health (4)
• Pacific health (3)
• Preventative health (2)
• Primary health care (3)
• Public health (16)
• Public Health Advisory Committee (2)
• Public health units (1)
• Radiation safety (3)
• Regulation (3)
• Screening (2)
• Sexual & reproductive health (3)
• Smokefree law (5)
• Statistics (5)
• Suicide (8)
• Support services (2)
• Surveys (3)
• Tobacco (7)
• Tobacco control (8)
• Vaping (13)
• Workforce (8)
• Youth health (1)
• email Email
• print Print
• chat Feedback

© Ministry of Health – Manatū Hauora

• Get new issue alerts Get alerts
• Submit a Manuscript

Secondary Logo

Journal logo.

Colleague's E-mail is Invalid

Your message has been successfully sent to your colleague.

Save my selection

The anti-neoplastic impact of thymoquinone from Nigella sativa on small cell lung cancer: In vitro and in vivo investigations

Khan, Mahjabin; Lam, Sze-Kwan; Yan, Sheng; Feng, Yuqian; Chen, Caoyang; Ko, Frankie Chi-Fat; Ho, James Chung-Man

Department of Medicine, The University of Hong Kong, Queen Mary Hospital, Hong Kong SAR, China

For correspondence: Dr. James Chung-Man Ho, Department of Medicine, The University of Hong Kong, Queen Mary Hospital, Pokfulam, Hong Kong SAR, China. E-mail: [email protected]

This is an open-access article distributed under the terms of the Creative Commons Attribution-Noncommercial-Share Alike 4.0 Unported, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Purpose: 

Malignant and aggressive, small cell lung cancer (SCLC) constitutes about 15% of all diagnosed lung cancer cases. With primary therapeutic options such as chemotherapy accompanied by debilitating side effects, interest has been soaring in the therapeutic competencies of herbs. The pharmacological driving force behind the beneficial properties of Nigella sativa is the quinone, thymoquinone (TQ). The anti-cancer effects of TQ on different cancers have been extensively studied. Nonetheless, only one paper in the entire National Center for Biotechnology Information (NCBI) database describes its effects on SCLC. A more detailed investigation is required.

Methods: 

The current study examined the impact of TQ in vitro on five SCLC cell lines and in vivo in a nude mouse xenograft model. The following in vitro effects of TQ on SCLC were evaluated: (a) cell viability; (b) apoptosis; (c) cell cycle arrest; (d) intracellular reactive oxygen species (ROS) levels, and (e) protein expression in concomitant signaling pathways. For the in vivo effects of TQ on SCLC, (a) tumor volume was measured, and (b) selected protein expression in selected concomitant signaling pathways was determined by Western blotting.

Result: 

In general, TQ reduced cell viability, induced apoptosis and cell cycle arrest, depleted ROS, and altered protein expression in associated signaling pathways. Furthermore, TQ exhibited a tumor-suppressive effect in an H446 SCLC xenograft model.

Conclusion: 

The cytotoxic impact of TQ arising from anti-cancer mechanisms was elucidated. The positive results obtained in this study warrant further investigation.

INTRODUCTION

Small cell lung cancer (SCLC) represents 15% of all clinically diagnosed lung tumor cases. [ 1 ] It typically has a clinically poor prognosis and an increased mortality statistic [ 2 ] with a five-year survival rate of only 7%. [ 3 ]

The methanolic extract of Nigella sativa seeds constitutes an active ingredient responsible for its medicinal properties, thymoquinone (TQ). [ 4 ] With regards to its anti-cancer effects, TQ has been shown to attack tumor cells in a variety of ways, including suppressing cell proliferation through cell cycle arrest, [ 5 ] suppressing STAT3, [ 6 ] interfering with tumor cell survival through suppressing Akt [ 7 ] and extracellular signal-regulated kinase (ERK) phosphorylation [ 8 ] and causing ER stress, resulting in mitochondrial dysfunction and cleavage of PARP and caspases 3, 8, and 7. [ 9 ] TQ has been shown to exhibit these diverse anti-cancer mechanisms in distinct types of cancers, including those of the lung, colon, pancreas, multiple myeloma, and myeloid leukemia. [ 10 ]

The antitumor effects of different compounds [ 11-15 ] and TQ have been investigated in NSCLC. Nonetheless, for TQ in SCLC, only one study exists in the entire National Center for Biotechnology Information (NCBI) repository. The current study probes the anti-cancer impact of TQ in SCLC. TQ reduced cell viability, induced apoptosis and cell cycle arrest, depleted reactive oxygen species (ROS), and altered protein expression in associated signaling pathways in a panel of SCLC cell lines. In addition, TQ exhibited a tumor-suppressive effect in an H446 SCLC xenograft model.

MATERIALS AND METHODS

Cell culture.

A panel of five human SCLC cell lines (H69, DMS79, H446, H841, and SW1271) [ Supplementary Table 1 ] was purchased from the American Type Culture Collection (Manassas, VA, USA). All cell lines were cultured in RPMI-1640 medium (Gibco®, Life Technologies, Waltham, Massachusetts, USA) supplemented with 10% fetal bovine serum (FBS) (Gibco®). They were maintained in a CO 2 incubator set in a humidified chamber with 5% CO 2 at 37°C. The study protocol was approved (dated 9 March 2021) by the Committee on the Use of Live Animals in Teaching and Research (CULATR) of the University of Hong Kong (The approved CULATR number is 5631-21).

Obtained as a suspension cell line, H69 was cultured over time, and an adherent subpopulation was acquired. This study employed this adherent subpopulation, which is referred to as H69-adherent. In comparison, H841 was obtained as a mixed cell line comprising both suspension and adherent subpopulations. Cultured over time, the adherent subpopulation dominated and was then used throughout the study and is referred to as H841-adherent. The remaining three cell lines – DMS79, H446, and SW1271 – remained in their original form (full suspension, mixed, and fully adherent, respectively) when cultured over time.

Cell viability assay

The effect of TQ (Merck), on the viability of SCLC cells was determined by two methods: the 3-(4,5-dimethyl-thiazoyl-2-yl)-2,5-diphenyl-tetrazolium bromide (MTT) method and the crystal violet staining method. [ 15 ] Cell lines containing suspended cells, DMS79 and H446, were exposed to the MTT method, and cell lines that comprised adherent cells, H69-adherent, H841-adherent, and SW1271, were subjected to the crystal violet staining protocol. H69-adherent (5000 cells/well), DMS79 (20,000 cells/well), H446 (10,000 cells/well), H841-adherent (5000 cells/well), and SW1271 (5000 cells/well) cells were seeded. Cells were treated with a serial dilution of TQ for 24, 48, and 72 hours to investigate its dose- and time-dependent effects. After TQ-exposure of DMS79 and H446 cells for the desired time, MTT solution was added to each well and incubated for two hours at 37°C. Dimethyl sulfoxide (DMSO) was added to solubilize the formazan crystals. H69-adherent, H841-adherent, and SW1271 cells were fixed with 35 μl 1% formaldehyde (Santa Cruz, Dallas, Texas, USA) at room temperature for 10 minutes and then incubated with 100-μl crystal violet (Merck) at room temperature for 15 minutes, followed by cell lysing with 1% sodium dodecyl sulfate for half an hour. The optical density was measured at 570 nm on the microplate reader (FLUOstar OPTIMA, BMG Labtech GmbH, Ortenberg, Germany). A minimum of three independent experiments were carried out, with each dose performed in sextuplicate in each experiment.

H841-adherent cells (5000 cells/well) were exposed to various concentrations of NAC (Sigma-Aldrich) for 24 hours. Cell viability was then determined.

Flow cytometry assays

Various cellular events including mitochondrial membrane depolarization (MMD), apoptosis, cell cycle arrest, and ROS level were evaluated by flow cytometry. CytoFLEX S (Beckman Coulter, CA, USA) was employed for all samples in all assays studied through flow. All processes were computed and deciphered using FlowJo software v10 (BD Biosciences).

Annexin V-PE/7-aminoactinomycin D staining

Apoptosis was evaluated using a phycoerythrin (PE)-conjugated annexin-V/7-aminoactinomycin D (AAD) kit (BD Biosciences). Briefly, 100,000 cells/well were seeded in six-well plates and exposed to TQ for 24 hours, then harvested, washed, and re-suspended in binding buffer. Cells were stained for one hour at room temperature with fluorescein isothiocyante (FITC)-annexin V (Ex/Em = 494 nm/518 nm)/7-AAD (Ex/Em = 546 nm/647 nm), and signals read by CytoFLEX S with FL1/FL12 (FITC and PC5.5, respectively) channels (Beckman Coulter, CA, USA). The cell populations within the FITC+/7-AAD+ and FITC+/7-AAD− quadrants were relevant to result analysis. [ 15 ]

Cell cycle arrest

Cell cycle arrest was examined in cells treated with TQ for 48 hours and stained with propidium iodide (PI) (Sigma-Aldrich). Briefly, cells were harvested, washed with phosphate-buffered saline (PBS), and fixed with 70% ethanol at −20°C overnight. Fixed cells were then centrifuged, washed once with PBS, and incubated at 4°C in the dark for 30–40 minutes with RNase A (50 μg/ml) and PI (25 μg/ml) in serum-free medium. The intensity of fluorescence was measured by the FL10 channel in CytoFLEX S. [ 15 , 16 ]

Measurement of MMD

Briefly, 100,000 cells/well were seeded in six-well plates and TQ-treated for 24 hours. They were then harvested, washed, and stained at 37°C in the dark for 25 minutes with 5,5’,6,6’-tetrachloro-1,1’,3,3’-tetraethylbenzimidazolylcarbocyanine iodide (JC-1, 2 μM) (Sigma-Aldrich). Cells were read through the FL1/FL10 (FITC and PE, respectively) channels of CytoFLEX S. [ 11 ]

Detection of ROS

ROS was examined by 2’,7’- dichlorodihydrofluorescein diacetate (H2DCFDA, Ex/Em = 500/520 nm, Thermo Fisher Scientific) staining. Briefly, 100,000 cells/well were seeded in 6-well plates and TQ-treated for 24 hours. They were then harvested, washed, and incubated with 1 μM of H2DCFDA in serum-free medium for 15 minutes at 37°C in the dark, and read out from the FL1 (FITC) channel on CytoFLEX S. [ 15 , 17 ]

Protein extraction, gel electrophoresis, and immunoblotting

Cells treated with respective concentrations of TQ for 72 hours were lysed with radio-immunoprecipitation assay (RIPA) buffer (Thermo Fisher Scientific) and total protein lysate was acquired. The Western blot was performed as previously described. [ 18 ] Blocking was performed according to the manufacturer’s directions (with either 5% BSA or 5% milk), after which membranes were incubated at 4°C overnight with primary antibodies: cyclin A2, (Cell Signaling Technology, 4656P), cyclin E1 (Cell Signaling Technology, 4129T), cyclin B1 (Cell Signaling Technology, 4138P), cyclin D3 (Cell Signaling Technology, 2936P), cdc2 (Cell Signaling Technology, 9116T), Bcl-2 (Cell Signaling Technology, 15071S), PARP-1 (Santa Cruz Biotechnology, sc-7150), and β-actin (Sigma-Aldrich, A1978), followed by incubation with the corresponding secondary antibody at room temperature for at least 2 hours. An enhanced chemiluminescence (ECL) kit (GE Healthcare) was used to detect protein expression. Beta-actin was selected as a house-keeping protein. [ 15 ] The band intensities were analyzed with GelQuant version 1.8.2.

Tumor growth inhibition in the H446 xenograft model

TQ was dissolved in 20% Kolliphor EL (Sigma-Aldrich) in PBS and sonicated for 20 minutes. A tumor xenograft model was established with nude mice (female, four weeks old, BALB/cAnN-nu, Charles River Laboratories, Wilmington, USA). H446 cells (5,000,000) were subcutaneously inoculated into the upper back of each mouse, causing a palpable tumor of approximately 50–100 mm 3 to develop 3–4 weeks later. Mice were then split into treatment groups, with five mice in each group: control (20% Kolliphor EL in PBS) and TQ (2.5 and 5 mg/kg). Treatments were administered intraperitoneally on alternate days. Tumor size (with a standard caliper) and body weight were measured. Tumor volume (V) was calculated using the formula V = (length × width × depth)/2. Mice were sacrificed when humane endpoints (tumor volume >600 mm 3 or 20% drop in body weight) were reached. The post-treatment weight was then determined. [ 15 , 18 ] The study protocol was approved by the Committee on the Use of Live Animals in Teaching and Research (CULATR) of the University of Hong Kong (CULATR 5631-21).

Statistical analysis

Experiments were repeated at least three times, and data were analyzed. Student’s two-tailed t -test was used for the comparison of pairs. The difference between groups (more than two groups) was analyzed using variance analysis (ANOVA) by Prism (GraphPad Software, La Jolla, Southern California, United States). A P value < 0.05 was considered statistically significant (*: P <0.05, **: P <0.01, ***: P <0.001).

Cell growth inhibition and apoptosis induced by TQ in SCLC cell lines

SCLC cell lines were incubated for 24, 48, and 72 hours with various concentrations of TQ. A dose-dependent reduction in cell viability was observed [ Figure 1a , Supplementary Table 2 ]. H446 cells were the most TQ-sensitive cell line at different incubation time points. DMS79 cells were the most TQ-resistant cell line in 24-hour incubation. H69-adherent, H841-adherent, and SW1271 cells showed a similar trend at different incubation time points.

Phosphatidylserine externalization is one of the signs of apoptosis, and can be determined through Annexin V-PE/7-aminoactinomycin D (AAD) staining. TQ induced dose-dependent apoptosis of all cell lines [ Figure 1b ]. A low dose of TQ (6 μM) was required to induce apoptosis in H446 cells, while a higher dose (32 μM) in SW1271 cells was needed. A moderate dose (18μ μM) was able to induce apoptosis in H69-adherent, DMS79, and H841 cells.

PARP downregulation was demonstrated in H446 cells [ Figure 1c ], but basal PARP expression was undetectable in other cell lines.

Cell cycle arrest induced by TQ in SCLC cell lines

TQ caused S-phase arrest in H69-adherent, H841-adherent, and SW1271 cells, as well as G1 arrest in DMS79 cells, all demonstrated by an increase in cell proportion in the corresponding cell cycle stages [ Figure 2a ]. TQ induced upregulation of cyclin B1 and cyclin D3 in H69-adherent and H446 cells, respectively. Cyclins A2, E1, and cdc2 were downregulated, while cyclin D3 was upregulated in H841-adherent cells [ Figure 2b ]. TQ had no significant effects on cyclin H, cdk2, cdk4, cdk7, or cdk9 (data not shown).

MMD induced by TQ in SCLC cell lines

JC-1 mainly accumulated as aggregates in the mitochondria (red fluorescence, Ex/Em = 550 nm/600 nm) and a few were present as monomers in the cytosol (green fluorescence, Ex/Em = 485 nm/535 nm) in non-apoptotic cells. When MMD occurred, the green fluorescence (monomeric JC-1) increased. MMD was elevated in all SCLC cell lines upon exposure to TQ [ Figure 3a ]. Simultaneously, Bcl-2 (anti-apoptotic protein) was downregulated in DSM79 cells only [ Figure 3b ] but not in H446 cells (data not shown). On the other hand, basal Bcl-2 was undetectable in other cell lines. Both elevation of MMD and downregulation of anti-apoptotic protein are signs of apoptosis.

Depletion of ROS was induced by TQ in SCLC cell lines

TQ demonstrated antioxidative behavior by depleting intracellular ROS in all SCLC cells except H446 cells [ Figure 4a ], consequentially driving cells towards suicide. In H69-adherent cells, ROS decreased to 65% and 40% by 9 and 45 μM, respectively. In DMS79 cells, 18 μM TQ downregulated ROS to very low levels (<10%). In H841 cells, ROS decreased to about 50% by 9μ μM TQ and gradually to about 25% by 45 μM TQ. In SW1271 cells, ROS was decreased to 50–60% when treated with 6–40 μM TQ.

Employing H841-adherent cells as a model, NAC was shown to dose-dependently decrease cell viability over 24 hours [ Figure 4b ]. This showed that scavenging free radicals and depleting intracellular ROS levels indeed impaired the viability of SCLC cells.

Tumor suppressive effect of TQ in the H446 xenograft model

In the H446 xenograft model, TQ was administered 28 days post-inoculation when the average palpable tumor size reached 50–100 mm 3 (15/18 mice). The tumor in mice treated with 2.5 mg/kg TQ demonstrated a similar growth rate to that in the control arm, whereas mice that received 5 mg/kg TQ demonstrated a significant tumor suppressive response on day seven relative to the control group [ Figure 5a ]. After eight days of treatment, 5 mg/kg TQ induced about 10% reduction in body weight but was not statistically significant. One mouse in the 5 mg/kg TQ-treated arm had to be sacrificed owing to a significant drop in body weight (20% reduction) on day 15 [ Figure 5b ]. The median survival of control, TQ (2.5 μM), and TQ (5μ μM) was 10 days, 10 days, and 13 days, respectively. However, the difference in median survival between the control and 5 mg/kg TQ groups was insignificant ( P = 0.2929) [ Figure 5c ]. Apart from weight loss, no other side effect of TQ was observed. With regards to significant molecular activity, TQ downregulated intratumoral PARP [ Figure 5d ], which may impair DNA damage repair.

The present study focused on a panel of five SCLC cell lines to scrutinize the effects of TQ on cell viability. All were found to be TQ-sensitive with IC 50 value around 20 μM. Similar results have been achieved by other research teams. [ 7 , 19 ] TQ was observed to dose-dependently increase the percentage of annexin-V positive cells in all SCLC cell lines, confirming the occurrence of apoptosis. These results are consistent with cell viability data. Collectively, these observations confirm that TQ causes cell death via apoptosis. The same results have been evidenced in another SCLC cell line H146 [ 20 ] as well as in other cancers, including breast neoplasm. [ 19 ]

With reference to cancer cell survival, TQ has been shown to exhibit a suppressive effect on different tumor types including breast cancer, [ 7 ] esophageal cancer, [ 21 ] cholangiocarcinoma, [ 22 ] and primary effusion lymphoma, [ 23 ] primarily via inhibition of Akt, a vital cell survival kinase. As well as PI3K-Akt pathway inhibition, TQ-mediated inhibition of cell survival involved stimulation of pro-apoptotic proteins, including Bax, and suppression of anti-apoptotic proteins including XIAP, survivin, Bcl-2, and Bcl-xL, as demonstrated in breast cancer cells. [ 24 ] Furthermore, a TQ-mediated reduction in cell survival has been shown in various cancers including NSCLC [ 8 ] and glioblastoma [ 25 ] to often involve modulation of the mitogen-activated protein kinase (MAPK) family of kinases including p38 MAPK, c-Jun N-terminal kinase (JNK), and ERK. Nonetheless, in SCLC, TQ did not alter protein expression in the PI3K-Akt and MAPK pathways (data not shown).

Focusing now on proliferation and the cell cycle, TQ may inhibit cell proliferation by causing cell cycle arrest. TQ has been shown to mediate G0/G1 cell cycle arrest in acute T-cell leukemia, [ 26 ] G1/S cell cycle arrest in NSCLC, [ 20 ] and prostate cancer, [ 5 ] and G2/M cell cycle arrest in cholangiosarcoma. [ 22 ] In our study of SCLC, TQ induced S-phase arrest in three cell lines and G1 arrest in one cell line. The associated cell cycle protein effects included cyclin B1 and D3 upregulation and cyclin A2, E1, and cdc2 downregulation in a cell line-specific manner. The upregulation of cyclin D3 may be attributed to the fact that cyclin D is expressed during the early G1 phase, and the observed augmentation of cyclin D3 may be the driving force for cells to transition from the G1 phase to the S phase where arrest occurred. No notable alteration of cyclin H expression was observed; its expression and activity remained constant throughout the cell cycle. [ 27 ] The principal process whereby TQ exerted its anti-proliferative effects was via the downregulation of cyclins A, [ 5 ] D1, [ 24 ] D2, [ 28 ] E, [ 24 ] and cyclin-dependent kinases (CDKs).

The downregulation of Bcl-2 in response to TQ administration in DMS79 cells in the present study is in concert with the effect of TQ on Bcl-2 in other cancers including colorectal, [ 29 ] breast, [ 19 ] and acute lymphoblastic leukemia. [ 30 ] This shows that TQ causes Bcl-2-mediated apoptosis in DMS79 cells. Bak was nonetheless independent of TQ dosage (data not shown). For MMD, TQ induced dose-dependent MMD in all SCLC cells. The same results have been obtained in other cancers, including bladder cancer. [ 9 , 31 ]

With regards to ROS in the current study, TQ dose-dependently decreased intracellular ROS levels in all SCLC cells except H446 cells upon 24-hour treatment with TQ. Therefore, TQ acted as an antioxidant in SCLC cells, depleting intracellular ROS levels and eventually triggering cell death. The antioxidative property of TQ is evident and well documented. TQ scavenges numerous ROS species including superoxide anion O 2 •− and hydroxyl radical (OH•), [ 32 ] and its scavenging ability parallels that of superoxide dismutase (SOD) against superoxide. [ 33 ] Additionally, TQ exerts its antioxidative effect by upregulating nuclear protein nuclear factor-erythroid 2 related factor 2 (Nrf2), hence amplifying antioxidant response element (ARE) expression. This is the proposed mechanism that underlies the antioxidative effect of TQ in triple-negative breast cancer cells. [ 34 ] NAC is a notable antioxidant and scavenger of free radicals. To confirm that the antioxidative action of TQ is anti-survival for cells, H841 cells were employed as a model and treated with NAC. NAC confirmed that ROS depletion led to a decrease in the cell viability of SCLC cells.

Turning to in vivo investigations, TQ has been scrutinized in xenograft models of various cancers. The A549 NSCLC cell line has been xenografted into CD-1 nude mice treated with 20 mg/kg TQ. Mice demonstrated a decrease in tumor size with an insignificant change in body weight. [ 35 ] Another study examined BALB/c athymic nude mice xenografted with gastric cancer cells. TQ was administered intraperitoneally at 10, 20, and 30 mg/kg three times per week for 30 days and resulted in a considerable reduction in tumor weight and size compared with mice in the control group. The TUNEL assay demonstrated the existence of an appreciably higher number of apoptotic bodies in mice treated with TQ compared with untreated mice. Moreover, TQ was seen to attenuate STAT3 phosphorylation. [ 36 ] In NOD-SCID and NOG mouse models of colorectal cancer stem/progenitor cells, 20 mg/kg TQ was administered intraperitoneally three times per week for 21 days. Tumor growth was significantly inhibited with no detrimental impact on body weight accompanied by the induction of apoptosis and suppression of NF-κB and MEK signaling. [ 37 ]

There are several limitations to this study. Although the main mechanisms by which TQ inhibited SCLC cell survival were deciphered in vitro and in vivo , in-depth studies are needed to decode the intricacies of the major signaling pathways. In addition, the SCLC cell lines used in the current study were all derived from a Western population, and the results may not be generalizable to other ethnicities, including the Far East population.

Our study is one of many to delve into the anti-neoplastic cellular effects of TQ, but it is the first detailed investigation to focus on its anti-tumoral effects in SCLC. Findings from the present study provide an elaborate foundation from which to interpret the anticancer effects of TQ in SCLC. Additional investigations are warranted to extend the present study.

CONCLUSIONS

TQ exhibits anti-cancer effects in SCLC by inducing apoptosis, MMD and cell cycle arrest as well as depleting intracellular ROS levels in vitro . In an H446 SCLC xenograft model, TQ decreased tumor size and downregulated PARP.

Financial support and sponsorship

Conflicts of interest.

There are no conflicts of interest.

• Cited Here |
• Google Scholar

Apoptosis; cell cycle arrest; small cell lung cancer; thymoquinone; xenografts

• + Favorites
• View in Gallery