ai in education a systematic literature review

• Review article
• Open access
• Published: 31 October 2023

Role of AI chatbots in education: systematic literature review

• Lasha Labadze   ORCID: orcid.org/0000-0002-8884-2792 1 ,
• Maya Grigolia   ORCID: orcid.org/0000-0001-9043-7932 2 &
• Lela Machaidze   ORCID: orcid.org/0000-0001-5958-5662 3  

International Journal of Educational Technology in Higher Education volume  20 , Article number:  56 ( 2023 ) Cite this article

80k Accesses

77 Citations

176 Altmetric

Metrics details

A Correction to this article was published on 15 April 2024

This article has been updated

AI chatbots shook the world not long ago with their potential to revolutionize education systems in a myriad of ways. AI chatbots can provide immediate support by answering questions, offering explanations, and providing additional resources. Chatbots can also act as virtual teaching assistants, supporting educators through various means. In this paper, we try to understand the full benefits of AI chatbots in education, their opportunities, challenges, potential limitations, concerns, and prospects of using AI chatbots in educational settings. We conducted an extensive search across various academic databases, and after applying specific predefined criteria, we selected a final set of 67 relevant studies for review. The research findings emphasize the numerous benefits of integrating AI chatbots in education, as seen from both students' and educators' perspectives. We found that students primarily gain from AI-powered chatbots in three key areas: homework and study assistance, a personalized learning experience, and the development of various skills. For educators, the main advantages are the time-saving assistance and improved pedagogy. However, our research also emphasizes significant challenges and critical factors that educators need to handle diligently. These include concerns related to AI applications such as reliability, accuracy, and ethical considerations.

Introduction

The traditional education system faces several issues, including overcrowded classrooms, a lack of personalized attention for students, varying learning paces and styles, and the struggle to keep up with the fast-paced evolution of technology and information. As the educational landscape continues to evolve, the rise of AI-powered chatbots emerges as a promising solution to effectively address some of these issues. Some educational institutions are increasingly turning to AI-powered chatbots, recognizing their relevance, while others are more cautious and do not rush to adopt them in modern educational settings. Consequently, a substantial body of academic literature is dedicated to investigating the role of AI chatbots in education, their potential benefits, and threats.

AI-powered chatbots are designed to mimic human conversation using text or voice interaction, providing information in a conversational manner. Chatbots’ history dates back to the 1960s and over the decades chatbots have evolved significantly, driven by advancements in technology and the growing demand for automated communication systems. Created by Joseph Weizenbaum at MIT in 1966, ELIZA was one of the earliest chatbot programs (Weizenbaum, 1966 ). ELIZA could mimic human-like responses by reflecting user inputs as questions. Another early example of a chatbot was PARRY, implemented in 1972 by psychiatrist Kenneth Colby at Stanford University (Colby, 1981 ). PARRY was a chatbot designed to simulate a paranoid patient with schizophrenia. It engaged in text-based conversations and demonstrated the ability to exhibit delusional behavior, offering insights into natural language processing and AI. Developed by Richard Wallace in 1995, ALICE (Artificial Linguistic Internet Computer Entity) was an early example of a chatbot using natural language processing techniques that won the Loebner Prize Turing Test in 2000–2001 (Wallace, 1995 ), which challenged chatbots to convincingly simulate human-like conversation. Later in 2001 ActiveBuddy, Inc. developed the chatbot SmarterChild that operated on instant messaging platforms such as AOL Instant Messenger and MSN Messenger (Hoffer et al., 2001 ). SmarterChild was a chatbot that could carry on conversations with users about a variety of topics. It was also able to learn from its interactions with users, which made it more and more sophisticated over time. In 2011 Apple introduced Siri as a voice-activated personal assistant for its iPhone (Aron, 2011 ). Although not strictly a chatbot, Siri showcased the potential of conversational AI by understanding and responding to voice commands, performing tasks, and providing information. In the same year, IBM's Watson gained fame by defeating human champions in the quiz show Jeopardy (Lally & Fodor, 2011 ). It demonstrated the power of natural language processing and machine learning algorithms in understanding complex questions and providing accurate answers. More recently, in 2016, Facebook opened its Messenger platform for chatbot development, allowing businesses to create AI-powered conversational agents to interact with users. This led to an explosion of chatbots on the platform, enabling tasks like customer support, news delivery, and e-commerce (Holotescu, 2016 ). Google Duplex, introduced in May 2018, was able to make phone calls and carry out conversations on behalf of users. It showcased the potential of chatbots to handle complex, real-time interactions in a human-like manner (Dinh & Thai, 2018 ; Kietzmann et al., 2018 ).

More recently, more sophisticated and capable chatbots amazed the world with their abilities. Among them, ChatGPT and Google Bard are among the most profound AI-powered chatbots. ChatGPT is an artificial intelligence chatbot developed by OpenAI. It was first announced in November 2022 and is available to the general public. ChatGPT’s rival Google Bard chatbot, developed by Google AI, was first announced in May 2023. Both Google Bard and ChatGPT are sizable language model chatbots that undergo training on extensive datasets of text and code. They possess the ability to generate text, create diverse creative content, and provide informative answers to questions, although their accuracy may not always be perfect. The key difference is that Google Bard is trained on a dataset that includes text from the internet, while ChatGPT is trained on a dataset that includes text from books and articles. This means that Google Bard is more likely to be up-to-date on current events, while ChatGPT is more likely to be accurate in its responses to factual questions (AlZubi et al., 2022 ; Rahaman et al., 2023 ; Rudolph et al., 2023 ).

Chatbots are now used across various sectors, including education. Most of the latest intelligent AI chatbots are web-based platforms that adapt to the behaviors of both instructors and learners, enhancing the educational experience (Chassignol et al., 2018 ; Devedzic, 2004 ; Kahraman et al., 2010 ; Peredo et al., 2011 ). AI chatbots have been applied in both instruction and learning within the education sector. Chatbots specialize in personalized tutoring, homework help, concept learning, standardized test preparation, discussion and collaboration, and mental health support. Some of the most popular AI-based tools /chatbots used in education are:

Bard, introduced in 2022, is a large language model chatbot created by Google AI. Its capabilities include generating text, language translation, producing various types of creative content, and providing informative responses to questions. (Rudolph et al., 2023 ). Bard is still under development, but it has the potential to be a valuable tool for education.

ChatGPT, launched in 2022 by OpenAI, is a large language model chatbot that can generate text, produce diverse creative content, and deliver informative answers to questions (Dergaa et al., 2023 ; Khademi, 2023 ; Rudolph et al., 2023 ). However, as discussed in the results section of this paper, there are numerous concerns related to the use of ChatGPT in education, such as accuracy, reliability, ethical issues, etc.

Ada, launched in 2017, is a chatbot that is used to provide personalized tutoring to students. It can answer questions, provide feedback, and facilitate individualized learning for students (Kabiljo et al., 2020 ; Konecki et al., 2023 ). However, the Ada chatbot has limitations in understanding complex queries. It could misinterpret context and provide inaccurate responses

Replika, launched in 2017, is an AI chatbot platform that is designed to be a friend and companion for students. It can listen to students' problems, offer advice, and help them feel less alone (Pentina et al., 2023 ; Xie & Pentina, 2022 ). However, given the personal nature of conversations with Replika, there are valid concerns regarding data privacy and security.

Socratic, launched in 2013, had the goal of creating a community that made learning accessible to all students. Currently, Socratic is an AI-powered educational platform that was acquired by Google in 2018. While not a chatbot per se, it has a chatbot-like interface and functionality designed to assist students in learning new concepts (Alsanousi et al., 2023 ; Moppel, 2018 ; St-Hilaire et al., 2022 ). Like with other chatbots, a concern arises where students might excessively rely on Socratic for learning. This could lead to a diminished emphasis on critical thinking, as students may opt to use the platform to obtain answers without gaining a genuine understanding of the underlying concepts.

Habitica, launched in 2013, is used to help students develop good study habits. It gamifies the learning process, making it more fun and engaging for students. Students can use Habitica to manage their academic tasks, assignments, and study schedules. By turning their to-do list into a game-like experience, students are motivated to complete their tasks and build productive habits (Sales & Antunes, 2021 ; Zhang, 2023 ). However, the gamified nature of Habitica could inadvertently introduce distractions, especially for students who are easily drawn into the gaming aspect rather than focusing on their actual academic responsibilities.

Piazza launched in 2009, is used to facilitate discussion and collaboration in educational settings, particularly in classrooms and academic institutions. It provides a space for students and instructors to engage in discussions, ask questions, and share information related to course content and assignments (Ruthotto et al., 2020 ; Wang et al., 2020 ). Because discussions on Piazza are user-generated, the quality and accuracy of responses can vary. This variability may result in situations where students do not receive accurate and helpful information.

We will likely see even more widespread adoption of chatbots in education in the years to come as technology advances further. Chatbots have enormous potential to improve teaching and learning. A large body of literature is devoted to exploring the role, challenges, and opportunities of chatbots in education. This paper gathers and synthesizes this vast amount of literature, providing a comprehensive understanding of the current research status concerning the influence of chatbots in education. By conducting a systematic review, we seek to identify common themes, trends, and patterns in the impact of chatbots on education and provide a holistic view of the research, enabling researchers, policymakers, and educators to make evidence-based decisions. One of the main objectives of this paper is to identify existing research gaps in the literature to pinpoint areas where further investigation is needed, enabling researchers to contribute to the knowledge base and guide future research efforts. Firstly, we aim to understand the primary advantages of incorporating AI chatbots in education, focusing on the perspectives of students. Secondly, we seek to explore the key advantages of integrating AI chatbots from the standpoint of educators. Lastly, we endeavor to comprehensively analyze the major concerns expressed by scholars regarding the integration of AI chatbots in educational settings. Corresponding research questions are formulated in the section below. Addressing these research questions, we aim to contribute valuable insights that shed light on the potential benefits and challenges associated with the utilization of AI chatbots in the field of education.

The paper follows a structured outline comprising several sections. Initially, we provide a summary of existing literature reviews. Subsequently, we delve into the methodology, encompassing aspects such as research questions, the search process, inclusion and exclusion criteria, as well as the data extraction strategy. Moving on, we present a comprehensive analysis of the results in the subsequent section. Finally, we conclude by addressing the limitations encountered during the study and offering insights into potential future research directions.

Summary of existing literature reviews

Drawing from extensive systematic literature reviews, as summarized in Table 1 , AI chatbots possess the potential to profoundly influence diverse aspects of education. They contribute to advancements in both teaching and learning processes. However, it is essential to address concerns regarding the irrational use of technology and the challenges that education systems encounter while striving to harness its capacity and make the best use of it.

It is evident that chatbot technology has a significant impact on overall learning outcomes. Specifically, chatbots have demonstrated significant enhancements in learning achievement, explicit reasoning, and knowledge retention. The integration of chatbots in education offers benefits such as immediate assistance, quick access to information, enhanced learning outcomes, and improved educational experiences. However, there have been contradictory findings related to critical thinking, learning engagement, and motivation. Deng and Yu ( 2023 ) found that chatbots had a significant and positive influence on numerous learning-related aspects but they do not significantly improve motivation among students. Contrary, Okonkwo and Ade-Ibijola (Okonkwo & Ade-Ibijola, 2021 ), as well as (Wollny et al., 2021 ) find that using chatbots increases students’ motivation.

In terms of application, chatbots are primarily used in education to teach various subjects, including but not limited to mathematics, computer science, foreign languages, and engineering. While many chatbots follow predetermined conversational paths, some employ personalized learning approaches tailored to individual student needs, incorporating experiential and collaborative learning principles. Challenges in chatbot development include insufficient training datasets, a lack of emphasis on usability heuristics, ethical concerns, evaluation methods, user attitudes, programming complexities, and data integration issues.

Although existing systematic reviews have provided valuable insights into the impact of chatbot technology in education, it's essential to acknowledge that the field of chatbot development is continually emerging and requires timely, and updated analysis to ensure that the information and assessments reflect the most recent advancements, trends, or developments in chatbot technology. The latest chatbot models have showcased remarkable capabilities in natural language processing and generation. Additional research is required to investigate the role and potential of these newer chatbots in the field of education. Therefore, our paper focuses on reviewing and discussing the findings of these new-generation chatbots' use in education, including their benefits and challenges from the perspectives of both educators and students.

There are a few aspects that appear to be missing from the existing literature reviews: (a) The existing findings focus on the immediate impact of chatbot usage on learning outcomes. Further research may delve into the enduring impacts of integrating chatbots in education, aiming to assess their sustainability and the persistence of the observed advantages over the long term. (b) The studies primarily discuss the impact of chatbots on learning outcomes as a whole, without delving into the potential variations based on student characteristics. Investigating how different student groups, such as age, prior knowledge, and learning styles, interact with chatbot technology could provide valuable insights. (c) Although the studies highlight the enhancements in certain learning components, further investigation could explore the specific pedagogical strategies employed by chatbots to achieve these outcomes. Understanding the underlying mechanisms and instructional approaches utilized by chatbots can guide the development of more effective and targeted educational interventions. (d) While some studies touch upon user attitudes and acceptance, further research can delve deeper into the user experience of interacting with chatbots in educational settings. This includes exploring factors such as usability, perceived usefulness, satisfaction, and preferences of students and teachers when using chatbot technology.

Addressing these gaps in the existing literature would significantly benefit the field of education. Firstly, further research on the impacts of integrating chatbots can shed light on their long-term sustainability and how their advantages persist over time. This knowledge is crucial for educators and policymakers to make informed decisions about the continued integration of chatbots into educational systems. Secondly, understanding how different student characteristics interact with chatbot technology can help tailor educational interventions to individual needs, potentially optimizing the learning experience. Thirdly, exploring the specific pedagogical strategies employed by chatbots to enhance learning components can inform the development of more effective educational tools and methods. Lastly, a deeper exploration of the user experience with chatbots, encompassing usability, satisfaction, and preferences, can provide valuable insights into enhancing user engagement and overall satisfaction, thus guiding the future design and implementation of chatbot technology in education.

Methodology

A systematic review follows a rigorous methodology, including predefined search criteria and systematic screening processes, to ensure the inclusion of relevant studies. This comprehensive approach ensures that a wide range of research is considered, minimizing the risk of bias and providing a comprehensive overview of the impact of AI in education. Firstly, we define the research questions and corresponding search strategies and then we filter the search results based on predefined inclusion and exclusion criteria. Secondly, we study selected articles and synthesize results and lastly, we report and discuss the findings. To improve the clarity of the discussion section, we employed Large Language Model (LLM) for stylistic suggestions.

Research questions

Considering the limitations observed in previous literature reviews, we have developed three research questions for further investigation:

What are the key advantages of incorporating AI chatbots in education from the viewpoint of students?

What are the key advantages of integrating AI chatbots in education from the viewpoint of educators?

What are the main concerns raised by scholars regarding the integration of AI chatbots in education?

Exploring the literature that focuses on these research questions, with specific attention to contemporary AI-powered chatbots, can provide a deeper understanding of the impact, effectiveness, and potential limitations of chatbot technology in education while guiding its future development and implementation. This paper will help to better understand how educational chatbots can be effectively utilized to enhance education and address the specific needs and challenges of students and educators.

Search process

The search for the relevant literature was conducted in the following databases: ACM Digital Library, Scopus, IEEE Xplore, and Google Scholar. The search string was created using Boolean operators, and it was structured as follows: (“Education” or “Learning” or “Teaching”) and (“Chatbot” or “Artificial intelligence” or “AI” or “ChatGPT”). Initially, the search yielded a total of 563 papers from all four databases. Search filters were applied based on predefined inclusion and exclusion criteria, followed by a rigorous data extraction strategy as explained below.

Inclusion and exclusion criteria

In our review process, we carefully adhered to the inclusion and exclusion criteria specified in Table 2 . Criteria were determined to ensure the studies chosen are relevant to the research question (content, timeline) and maintain a certain level of quality (literature type) and consistency (language, subject area).

Data extraction strategy

All three authors collaborated to select the articles, ensuring consistency and reliability. Each article was reviewed by at least two co-authors. The article selection process involved the following stages: Initially, the authors reviewed the studies' metadata, titles, abstracts, keywords and eliminated articles that were not relevant to research questions. This reduced the number of studies to 139. Next, the authors evaluated the quality of the studies by assessing research methodology, sample size, research design, and clarity of objectives, further refining the selection to 85 articles. Finally, the authors thoroughly read the entire content of the articles. Studies offering limited empirical evidence related to our research questions were excluded. This final step reduced the number of papers to 67. Figure  1 presents the article selection process.

Flow diagram of selecting studies

In this section, we present the results of the reviewed articles, focusing on our research questions, particularly with regard to ChatGPT. ChatGPT, as one of the latest AI-powered chatbots, has gained significant attention for its potential applications in education. Within just eight months of its launch in 2022, it has already amassed over 100 million users, setting new records for user and traffic growth. ChatGPT stands out among AI-powered chatbots used in education due to its advanced natural language processing capabilities and sophisticated language generation, enabling more natural and human-like conversations. It excels at capturing and retaining contextual information throughout interactions, leading to more coherent and contextually relevant conversations. Unlike some educational chatbots that follow predetermined paths or rely on predefined scripts, ChatGPT is capable of engaging in open-ended dialogue and adapting to various user inputs. Its adaptability allows it to write articles, stories, and poems, provide summaries, accommodate different perspectives, and even write and debug computer code, making it a valuable tool in educational settings (Baidoo-Anu & Owusu Ansah, 2023 ; Tate et al., 2023 ; Williams, 2023 ).

Advantages for students

Research question 1. what are the key advantages of incorporating ai chatbots in education from the viewpoint of students.

The integration of chatbots and virtual assistants into educational settings has the potential to transform support services, improve accessibility, and contribute to more efficient and effective learning environments (Chen et al., 2023 ; Essel et al., 2022 ). AI tools have the potential to improve student success and engagement, particularly among those from disadvantaged backgrounds (Sullivan et al., 2023 ). However, the existing literature highlights an important gap in the discussion from a student’s standpoint. A few existing research studies addressing the student’s perspective of using ChatGPT in the learning process indicate that students have a positive view of ChatGPT, appreciate its capabilities, and find it helpful for their studies and work (Kasneci et al., 2023 ; Shoufan, 2023 ). Students acknowledge that ChatGPT's answers are not always accurate and emphasize the need for solid background knowledge to utilize it effectively, recognizing that it cannot replace human intelligence (Shoufan, 2023 ). Common most important benefits identified by scholars are:

Homework and Study Assistance. AI-powered chatbots can provide detailed feedback on student assignments, highlighting areas of improvement and offering suggestions for further learning (Celik et al., 2022 ). For example, ChatGPT can act as a helpful study companion, providing explanations and clarifications on various subjects. It can assist with homework questions, offering step-by-step solutions and guiding students through complex problems (Crawford et al., 2023 ; Fauzi et al., 2023 ; Lo, 2023 ; Qadir, 2023 ; Shidiq, 2023 ). According to Sedaghat ( 2023 ) experiment, ChatGPT performed similarly to third-year medical students on medical exams, and could write quite impressive essays. Students can also use ChatGPT to quiz themselves on various subjects, reinforcing their knowledge and preparing for exams (Choi et al., 2023 ; Eysenbach, 2023 ; Sevgi et al., 2023 ; Thurzo et al., 2023 ).

Flexible personalized learning. AI-powered chatbots in general are now able to provide individualized guidance and feedback to students, helping them navigate through challenging concepts and improve their understanding. These systems can adapt their teaching strategies to suit each student's unique needs (Fariani et al., 2023 ; Kikalishvili, 2023 ; Schiff, 2021 ). Students can access ChatGPT anytime, making it convenient. According to Kasneci et al. ( 2023 ), ChatGPT's interactive and conversational nature can enhance students' engagement and motivation, making learning more enjoyable and personalized. (Khan et al., 2023 ) examine the impact of ChatGPT on medical education and clinical management, highlighting its ability to offer students tailored learning opportunities.

Skills development. It can aid in the enhancement of writing skills (by offering suggestions for syntactic and grammatical corrections) (Kaharuddin, 2021 ), foster problem-solving abilities (by providing step-by-step solutions) (Benvenuti et al., 2023 ), and facilitate group discussions and debates (by furnishing discussion structures and providing real-time feedback) (Ruthotto et al., 2020 ; Wang et al., 2020 ).

It's important to note that some papers raise concerns about excessive reliance on AI-generated information, potentially leading to a negative impact on student’s critical thinking and problem-solving skills (Kasneci et al., 2023 ). For instance, if students consistently receive solutions or information effortlessly through AI assistance, they might not engage deeply in understanding the topic.

Advantages for educators

Research question 2. what are the key advantages of integrating ai chatbots in education from the viewpoint of educators.

With the current capabilities of AI and its future potential, AI-powered chatbots, like ChatGPT, can have a significant impact on existing instructional practices. Major benefits from educators’ viewpoint identified in the literature are:

Time-Saving Assistance. AI chatbot administrative support capabilities can help educators save time on routine tasks, including scheduling, grading, and providing information to students, allowing them to allocate more time for instructional planning and student engagement. For example, ChatGPT can successfully generate various types of questions and answer keys in different disciplines. However, educators should exercise critical evaluation and customization to suit their unique teaching contexts. The expertise, experience, and comprehension of the teacher are essential in making informed pedagogical choices, as AI is not yet capable of replacing the role of a science teacher (Cooper, 2023 ).

Improved pedagogy. Educators can leverage AI chatbots to augment their instruction and provide personalized support. According to Herft ( 2023 ), there are various ways in which teachers can utilize ChatGPT to enhance their pedagogical approaches and assessment methods. For instance, Educators can leverage the capabilities of ChatGPT to generate open-ended question prompts that align precisely with the targeted learning objectives and success criteria of the instructional unit. By doing so, teachers can tailor educational content to cater to the distinct needs, interests, and learning preferences of each student, offering personalized learning materials and activities (Al Ka’bi,  2023 ; Fariani et al., 2023 ).

Concerns raised by scholars

Research question 3. what are the main concerns raised by scholars regarding the integration of ai chatbots in education.

Scholars' opinions on using AI in this regard are varied and diverse. Some see AI chatbots as the future of teaching and learning, while others perceive them as a potential threat. The main arguments of skeptical scholars are threefold:

Reliability and Accuracy. AI chatbots may provide biased responses or non-accurate information (Kasneci et al., 2023 ; Sedaghat, 2023 ). If the chatbot provides incorrect information or guidance, it could mislead students and hinder their learning progress. According to Sevgi et al. ( 2023 ), although ChatGPT exhibited captivating and thought-provoking answers, it should not be regarded as a reliable information source. This point is especially important for medical education. Within the field of medical education, it is crucial to guarantee the reliability and accuracy of the information chatbots provide (Khan et al., 2023 ). If the training data used to develop an AI chatbot contains biases, the chatbot may inadvertently reproduce those biases in its responses, potentially including skewed perspectives, stereotypes, discriminatory language, or biased recommendations. This is of particular concern in an educational context.

Fair assessments. One of the challenges that educators face with the integration of Chatbots in education is the difficulty in assessing students' work, particularly when it comes to written assignments or responses. AI-generated text detection, while continually improving, is not yet foolproof and can produce false negatives or positives. This creates uncertainty and can undermine the credibility of the assessment process. Educators may struggle to discern whether the responses are genuinely student-generated or if they have been provided by an AI, affecting the accuracy of grading and feedback. This raises concerns about academic integrity and fair assessment practices (AlAfnan et al., 2023 ; Kung et al., 2023 ).

Ethical issues. The integration of AI chatbots in education raises several ethical implications, particularly concerning data privacy, security, and responsible AI use. As AI chatbots interact with students and gather data during conversations, necessitating the establishment of clear guidelines and safeguards. For example, medical education frequently encompasses the acquisition of knowledge pertaining to delicate and intimate subjects, including patient confidentiality and ethical considerations within the medical field and thus ethical and proper utilization of chatbots holds significant importance (Masters, 2023 ; Miao & Ahn, 2023 ; Sedaghat, 2023 ; Thurzo et al., 2023 ).

For these and other geopolitical reasons, ChatGPT is banned in countries with strict internet censorship policies, like North Korea, Iran, Syria, Russia, and China. Several nations prohibited the usage of the application due to privacy apprehensions. Meanwhile, North Korea, China, and Russia, in particular, contended that the U.S. might employ ChatGPT for disseminating misinformation. Conversely, OpenAI restricts access to ChatGPT in certain countries, such as Afghanistan and Iran, citing geopolitical constraints, legal considerations, data protection regulations, and internet accessibility as the basis for this decision. Italy became the first Western country to ban ChatGPT (Browne, 2023 ) after the country’s data protection authority called on OpenAI to stop processing Italian residents’ data. They claimed that ChatGPT did not comply with the European General Data Protection Regulation. However, after OpenAI clarified the data privacy issues with Italian data protection authority, ChatGPT returned to Italy. To avoid cheating on school homework and assignments, ChatGPT was also blocked in all New York school devices and networks so that students and teachers could no longer access it (Elsen-Rooney, 2023 ; Li et al., 2023 ). These examples highlight the lack of readiness to embrace recently developed AI tools. There are numerous concerns that must be addressed in order to gain broader acceptance and understanding.

To summarize, incorporating AI chatbots in education brings personalized learning for students and time efficiency for educators. Students benefit from flexible study aid and skill development. However, concerns arise regarding the accuracy of information, fair assessment practices, and ethical considerations. Striking a balance between these advantages and concerns is crucial for responsible integration in education.

The integration of artificial intelligence (AI) chatbots in education has the potential to revolutionize how students learn and interact with information. One significant advantage of AI chatbots in education is their ability to provide personalized and engaging learning experiences. By tailoring their interactions to individual students’ needs and preferences, chatbots offer customized feedback and instructional support, ultimately enhancing student engagement and information retention. However, there are potential difficulties in fully replicating the human educator experience with chatbots. While they can provide customized instruction, chatbots may not match human instructors' emotional support and mentorship. Understanding the importance of human engagement and expertise in education is crucial. A teacher's role encompasses more than just sharing knowledge. They offer students guidance, motivation, and emotional support—elements that AI cannot completely replicate.

We find that AI chatbots may benefit students as well as educators in various ways, however, there are significant concerns that need to be addressed in order to harness its capabilities effectively. Specifically, educational institutions should implement preventative measures. This includes (a) creating awareness among students, focusing on topics such as digital inequality, the reliability and accuracy of AI chatbots, and associated ethical considerations; and (b) offering regular professional development training for educators. This training should initially focus on enabling educators to integrate diverse in-class activities and assignments into the curriculum, aimed at nurturing students’ critical thinking and problem-solving skills while ensuring fair performance evaluation. Additionally, this training should cover educating educators about the capabilities and potential educational uses of AI chatbots, along with providing them with best practices for effectively integrating these tools into their teaching methods.

As technology continues to advance, AI-powered educational chatbots are expected to become more sophisticated, providing accurate information and offering even more individualized and engaging learning experiences. They are anticipated to engage with humans using voice recognition, comprehend human emotions, and navigate social interactions. Consequently, their potential impact on future education is substantial. This includes activities such as establishing educational objectives, developing teaching methods and curricula, and conducting assessments (Latif et al., 2023 ). Considering Microsoft's extensive integration efforts of ChatGPT into its products (Rudolph et al., 2023 ; Warren, 2023 ), it is likely that ChatGPT will become widespread soon. Educational institutions may need to rapidly adapt their policies and practices to guide and support students in using educational chatbots safely and constructively manner (Baidoo-Anu & Owusu Ansah, 2023 ). Educators and researchers must continue to explore the potential benefits and limitations of this technology to fully realize its potential.

The widespread adoption of chatbots and their increasing accessibility has sparked contrasting reactions across different sectors, leading to considerable confusion in the field of education. Among educators and learners, there is a notable trend—while learners are excited about chatbot integration, educators’ perceptions are particularly critical. However, this situation presents a unique opportunity, accompanied by unprecedented challenges. Consequently, it has prompted a significant surge in research, aiming to explore the impact of chatbots on education.

In this article, we present a systematic review of the latest literature with the objective of identifying the potential advantages and challenges associated with integrating chatbots in education. Through this review, we have been able to highlight critical gaps in the existing research that warrant further in-depth investigation. Addressing these gaps will be instrumental in optimizing the implementation of chatbots and harnessing their full potential in the educational landscape, thereby benefiting both educators and students alike. Further research will play a vital role in comprehending the long-term impact, variations based on student characteristics, pedagogical strategies, and the user experience associated with integrating chatbots in education.

From the viewpoint of educators, integrating AI chatbots in education brings significant advantages. AI chatbots provide time-saving assistance by handling routine administrative tasks such as scheduling, grading, and providing information to students, allowing educators to focus more on instructional planning and student engagement. Educators can improve their pedagogy by leveraging AI chatbots to augment their instruction and offer personalized support to students. By customizing educational content and generating prompts for open-ended questions aligned with specific learning objectives, teachers can cater to individual student needs and enhance the learning experience. Additionally, educators can use AI chatbots to create tailored learning materials and activities to accommodate students' unique interests and learning styles.

Incorporating AI chatbots in education offers several key advantages from students' perspectives. AI-powered chatbots provide valuable homework and study assistance by offering detailed feedback on assignments, guiding students through complex problems, and providing step-by-step solutions. They also act as study companions, offering explanations and clarifications on various subjects. They can be used for self-quizzing to reinforce knowledge and prepare for exams. Furthermore, these chatbots facilitate flexible personalized learning, tailoring their teaching strategies to suit each student's unique needs. Their interactive and conversational nature enhances student engagement and motivation, making learning more enjoyable and personalized. Also, AI chatbots contribute to skills development by suggesting syntactic and grammatical corrections to enhance writing skills, providing problem-solving guidance, and facilitating group discussions and debates with real-time feedback. Overall, students appreciate the capabilities of AI chatbots and find them helpful for their studies and skill development, recognizing that they complement human intelligence rather than replace it.

The presence of AI chatbots also brought lots of skepticism among scholars. While some see transformative potential, concerns loom over reliability, accuracy, fair assessments, and ethical dilemmas. The fear of misinformation compromised academic integrity, and data privacy issues cast an eerie shadow over the implementation of AI chatbots. Based on the findings of the reviewed papers, it is commonly concluded that addressing some of the challenges related to the use of AI chatbots in education can be accomplished by introducing preventative measures. More specifically, educational institutions must prioritize creating awareness among students about the risks associated with AI chatbots, focusing on essential aspects like digital inequality and ethical considerations. Simultaneously, investing in the continuous development of educators through targeted training is key. Empowering educators to effectively integrate AI chatbots into their teaching methods, fostering critical thinking and fair evaluation, will pave the way for a more effective and engaging educational experience.

The implications of the research findings for policymakers and researchers are extensive, shaping the future integration of chatbots in education. The findings emphasize the need to establish guidelines and regulations ensuring the ethical development and deployment of AI chatbots in education. Policies should specifically focus on data privacy, accuracy, and transparency to mitigate potential risks and build trust within the educational community. Additionally, investing in research and development to enhance AI chatbot capabilities and address identified concerns is crucial for a seamless integration into educational systems. Researchers are strongly encouraged to fill the identified research gaps through rigorous studies that delve deeper into the impact of chatbots on education. Exploring the long-term effects, optimal integration strategies, and addressing ethical considerations should take the forefront in research initiatives.

Availability of data and materials

The data and materials used in this paper are available upon request. The comprehensive list of included studies, along with relevant data extracted from these studies, is available from the corresponding author upon request.

Change history

15 april 2024.

A Correction to this paper has been published: https://doi.org/10.1186/s41239-024-00461-6

Al Ka’bi, A. (2023). Proposed artificial intelligence algorithm and deep learning techniques for development of higher education. International Journal of Intelligent Networks, 4 , 68–73.

Article   Google Scholar  

AlAfnan, M. A., Dishari, S., Jovic, M., & Lomidze, K. (2023). Chatgpt as an educational tool: Opportunities, challenges, and recommendations for communication, business writing, and composition courses. Journal of Artificial Intelligence and Technology, 3 (2), 60–68.

Google Scholar  

Alsanousi, B., Albesher, A. S., Do, H., & Ludi, S. (2023). Investigating the user experience and evaluating usability issues in ai-enabled learning mobile apps: An analysis of user reviews. International Journal of Advanced Computer Science and Applications , 14(6).

AlZubi, S., Mughaid, A., Quiam, F., & Hendawi, S. (2022). Exploring the Capabilities and Limitations of ChatGPT and Alternative Big Language Models. Artificial Intelligence and Applications .

Aron, J. (2011). How innovative is Apple’s new voice assistant. Siri, NewScientist , 212 (2836), 24

Baidoo-Anu, D., & Owusu Ansah, L. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Available at SSRN 4337484 .

Benvenuti, M., Cangelosi, A., Weinberger, A., Mazzoni, E., Benassi, M., Barbaresi, M., & Orsoni, M. (2023). Artificial intelligence and human behavioral development: A perspective on new skills and competencies acquisition for the educational context. Computers in Human Behavior, 148 , 107903.

Browne, R. (2023). Italy became the first Western country to ban ChatGPT. Here’s what other countries are doing . CNBC (Apr. 4, 2023).

Celik, I., Dindar, M., Muukkonen, H., & Järvelä, S. (2022). The promises and challenges of artificial intelligence for teachers: A systematic review of research. TechTrends, 66 (4), 616–630.

Chassignol, M., Khoroshavin, A., Klimova, A., & Bilyatdinova, A. (2018). Artificial Intelligence trends in education: A narrative overview. Procedia Computer Science, 136 , 16–24.

Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. IEEE Access, 8 , 75264–75278.

Chen, Y., Jensen, S., Albert, L. J., Gupta, S., & Lee, T. (2023). Artificial intelligence (AI) student assistants in the classroom: Designing chatbots to support student success. Information Systems Frontiers, 25 (1), 161–182.

Choi, J. H., Hickman, K. E., Monahan, A., & Schwarcz, D. (2023). Chatgpt goes to law school. Available at SSRN .

Colby, K. M. (1981). PARRYing. Behavioral and Brain Sciences, 4 (4), 550–560.

Cooper, G. (2023). Examining science education in chatgpt: An exploratory study of generative artificial intelligence. Journal of Science Education and Technology, 32 (3), 444–452.

Crawford, J., Cowling, M., & Allen, K.-A. (2023). Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). Journal of University Teaching and Learning Practice, 20 (3), 02.

Crompton, H., & Burke, D. (2023). Artificial intelligence in higher education: The state of the field. International Journal of Educational Technology in Higher Education, 20 (1), 1–22.

Deng, X., & Yu, Z. (2023). A meta-analysis and systematic review of the effect of chatbot technology use in sustainable education. Sustainability, 15 (4), 2940.

Dergaa, I., Chamari, K., Zmijewski, P., & Saad, H. B. (2023). From human writing to artificial intelligence generated text: Examining the prospects and potential threats of ChatGPT in academic writing. Biology of Sport, 40 (2), 615–622.

Devedzic, V. (2004). Web intelligence and artificial intelligence in education. Journal of Educational Technology and Society, 7 (4), 29–39.

Dinh, T. N., & Thai, M. T. (2018). AI and blockchain: A disruptive integration. Computer, 51 (9), 48–53.

Elsen-Rooney, M. (2023). NYC education department blocks ChatGPT on school devices, networks. Retrieved Jan , 25 , 2023.

Essel, H. B., Vlachopoulos, D., Tachie-Menson, A., Johnson, E. E., & Baah, P. K. (2022). The impact of a virtual teaching assistant (chatbot) on students’ learning in Ghanaian higher education. International Journal of Educational Technology in Higher Education, 19 (1), 1–19.

Eysenbach, G. (2023). The role of ChatGPT, generative language models, and artificial intelligence in medical education: A conversation with ChatGPT and a call for papers. JMIR Medical Education, 9 (1), e46885.

Fariani, R. I., Junus, K., & Santoso, H. B. (2023). A systematic literature review on personalised learning in the higher education context. Technology, Knowledge and Learning, 28 (2), 449–476.

Fauzi, F., Tuhuteru, L., Sampe, F., Ausat, A. M. A., & Hatta, H. R. (2023). Analysing the role of ChatGPT in improving student productivity in higher education. Journal on Education, 5 (4), 14886–14891.

Herft, A. (2023). A Teacher’s Prompt Guide to ChatGPT aligned with’What Works Best’ .

Hoffer, R., Kay, T., Levitan, P., & Klein, S. (2001). Smarterchild . ActiveBuddy.

Holotescu, C. (2016). MOOCBuddy: A Chatbot for personalized learning with MOOCs. RoCHI , 91–94.

Kabiljo, M., Vidas-Bubanja, M., Matic, R., & Zivkovic, M. (2020). Education system in the republic of serbia under COVID-19 conditions: Chatbot-acadimic digital assistant of the belgrade business and arts academy of applied studies. Knowledge-International Journal, 43 (1), 25–30.

Kaharuddin, A. (2021). Assessing the effect of using artificial intelligence on the writing skill of Indonesian learners of English. Linguistics and Culture Review, 5 (1), 288.

Kahraman, H. T., Sagiroglu, S., & Colak, I. (2010). Development of adaptive and intelligent web-based educational systems. In 2010 4th International Conference on Application of Information and Communication Technologies , 1–5.

Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., & Hüllermeier, E. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103 , 102274.

Khademi, A. (2023). Can ChatGPT and bard generate aligned assessment items? A reliability analysis against human performance. ArXiv Preprint ArXiv:2304.05372.

Khan, R. A., Jawaid, M., Khan, A. R., & Sajjad, M. (2023). ChatGPT-Reshaping medical education and clinical management. Pakistan Journal of Medical Sciences, 39 (2), 605.

Kietzmann, J., Paschen, J., & Treen, E. (2018). Artificial intelligence in advertising: How marketers can leverage artificial intelligence along the consumer journey. Journal of Advertising Research, 58 (3), 263–267.

Kikalishvili, S. (2023). Unlocking the potential of GPT-3 in education: Opportunities, limitations, and recommendations for effective integration. Interactive Learning Environments , 1–13.

Konecki, M., Konecki, M., & Biškupić, I. (2023). Using artificial intelligence in higher education. In Proceedings of the 15th International Conference on Computer Supported Education .

Krstić, L., Aleksić, V., & Krstić, M. (2022). Artificial intelligence in education: A review .

Kuhail, M. A., Alturki, N., Alramlawi, S., & Alhejori, K. (2023). Interacting with educational chatbots: A systematic review. Education and Information Technologies, 28 (1), 973–1018.

Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., et al. (2023). Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLoS Digital Health, 2 (2), e0000198.

Lally, A., & Fodor, P. (2011). Natural language processing with prolog in the ibm watson system. The Association for Logic Programming (ALP) Newsletter , 9 , 2011.

Latif, E., Mai, G., Nyaaba, M., Wu, X., Liu, N., Lu, G., ... & Zhai, X. (2023). Artificial general intelligence (AGI) for education. arXiv preprint arXiv:2304.12479.

Li, L., Ma, Z., Fan, L., Lee, S., Yu, H., & Hemphill, L. (2023). ChatGPT in education: A discourse analysis of worries and concerns on social media. ArXiv Preprint ArXiv:2305.02201.

Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13 (4), 410.

Masters, K. (2023). Ethical use of artificial intelligence in health professions education: AMEE Guide No. 158. Medical Teacher , 45 (6), 574–584.

Miao, H., & Ahn, H. (2023). Impact of ChatGPT on interdisciplinary nursing education and research. Asian/pacific Island Nursing Journal, 7 (1), e48136.

Moppel, J. (2018). Socratic chatbot . University Of Tartu, Institute of Computer Science, Bachelor’s Thesis.

Okonkwo, C. W., & Ade-Ibijola, A. (2021). Chatbots applications in education: A systematic review. Computers and Education: Artificial Intelligence, 2 , 100033.

Pentina, I., Hancock, T., & Xie, T. (2023). Exploring relationship development with social chatbots: A mixed-method study of replika. Computers in Human Behavior, 140 , 107600.

Peredo, R., Canales, A., Menchaca, A., & Peredo, I. (2011). Intelligent Web-based education system for adaptive learning. Expert Systems with Applications, 38 (12), 14690–14702.

Pérez, J. Q., Daradoumis, T., & Puig, J. M. M. (2020). Rediscovering the use of chatbots in education: A systematic literature review. Computer Applications in Engineering Education, 28 (6), 1549–1565.

Qadir, J. (2023). Engineering education in the era of ChatGPT: Promise and pitfalls of generative AI for education. IEEE Global Engineering Education Conference (EDUCON), 2023 , 1–9.

Rahaman, M. S., Ahsan, M. M., Anjum, N., Rahman, M. M., & Rahman, M. N. (2023). The AI race is on! Google’s Bard and OpenAI’s ChatGPT head to head: An opinion article. Mizanur and Rahman, Md Nafizur, The AI Race Is On .

Rudolph, J., Tan, S., & Tan, S. (2023). War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education. Journal of Applied Learning and Teaching, 6 (1).

Ruthotto, I., Kreth, Q., Stevens, J., Trively, C., & Melkers, J. (2020). Lurking and participation in the virtual classroom: The effects of gender, race, and age among graduate students in computer science. Computers & Education, 151 , 103854.

de Sales, A. B., & Antunes, J. G. (2021). Evaluation of educational games usage satisfaction. 2021 16th Iberian Conference on Information Systems and Technologies (CISTI) , 1–6.

Schiff, D. (2021). Out of the laboratory and into the classroom: the future of artificial intelligence in education. AI & Society, 36 (1), 331–348.

Sedaghat, S. (2023). Success through simplicity: What other artificial intelligence applications in medicine should learn from history and ChatGPT. Annals of Biomedical Engineering , 1–2.

Sevgi, U. T., Erol, G., Doğruel, Y., Sönmez, O. F., Tubbs, R. S., & Güngor, A. (2023). The role of an open artificial intelligence platform in modern neurosurgical education: A preliminary study. Neurosurgical Review, 46 (1), 86.

Shidiq, M. (2023). The use of artificial intelligence-based chat-gpt and its challenges for the world of education; from the viewpoint of the development of creative writing skills. Proceeding of International Conference on Education, Society and Humanity, 1 (1), 353–357.

Shoufan, A. (2023). Exploring Students’ Perceptions of CHATGPT: Thematic Analysis and Follow-Up Survey. IEEE Access .

St-Hilaire, F., Vu, D. D., Frau, A., Burns, N., Faraji, F., Potochny, J., Robert, S., Roussel, A., Zheng, S., & Glazier, T. (2022). A new era: Intelligent tutoring systems will transform online learning for millions. ArXiv Preprint ArXiv:2203.03724.

Sullivan, M., Kelly, A., & McLaughlan, P. (2023). ChatGPT in higher education: Considerations for academic integrity and student learning .

Tahiru, F. (2021). AI in education: A systematic literature review. Journal of Cases on Information Technology (JCIT), 23 (1), 1–20.

Tate, T., Doroudi, S., Ritchie, D., & Xu, Y. (2023). Educational research and AI-generated writing: Confronting the coming tsunami .

Thurzo, A., Strunga, M., Urban, R., Surovková, J., & Afrashtehfar, K. I. (2023). Impact of artificial intelligence on dental education: A review and guide for curriculum update. Education Sciences, 13 (2), 150.

Wallace, R. (1995). Artificial linguistic internet computer entity (alice). City .

Wang, Q., Jing, S., Camacho, I., Joyner, D., & Goel, A. (2020). Jill Watson SA: Design and evaluation of a virtual agent to build communities among online learners. Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems , 1–8.

Warren, T. (2023). Microsoft is looking at OpenAI’s GPT for Word, Outlook, and PowerPoint. The Verge .

Weizenbaum, J. (1966). ELIZA—A computer program for the study of natural language communication between man and machine. Communications of the ACM, 9 (1), 36–45.

Williams, C. (2023). Hype, or the future of learning and teaching? 3 Limits to AI’s ability to write student essays .

Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., & Drachsler, H. (2021). Are we there yet?—A systematic literature review on chatbots in education. Frontiers in Artificial Intelligence, 4 , 654924.

Xie, T., & Pentina, I. (2022). Attachment theory as a framework to understand relationships with social chatbots: A case study of Replika .

Zhang, Q. (2023). Investigating the effects of gamification and ludicization on learning achievement and motivation: An empirical study employing Kahoot! and Habitica. International Journal of Technology-Enhanced Education (IJTEE), 2 (1), 1–19.

Download references

Acknowledgements

Not applicable.

The authors declare that this research paper did not receive any funding from external organizations. The study was conducted independently and without financial support from any source. The authors have no financial interests or affiliations that could have influenced the design, execution, analysis, or reporting of the research.

Author information

Authors and affiliations.

Finance Department, American University of the Middle East, Block 6, Building 1, Egaila, Kuwait

Lasha Labadze

Statistics Department, American University of the Middle East, Block 6, Building 1, Egaila, Kuwait

Maya Grigolia

Caucasus School of Business, Caucasus University, 1 Paata Saakadze St, 0102, Tbilisi, Georgia

Lela Machaidze

You can also search for this author in PubMed   Google Scholar

Contributions

LL provided a concise overview of the existing literature and formulated the methodology. MG initiated the initial search process. LM authored the discussion section. All three authors collaborated on the selection of the final paper collection and contributed to crafting the conclusion. The final version of the paper received approval from all authors.

Corresponding author

Correspondence to Lasha Labadze .

Ethics declarations

Competing interests.

Authors have no competing interests to declare.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original version of this article was revised: “A sentence has been added to the Methodology section of the article to acknowledge use of LLM”

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Labadze, L., Grigolia, M. & Machaidze, L. Role of AI chatbots in education: systematic literature review. Int J Educ Technol High Educ 20 , 56 (2023). https://doi.org/10.1186/s41239-023-00426-1

Download citation

Received : 22 August 2023

Accepted : 18 October 2023

Published : 31 October 2023

DOI : https://doi.org/10.1186/s41239-023-00426-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Systematic literature review
• Artificial intelligence
• AI chatbots
• Chatbots in education

• Open access
• Published: 19 April 2023

AI literacy in K-12: a systematic literature review

• Lorena Casal-Otero   ORCID: orcid.org/0000-0002-0906-4321 1 ,
• Alejandro Catala   ORCID: orcid.org/0000-0002-3677-672X 2 , 3 ,
• Carmen Fernández-Morante   ORCID: orcid.org/0000-0003-4398-3361 1 ,
• Maria Taboada   ORCID: orcid.org/0000-0002-2353-596X 2 ,
• Beatriz Cebreiro   ORCID: orcid.org/0000-0003-2064-915X 1 &
• Senén Barro   ORCID: orcid.org/0000-0001-6035-540X 3  

International Journal of STEM Education volume  10 , Article number:  29 ( 2023 ) Cite this article

54k Accesses

89 Citations

28 Altmetric

Metrics details

The successful irruption of AI-based technology in our daily lives has led to a growing educational, social, and political interest in training citizens in AI. Education systems now need to train students at the K-12 level to live in a society where they must interact with AI. Thus, AI literacy is a pedagogical and cognitive challenge at the K-12 level. This study aimed to understand how AI is being integrated into K-12 education worldwide. We conducted a search process following the systematic literature review method using Scopus. 179 documents were reviewed, and two broad groups of AI literacy approaches were identified, namely learning experience and theoretical perspective. The first group covered experiences in learning technical, conceptual and applied skills in a particular domain of interest. The second group revealed that significant efforts are being made to design models that frame AI literacy proposals. There were hardly any experiences that assessed whether students understood AI concepts after the learning experience. Little attention has been paid to the undesirable consequences of an indiscriminate and insufficiently thought-out application of AI. A competency framework is required to guide the didactic proposals designed by educational institutions and define a curriculum reflecting the sequence and academic continuity, which should be modular, personalized and adjusted to the conditions of the schools. Finally, AI literacy can be leveraged to enhance the learning of disciplinary core subjects by integrating AI into the teaching process of those subjects, provided the curriculum is co-designed with teachers.

Introduction

In recent years, the convergence of huge computing power, massive amounts of data and improved machine learning algorithms have led to remarkable advances in Artificial Intelligence (AI) based technologies, which are set to be the most socially and economically disruptive technologies ever developed (Russell, 2021 ). The irruption of AI-based technology in our daily lives (e.g., robot vacuum cleaners, real-time location and search systems, virtual assistants, etc.) has generated a growing social and political interest in educating citizens about AI. The scientific community has also begun to engage in this education after detecting a significant gap in the understanding of AI, based on comments and fears expressed by citizens about this technology (West & Allen, 2018 ). Therefore, integrating AI into curricula is necessary to train citizens who must increasingly live and act in a world with a significant presence of AI.

It is worth noting that AI education addresses not only the learning of the scientific and technological foundations of AI, but also the knowledge and critical reflection on how a trustworthy AI should be developed and the consequences of not doing so. Hence, it is crucial to incorporate AI teaching from the earliest stages of education (Heintz, 2021 ). However, although some countries are making significant efforts to promote AI teaching in K-12 (Touretzky et al., 2019a ), this is being implemented through highly varied AI training experiences, such as data-driven design (Vartiainen et al., 2021 ), interactive data visualizations (Chittora & Baynes, 2020 ; von Wangenheim et al., 2021 ), virtual reality and robotics (Narahara & Kobayashi, 2018 ), games (Giannakos et al., 2020 ), or even based on combined workshop series (Lee et al., 2021 ). To date, there are very few methodological proposals on how to introduce the AI curriculum in K-12 education (Lee et al., 2020 ).

Since the development of a field requires prior research, we propose in this paper to identify and examine the way in which AI literacy is developing in K-12 around the world, to draw conclusions and guide teaching proposals for AI literacy in K-12. By highlighting and discussing the pros and cons of the different approaches and experiences in the literature, we aim to inspire new initiatives and guide the actors involved, from decisions-makers, for example in education policy, to teachers involved in their conception, design and implementation. We also hope to raise awareness of the importance of learning about AI from an early age, emphasizing the key aspects of this training and, hopefully, fueling the debate that needs to be fostered in our research community.

Integration of AI into the K12 curriculum

As a scientific-technological field, AI is just a few decades old. The name was coined in 1956, and since then different disciplines (such as computer science, mathematics, philosophy, neuroscience, or psychology) have contributed to its development from an interdisciplinary focus. AI is oriented to comprehend, model, and replicate human intelligence and cognitive processes into artificial systems. Currently, it covers a wide range of subfields such as machine learning, perception, natural language processing, knowledge representation and reasoning, computer vision, among many others (Russell & Norvig, 2021 ).

Starting in the 1970s, AI began to emerge in educational contexts through tools specifically designed to support learning, teaching, and the management of educational institutions. Since many jobs are now AI-related and will continue to increase in the coming years, some researchers believe that AI education should be considered as important as literacy in reading and writing (Kandlhofer et al., 2016 ). The highly interdisciplinary character is also another factor to consider. AI literacy can be defined as a set of skills that enable a solid understanding of AI through three priority axes: learning about AI, learning about how AI works, and learning for life with AI (Long & Magerko, 2020 ; Miao et al., 2021 ). The first axis focuses on understanding AI concepts and techniques to enable the recognition of which artifacts/platforms use AI and which do not. The second axis addresses the understanding of how AI works, to effectively interact with it. The third axis seeks to understand how AI can affect our lives, allowing us to critically evaluate its technology. Thus, AI literacy goes beyond the use of AI applications in education, such as Intelligent Tutoring Systems (ITS) (du Boulay, 2016 ).

The teaching of knowledge in AI has traditionally been carried out at the university level, focused on students who study disciplines closely related to computing and ICT in general. In recent years, AI learning has also started to be relevant both in university programs with diverse study backgrounds (Kong et al., 2021 ), as well as at the K-12 level (Kandlhofer & Steinbauer, 2021 ; Tedre et al., 2021 ). However, teaching AI at the K-12 level is not yet prevalent in formal settings and is considered challenging. Experts believe it is important to have some thought on what AI education should look like at the K-12 level so that future generations can become informed citizens who understand the technologies they interact with in their daily lives (Touretzky et al., 2019a ). Training children and teenagers will allow them to understand the basics of the science and technology that underpins AI, its possibilities, its limits and its potential social and economic impact. It also stimulates and better prepares them to pursue further studies related to AI or even to become creators and developers of AI themselves (Heintz, 2021 ).

Nowadays, research on AI teaching is still scarce (Chai et al., 2020a , 2020b ; Lee et al., 2020 ). The acquisition of knowledge in AI represents a great pedagogical challenge for both experts and teachers, and a cognitive challenge for students (Micheuz, 2020 ). Some countries are making significant efforts to promote AI education in K-12 (Touretzky et al., 2019b ), by developing relatively comprehensive curriculum guidelines (Yue et al., 2021 ). Through interviews with practitioners and policy makers from three different continents (America, Asia and Europe), some studies report on continuing works to introduce AI in K-12 education (He et al., 2020 ). Some other work focuses on examining and comparing AI curricula in several countries (Yue et al., 2021 ). In addition, there are a growing number of AI training experiences that explore pathways to optimize AI learning for K-12 students. However, most of them are somehow thematically limited as they do not adequately address key areas of AI, such as planning, knowledge representation and automated reasoning (Nisheva-Pavlova, 2021 ). Additionally, due to the rapid growth of AI, there is a need to understand how educators can best leverage AI techniques for the academic success of their students. Zhai et al. ( 2021 ) recommend that educators work together with AI experts to bridge the gap between technique and pedagogy.

Using a systematic review method, our research aims to present an overview of current approaches to understand how AI is taught worldwide. Several studies have conducted systematic reviews concerning applications of AI in education. Zhai et al. ( 2021 ) analyzed how AI was applied to the education domain from 2010 to 2020. Their review covers research on AI-based learning environments, from their construction to their application and integration in the educational environment. Guan et al. ( 2020 ) reviewed the main themes and trends in AI research in education over the past two decades. The authors found that research on the use of AI techniques to support teaching or learning has stood the test of time and that learner profiling models and learning analytics have proliferated in the last two decades. Ng et al. ( 2022 ) examined learner types, teaching tools and pedagogical approaches in AI teaching and learning, mainly in university computer science education. Chen et al. ( 2020 ) covered education enhanced by AI techniques aimed to back up teaching and learning. All these studies have focused on the main role that AI has played in educational applications over the last decades. However, in light of the recent need to consider how AI education should be approached at the K-12 level (Kandlhofer et al., 2016 ; Long & Magerko, 2020 ; Miao et al., 2021 ; Touretzky et al., 2019b ), it would be of great value to structure and characterize the different approaches used so far to develop AI literacy in K-12, as well as to identify research gaps to be explored. Recently, Yue et al. ( 2022 ) analyzed the main components of the pedagogical design in 32 empirical studies in K-12 AI education and Su et al. ( 2022 ) examined 14 learning experiences carried out in the Asian-Pacific region. These components included target audience, setting, duration, contents, pedagogical approaches to teaching, and assessment methods. Sanusi et al. ( 2022 ) reviewed research on teaching machine learning in K-12 from four perspectives: curriculum development, technology development, pedagogical development, and teacher training development. The findings of the study revealed that more studies are needed on how to integrate machine learning into subjects other than computer science. Crompton et al. ( 2022 ) carried out a systematic review on the use of AI as a supporting tool in K-12 teaching, which entails an interesting but narrower scope. Our study extends previous reviews on K-12 AI research by emphasizing how the current approaches are integrating AI literacy in K-12 education worldwide.

Research question

To begin the systematic review, a single research question (RQ) was formulated.

RQ: How are current approaches integrating AI literacy into K-12 education worldwide?

In essence, the RQ aims to investigate the characterization of the different approaches being employed to incorporate AI education in K-12. The following subsections in the methodology describe the search and the data collection process in such a way that an answer to the RQ can be provided in a replicable and objective fashion.

The research method chosen to conduct this research was the systematic literature review (SLR), following the guidelines posed by Kitchenham ( 2004 ). Accordingly, the following subsections summarize and document the key steps implemented in this research method.

Search process

We used Scopus to implement the search process. Scopus provides an integrated search facility to find relevant papers in its database based on curated metadata. It includes primary bibliographic sources published by Elsevier , Springer , ACM , and IEEE , among others. It provides a comprehensive coverage of journals and top-ranked conferences within fields of interest. We did not limit our search to specific journals or regular conference proceedings, as there is not yet a clearly established body of literature on the subject. All searches were performed based on title, keywords and abstract, and conducted between 21 October 2021 and 9 March 2023.

To decide the search string, we ran an initial search and found only a few papers focused on ‘literacy’ whereas the vast majority referred to the broader term ‘education’. Therefore, we decided to use both search terms (key issue 1 in Table 1 ). As some recent works combine the terms ‘Artificial Intelligence’ and ‘education’/’literacy’ into single terms such as ‘AI literacy’ or ‘AI education’, these were added to the search string (key issue 2 in Table 1 ). The educational stage was also included in the search string (key issue 3 in Table 1 ). As the search term ‘education’ also returns AI-based learning environments which are outside the scope of our review, we explicitly considered negated terms to leave out both computer-based learning and intelligent tutoring systems (key issue 4 in Table 1 ). A final decision was whether to use the term ‘Artificial Intelligence’ as a single umbrella term or to add narrower terms related to AI subfields (e.g., machine learning). After a preliminary inspection of a few relevant papers, we observed that such additional specific terms usually co-occur with the string ‘Artificial Intelligence’ in education, and they were therefore regarded as unnecessary. Thus, to capture the essence of our RQ and to build up the complete search string, we considered the search terms as shown in Table 1 . Eventually, this resulted in the following complete search string in Scopus:

TITLE-ABS-KEY ( ( ( ( literacy OR education) AND ( ( artificial AND intelligence))) OR ( "AI literacy" OR "AI education")) AND ( "primary school" OR "secondary school" OR k-12 OR "middle school"

OR "high school") AND NOT ( "computer-based learning") AND NOT ( "intelligent tutoring system")).

We included peer-reviewed papers published on topics related to literacy and education on AI at school. Then we excluded papers whose usage of AI was limited to 1) supporting computer-based learning only, with no focus on learning about AI; 2) supporting assessment/tutoring based on AI. We also excluded papers that targeted college students and those that were limited to K-12 programming/CS concepts as a prerequisite for learning about AI in the future. Following these inclusion and exclusion criteria, our search in Scopus returned an initial list of 750 documents. After we inspected the title, abstract, keywords and full-text screening, we obtained a final list of 179 documents.

Data collection extraction and synthesis strategy

Data collection extraction was performed, discussed, and coordinated through regular meetings. After inspecting and discussing 10% of the papers over multiple meetings, the authors agreed on the annotations presented in Table 2 . This process is important as it allowed us to build a data annotation scheme empirically emerging from the sampled papers. A copy of the papers was also kept for easy review in case of doubts or disagreements.

The data resulted in a spreadsheet with the metadata of the papers which passed the inclusion and exclusion criteria, and a document with the list of paper IDs together with the rest of annotations. Some Python scripts were used to further process metadata (e.g., counting participating countries, frequencies, etc.) and produce a more complete bibliographic report with histograms and overview counting. A more qualitative analysis was carried out to answer the research question based on paper reading and annotations.

The results were organized into two subsections. The first subsection is a bibliometric analysis of the reviewed studies, which is based on the metadata provided by Scopus. The second subsection provides a qualitative analysis of the studies, which is based on the extracted data annotations (see Table 2 ). Both analyses are complementary and together deliver a better understanding of the research articles retrieved.

Bibliometric analysis

Figure  1 shows that the annual scientific production has been modest. It gained traction in 2016 and increased sharply in 2020.

Annual scientific production: number of papers by year

Most of the contributions are conference publications (126 papers), while 52 are journal articles and one is a book chapter (Fig.  2 ).

Type of contributions: number of papers by type

Eighty out of 179 papers have at least a citation in Scopus. There are 13 papers that have 10 or more citations, and the most cited papers are Long and Magerko ( 2020 ) and Touretzky et al. ( 2019b ). Figure  3 summarizes the number of contributions by publishers, where Springer, IEEE and ACM stand out, followed by Elsevier. As for journals, there are no single journals concentrating the publication of articles. Nevertheless, there are some journals that are especially relevant and well-known by the community such as the International Journal of Child-Computer Interaction, Computers and Education: Artificial Intelligence, International Journal of Artificial Intelligence in Education, or IEEE Transactions on Education.

Frequency of publishers: number of papers by publisher

As for conferences, Fig.  4 summarizes the main conference events where papers are published. It includes flagship conferences Footnote 1 such as CHI and AAAI, top-ranked conferences such as HRI or SIGCSE and several noteworthy events (IDC, ICALT, ITiCSE, VL/HCC, to name a few). It is worth mentioning that AAAI is receiving contributions from recent years, which confirms the interest in the field in broadening the discussion to education. There are some additional publications associated with satellite AAAI events, such as workshops in CEUR-WS that deal with the issue under study. Although such contributions may sometimes be short, we decided to include them as they were relevant. For instance, the works published in (Herrero et al., 2020 ) and (Micheuz, 2020 ) include the German countrywide proposal for educating about AI, through a 6-module course focusing on explaining how AI works, the social discourse on AI and reducing existing misconceptions. On the other hand, Aguar et al. ( 2016 ) talk about teaching AI via an optional course which does not contribute to the final grades.

Main conference events: number of papers by conference

The analysis did not reveal particularly outstanding institutions (see Table 3 for a summary). Among the 299 affiliated institutions, we mostly find universities and research centers along with a few collaboration associations. The most active institutions are the Chinese University of Hong Kong, University of Eastern Finland and MIT, whose authors participated in a total of 19, 11 and 10 contributions, respectively.

Finally, the retrieved papers were co-authored by 643 different authors affiliated to research institutions from 42 countries. Figure  5 shows the histogram of participation by country. Of the 179 papers reviewed, most papers were written by authors affiliated with institutions in the same country. Only 32 papers involved authors from several countries. It is remarkable that in these cases at least one author is from the US, Hong Kong or China.

Country participation: number of papers by country

Literature analysis

By analyzing the data extracted, the papers were classified into two broad thematic categories according to the type of educational approach, namely, learning experience and theoretical perspective. The first category covers AI learning experiences focused on understanding a particular AI concept/technique or using specific tools/platforms to illustrate some AI concepts. The second category involves initiatives for the implementation of AI education for K-12 through the development of guidelines, curriculum design or teacher training, among others. Each main category was further subdivided into other subcategories to structure the field and characterize the different approaches used in developing AI literacy in K-12. Figure  6 shows all the identified categories and subcategories.

Taxonomy of approaches to AI learning in K-12

Learning experiences focused on understanding AI

This category covers learning experiences aimed at experimenting and becoming familiar with AI concepts and techniques. Based on the priority axes in AI literacy (Long & Magerko, 2020 ; Miao et al., 2021 ), we identified experiences aimed at acquiring basic AI knowledge to recognize artifacts using AI, learning how AI works, learning tools for AI and learning to live with AI.

Learning to recognize artifacts using AI

This subcategory refers to experiences that aim to understand AI concepts and techniques enabling the recognition of which artifacts/platforms use AI and which do not. Four studies were found in this subcategory. They are proposals aimed at helping young people to understand and demystify AI through different types of activities. These activities included conducting discussions after watching AI-related movies (Tims et al., 2012 ), carrying out computer-based simulations of human-like behaviors (Ho et al., 2019 ), experimenting as active users of social robots (Gonzalez et al., 2017 ) and programming AI-based conversational agents (Van Brummelen et al., 2021b ).

Learning about how AI works

This topic covers proposals designed to understand how AI works to make user interaction with AI easier and more effective. In this type of proposal, the focus is on methodology and learning is achieved through technology (Kim et al., 2023 ). The objective is to provide a better understanding of a particular aspect of reality in order to carry out a project or solve a problem (Lenoir & Hasni, 2016 ). The activities are supported by active experiences based on building and creating intelligent devices to achieve the understanding of AI concepts following the idea of Papert’s constructionism.

These experiences are mainly focused on teaching AI subfields such as ML or AI algorithms applied to robotics. Understanding the principles of ML, its workflows and its role in everyday practices to solve real-life problems has been the main objective of some studies (Burgsteiner et al., 2016 ; Evangelista et al., 2019 ; Lee et al., 2020 ; Sakulkueakulsuk et al., 2019 ; Vartiainen et al., 2021 ). In addition, there are also experiences focused on unplugged activities that simulate AI algorithms. For example, through classic games such as Mystery Hunt, one can learn how to traverse a graph without being able to see beyond the next path to be traversed (blind search) (Kandlhofer et al., 2016 ). Similarly, the AI4K12 initiative (Touretzky et al., 2019b ) collects a large set of activities and resources to simulate AI algorithms.

Learning tools for AI

This topic includes approaches that involve learning about AI support tools. The development of intelligent devices in the context of teaching AI requires specific programming languages or age-appropriate tools. Many of the tools currently available are focused on ML, with the aim of demystifying this learning in K-12 education (Wan et al., 2020 ). Some of them are integrated into block-based programming languages (such as Scratch or App Inventor) (Toivonen et al., 2020 ; von Wangenheim et al., 2021 ), enabling the deployment of the ML models built into games or mobile applications. Other approaches use data visualization and concepts of gamification to engage the student in the learning process (Reyes et al., 2020 ; Wan et al., 2020 ) or combine traditional programming activities with ML model building (Rodríguez-García et al., 2020 ).

This type of proposal aims to introduce AI through tools that enable the use of AI techniques. It is therefore an approach focused on learning by using AI-oriented tools. In this vein, different experiences have focused on learning programming tools for applications based on Machine Learning (Reyes et al., 2020 ; Toivonen et al., 2020 ; von Wangenheim et al., 2021 ; Wan et al., 2020 ), robotics (Chen et al., 2017 ; Eguchi, 2021 ; Eguchi & Okada, 2020 ; Holowka, 2020 ; Narahara & Kobayashi, 2018 ; Nurbekova et al., 2018 ; Verner et al., 2021 ), programming and the creation of applications (Chittora & Baynes, 2020 ; Giannakos et al., 2020 ; Kahn et al., 2018 ; Kelly et al., 2008 ; Park et al., 2021 ). Some of these tools use Scratch-based coding platforms to make AI-based programming attractive to children. In (Kahn et al., 2018 ), students play around with machine learning to classify self-captured images, using a block-based coding platform.

There are also experiences in which other types of environments are used to facilitate learning (Aung et al., 2022 ). In (Holowka, 2020 ; Verner et al., 2021 ), students can learn reinforcement learning through online simulation. In (Narahara & Kobayashi, 2018 ), a virtual environment helps students generate data in a playful setting, which is then used to train a neural network for the autonomous driving of a toy car-lab. In (Avanzato, 2009 ; Croxell et al., 2007 ), students experiment with different AI-based tasks through robotics-oriented competitions.

Learning for life with AI

This subcategory covers experiences aimed at understanding how AI can affect our lives thus providing us with skills to critically assess its technology. In (Vachovsky et al., 2016 ), technically rigorous AI concepts are contextualized through the impact on society. There are also experiences where students explore how a robot equipped with AI components can be used in society (Eguchi & Okada, 2018 ), program conversational agents (Van Brummelen et al., 2021b ), or learn to recognize credible but fake media products (video, photos), which have been generated using AI-based techniques ( 2021b ; Ali et al., 2021a ).

The ethical and philosophical implications of AI have also been addressed in some experiences ( 2021b ; Ali et al., 2021a ; Ellis et al., 2005 ), whereas others focus on training students to participate in present-day society and become critical consumers of AI (Alexandre et al., 2021 ; Cummings et al., 2021 ; Díaz et al., 2015 ; Kaspersen et al., 2022 ; Lee et al., 2021 ; Vartiainen et al., 2020 ).

Proposals for implementation of AI learning at the K-12 level

Some countries are making efforts to promote AI education in K-12. In the U.S., intense work is being carried out on the integration of AI in schools and among these schemes, AI4K12 stands out (Heintz, 2021 ). This scheme is especially interesting since it defines the national guidelines for future curricula, highlighting the essential collaborative work between developers, teachers and students (Touretzky et al., 2019a ). This idea of co-creation is also stressed in other schemes (Chiu, 2021 ). In the U.S. we can also mention the proposal made by the Massachusetts Institute of Technology, which is an AI curriculum that aims to engage students with its social and ethical implications (Touretzky et al., 2019a ). Although the United States is working intensively on the design of integrating this knowledge into the curriculum, so far AI is not widely offered in most K-12 schools (Heintz, 2021 ).

In China, the Ministry of Education has integrated AI into the compulsory secondary school curriculum (Ottenbreit-Leftwich et al., 2021 ; Xiao & Song, 2021 ). Among their schemes we can reference the AI4Future initiative of the Chinese University of Hong Kong (CUHK), which promotes the co-creation process to implement AI education (Chiu et al., 2021 ). In Singapore, a program for AI learning in schools has also been developed, where K-12 children learn AI interactively. However, the program is hindered by a lack of professionals (teachers) with adequate training (Heintz, 2021 ). In Germany, there are also several initiatives to pilot AI-related projects and studies (Micheuz, 2020 ), including the launch of a national initiative to teach a holistic view of AI. This initiative consists of a 6-module course aimed at explaining how AI works, stimulating a social discourse on AI and clarifying the abundant existing misconceptions (Micheuz, 2020 ). Canada has also designed an AI course for high schools. The course is intended to empower students with knowledge about AI, covering both its philosophical and conceptual underpinnings as well as its practical aspects. The latter are achieved by building AI projects that solve real-life problems (Nisheva-Pavlova, 2021 ).

The literature also highlights the different approaches that AI literacy should focus on: curriculum design, AI subject design, student perspective, teacher training, resource design and gender diversity. All these approaches are described in depth below.

AI literacy curriculum design

Approaches to curriculum development differ widely, ranging from the product-centered model (technical-scientific perspective) to the process-centered model (learner perspective) (Yue et al., 2021 ). AI literacy can be launched in primary and secondary education depending on the age and computer literacy of the students. To do this, it is necessary to define the core competencies for AI literacy according to three dimensions: AI concepts, AI applications and AI ethics and security (Long & Magerko, 2020 ; Wong et al., 2020 ). Research has focused on the understanding of the concepts, the functional roles of AI, and the development of problem-solving skills (Woo et al., 2020 ). This has led to proposing a redefinition of the curriculum (Han et al., 2019 ; Malach & Vicherková, 2020 ; Zhang et al., 2020 ) supported by different ideas that K-12 students should know (Chiu et al., 2021 ; Sabuncuoglu, 2020 ; Touretzky et al., 2019b ). Several countries have already made different curricular proposals (Alexandre et al., 2021 ; Micheuz, 2020 ; Nisheva-Pavlova, 2021 ; Ottenbreit-Leftwich et al., 2021 ; Touretzky et al., 2019b ; Xiao & Song, 2021 ), where they argue that the curricular design must include different elements such as content, product, process and praxis (Chiu, 2021 ). It is also convenient for learning in AI to follow the computational thinking model (Shin, 2021 ), contextualizing the proposed curriculum (Eguchi et al., 2021 ; Wang et al., 2020 ) and providing it with the necessary resources for teachers (Eguchi et al., 2021 ). In this sense, emerging initiatives highlight the need to involve teachers in the process of co-creating a curriculum associated to their context (Barlex et al., 2020 ; Chiu et al., 2021 ; Dai et al., 2023 ; Lin & Brummelen, 2021 ; Yau et al., 2022 ).

AI as a subject in K-12 education

Traditionally, including computer science or new technologies in the educational system has been carried out through a specific subject integrated into the curriculum or through the offer of extracurricular activities. In this sense, different proposals have suggested the integration of AI as a subject in K-12 education (Ellis et al., 2009 ; Knijnenburg et al., 2021 ; Micheuz, 2020 ; Sperling & Lickerman, 2012 ), in short-term courses (around 15 h) and divided into learning modules focused on classical and modern AI (Wong, 2020 ) or through MOOCs (Alexandre et al., 2021 ).

Student perspective on AI Literacy

Student-focused studies explore and analyze attitudes and previous knowledge to make didactic proposals adapted to the learner. Some of them measure their intention and interest in learning AI (Bollin et al., 2020 ; Chai et al., 2021 , 2020a , 2020b ; Gao & Wang, 2019 ; Harris et al., 2004 ; Sing, et al., 2022 ; Suh & Ahn, 2022 ), whereas others discuss their views on the integration of technologies in the education system (Sorensen & Koefoed, 2018 ) and on teaching–learning support tools in AI (Holstein et al., 2019 ).

Teacher training in AI

Teachers are key players for the integration of AI literacy in K-12, as proven by the numerous studies that examine this issue (An et al., 2022 ; Bai & Yang, 2019 ; Chiu & Chai, 2020 ; Chiu et al., 2021 ; Chounta et al., 2021 ; Judd, 2020 ; Kandlhofer et al., 2019 , 2021 ; Kim et al., 2021 ; Korenova, 2016 ; Lin et al., 2022 ; Lindner & Berges, 2020 ; Oh, 2020 ; Summers et al., 1995 ; Wei et al., 2020 ; Wu et al., 2020 ; Xia & Zheng, 2020 ). This approach places teachers at the center, bearing in mind what they need to know so as to integrate AI into K-12 (Itmazi & Khlaif, 2022 ; Kim et al., 2021 ). The literature analyzed reports on the factors that influence the knowledge of novice teachers (Wei, 2021 ) and focuses on teacher training in AI (Lindner & Berges, 2020 ; Olari & Romeike, 2021 ). Thus, AI training proposals can be found aimed at both teachers in training (Xia & Zheng, 2020 ) and practicing educators. Training schemes focus on their knowledge in technologies to facilitate their professional development (Wei et al., 2020 ) through the TPACK (Technological, Pedagogical and Content Knowledge) teaching knowledge model (Gutiérrez-Fallas & Henriques, 2020 ). Studies focusing on teachers’ opinions on curriculum development in AI are relevant (Chiu & Chai, 2020 ), as are their self-efficacy in relation to ICT (Wu et al., 2020 ), their opinions on the tools that support the teaching–learning process in AI (Holstein et al., 2019 ) and their teacher training in technologies (Cheung et al, 2018 ; Jaskie et al., 2021 ). These elements are central to the design of an AI literacy strategy in K-12. Both the co-design of ML curricula between AI researchers and K-12 teachers, and the assessment of the impact of these educational interventions on K-12 are important issues today. At present, there is a shortage of teachers with training in AI and working with teachers in training (Xia & Zheng, 2020 ) or with teachers in schools (Chiu et al., 2021 ) is proposed as an effective solution. One of the most interesting analyses of teacher competency proposes the acquisition of this skill for the teaching of AI in K-12, through the analysis of the curricula and resources of AI using TPACK. This model was formulated by (Mishra & Koehler, 2006 ) and aims to define the different types of knowledge that teachers need to integrate ICT effectively in the classroom. In this regard, it is suggested that teachers imparting AI to K-12 students require TPACK to build an environment and facilitate project-based classes that solve problems using AI technologies (Kim et al., 2021 ).

AI literacy support resources

Research using this approach focuses on presenting resources that support AI literacy (Kandlhofer & Steinbauer, 2021 ), considering that the creation of resources and repositories is a priority in supporting this teaching–learning process (Matarić et al., 2007 ; Mongan & Regli, 2008 ). However, these resources largely do not meet an interdisciplinary approach and do not embody a general approach to AI development (Sabuncuoglu, 2020 ).

Gender diversity in AI literacy

AI education, as a broad branch of computer science, also needs to address the issue of gender diversity. Lack of gender diversity can impact the lives of the people for whom AI-based systems are developed. The literature highlights the existence of proposals designed with a perspective toward gender, where the activities designed are specifically aimed at girls (Ellis et al., 2009 ; Jagannathan & Komives, 2019 ; Perlin et al., 2005 ; Summers et al., 1995 ; Vachovsky et al., 2016 ; Xia et al., 2022 ).

The huge impact that AI is having on our lives, at work and in every type of organization and business sector is easily recognizable today. No one doubts that AI is one of the most disruptive technologies in history, if not the most. In recent years, the expectations generated by AI, far from being deflated, have only grown. We are still a long way from general-purpose AI, but the application of AI to solve real problems has already taken hold for a wide range of purposes. It is therefore necessary for young people to know how AI works, as this learning will make it easier for them to use these technologies in their daily lives, both to learn and to interact with others.

Like any other technology, the potential uses and abuses of AI go hand in hand with its disruptive capacity. Many social groups and governments are expressing concern about the possible negative consequences of AI misuse. Although it is crucial to adequately regulate the use of AI, education is as important, if not more important, than regulation. Everything, whether good or bad, stems from the education received. Thus, education systems must prepare students for a society in which they will have to live and interact with AI. AI education will enable young people to discover how these tools work and, consequently, to act responsibly and critically. Therefore, AI literacy has become a relevant and strategic issue (Chiu & Chai, 2020 ).

This systematic review has focused on analyzing AI teaching–learning proposals in K-12 globally. The results confirm that the teaching of basic AI- related concepts and techniques at the K-12 level is scarce (Kandlhofer et al., 2016 ). Our work shows that there have been, on the one hand, different AI learning experiences and, on the other hand, proposals for the implementation of AI literacy, made at the political level and by different experts. The learning experiences described show that AI literacy in schools has focused on technical, conceptual, and applied skills in some domains of interest. Proposals for AI implementation, especially those defined by the US and China, reveal that significant efforts are being made to design models that frame AI literacy proposals.

We also found that there are hardly any AI learning experiences that have analyzed learning outcomes, e.g., through assessments of learners’ understanding of AI concepts. Obviously, this is a result of the infancy of these AI learning experiences at the K-12 level. However, it is important for learning experiences to be based on clearly defined competencies in a particular AI literacy framework, such as those proposed in the literature (Alexandre et al., 2021 ; Han et al., 2019 ; Long & Magerko, 2020 ; Malach & Vicherková, 2020 ; Micheuz, 2020 ; Ottenbreit-Leftwich et al., 2021 ; Touretzky et al., 2019a ; Wong et al., 2020 ; Xiao & Song, 2021 ; Zhang et al., 2020 ). Recently, Van Brummelen et al. ( 2021a ) designed a curriculum for a five-day online workshop based on the specific AI competencies proposed by Long and Magerko ( 2020 ). They used several types of questionnaires to assess the quality of the program through the knowledge acquired by the students in these competencies. Therefore, clearly defined competency-based learning experiences can provide a rigorous assessment of student learning outcomes.

The research shows that clear guidelines are needed on what students are expected to learn about AI in K-12 (Chiu, 2021 ; Chiu & Chai, 2020 ; Lee et al., 2020 ). These studies highlight the need for a competency framework to guide the design of didactic proposals for AI literacy in K-12 in educational institutions. This framework would provide a benchmark for describing the areas of competency that K-12 learners should develop and which specific educational projects can be designed. Furthermore, it would support the definition of a curriculum reflecting sequence and academic continuity (Woo et al., 2020 ). Such a curriculum should be modular and personalized (Gong et al., 2019 ) and adjusted to the conditions of the schools (Wang et al., 2020 ). In the teaching of AI, an exploratory education should be adopted, which integrates science, computer science and integral practice (Wang et al., 2020 ). It should also address issues related to the ethical dimension, which is fundamental to the literacy of K-12 students as it enables them to understand the basic principles of AI (Henry et al., 2021 ). This training facilitates the development of students’ critical capacity, and this is necessary to understand that technology is not neutral and to benefit from and make appropriate use of it. Ethics, complementary to legal norms, enhances the democratic quality of society by setting legitimate limits in the shaping of technological life. In this sense, different AI literacy proposals in K-12 already support the addressing of ethical, social and security issues linked to AI technologies (Eguchi et al., 2021 ; Micheuz, 2020 ; Wong et al., 2020 ). Moreover, considering designing for social good could foster or help to motivate learning about AI (Chai et al., 2021 ). Without a doubt, all this will impact on the achievement of a more democratic society. Due to the gender gap in issues related to computer science, it is also necessary to address the gender perspective. In this vein, the research proposes, among other strategies, to focus AI literacy on real-world elements since this approach favors the motivation of girls and greater involvement in learning (Jagannathan & Komives, 2019 ). However, little attention is paid to the undesirable consequences of an indiscriminate and insufficiently thought-out application of AI, both in higher education and especially in K-12. For example, the increase in socio-economic inequality between countries and within countries, resulting from the increasing automation of employment, is of particular concern. This is leading to growing inequality in wages and preservation of human employment, but it is not usually a subject of interest in education.

Currently, the challenges of this AI literacy require an interdisciplinary and critical approach (Henry et al., 2021 ). We believe that AI literacy can be leveraged to enhance the learning of disciplinary core subjects by integrating AI into the teaching process of those subjects. AI literacy should rely on transferring AI knowledge and methods to core subjects, allowing education to cross disciplinary boundaries, but staying within the framework of disciplinary core subjects. To achieve this change, educators need to take a closer look at the current capabilities of AI. This would enable them to identify all options to improve the core of educational practice and thus optimize the educational process. For example, understanding and using word clouds is a powerful educational strategy to enhance education in core subjects such as science (e.g., to facilitate object classification), language (e.g. to enable the matching of different topics or authors’ works), music (e.g., to support the analysis of song lyrics) or social sciences (e.g., to assist in comparing different discourses). Since AI is highly interdisciplinary in nature, it has a broad projection on multiple fields and problems that require a transversal and applied approach. For example, the basic algorithms of ML could be taught in Mathematics and related disciplines, the design of supervised classifiers could be performed for the study of taxonomies in Biology, natural language processing could be used to make the study of a language more attractive, or the ethical issues surrounding AI could be discussed in Philosophy and Social Sciences subjects.

Finally, for this meaningful learning to take place, AI teaching must be addressed through holistic, active, and collaborative pedagogical strategies in which real problem solving is the starting point of the learning process. An important gap regarding the integration of AI in K-12 concerns teachers, as it is unclear how to prepare and involve them in the process (Chiu & Chai, 2020 ). Teachers’ attitudes towards AI have a significant influence on the effectiveness of using AI in education. Teachers can swing between total resistance and overconfidence. The first could arise from inadequate, inappropriate, irrelevant, or outdated professional development. On the one hand, teachers must be digitally-competent enough to integrate AI into the teaching–learning processes of their subjects. Therefore, teacher training is also necessary following a framework of standard competencies. This should include new ways of organizing the professional role of teachers, as well as enhancing students’ attitudes towards these changes. On the other hand, research reveals that it is essential for didactic proposals to be co-designed and implemented by the teachers at those schools involved (Henry et al., 2021 ), to undergo training in the specific AI subjects and for this knowledge to be integrated into non-computer subjects (Lin & Brummelen, 2021 ). To this end, it is crucial to identify the perception and knowledge that teachers have about AI and involve them in the design of curricular proposals (Chiu, 2021 ; Chiu & Chai, 2020 ; Chiu et al., 2021 ).

This study aimed to understand how AI literacy is being integrated into K-12 education. To achieve this, we conducted a search process following the systematic literature review method and using Scopus. Two broad groups of AI literacy approaches were identified, namely learning experiences and theoretical perspective. The study revealed that learning experiences in schools have focused mainly on technical and applied skills limited to a specific domain without rigorously assessing student learning outcomes. In contrast, the US and China are leading the way in AI literacy implementation schemes which are broader in scope and involve a more ambitious approach. However, there is still a need to test these initiatives through comprehensive learning experiences that incorporate an analysis of learning outcomes. This work has allowed us to draw several conclusions that can be considered in the design of AI literacy proposals in K-12. Firstly, AI literacy should be based on an interdisciplinary and competency-based approach and integrated into the school curriculum. There is no need to include a new AI subject in the curriculum, but rather to build on the competencies and content of disciplinary subjects and then integrate AI literacy into those subjects. Given the interdisciplinary nature of AI, AI education can break disciplinary boundaries and adopt a global, practical, and active approach in which project-based and contextualized work plays an important role. Secondly, AI literacy should be leveraged to extend and enhance learning in curricular subjects. As a final point, AI literacy must prioritize the competency of teachers and their active participation in the co-design of didactic proposals, together with pedagogues and AI experts.

Availability of data and materials

Last revision round required update the review. Thus, Additional file 1 contains a.csv file with the listing of papers that are not cited but are part of the reviewed papers. The papers cited in text already appear in the Reference section and, therefore, not in the Additional file.

1 Conference categorization and ranking based on the GII-GRIN-SCIE (GGS) Conference Ratings: https://scie.lcc.uma.es/

Aguar, K., Arabnia, H. R., Gutierrez, J. B., Potter, W. D., & Taha, T. R. (2016). Making cs inclusive: An overview of efforts to expand and diversify cs education. In International Conference on Computational Science and Computational Intelligence (CSCI). (pp. 321–326). https://doi.org/10.1109/CSCI.2016.0067

Alexandre, F., Becker, J., Comte, M. H., Lagarrigue, A., Liblau, R., Romero, M., & Viéville, T. (2021). Why, What and How to help each citizen to understand artificial intelligence? KI Kunstliche Intelligenz, 35 (2), 191–199. https://doi.org/10.1007/s13218-021-00725-7

Article   Google Scholar  

Ali, S., DiPaola, D., Lee, I., Hong, J., & Breazeal, C. (2021a). Exploring generative models with middle school students. In Proceedings of the 2021a CHI Conference on Human Factors in Computing Systems. (pp. 1–13). https://doi.org/10.1145/3411764.3445226

Ali, S., DiPaola, D., Lee, I., Sindato, V., Kim, G., Blumofe, R., & Breazeal, C. (2021b). Children as creators, thinkers and citizens in an AI-driven future. Computers and Education Artificial Intelligence . https://doi.org/10.1016/j.caeai.2021.100040

An, X., Chai, C. S., Li, Y., Zhou, Y., Shen, X., Zheng, C., & Chen, M. (2022). Modeling English teachers’ behavioral intention to use artificial intelligence in middle schools. Education and Information Technologies . https://doi.org/10.1007/s10639-022-11286-z

Aung, Z. H., Sanium, S., Songsaksuppachok, C., Kusakunniran, W., Precharattana, M., Chuechote, S., & Ritthipravat, P. (2022). Designing a novel teaching platform for AI: A case study in a Thai school context. Journal of Computer Assisted Learning, 38 (6), 1714–1729. https://doi.org/10.1111/jcal.12706

Avanzato, R. L. (2009). Autonomous Outdoor Mobile Robot Challenge. Computer in Education Journal (July-September 2009).

Bai, H., & Yang, S. (2019, October). Research on the Sustainable Development Model of Information Technology Literacy of Normal Students Based on Deep Learning Recommendation System. In 2019 4th International Conference on Mechanical, Control and Computer Engineering (ICMCCE). (pp. 837–840). https://doi.org/10.1109/ICMCCE48743.2019.00192

Barlex, D., Steeg, T., & Givens, N. (2020). Teaching about disruption: A key feature of new and emerging technologies. Learning to Teach Design and Technology in the Secondary School, 4 , 137–154. https://doi.org/10.4324/9780429321191-9

Bollin, A., Kesselbacher, M., & Mößlacher, C. (2020). Ready for computing science? A closer look at personality, interests and self-concept of girls and boys at secondary level. In  Informatics in Schools. Engaging Learners in Computational Thinking: 13th International Conference, ISSEP. (pp. 107–118). https://doi.org/10.1007/978-3-030-63212-0_9

Burgsteiner, H., Kandlhofer, M., & Steinbauer, G. (2016). IRobot: teaching the basics of artificial intelligence in high schools. Proceedings of the AAAI Conference on Artificial Intelligence . https://doi.org/10.1609/aaai.v30i1.9864

Chai, C.S., Lin, P.-Y., Jong, M.S.-Y., Dai, Y., Chiu, T.K., & Huang, B. (2020a). Factors influencing students' behavioral intention to continue artificial intelligence learning. In 2020a International Symposium on Educational Technology (ISET). (pp. 147–150).  https://doi.org/10.1109/ISET49818.2020.00040

Chai, C. S., Wang, X., & Xu, C. (2020b). An extended theory of planned behavior for the modelling of chinese secondary school students’ intention to learn artificial intelligence. Mathematics, 8 (11), 1–18. https://doi.org/10.3390/math8112089

Chai, C.S., Lin, P.-Y., Jong, M.S.-Y., Dai, Y., Chiu, T.K., & Qin, J. (2021). Perceptions of and behavioral intentions towards learning artificial intelligence in primary school students. Educational Technology & Society , 24 (3), 89–101. Retrieved from https://www.jstor.org/stable/27032858

Chen, S., Qian, B., & Cheng, H. (2017). Voice recognition for STEM education using robotics. In Volume 9: 13th ASME/IEEE International Conference on Mechatronic and Embedded Systems and Applications. ASME . https://doi.org/10.1115/DETC2017-68368

Chen, M., Zhou, C., & Wu, Y. (2020). Research on the model and development status of information literacy self-improvement ability of primary and secondary school teachers. In Ninth International Conference of Educational Innovation through Technology (EITT). (pp. 87–91). https://doi.org/10.1109/EITT50754.2020.00021

Cheung, S. K., Lam, J., Li, K. C., Au, O., Ma, W. W., & Ho, W. S. (Eds.). (2018).  Technology in Education. Innovative Solutions and Practices: Third International Conference, ICTE 2018. Springer.

Chittora, S., & Baynes, A. (2020, October). Interactive Visualizations to Introduce Data Science for High School Students. In  Proceedings of the 21st Annual Conference on Information Technology Education.  (pp. 236–241). https://doi.org/10.1145/3368308.3415360

Chiu, T. K. F. (2021). A holistic approach to the design of artificial intelligence (AI) education for k-12 schools. TechTrends, 65 (5), 796–807. https://doi.org/10.1007/s11528-021-00637-1

Chiu, T. K., & Chai, C.-S. (2020). Sustainable curriculum planning for artificial intelligence education: A self-determination theory perspective. Sustainability (switzerland) . https://doi.org/10.3390/su12145568

Chiu, T. K., Meng, H., Chai, C. S., King, I., Wong, S., & Yam, Y. (2021). Creation and evaluation of a pretertiary artificial intelligence (AI) curriculum. IEEE Transactions on Education, 65 (1), 30–39. https://doi.org/10.1109/TE.2021.3085878

Chounta, I.-A., Bardone, E., Raudsep, A., & Pedaste, M. (2021). Exploring teachers’ perceptions of artificial intelligence as a tool to support their practice in estonian k-12 education. International Journal of Artificial Intelligence in Education, 32 , 725–755. https://doi.org/10.1007/s40593-021-00243-5

Crompton, H., Jones, M. V., & Burke, D. (2022). Affordances and challenges of artificial intelligence in K-12 education: a systematic review. Journal of Research on Technology in Education . https://doi.org/10.1080/15391523.2022.2121344

Croxell, J., Mead, R., & Weinberg, J. (2007). Designing robot competitions that promote ai solutions: Lessons learned competing and designing. Technical Report of the 2007 American Association of Artificial Intelligence. Spring Symposia , SS-07–09. (pp. 29–34).

Cummings D., Anthony M., Watson C., Watson A., & Boone S. (2021). Combating social injustice and misinformation to engage minority youth in computing sciences. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. (pp.1006–1012). https://doi.org/10.1145/3408877.3432452 .

Dai, Y., Liu, A., Qin, J., Guo, Y., Jong, M. S. Y., Chai, C. S., & Lin, Z. (2023). Collaborative construction of artificial intelligence curriculum in primary schools. Journal of Engineering Education, 112 (1), 23–42. https://doi.org/10.1002/jee.20503

Díaz, J., Queiruga, C., Tzancoff, C., Fava, L., & Harari, V. (2015). Educational robotics and videogames in the classroom. In 2015 10th Iberian Conference on Information Systems and Technologies (CISTI) . Aveiro, Portugal. (pp. 1–6). https://doi.org/10.1109/CISTI.2015.7170616

du Boulay, B. (2016). Artificial intelligence as an effective classroom assistant. IEEE Intelligent Systems, 31 (6), 76–81. https://doi.org/10.1109/MIS.2016.93

Eguchi, A. (2021). AI-robotics and ai literacy. Studies in Computational Intelligence, 982 , 75–85. https://doi.org/10.1007/978-3-030-77022-8

Eguchi, A., & Okada, H. (2018). If you give students a social robot? - world robot summit pilot study. In  Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction.  (pp. 103–104). https://doi.org/10.1145/3173386.3177038

Eguchi, A., & Okada, H. (2020). Imagine the Future with Social Robots - World Robot Summit’s Approach: Preliminary Investigation. In M. Moro, D. Alimisis, & L. Iocchi, L. (Eds) Educational Robotics in the Context of the Maker Movement. Edurobotics 2018. Advances in Intelligent Systems and Computing, (p. 946). Springer. https://doi.org/10.1007/978-3-030-18141-3_10

Eguchi, A., Okada, H., & Muto, Y. (2021). Contextualizing AI education for k- 12 students to enhance their learning of ai literacy through culturally responsive approaches. KI Kunstliche Intelligenz, 35 (2), 153–161. https://doi.org/10.1007/s13218-021-00737-3

Ellis, G., Ory, E., Bhushan, N. (2005). Organizing a K-12 AI curriculum using philosophy of the mind. Engineering: Faculty Publications, Smith College. Retrieved from https://scholarworks.smith.edu/egr_facpubs/96

Ellis, G., Silva, K., Epstein, T., & Giammaria, N. (2009). Artificial intelligence in pre-college education: Learning within a philosophy of the mind framework. International Journal of Engineering Education, 25 (3), 511–522.

Google Scholar  

Evangelista, I., Blesio, G., & Benatti, E. (2019). Why are we not teaching machine learning at high school? a proposal. In 2018 World Engineering Education Forum - Global Engineering Deans Council (WEEF-GEDC). (pp. 1–6). https://doi.org/10.1109/WEEF-GEDC.2018.8629750

Gao, J., & Wang, L. (2019). Reverse thinking teaching discussion in high school information technology under new curriculum standards. In 14th International Conference on Computer Science & Education (ICCSE). (pp. 222–226). https://doi.org/10.1109/ICCSE.2019.8845429

Giannakos, M., Voulgari, I., Papavlasopoulou, S., Papamitsiou, Z., & Yannakakis, G. (2020). Games for artificial intelligence and machine learning education: Review and perspectives. Lecture Notes in Educational Technology . https://doi.org/10.1007/978-981-15-6747-6_7

Gong, X., Zhao, L., Tang, R., Guo, Y., Liu, X., He, J., … Wang, X. (2019). AI education system for primary and secondary schools. In  2019 ASEE Annual Conference & Exposition .

Gonzalez, A. J., Hollister, J. R., DeMara, R. F., Leigh, J., Lanman, B., Lee, S. Y., & Wilder, B. (2017). AI in informal science education: bringing turing back to life to perform the turing test. International Journal of Artificial Intelligence in Education, 27 (2), 353–384. https://doi.org/10.1007/s40593-017-0144-1

Guan, C., Mou, J., & Jiang, Z. (2020). Artificial intelligence innovation in education: A twenty-year data-driven historical analysis. International Journal of Innovation Studies, 4 (4), 134–147. https://doi.org/10.1016/j.ijis.2020.09.001

Gutiérrez, L. F., & Henriques, A. (2020). Prospective mathematics teachers’ tpack in a context of a teacher education experiment. Revista Latinoamericana De Investigación En Matemática Educativa, 23 (2), 175–202. https://doi.org/10.12802/relime.20.2322

Han, X., Hu, F., Xiong, G., Liu, X., Gong, X., Niu, X., … Wang, X. (2019). Design of AI + curriculum for primary and secondary schools in Qingdao. In Chinese Automation Congress (CAC) . (pp. 4135–4140). https://doi.org/10.1109/CAC.2018.8623310

Harris, E., Lamonica, A., & Weinberg. JB. (2004) Interfacing the public and technology: a web controlled mobile robot. In Accessible hands-on artificial intelligence and robotics education: working papers of the 2004. AAAI spring symposium series. AAAI Press. (pp.106–110)

He, Y.-T., Guo, B.-J., Lu, J., Xu, Y.-P., & Gong, M. (2020). Research of scratch programming recommendation system based on med and knowledge graph. In 2020 5th International Conference on Mechanical, Control and Computer Engineering (ICMCCE) . (pp. 2158–2163). https://doi.org/10.1109/ICMCCE51767.2020.00469

Heintz, F. (2021). Three interviews about k-12 ai education in america, europe, and singapore. KI Kunstliche Intelligenz, 35 (2), 233–237. https://doi.org/10.1007/s13218-021-00730-w

Henry, J., Hernalesteen, A., & Collard, A.-S. (2021). Teaching artificial intelli- gence to k-12 through a role-playing game questioning the intelligence concept. KI Kunstliche Intelligenz, 35 (2), 171–179. https://doi.org/10.1007/s13218-021-00733-7

Herrero, Á., Cambra, C., Urda, D., Sedano, J., Quintián, H., & Corchado, E. (Eds) (2020). 11th International Conference on European Transnational Educational (ICEUTE 2020). ICEUTE 2020. Advances in Intelligent Systems and Computing, 1266 . Springer. https://doi.org/10.1007/978-3-030-57799-5_8

Ho, J. W., Scadding, M., Kong, S. C., Andone, D., Biswas, G., Hoppe, H. U., & Hsu, T. C. (2019). Classroom activities for teaching artificial intelligence to primary school students. In  Proceedings of international conference on computational thinking education. The Education University of Hong Kong. (pp. 157–159).

Holowka, P. (2020). Teaching robotics during COVID-19: Machine learning, simulation, and aws deepracer. In 17th International Conference on Cognition and Exploratory Learning in Digital Age, CELDA .

Holstein, K., McLaren, B. M., & Aleven, V. (2019). Designing for complementarity: Teacher and student needs for orchestration support in ai-enhanced classrooms. In S. Isotani, E. Millán, A. Ogan, P. Hastings, B. McLaren, & R. Luckin (Eds.), Artificial intelligence in education. AIED 2019. Lecture notes in computer science (p. 11625). Springer. 10.1007/978-3-030-23204-7_14.

Itmazi, J., & Khlaif, Z. N. (2022). Science education in Palestine: Hope for a better future. Lecture Notes in Educational Technology . https://doi.org/10.1007/978-981-16-6955-2_9

Jagannathan, R. K., & Komives, C. (2019). Teaching by induction: Project- based learning for Silicon Valley. Journal of Engineering Education Transformations, 33 (1), 22–26. https://doi.org/10.16920/jeet/2019/v33i1/149003

Jaskie, K., Larson, J., Johnson, M., Turner, K., O’Donnell, M., Christen, J.B., & Spanias, A. (2021). Research experiences for teachers in machine learning. In IEEE Frontiers in Education Conference (FIE) . Lincoln, NE, USA. (pp. 1–5). https://doi.org/10.1109/FIE49875.2021.9637132

Judd, S. (2020). Activities for Building Understanding: How AI4ALL Teaches AI to Diverse High School Students. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. (pp. 633–634). https://doi.org/10.1145/3328778.3366990

Kahn, K., Megasari, R., Piantari, E., & Junaeti, E. (2018). AI programming by children using Snap! block programming in a developing country. In Thirteenth European Conference on Technology Enhanced Learning . (p. 11082). https://doi.org/10.1007/978-3-319-98572-5

Kandlhofer, M., Steinbauer, G., Hirschmugl-Gaisch, S., & Huber, P. (2016). Artificial intelligence and computer science in education: From kinder- garten to university. In IEEE Frontiers in Education Conference. (pp. 1–9). https://doi.org/10.1109/FIE.2016.7757570

Kandlhofer, M., Steinbauer, G., Lasnig, J.P., Baumann, W., Plomer, S., Ballagi, A., & Alfoldi, I. (2019). Enabling the creation of intelligent things: Bringing artificial intelligence and robotics to schools. In IEEE Frontiers in Education Conference (FIE) . (pp. 1–5). https://doi.org/10.1109/FIE43999.2019.9028537

Kandlhofer, M., & Steinbauer, G. (2021). AI k-12 education service. KI Kunstliche Intelligenz, 35 (2), 125–126. https://doi.org/10.1007/s13218-021-00715-9

Kandlhofer, M., Steinbauer, G., Lassnig, J., Menzinger, M., Baumann, W., Ehardt-Schmiederer, M., & Szalay, I. (2021). EDLRIS: A European driving license for robots and intelligent systems. KI Kunstliche Intelligenz, 35 (2), 221–232. https://doi.org/10.1007/s13218-021-00716-8

Kaspersen, M. H., Bilstrup, K. E. K., Van Mechelen, M., Hjort, A., Bouvin, N. O., & Petersen, M. G. (2022). High school students exploring machine learning and its societal implications Opportunities and challenges. International Journal of Child-Computer Interaction, 34 , 1–12. https://doi.org/10.1016/j.ijcci.2022.100539

Kelly, J., Binney, J., Pereira, A., Khan, O., & Sukhatme, G. (2008). Just add wheels: Leveraging commodity laptop hardware for robotics and ai education. In  Proceedings of AAAI Education Colloquium , 22.

Kim, K., Kwon, K., Ottenbreit-Leftwich, A., Bae, H., & Glazewski, K. (2023). Exploring middle school students’ common naive conceptions of Artificial Intelligence concepts, and the evolution of these ideas. Education and Information Technologies . https://doi.org/10.1007/s10639-023-11600-3

Kim, S., Jang, Y., Choi, S., Kim, W., Jung, H., Kim, S., & Kim, H. (2021). Analyzing teacher competency with tpack for k-12 ai education. KI Kunstliche Intelligenz, 35 (2), 139–151. https://doi.org/10.1007/s13218-021-00731-9

Kitchenham, B. (2004). Procedures for performing systematic reviews (Vol. 33, pp. 1–26). Keele: Keele University.

Knijnenburg, B., Bannister, N., & Caine, K. (2021). Using mathematically- grounded metaphors to teach ai-related cybersecurity. In IJCAI-21 Workshop on Adverse Impacts and Collateral Effects of Artificial Intelligence Technologies (AIofAI) .

Kong, S. C., ManYinCheung, W., & Zhang, G. (2021). Evaluation of an artificial intelligence literacy course for university students with diverse study backgrounds. Computers and Education: Artificial Intelligence . https://doi.org/10.1016/j.caeai.2021.100026

Korenova, L. (2016). Digital technologies in teaching mathematics on the faculty of education of the Comenius University in Bratislava. In 15 Conference on Applied Mathematics. Slovak University of Technology in Bratislava. (p. 690–699).

Lee, S., Mott, B., Ottenbriet-Leftwich, A., Scribner, A., Taylor, S., Glazewski, K.,…Lester, J. (2020). Designing a collaborative game-based learning environment for ai-infused inquiry learning in elementary school class- rooms. In Proceedings of the 2020 ACM conference on innovation and technology in computer science education . (pp. 566–566). https://doi.org/10.1145/3341525.3393981

Lee, I., Ali, S., Zhang, H., Dipaola, D., & Breazeal, C. (2021). Developing middle school students’ ai literacy. In Association for Computing Machinery, Inc. (pp. 191–197). https://doi.org/10.1145/3408877.3432513

Lenoir, Y., & Hasni, A. (2016). Interdisciplinarity in primary and secondary school: Issues and perspectives. Creative Education, 7 (16), 2433–2458. https://doi.org/10.4236/ce.2016.716233

Lin, P., & Brummelen, J. (2021). Engaging teachers to co-design integrated ai curriculum for k-12 classrooms. In CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . (pp.1–12). https://doi.org/10.1145/3411764.3445377

Lin, X. F., Chen, L., Chan, K. K., Peng, S., Chen, X., Xie, S., & Hu, Q. (2022). Teachers’ perceptions of teaching sustainable artificial intelligence: A design frame perspective. Sustainability, 14 (13), 1–20. https://doi.org/10.3390/su14137811

Lindner, A., & Berges, M. (2020). Can you explain ai to me? teachers’ pre- concepts about artificial intelligence. In IEEE Frontiers in Education Conference (FIE). (pp. 1–9). https://doi.org/10.1109/FIE44824.2020.9274136

Long, D., & Magerko, B. (2020). What is AI literacy? Competencies and design considerations. In Proceedings of the 2020 chi conference on human factors in computing systems. (pp. 1–16). https://doi.org/10.1145/3313831.3376727

Malach, J., & Vicherková, D. (2020). Background of the Revision of the Secondary School Engineering Curriculum in the Context of the Society 4.0. In M. Auer, H. Hortsch & P. Sethakul (Eds). The Impact of the 4th Industrial Revolution on Engineering Education. ICL Advances in Intelligent Systems and Computing, vol 1135. Springer. https://doi.org/10.1007/978-3-030-40271-6_27

Matarić, M.J., Koenig, N., & Feil-Seifer, D. (2007). Materials for enabling hands- on robotics and stem education. In AAAI Spring Symposium: Semantic Scientific Knowledge Integration. (pp. 99–102). http://www.aaai.org/Papers/Symposia/Spring/2007/SS-07-09/SS07-09-022.pdf

Miao, F., Holmes, W., Huang, R., & Zhang, H. (2021). AI and education: A guidance for policymakers . UNESCO Publishing.

Micheuz, P. (2020). Approaches to Artificial Intelligence as a Subject in School Education. In T. Brinda, D. Passey, & T. Keane (Eds), Empowering Teaching for Digital Equity and Agency. OCCE 2020. IFIP Advances in Information and Communication Technology, 595. Springer. https://doi.org/10.1007/978-3-030-59847-1_1

Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108 (6), 1017–1054. https://doi.org/10.1111/j.1467-9620.2006.00684.x

Mongan, W.M., & Regli, W.C. (2008). A cyber-infrastructure for supporting k-12 engineering education through robotics, WS-08-02, 68–73.

Narahara, T., & Kobayashi, Y. (2018). Personalizing homemade bots with plug & play ai for steam education. In SIGGRAPH Asia 2018 technical briefs . (pp. 1–4). https://doi.org/10.1145/3283254.3283270

Ng, D. T. K., Lee, M., Tan, R. J. Y., Hu, X., Downie, J. S., & Chu, S. K. W. (2022). A review of AI teaching and learning from 2000 to 2020. Education and Information Technologies . https://doi.org/10.1007/s10639-022-11491-w

Nisheva-Pavlova, M.M. (2021). Ai courses for secondary and high school - comparative analysis and conclusions. In CEUR Workshop Proceedings, 3061. (pp. 9–16).

Nurbekova, Z., Mukhamediyeva, K., & Assainova, A. (2018). Educational robotics technologies in Kazakhstan and in the world: Comparative analysis, current state and perspectives. Astra Salvensis, 6 (1), 665–686.

Oh, W. (2020). Physics teachers’ perception of it convergence-based physics education. New Physics: Sae Mulli, 70 (8), 660–666. https://doi.org/10.3938/NPSM.70.660

Olari, V., & Romeike, R. (2021). Addressing ai and data literacy in teacher education: A review of existing educational frameworks. In WiPSCE '21: The 16th Workshop in Primary and Secondary Computing Education, 17. (pp. 1–2) https://doi.org/10.1145/3481312.3481351

Ottenbreit-Leftwich, A., Glazewski, K., Jeon, M., Hmelo-Silver, C., Mott, B., Lee, S., & Lester, J. (2021). How do elementary students conceptualize artificial intelligence? In SIGCSE '21: Proceedings of the 52nd ACM Technical Symposium on Computer Science Education . (pp. 1261). https://doi.org/10.1145/3408877.3439642

Park, K., Mott, B., Lee, S., Glazewski, K., Scribner, J., Ottenbreit-Leftwich, A., & Lester, J. (2021). Designing a visual interface for elementary students to formulate ai planning tasks. In IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). (pp. 1–9). https://doi.org/10.1109/VL/HCC51201.2021.9576163

Perlin, K., Flanagan, M., & Hollingshead, A. (2005). The Rapunsel Project. In Subsol, G. (Eds). Virtual Storytelling. Using Virtual Reality Technologies for Storytelling. ICVS 2005. Lecture Notes in Computer Science , 3805. Springer. https://doi.org/10.1007/11590361_29

Reyes, A., Elkin, C., Niyaz, Q., Yang, X., Paheding, S., & Devabhaktuni, V. (2020). A preliminary work on visualization-based education tool for high school machine learning education. In IEEE Integrated STEM Education Conference (ISEC). (pp. 1–5). https://doi.org/10.1109/ISEC49744.2020.9280629

Rodríguez-García, J., Moreno-León, J., Román-González, M., & Robles, G. (2020). Introducing artificial intelligence fundamentals with learning ML: Artificial intelligence made easy. In TEEM'20: Eighth International Conference on Technological Ecosystems for Enhancing Multiculturality . (pp. 18–20). https://doi.org/10.1145/3434780.3436705

Russell, S. (2021). The history and future of AI. Oxford Review of Economic Policy, 37 (3), 509–520. https://doi.org/10.1093/oxrep/grab013

Russell, S., & Norvig, P. (2021). Artificial Intelligence, global edition a modern approach . Pearson Deutschland.

Sabuncuoglu, A. (2020). Designing one year curriculum to teach artificial intelligence for middle school. In Proceedings of the 2020 ACM conference on innovation and technology in computer science education . (pp. 96–102). https://doi.org/10.1145/3341525.3387364

Sakulkueakulsuk, B., Witoon, S., Ngarmkajornwiwat, P., Pataranutapom, P., Surareungchai, W., Pataranutaporn, P., & Subsoontorn, P. (2019). Kids making ai: Integrating machine learning, gamification, and social context. In 2018 IEEE international conference on teaching, assessment, and learning for engineering (TALE) . (pp. 1005–1010). https://doi.org/10.1109/TALE.2018.8615249

Sanusi, I. T., Oyelere, S. S., Vartiainen, H., Suhonen, J., & Tukiainen, M. (2022). A systematic review of teaching and learning machine learning in K-12 education. Education and Information Technologies . https://doi.org/10.1007/s10639-022-11416-7

Shin, S. (2021). A study on the framework design of artificial intelligence thinking for artificial intelligence education. International Journal of Information and Education Technology, 11 (9), 392–397. https://doi.org/10.18178/ijiet.2021.11.9.1540

Sing, C. C., Teo, T., Huang, F., Chiu, T. K., & Xing Wei, W. (2022). Secondary school students’ intentions to learn AI: Testing moderation effects of readiness, social good and optimism. Educational Technology Research and Development, 70 (3), 765–782. https://doi.org/10.1007/s11423-022-10111-1

Sorensen, L., & Koefoed, N. (2018). The future of teaching—what are students’ expectations. In 2018 11th CMI International Conference: Prospects and Challenges Towards Developing a Digital Economy within the EU . (pp. 62–66). https://doi.org/10.1109/PCTDDE.2018.8624771

Sperling, A., & Lickerman, D. (2012). Integrating AI and machine learning in software engineering course for high school students. In Proceedings of the 17th ACM annual conference on Innovation and technology in computer science education . (pp. 244–249). https://doi.org/10.1145/2325296.2325354

Su, J., Zhong, Y., & Ng, D. T. K. (2022). A meta-review of literature on educational approaches for teaching AI at the K-12 levels in the Asia-Pacific region. Computers and Education: Artificial Intelligence, 3 , 1–18. https://doi.org/10.1016/j.caeai.2022.100065

Suh, W., & Ahn, S. (2022). Development and validation of a scale measuring student attitudes toward artificial intelligence. SAGE Open, 12 (2), 1–12. https://doi.org/10.1177/21582440221100463

Summers, B.G., Hicks, H., & Oliver, C. (1995). Reaching minority, female and disadvantaged students. In Proceedings Frontiers in Education 1995 25th Annual Conference. Engineering Education for the 21st Century , 1 . (992a4–16). https://doi.org/10.1109/FIE.1995.483030

Tedre, M., Toivonen, T., Kahila, J., Vartiainen, H., Valtonen, T., Jormanainen, I., & Pears, A. (2021). Teaching machine learning in k-12 classroom: Pedagogical and technological trajectories for artificial intelligence education. IEEE Access, 9 , 110558–110572. https://doi.org/10.1109/ACCESS.2021.3097962

Tims, H., Turner III, G., Cazes, G., & Marshall, J. (2012). Junior cyber discovery: Creating a vertically integrated middle school cyber camp. In  2012 ASEE Annual Conference & Exposition.  (pp. 25–867). Retrieved from https://peer.asee.org/21624

Toivonen, T., Jormanainen, I., Kahila, J., Tedre, M., Valtonen, T., Vartiainen, H. (2020). Co-designing machine learning apps in k-12 with primary school children. In  2020 IEEE 20th International Conference on Advanced Learning Technologies (ICALT). IEEE. (pp. 308–310). https://doi.org/10.1109/ICALT49669.2020.00099

Touretzky, D., Gardner-McCune, C., Breazeal, C., Martin, F., & Seehorn, D. (2019a). A year in k-12 ai education. AI Magazine, 40 (4), 88–90. https://doi.org/10.1609/aimag.v40i4.5289

Touretzky, D., Gardner-McCune, C., Martin, F., & Seehorn, D. (2019b). Envisioning ai for k-12: What should every child know about ai? Proceedings of the AAAI Conference on Artificial Intelligence, 33 (01), 9795–9799. https://doi.org/10.1609/aaai.v33i01.33019795

Vachovsky, M., Wu, G., Chaturapruek, S., Russakovsky, O., Sommer, R., & Fei-Fei, L. (2016). Towards more gender diversity in cs through an arti- ficial intelligence summer program for high school girls. In  Proceedings of the 47th ACM technical symposium on computing science education.  (pp. 303–308). https://doi.org/10.1145/2839509.2844620

Van Brummelen, J., Heng, T., & Tabunshchyk, V. (2021a). Teaching tech to talk: K-12 conversational artificial intelligence literacy curriculum and development tools. Proceedings of the AAAI Conference on Artificial Intelligence, 35 (17), 15655–15663. https://doi.org/10.1609/aaai.v35i17.17844

Van Brummelen, J., Tabunshchyk, V., & Heng, T. (2021b). Alexa, can i program you? Student perceptions of conversational artificial intelligence before and after programming Alexa. In IDC '21: Interaction Design and ChildrenJune. (pp. 305–313) https://doi.org/10.1145/3459990.3460730

Vartiainen, H., Tedre, M., & Valtonen, T. (2020). Learning machine learning with very young children: Who is teaching whom? International Journal of Child-Computer Interaction, 25 , 1–11. https://doi.org/10.1016/j.ijcci.2020.100182

Vartiainen, H., Toivonen, T., Jormanainen, I., Kahila, J., Tedre, M., & Valtonen, T. (2021). Machine learning for middle schoolers: Learning through data- driven design. International Journal of Child-Computer Interaction, 29 , 1–12. https://doi.org/10.1016/j.ijcci.2021.100281

Verner, I., Cuperman, D., & Reitman, M. (2021). Exploring robot connectivity and collaborative sensing in a high-school enrichment program. Robotics, 10 (1), 1–19. https://doi.org/10.3390/robotics10010013

von Wangenheim, C. G., Hauck, J. C., Pacheco, F. S., & Bueno, M. F. B. (2021). Visual tools for teaching machine learning in K-12: A ten-year systematic mapping. Education and Information Technologies, 26 (5), 5733–5778. https://doi.org/10.1007/s10639-021-10570-8

Wan, X., Zhou, X., Ye, Z., Mortensen, C., & Bai, Z. (2020). Smileyclus- ter: Supporting accessible machine learning in k-12 scientific discovery. In  proceedings of the Interaction Design and Children Conference.  (pp. 23–35). https://doi.org/10.1145/3392063.3394440

Wang, H., Liu, Y., Han, Z., & Wu, J. (2020). Extension of media literacy from the perspective of artificial intelligence and implementation strategies of artificial intelligence courses in junior high schools. In  2020 International Conference on Artificial Intelligence and Education (ICAIE).  (pp. 63–66). https://doi.org/10.1109/ICAIE50891.2020.00022

Wei, Y. (2021). Influence factors of using modern teaching technology in the classroom of junior middle school teachers under the background of artificial intelligence-analysis based on HLM. Advances in Intelligent Systems and Computing, 1282 , 110–118. https://doi.org/10.1007/978-3-030-62743-0_16

Wei, Q., Li, M., Xiang, K., & Qiu, X. (2020). Analysis and strategies of the professional development of information technology teachers under the vision of artificial intelligence. In  2020 15th International Conference on Computer Science & Education (ICCSE).  (pp. 716–721). https://doi.org/10.1109/ICCSE49874.2020.9201652

West, D.M., & Allen, J.R. (2018). How artificial intelligence is transforming the world. Report. Retrieved April 24, 2018, f rom https://www.brookings.edu/research/how-artificial-intelligence-is-transforming-the-world/

Wong, K.-C. (2020). Computational thinking and artificial intelligence education: A balanced approach using both classical AI and modern AI. CoolThink@ JC , 108.

Wong, G. K., Ma, X., Dillenbourg, P., & Huen, J. (2020). Broadening artificial intelligence education in k-12: Where to start? ACM Inroads, 11 (1), 20–29. https://doi.org/10.1145/3381884

Woo, H., Kim, J., Kim, J., & Lee, W. (2020). Exploring the ai topic composition of k-12 using nmf-based topic modeling. International Journal on Advanced Science, Engineering and Information Technology, 10 (4), 1471–1476. https://doi.org/10.18517/ijaseit.10.4.12787

Wu, D., Zhou, C., Meng, C., & Chen, M. (2020). Identifying multilevel factors influencing ICT self-efficacy of k-12 teachers in China. In  Blended Learning. Education in a Smart Learning Environment: 13th International Conference, ICBL 2020. (pp. 303–314). Springer International Publishing. https://doi.org/10.1007/978-3-030-51968-1

Xia, Q., Chiu, T. K., & Chai, C. S. (2022). The moderating effects of gender and need satisfaction on self-regulated learning through Artificial Intelligence (AI). Education and Information Technologies . https://doi.org/10.1007/s10639-022-11547-x

Xia, L., & Zheng, G. (2020). To meet the trend of AI: The ecology of developing ai talents for pre-service teachers in China. International Journal of Learning, 6 (3), 186–190. https://doi.org/10.18178/IJLT.6.3.186-190

Xiao, W., & Song, T. (2021). Current situation of artificial intelligence education in primary and secondary schools in China. In  The Sixth International Conference on Information Management and Technology. (pp. 1–4). https://doi.org/10.1145/3465631.3465980

Yau, K. W., Chai, C. S., Chiu, T. K., Meng, H., King, I., Wong, S. W. H., & Yam, Y. (2022). Co-designing artificial intelligence curriculum for secondary schools: A grounded theory of teachers' experience. In 2022 International Symposium on Educational Technology (ISET). (pp. 58–62). https://doi.org/10.1109/ISET55194.2022.00020

Yue, M., Dai, Y., Siu-Yung, M., & Chai, C.-S. (2021). An analysis of k-12 artificial intelligence curricula in eight countries. In  Proceedings of the 29th International Conference on Computers in Education.  (pp. 22–26).

Yue, M., Jong, M. S. Y., & Dai, Y. (2022). Pedagogical design of K-12 artificial intelligence education: A systematic review. Sustainability, 14 (23), 15620. https://doi.org/10.3390/su142315620

Zhai, X., Chu, X., Chai, C. S., Jong, M. S. Y., Istenic, A., Spector, M., & Li, Y. (2021). A review of artificial intelligence (AI) in education from 2010 to 2020. Complexity . https://doi.org/10.1155/2021/8812542

Zhang, N., Biswas, G., McElhaney, K.W., Basu, S., McBride, E., & Chiu, J.L. (2020). Studying the interactions between science, engineering, and computational thinking in a learning-by-modeling environment. In  International conference on artificial intelligence in education. (pp. 598–609). Springer.

Download references

Acknowledgements

Authors would like to thank the reviewers and editors, whose comments and feedback helped us to improve the original manuscript.

This work has partially been funded by the Spanish Ministry of Science, Innovation and Universities (PID2021-123152OB-C21), and the Consellería de Educación, Universidade e Formación Profesional (accreditation 2019–2022 ED431C2022/19 and reference competitive group, ED431G2019/04) and the European Regional Development Fund (ERDF), which acknowledges the CiTIUS—Centro Singular de Investigación en Tecnoloxías Intelixentes da Universidade de Santiago de Compostela as a Research Center of the Galician University System. This work also received support from the Educational Knowledge Transfer (EKT), the Erasmus + project (reference number 612414-EPP-1-2019-1-ES-EPPKA2-KA) and the Knowledge Alliances call (Call EAC/A03/2018).

Author information

Authors and affiliations.

Pedagogy and Didactics Department, University of Santiago de Compostela, 15782, Santiago de Compostela, Spain

Lorena Casal-Otero, Carmen Fernández-Morante & Beatriz Cebreiro

Departamento de Electrónica e Computación, Universidade de Santiago de Compostela, 15782, Santiago de Compostela, Spain

Alejandro Catala & Maria Taboada

Centro Singular de Investigación en Tecnoloxías Intelixentes (CiTIUS), Universidade de Santiago de Compostela, 15782, Santiago de Compostela, Spain

Alejandro Catala & Senén Barro

You can also search for this author in PubMed   Google Scholar

Contributions

All authors have contributed significantly to the authorship of this work in all stages of conceptualization, discussions, definition of the methodology, carrying out the analysis as well as writing—review & editing. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Alejandro Catala .

Ethics declarations

Ethics approval and consent to participate.

This research is carried out in accordance to ethics recommendations. As it focuses on a systematic literature review as a research method, ethics approval by the University ethics committee does not apply.

Competing interests

The authors declare that they have no competing interests. The authors have no other relevant financial or non-financial interests to disclose and no further competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Additional listing.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Casal-Otero, L., Catala, A., Fernández-Morante, C. et al. AI literacy in K-12: a systematic literature review. IJ STEM Ed 10 , 29 (2023). https://doi.org/10.1186/s40594-023-00418-7

Download citation

Received : 06 September 2022

Accepted : 30 March 2023

Published : 19 April 2023

DOI : https://doi.org/10.1186/s40594-023-00418-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Secondary education
• Teaching/learning strategies
• Twenty-first century skills
• Cultural and social implications

Artificial Intelligence in Education: A Review

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 25 October 2024

Education in the AI era: a long-term classroom technology based on intelligent robotics

• Francisco Bellas   ORCID: orcid.org/0000-0001-6043-1468 1 ,
• Martin Naya-Varela 1 ,
• Alma Mallo 1 &
• Alejandro Paz-Lopez 1  

Humanities and Social Sciences Communications volume  11 , Article number:  1425 ( 2024 ) Cite this article

98 Accesses

2 Altmetric

Metrics details

• Science, technology and society

Artificial Intelligence (AI) will have a major social impact in the coming years, affecting today’s professions and our daily routines. In the short-term, education is one of the most impacted areas. The autonomous decision making that can be achieved with tools based on AI implies that some of the traditional methodologies associated with the fundamentals of the learning process in students, must be reviewed. Consequently, the role of teachers in the classroom may change, as they will have to deal with such AI tools performing parts of their work, and with students making a common use of them. In this scope, the AI in Education (AIEd) community agrees on the key relevance of developing AI literacies to train teachers and students of all educational levels in the fundamentals of this new technological discipline, so they can understand how these tools based on AI work and pilot the adaptation in an informed way. This implies teaching students about the fundamentals of topics like perception, representation, reasoning, learning, and the impact of AI, with the aim of delivering a solid formation in this area. To support them, formal teaching and learning resources must be developed and tested with students, properly adapted to different educational levels. The main contribution of this proposal lies in the presentation of the Robobo Project, a technological tool based on intelligent robotics that supports such formal AI literacy training for a wide range of ages, from secondary school to higher education. The core part of this paper is focused on showing the possibilities the Robobo Project offers to teachers in a simple way, and how it can be adapted to different levels and skills, leading to a long-term educational proposal. Validation results that support the feasibility of this technology in the education about AI, obtained with students and teachers in different educational levels during a period of six years, are presented and discussed.

Similar content being viewed by others

Development of an artificial intelligence curriculum design for children in Taiwan and its impact on learning outcomes

Exploring the impact of artificial intelligence on higher education: The dynamics of ethical, social, and educational implications

RETRACTED ARTICLE: The impact of augmented reality on student attitudes, motivation, and learning achievements—a meta-analysis (2016–2023)

Introduction.

The Artificial Intelligence in education (hereafter referred to as AIEd) community agrees on the key relevance of developing an AI literacy with the aim of training teachers and students, of all educational levels, in the fundamentals of this discipline, so they can face in an informed fashion the new technological revolution that is up to come (Global Education Monitoring Report Team 2023 ; UNESCO 2023 ).

We can find many initiatives focused on the development of such AI literacy, mainly from secondary school to higher education (Ng et al. 2021 ). In the case of higher education, two main situations can be distinguished (Laupichler et al. 2022 ; Brew et al. 2023 ; Hornberger et al. 2023 ). First, on educational programmes focused on training “AI engineers”, like computer science and similar ones, AI has been part of the syllabus for years and the core topics are clearly established in classical textbooks (Russell and Norvig 2021 ). Second, addressing university programmes in many other areas, like natural sciences or humanities, there is not a proper AI literacy as such.

The most advanced contributions for AI literacy can be found in secondary school and high school (K12 to K16) (Ng et al. 2021 ). A recent report from UNESCO has reviewed the main official K12 AI curricula developments of different countries (UNESCO 2022 ), concluding that all of them are based on similar topics, basically those included on the guidelines established in the AI4K12 USA’s initiative (AI4K12 2023 ). Based on them, AI literacy is structured around 5 big ideas: perception, natural interaction, representation & reasoning, learning, and the impact of AI. These ideas revolve around the “intelligent agent” perspective of AI, understood as computational systems situated in an environment (real or virtual) where they interact and operate in an autonomous fashion (Russell and Norvig 2021 ; Dignum et al. 2021 ). This approach implies training students, not in trending topics or specific tools like generative AI, but rather on the fundamentals of AI contained in these five ideas, from which all the AI technologies arise. Although generated in the frame of K12 education, this is the perspective towards AI literacy that should be adopted at all levels.

The goal now is how to transfer the AI literacies to the real educational centres, a key point that still has no answer. Some critical questions emerge when addressing this problem: (1) How teaching AI can fit into current education plans? (2) How can educators face teaching in this new discipline? and (3) Which educational resources can be used at different levels?

Policy makers worldwide and global organizations as UNESCO are actively working on these issues, but the heterogeneity of the educational systems and the wide spectrum of target students makes it complicated to design a common strategy. In the meantime, researchers and educators in the AIEd field have been very active on the two last questions, with the development and testing of specific classroom resources for the teacher that can fit into the five topics presented above. In this realm, machine learning is clearly the topic where the most remarkable contributions have been done, especially in the case of supervised learning (AI at code.org 2023 ; Machine Learning for Kids 2023 ; MIT 2024 ). Other initiatives like Elements of AI (The University of Helsinki 2024 ), ISTE (ISTE 2023 ) or MIT Raise (Lee et al. 2021 ) have followed a more general perspective, with resources also for the topics of representation, reasoning or the impact of AI.

However, in key topics as perception, natural interaction, or reinforcement learning (a very relevant approach in machine learning), the number of formal contributions is lower. The main reason behind this is motivated by the nature of these topics, highly coupled with the online interaction of the agent with the environment. Therefore, to properly learn about them, it is required to handle access to data in real time, dealing with uncertainty in many cases, which increases the complexity on the implementation of the teaching resources.

To face this problem, robots have shown to be a feasible solution. They clearly frame the concept of intelligent agent because they are situated and operate in the real world. Robots are continuously getting data and executing actions conditioned by the uncertainty of the environment. Moreover, the human-robot interaction (HRI) is a key field in robotics, so all the topics related to natural interaction fit also naturally. Finally, robotics is a great example of AI application field, and students clearly perceive the relevance of having autonomous robots in their lives, being interested about understanding how they can operate in such a way (Xu and Ouyang 2022 ). Of course, the use of robots in education is not new, but what is relevant in the scope of this paper is to realize to what extent they have been used in the realm of education about AI, and for what reasons.

Literature review

Thanks to the technological development and cheaper technology, robots have been extending their field of application beyond technical university levels, covering general education too. This has brought the emergence of the Educational Robotics (ER) field with its own entity, pursuing the goal of developing robots and associated resources to serve as educational tools at different grades. The characteristics of the ER, such as the application of digital technologies for problem real solving, its suitability for STEM (Science, Technology, Engineering, Mathematics) education and Project Based Learning (PBL) methodologies, or the existence of many student competitions, make this field optimal for fostering the development of the 21st century skills (Ananiadou and Claro 2009 ), especially the learning ones, called the “4Cs”: Critical thinking, Communication, Collaboration, and Creativity (Kivunja, 2015 ).

After a deep review of the literature in the scope of ER, some remarkable initiatives and platforms can be highlighted (Bezerra Junior et al. 2018 ; Anwar et al. 2019 ; Darmawansah et al. 2023 ). In kindergarten and first levels of primary school, the robots that have been used are very simple, usually with a friendly form similar to an animal, to be familiar to children and favour the HRI, like the Bee robot (Castro et al. 2018 ; Seckel et al. 2023 ) or Blue-bot (Aranda et al. 2019 ). For the highest levels of primary school and the lowest of secondary school, the KIBO Bot (Sullivan and Bers 2019 ) and the MBot robot (Sáez–López et al. 2019 ) are two representative examples of platforms, being the LEGO MINDSTORMS kit (LMkit) the most used robotic tool worldwide. Moving to high school and university levels we can find the iRobot Create (Gomoll et al. 2017 ), the NAO (Konijn and Hoorn 2020 ), the Turteblot (Amsters and Slaets 2020 ; Weiss et al. 2023 ), the Kephera IV (Farias et al. 2019 ), or different arm robots.

All the previous robots were not designed for teaching AI, so they lack of basic features required in this scope to deal with the five ideas commented above (Naya–Varela et al. 2023 ): (1) In terms of sensors, they should allow natural interaction with humans and the environment, which means including camera, microphone, and tactile sensing; (2) Support a wide range of actuations, like locomotion, manipulation, speech production (speaker) or visual communication (LCD screens); (3) Support the execution of complex algorithms, like those related to computer vision, reasoning, or machine learning; (4) Being equipped with a wide spectrum of communication technologies, like WiFi, Bluetooth, 5G or similar, and support internet connection; (5) Support different programming languages adapted to the educational levels, which allow to integrate external AI libraries or functionalities; (6) Have a simulation model, so students can work in a controlled and simple fashion before moving to the real robot, reducing also the number of physical platforms at schools and their required investment; (7) Including adapted teaching materials to face the five main AI topics, that teachers can use directly.

If we move to ER aimed for AI education (Chen et al. 2022 ; Chu et al. 2022 ), which should include the previous features, two main platforms must be highlighted. The Thymio, (Fig. 1 left) is a robotic platform that began as a tool to teach robotics to children between the ages of 6 and 12 (Mondada et al. 2017 ), and over time, has also become applicable at the university, being even used in research articles by professors and graduate students (Heinerman et al. 2015 ). It is a mobile robotic platform with two wheels and no camera, designed to be compatible with LEGO bricks. Over time new functionalities have been added to the robot, such as a Block Based Programming Language (BBPL) to control the robot (Shin et al. 2014 ), wireless communication to operate the robot in real time (Rétornaz et al. 2013 ) and a tangible programming language for the youngest students (Mussati et al. 2019 ). In addition, it offers a wide range of didactic materials, classified for languages and age range, oriented to all educational levels, but mainly focused on K12 (Thymio robot 2024 ).

Image of Thymio (left) and Fable (right) educational robots.

Fable (Fig. 1 right) is a modular robot targeted to teach robotics to children from 6 years of age (Pacheco et al. 2013 , 2015 ). It has various modules that can be joined together to form different types of robots. It allows a Learning by Doing methodology so that the student can select the morphology that best suits the challenge to be overcome. It can be programmed using BBPL, Python or Java. In addition, it has on its website teaching content grouped by concepts with various lessons (Pacheco 2024a ). Finally, this initiative has the “STEAM Lab”, where a set of combined tools for fully integrated STEM teaching, such as Fable robot modules, virtual reality glasses, 3D printers, etc. are provided (Pacheco 2024b ).

These two ER initiatives encompass most of the features commented above, but not all, as it will be discussed in section “Analysis and discussion”. As a conclusion, although the ER field has been very active in the last years for a wide spectrum of educational levels, none of the existing platforms support a clear roadmap towards AI literacy that can help teachers and educators from the AIEd community in a feasible fashion.

Methodology

Research contribution.

The current paper aims to fill the gap exposed in the previous section by proposing a technological didactical tool, called Robobo, developed within the scope of the Robobo Project (The Robobo Project 2024 ), to train AI topics at different education levels, from K12 to higher education, and aligned with the last recommendations about AI literacy. This tool is grounded on the idea of using robots for learning about AI as they perfectly represent the concept of intelligent agent, so they can be used for teaching all the core topics in this discipline. The Robobo robot contains all the hardware and software features to support such goal, and also a range of materials that can be adapted to different educational levels and skills, leading to a long-term educational proposal.

To simplify the target of this paper, the application of AI techniques to robots will be named here as Intelligent Robotics , and the following sections will be devoted with the presentation and analysis of the Robobo Project within this scope.

Theoretical basis

The intelligent agent perspective of ai.

The intelligent agent perspective of AI is typically illustrated through a cycle like the one shown in Fig. 2 (green, blue, pink, and orange blocks) (Russell and Norvig 2021 ). It is based on four main elements: the environment , real or simulated, where the agent “lives” and interacts with humans; the sensing stage where it perceives information from the environment; the acting stage where it executes actions on the environment, and the agent itself, which encompasses different internal elements and processes that allow it to reach goals in an autonomous fashion. In a more general setup, this agent can be situated in an environment with other intelligent agents or even people, and it can communicate with them, so a multiagent system arises (as displayed in Fig. 2 with three agent cycles). In this scope, robots can be naturally perceived as intelligent agents, embodied, and situated in real or simulated bodies and environments (Murphy 2019 ). In addition, multirobot systems can also be easily presented to students.

The multiagent system interaction cycle.

Taking this scheme as the background approach to the type of AI perspective we aim to teach to students in the Robobo Project, the following fundamental topics have been considered: (1) Sensing, (2) Acting, (3) Representation, (4) Reasoning, (5) Learning, (6) Collective AI and (7) Ethical and legal aspects. Most of these topics are those of the AI4K12 initiative (AI4K12 2023 ), but some modifications have been done to adapt them to ER, and a new one has been included: collective AI.

Sensing the environment

Artificial agents obtain data from their sensors and from direct communication channels (see Section “Collective AI”). For the first case, students must learn the difference between sensing (information provided by the sensor) and perception (information in context), and how the properties of the real environments imply some kind of uncertainty in the perception. This is a key concept for real AI systems, as it imposes one of the classical limits of symbolic approaches (section “Introduction”). In the realm of AI literacy, the fundamentals of cameras, microphones, distance sensors and tactile screens must be covered.

Agent actions

Students must be aware of the main types of actuators and effectors that AI systems may have, such as motors, speakers, or screens, and how they are found in smartphones, autonomous cars, or videogames. The intelligent agent selects internally the actions to be applied in order to fulfil its tasks through a series of complex processes that will be later explained. Consequently, it is necessary to understand the types of actions that can be executed and their effects on the environment.

Inside the agent itself, many processes are executed, but following the five ideas, we could reduce them to three: representation, reasoning, and learning.

Knowledge representation

How digital data is stored and what is represents in the computational system that constitutes the core of the agent must be addressed. It is a new topic for most of students who do not belong to the computer science field. It encompasses one of the main issues in AI, how to represent knowledge, and the differentiation between symbolic and sub symbolic approaches. This topic encompasses learning about the representation of images, sounds, speech, and other structures like graphs or trees that make up core aspects in AI.

Reasoning and decision making

Reasoning comprises all the methods involved in action selection and decision making. It is related with computational thinking and problem solving, although it includes specific algorithms and approaches to achieve an autonomous response. Considering the objective that the agent must achieve (the task to solve), it must reason how to fulfil it in an autonomous way using the available sensorial information and the established representation. Function optimization, graph search, probabilistic reasoning or reasoning based on rules are examples of topics to be learned in this scope.

Machine learning

This is the most popular topic in AIEd, as machine learning algorithms (ML) have evolved and improved notably in the last decades. They allow intelligent agents to create models from data which can be used for decision making and prediction. There are three main types of methods in this realm that students must understand: supervised, unsupervised and reinforcement learning. Learning about these methods must be faced in a general way, because there are many new algorithms constantly arising, and it is not possible to cover them all. Thus, AI literacy should focus on general and global processes such as data capture, data preparation, algorithm selection and parameterization, training stage, testing stage, and results analysis.

ER is especially suitable for reinforcement learning teaching because the algorithms behind this approach are designed to operate in systems based on agents, interacting with the environment in a continuous cycle of state, action, and reward. This is a very relevant property of ER, because it is not simple to find practical application cases where to train reinforcement learning with students, mainly in lower educational levels.

Collective AI

The general perspective of AI system illustrated in Fig. 2 implies that individual agents are situated in an environment where they interact with other agents and people. It is important for students to understand such AI ecosystem, because in the future, we will live surrounded by intelligent agents (smart houses, buildings, or cities), performing a natural interaction with them, so the fundamentals of these global systems like communication modalities, cloud computing, cybersecurity, interface design or biometric data must be addressed in AI literacy.

Ethical and legal aspects

The ethical and legal aspects behind AI are still under study and development, but it is important to enhance a critical view of students about AI technologies that is supported by their own knowledge of AI principles, so they can have an independent and solid opinion of the autonomous decision making of these systems. Many questions arise about the type of activities that should be faced with AI or restricted, the benefits and drawbacks of the commercial use of this technology, and the central role that humans should play in this scope (Huang et al. 2023 ).

All the previous topics are relevant in the scope of AI, with enough impact to be considered as independent research areas. Depending on the educational level, it is necessary to cover them with more or less technical detail, so the bibliography that teachers must use will be different. A general recommendation is to follow classical textbooks on AI, like (Russell and Norvig 2021 ) or (Poole and Mackworth 2010 ), as the starting point to learn the fundamentals of these seven topics at all levels. Specific textbooks, adapted to pre university ones, are still under development, while specific ones for the highest levels are very common.

To sum up, within the Robobo Project, the previous seven topics have been considered to frame the AI literacy. Obviously, their particular contents have been adapted to the different education levels, as it will be explained later.

Methods and materials

To reach the research goal established in Section “Research contribution”, and considering the theoretical frame explained in Section “Theoretical basis”, the research methodology applied has been a combination of educational and engineering methods. The materials developed in this work were created in the scope of the Robobo Project, a long-term educational and technological initiative, composed of three main elements: (1) a mobile robot, (2) a set of libraries and programming frameworks, and (3) specific documentation plus tutorials to support teachers and students in AIEd.

To develop the first one, mainly hardware, a classical engineering cycle of conceptualization, design, prototyping, testing, and refining was applied during the first year of the project in 2016 (Bellas et al. 2017 ). The validation was carried out in controlled sessions with students until reliable hardware functioning was achieved. To develop the software components, the Unified Process for software development was applied. This methodology proposes the development of an iterative and incremental scheme, in which the different iterations focus on relevant aspects of the Robobo software. In each iteration, the classic phases of analysis, design, implementation, and testing were carried out, so that by the end of the process, weaknesses and performance issues that need to be addressed in the next iteration were detected. Each iteration incorporated more functionalities than the previous one until the last iteration ended with the Robobo software implementing all the functionalities (Esquivel–Barboza et al. 2020 ). The software development was mainly carried out between 2016 and 2019, although improvements have continued to be incorporated up to the present day. Finally, for the educational materials, we have followed a typical educational design research methodology. Specific teaching units and classroom activities were created and evaluated by studying the learning outcomes they provided to the students, as well as the acceptance level of teachers, in an iterative cycle of testing and improvement in real sessions carried out in formal an informal education from 2017 to 2023. To this end, quantitative and qualitative analyses were performed using surveys and direct teacher feedback, leading to a set of materials that have been validated in several cycles (Bellas et al. 2022 ).

Focusing now on the explanation of the main materials used in this work, it must be pointed out that the key element of the Robobo Project is the smartphone that is used in the real platform, which provides most of the features established in Section “Theoretical basis” to support AIEd. The robotic base has a holder on its top (Fig. 3 ) to attach a standard smartphone. Communication between the smartphone and the robotic base is handled via Bluetooth. Students can use their own smartphone, which reduces the investment cost of schools and promotes a positive use of this type of device at schools, as recommended by the UNESCO (Global Education Monitoring Report Team 2023 ) Footnote 1 .

Black typeface represents those of the smartphone and blue ones those of the platform. Left: General view of the front of Robobo. Top right: Detailed view of the pan-tilt unit. Bottom right: Representation of the back of Robobo, with LEDs and infrared sensors.

Robobo: the real robot

The electronic and mechanical design of the mobile physical platform is detailed in (Bellas et al. 2017 ). The core concept behind the Robobo robot consists of combining the simple sensors and actuators of the platform with technologically advanced sensors, actuators, computational power, and connectivity the smartphone provides. From a functional perspective, the different components of the Robobo hardware (displayed in Fig. 3 ) can be classified into five categories: sensing, actuation, control, communications, and body.

Robobo sensors:

On the base :

Infrared (IR): It has 8 IR sensors, 5 in the front and 3 in the back, used for distance detection, as well as for avoiding collisions or falls from high surfaces.

Encoders: They provide the position in which the motor shaft is located. They can be utilized for odometry, to correct the trajectory of the movement, or just to realize if the robot is really moving.

Battery level: To know the degree of autonomy of the robot, useful for long-term experiments or for challenges related with energy consumption.

On the smartphone :

The characteristics of the smartphone’s sensors change depending on the model, but these are the most common:

Camera: Perhaps the most relevant Robobo sensor for teaching AIEd, due to the number of applications it supports, such as colour detection, object identification, face detection, among others.

Microphone: It allows to detect specific sounds, ambient noise, and of course, the user voice for speech recognition.

Tactile screen: This sensor allows to detect different types of touches, like tap or fling, which are typically carried out by users to allow natural HRI.

Illumination: This sensor provides the ambient light level, which is. useful in different applications to adapt the robot response according to it, or for energy saving purposes.

Gyroscope: It allows to identify the orientation of Robobo in the space. It can be used for navigation, map tracking, inclinations, and changes in the slopes.

Accelerometer: It calculates Robobo’s acceleration, identifying the real movement of Robobo even when no actuation is performed by the hardware.

GPS: It provides the global positioning of the robot using this popular technology, although it just works outdoors.

Robobo actuators

On the base:

Motors: Perhaps, they are the most relevant Robobo actuators. Two motors are attached to the wheels for navigation purpose. Other two motors are in the pan-tilt unit, enabling the horizontal and vertical rotation of the smartphone, which provides Robobo with wider range of possible movements. This is normally understood by students as “head” movements, increasing the personality of the robot expressions.

LEDs: They are utilized to transmit simple information to the user in a visual way. For instance, a warning condition when the battery is low, or a different colour depending on the distance to the walls.

On the smartphone:

The characteristics of the smartphone actuation change with the model, but again, we can find some common ones:

Speaker: It can be used to play sounds or produce speech, which is fundamental in natural interaction.

Torch: It is an adjustable light which is useful in many cases, for instance, to increase the illumination of a scenario to improve the camera response.

LCD screen: It is very useful for displaying visual information. Usually, the screen shows Robobo’s “face”, which can be changed to display different emotions.

Robobo control unit

Robobo’s control unit is the smartphone. It runs all the processes related with receiving information from the base and sending commands to the actuators through Bluetooth. In addition, it receives and sends commands to an external computer, as detailed in Section “Software and development tools”. Finally, it runs some algorithms onboard, related with image and sound processing. The computational power of the smartphone models can be very different, but most of the existing models have processors with more capability than required in most of the educational challenges, as it has been tested with students.

Robobo communication system

Current smartphones are equipped with WiFi, 5 G and Bluetooth connections. The first one is the most relevant in the educational scope, as it allows to connect the robot to the internet through the schools’ network, which is very common nowadays. As a consequence, students can use information taken from internet sources on their programs, as weather forecast, news, music, and they can also carry out direct communication by sending/receiving messages or emails.

Robobo body extensions

Finally, it must be highlighted that the Robobo base has a series of holes in its lower part to attach different types of 3D printed accessories (Fig. 4 ). Only the holes are provided to serve as structural support for the accessories, leaving the design completely free to the users, opening the possibility of multiple solutions to the challenges proposed to students while learning AIEd concepts under a STEM methodology.

Top middle: Example of 3D printed accessories. Top right and bottom: Different applications that can be performed with Robobo and the accessories, such as pushing, drawing, or even developing and outdoor version with bigger wheels.

Software and development tools

The Robobo software includes an entire ecosystem of applications, developer/user libraries, and simulators that allow easy adaptation to different learning objectives (Esquivel–Barboza et al. 2020 ). The software has been designed following a modular architecture that facilitates the addition of new capabilities in the future, as well as the configuration of which of these capabilities are available in a particular learning context. Therefore, it provides the technological foundations and functionalities that make Robobo an adaptable learning tool for different levels which is also in continuous evolution.

The core software runs on the smartphone and provides all the intrinsic sensing, actuation, and control capabilities of the robot. It also provides standard programming interfaces for local or remote access to these capabilities. Having the core of the software of the robot running on a regular smartphone allows to upgrade the hardware (the smartphone) of the robot in an almost unlimited way, leading to a long-term investment for educational institutions.

Figure 5 shows the software architecture from the perspective of a user of the Robobo platform. In this context, a user is a student who uses the robot to solve AI tasks by means of programming. However, teachers could also be considered users, who focus on designing tasks for the students (including the necessary teaching units and physical or simulated environments). As can be seen in the figure, depending on the educational level, a student can use different software libraries and programming languages to develop the challenges with Robobo. For working in the real environment, it is required to have the programming computer connected to the same local network as the robot (no cables are required). At any given time, the students can choose to run their program in the simulated environment or in the real robot. Moreover, the robot is entirely functional in a context where there is no Internet connection available (restricted local network), since even functionalities such as voice recognition have an implementation suitable for offline operation. Finally, it should also be noted that the problems to be solved usually involve a single robot interacting with its environment but several robots collaborating to solve a common task are also supported.

Representation of the software architecture of the Robobo platform from the user’s point of view.

What makes Robobo a suitable platform for long-term AIEd education are the functionalities it provides and how they have been adapted to different skills and educational levels, together with the set of teaching units designed to exploit them. A high level of semantic and conceptual homogeneity is always maintained in order to facilitate a progressive learning experience. Thus, at K12 levels, we propose the use of a programming model based on blocks, supported by Scratch, with a limited set of available functionalities that nevertheless allow experimenting with a multitude of AI topics. In the case of K16 and higher education levels, the use of the Python programming language is proposed along with a growing set of sensing, actuation and control functionalities. At this point it is important to mention that the use of Python is justified by the fact that it is currently the language of preference for data scientists and artificial intelligence developers (TIOBE Index 2024 ) and by the abundance of freely available AI libraries and tools that can be used together with the software provided by the Robobo.

Table 1 shows a simplified view of the main functionalities provided by the Robobo software platform, classified into three categories: perception (including self perception and sensing of the environment), actuation and control. As can be seen, even at the lowest level, the robotic platform provides a range of capabilities that are not usually available in other robots commonly used for educational purposes. Thus, from an early learning stage, students can explore fundamental AI concepts by creating adaptive behaviours supported by the variety of possibilities to sense the environment (through visual, acoustic, or tactile inputs) while interacting and even expressing internal emotional states (using speech, sounds or facial expressions).

In the Scratch 3 environment, an extensive set of new blocks that allow the student to work with all available Robobo functionalities, have been defined 1 . In Fig. 6 , it is displayed a subset of programming blocks that provide access to some of the sensing capabilities, as well as a simple program that defines the speed of the robot in response to its environment (namely depending on the distance at which a green object is detected using the camera).

Left: a subset of the new programming blocks we have designed for Robobo. Right: a simple program that uses some of these new blocks together with the generic Scratch blocks.

Regarding Python language, Robobo contains advanced capabilities that allow teachers to imagine an almost unlimited number of tasks to solve in different scenarios Footnote 2 . For example, they can propose challenges in the scope of computer vision, autonomous driving, or natural interaction, which can be faced in a simple fashion. Furthermore, it is always possible for students to combine these intrinsic capabilities of Robobo with other AI technologies and tools that are common in the field and are available through external libraries.

To briefly exemplify the way of programming using the Python interface, a code fragment is included in Fig. 7 . First of all, it is necessary to import the classes that provide access to the functionalities we want to use. Likewise, if desired, any external library commonly used in the field of AI, such as TensorFlow, OpenCV, Scikit-learn, etc., can be imported. Then remote communication with the Robobo robot (in a real or simulated environment) is established. At this point it is possible to start programming the behaviour of the robot using the functions available in our Robobo.py library. It is important to clarify that if we want to change the execution of our program between the real robot and the simulator, it is only necessary to modify the line of code that establishes communication with the robot (Fig. 7 ).

Simple Python code illustrating how to program a controller for Robobo, that can be executed both using a real robot or a robot in a simulated environment.

As shown in Fig. 5 , a dedicated simulator, called RoboboSim, was developed adjusted to the necessities of schools and students Footnote 3 . It was built on top of the Unity engine, and it was designed to be easy to use, requiring minimal training. The students and teachers only need to download the application, choose one of the available virtual environments (worlds) and start a remote connection with the simulated robot, in the same way as with a real robot. The user interface requires minimal configuration, and it was designed following a video game like design aimed to be user friendly for young students. The students can solve different challenges on the available worlds using Scracth3 or Python, since both are supported by RoboboSim.

RoboboSim can be configured in two levels of realism. (1) The standard one includes basic physical modelling of the robot (weight, friction, motors). The real robot response was empirically characterized, and different models were created. Therefore, in this realism level, RoboboSim is reasonably faithful to the real robot model, and the programs developed in this level do not require much adjustment to run in the real platform. (2) The simplified realism level does not perform physical simulation, and the response is deterministic. It is recommended for those students who are starting with robotic simulations, or for those who prefer focusing on programming concepts, facilitating the accomplishment of the challenges proposed.

A key property of the RoboboSim in the realm of autonomous robotics is that it includes an optional random mode of operation. This mode implies that particular objects in the simulation appear in slightly different positions each time the environment is restarted. The purpose of this property is to force students to develop more robust programs, leading to autonomous behaviours that can adapt to changes in the environment.

Educational resources

Two main types of educational resources have been developed and tested in the last years in the scope of the Robobo Project. First, Teaching Units (TU), which include specific activities, guides, and solutions to learn about different AI topics. Second, documentation, which allows students and teachers to use the robot in a more independent fashion, adapting it to their particular needs and exploring new possibilities.

Teaching units and lessons

In the case of TU, the Robobo Project has been mainly applied to formal education. Consequently, these materials have been designed to be aligned with the latest AI literacies introduced in Section “Introduction” and including the AI topics explained in Section “Theoretical basis”, although they can be used as independent lessons in informal education, like extracurricular activities, workshops, specific AI courses or summer courses. The main target for these units has been the teachers, who are the main actors in the real introduction of AI in education.

For secondary school and high school (from 14 to 18 years old), 7 TUs based on Robobo have been developed in the scope of the AI+ educational project (Bellas et al. 2022 ), specifically focused on intelligent robotics. In higher education, more specialized teaching units have been developed for Vocational Education and Training (VET) through the AIM@VET project (Renda et al. 2024 ), and others for the different courses on intelligent robotics belonging to the University of A Coruña (Llamas et al. 2020 ). Table 2 includes a set of representative TU. It aims to provide an overall view of the type of activity that can be implemented at classes with this robot. They have been organized in a sort of incremental complexity level, although they could be adapted to increase or decrease it.

Documentation and tutorials

The Robobo Project includes a solid documentation adapted to different educational levels, including manuals, reference guides and programming examples. Table 3 shows a description of these support materials, and the links to them.

All the documentation is open access, and available at the Robobo Wiki Footnote 4 . New users should start by downloading and configuring the RoboboSim. It is recommended to try the first programs using the scratch framework, for instance, by trying the sample projects available Footnote 5 . A next step would be to try them in the real robot, which requires following the documentation about the Android app and the platform initial configuration. Finally, for those students of higher levels, the Python documentation should be reviewed. It includes a complete library reference, as well as specific documents for the most advanced features, like object recognition, lane detection, or video streaming.

Teaching methodology

The methodology used in all the TUs of the Robobo Project is based on Project Based Learning (PBL), as it best fits the STEM approach in which educational robotics have shown clear advantages. Each TU presents a challenge that students must solve with the robot, organized in teams or individually, depending on the specific learning objectives and the teacher’s criterion. The challenge faces a real problem, which must be solved in a real or simulated environment, as established by the teacher depending on the learning focus (dealing with the real robot implies using more time dealing with technical issues). To apply the PBL methodology in the Robobo TUs, five typical phases of an engineering project are carried out by students:

Problem analysis and requirement capture : understanding the problem to solve and its relevance.

Organization and planning : dividing the whole problem into subproblems.

Solution design : programming the solution.

Solution validation : testing the solution in the robot.

Presentation of results and documentation : showing the final response, submitting the solution, and answering whatever question from the teacher.

The teacher’s role in each of these steps is very relevant, as he/she must monitor the students’ advance and solve their questions, which can be open as this type of project allows for different valid solutions. The teacher must evaluate the final solution of the student, which can be based on the robot’s performance, on the code, on the progress and attitude, or on a specific exam or test. In terms of the specific background on AI topics, teachers should have previous training on them according to the educational level, which is out of the scope of the Robobo Project. In the case of secondary school teachers, to support them in this new discipline, the TUs developed in the realm of the AI+ project make up a teacher guide, with a recommendation about the theoretical contents to be taught, a possible organization of the TU into activities and tasks, and all the code solutions to the challenge. As an example, Fig. 8 displays the organization of a TU focused on natural interaction with Robobo that is included in the teacher guide. It can be observed how the division of the final goal into activities and tasks is provided, including an initial stage for theoretical contents.

An example of TU organization that is provided to teachers in a project based on Robobo.

Experimental results. Robobo use cases with students

The Robobo Project has been validated in several training activities since 2017 in different European countries, from secondary school to master’s degree, both in informal (workshops) and formal education (official curricula). As commented above, in secondary school and high school, it has been used in the format of short sessions in the realm of the AI+ project. At University level, in Spain, it was applied in the Industrial Engineering degree and master at University of A Coruña, from 2019/20 academic year onwards, in general subjects as “Informatics” and other more specific as “Intelligent Robotics” or “Mobile Robotics”. In Nederland, the VU University of Amsterdam has also been using Robobo in the “Learning Machines” course of the Faculty of Sciences (Miras 2024 ) since 2018/19. Apart from official subjects, it has been applied in more than ten final degree and master projects at the University of Coruña (Spain) since 2018 to 2022, focused on advanced features like computer vision using Deep Learning (Esquivel–Barboza et al. 2020 ) or multirobot coordination using machine learning (Llamas et al. 2020 ).

The following three sections will describe the results obtained on specific use cases for (1) secondary school, (2) initial level of higher education, and (3) master education level. In the three use cases, the validation instruments of the implementation were the same: online questionnaires filled by students at the completion of the task or lesson. Through this validation stage, two research questions are faced:

RQ1: Is the Robobo Project a suitable tool for learning the fundamentals of AI in the long-term?

To know if the tool is perceived as another educational robot for learning about technology or as a more formal and powerful resource.

To know if the project is perceived as an educational framework that could be used in several courses throughout the academic life.

RQ2: Is the methodological approach and the materials adequate to learn intelligent robotics?

To know if the teaching resources are adequate to each education level.

To know if the use of the student’s smartphone for didactic could be problematic.

To know if the teaching approach based on simulated/real robot is well perceived by students.

Regarding the general profile of the students who participated in this study, at all three levels of education we had students from different countries, mainly from Europe and Latin America (Fig. 9 ). This introduced more variability in students’ background on programming, digital education, or AI topics. Furthermore, Table 4 shows the ratio between female and male students along with their age ranges. As it can be observed, there is a majority of male students, in line with the usual statistics in the fields of computer sciences and engineering, within these geographic areas.

Students’ country of origin.

Secondary school

This use case summarizes the overall impression of teachers and students about Robobo in the scope of the AI+ project (AI+ ( 2019 )). The goal was training students on the seven fundamental topics of AI:

Perception and actuation in AI, corresponding to TU7, TU8, and TU10 of the AI+ curriculum.

Natural interaction , corresponding to TU9 of the AI+ curriculum.

Reinforcement learning , corresponding to TU13 of the AI+ curriculum.

Representation (computer vision) , corresponding to TU12 of the AI+ curriculum.

Reasoning (path planning) , corresponding to TU14 of the AI+ curriculum.

Collective AI (multiagent system) , corresponding to TU17 of the AI+ curriculum.

Social Impact (AGI and cognitive robotics) , corresponding to TU15 of the AI+ curriculum.

The specific questionnaires related to RQ1 and RQ2 were filled in the final training activity of the AI+ project, held in Slovenia in May 2022, in which participated 30 students and 12 teachers. Starting with RQ1, representative results are displayed in the two graphs of Fig. 10 . The left one was answered only by teachers, and it shows how they rate the importance of the different topics covered in the AI+ curriculum by their relevance for the students’ training. All these topics were covered with the TUs commented above.

Teachers (left) and secondary school students’ (right) answers to sub questions of RQ1.

As it can be observed, machine learning and the social impact of AI are the top ones, but perception and actuation are the following, above representation and reasoning. These results reinforce both the idea that even teachers that are not experts in AI topics easily understand the intelligent agent approach to AI developed along the different TUs and the current proposal of using robotics as a key application field to learn about AI. On the other hand, the right graph of Fig. 10 displays the results of a questionnaire with multiple choices, answered only by students, and it shows their perspective of the robot’s capabilities for learning about intelligent robotics. Most students highlight the possibility of using computer vision and support HRI, two completely new features for them when the project started.

Regarding RQ2, two questions were passed to teachers in the final activity (Fig. 11 ). Figure 11 left is related to the use of Python in secondary school. The results show how teachers have an overall positive opinion of it, considering that it is adequate for learning AI. Regarding its complexity, the opinions are divided, encountering teachers who consider that Python could be integrated within the AI curriculum, but others consider that it should be learned separately, due to its complexity. However, none of them consider that Python is too complex to not be addressed in secondary and high school. Figure 11 right summarizes their opinion with regards to use of the smartphone in Robobo. All participants express a favourable point of view, though some believe that dedicated devices are preferable instead of the own smartphone of the students. This preference proceeds not only from potential technical complications arising from using various smartphones but also from ethical considerations related to performance disparities between inexpensive and costly devices when carrying out tasks.

Teachers’ answers to sub questions of RQ2.

University (degree level)

This second use case corresponds to a specific activity proposed to students in the “Informatics” subject of the 1st year of the Industrial Engineering degree at UDC during the course 2022/23. This subject is focused on learning the fundamentals of programming, using the Python language, during a period of four months. Traditionally, students had to solve different programming exercises, which focus on learning the algorithmic and structural aspects of the language, avoiding the problems derived from more advanced and realistic applications. However, it has been shown that such a methodology decreases students’ interest and motivation for the topic, mainly in degrees different from computer science, where students are more interested on practical issues (Fontenla–Romero et al. 2022 ).

Specifically, the following activity was proposed within this subject: To develop a Python program so Robobo can grab a red cylinder in a closed environment independently of its initial position . The activity was carried out in asynchronous fashion, so teachers proposed it and students had two weeks to solve it autonomously (homework). In this period, they were tutored on demand, but encouraged to use the Robobo documentation and solve the issues on their own. None of the students had previous experience with AI or robotics, and the main goal of the activity was to evaluate their programming skills developing a feasible and usable solution. Consequently, they had to learn about sensors (distance, colour) and actuators (motors and speech) to create an autonomous response on the robot. At the end of the activity, they also had to carry out a small presentation of their solution to the teachers. The total number of participants in the activity was 56, with ages ranging 18–20, who answered the survey after the deadline date.

Two representative questions related to RQ1 are displayed in Fig. 12 . The left one shows how students perceive that solving the proposed problem is closer to their work as engineers (only 14.5% of answers are 1 and 2), even with the childlike appearance of the robot, which could seem inadequate for this age. The right question reinforces the original claim about this practical methodology and the focus on validation aspects of a program when learning the fundamentals of programming (64.3% of students’ answers are 1 or 2, meaning disagreement with the use of traditional challenges).

1 means totally disagree while 5 means totally agree.

In Fig. 13 we can observe students’ answers about two questions related with RQ2. The left one is very important for this research question, as it is a typical problem of educational robotics: students put too much effort on solving practical issues related to robotics and they lose the point from the goal of learning about programming. In this case, the Robobo Python library and its integration with RoboboSim seems to be adequate for students according to their answers (only 12.7% of negative feedback). It must be pointed out here that the “level of realism” of the simulator was set to the minimum, to simplify programming. Regarding the right question of Fig. 13 , it clearly shows that most students consider the library documentation adequate and feel that it was helpful for the programming of the activity (78.2%).

First year university students’ answers to sub questions of the RQ2.

University (master level)

As in the previous section, these results correspond to specific subjects at master level in course 2022/23. In particular, five students from the “Industrial Robotics” course of the 4th level at the Industrial Technologies Engineering degree from the UDC, and 28 students from the “Intelligent Robotics and Autonomous Systems” course in the master’s degree on Industrial Informatics and Robotics from the UDC participated in this use case. The students belong to two different groups, but their background skills in terms of programming and mathematics were similar.

In both courses, students learned about the fundamentals of intelligent robotics, mainly covering perception, actuation, representation, and reasoning. In these subjects, the proposed challenge is focused on the architectural aspects of intelligent robotics, that is, how to organize the control system to achieve the desired autonomy in a feasible and modular fashion. The difference between reactive, deliberative and hybrid approaches are trained with detail.

The students were organized in pairs, and they had a period of 8 weeks to solve an intelligent robotics challenge with the following goals:

To define a practical goal for Robobo in one of the environments of RoboboSim. This point implies defining also the set of environmental rules that condition the robot’s behaviour. For example, students can specify that blue objects “burn” and the Robobo should avoid them, or that Robobo cannot touch walls or collide with objects.

To implement and test a reactive or hybrid architecture with at least four autonomous behaviours. The architecture must allow Robobo to act in the proposed environment to fulfil its objectives independently of its initial position or orientation, and independently of the fact that the positions of some objects change too.

Developing the proposed architecture implied: (1) Sensing: Using infrared, encoders and gyroscope for navigation and localization (odometry), as well as computer vision for guidance; (2) Actuation: Using the wheel and pan-tilt unit motors for actuation, implementing specific control programs for each behaviour, using a PID or a similar approach; (3) Representation: To develop an appropriate internal representation for the state space that could include a map and (4) Reasoning; the main goal was on the design and implementation of the decision architecture, to select the most adequate action on each case. Students could program independent behaviours “by hand” or use some machine learning approach (like reinforcement learning).

The final assessment was obtained after showing the correct response of the robot in simulation and, as extra value, in the real robot. The validation in the real robot was performed by all students, although the adjustment period was not enough to obtain reliable responses. Figure 14 contains one example of scenario selected and solved by students, both in simulation (left) and real world (right). In this case, the goal consisted of creating autonomously a map of the environment using the tags for the own robot location, with the aim of performing later operations in an optimal fashion.

Example of problem solved by master level students in simulation and real robot.

In this case, the answers provided by students are shown in Figs. 15 and 16 , corresponding to representative topics for RQ1 and RQ2. Left question on Fig. 15 relies on the concept that, even with the simplistic aspect of the robot, students clearly perceive it as an adequate tool for learning about intelligent robotics in a technical scope. Figure 15 right reinforces a key idea for specialized students, who must understand that not only complex sensors as cameras or microphones are required for AI, but also basic ones as distance or orientation sensors.

Master students’ answers to sub questions of RQ1. DK/DA means Don’t Know or Don’t Answer.

Master students’ answers to sub questions of RQ2. DK/DA means Don’t Know or Don’t Answer.

In terms of RQ2, left graph of Fig. 16 displays that the students valued as important the use of a robot that allowed them to use computer vision algorithms. This is very relevant for RQ2, as these students could require low level libraries and resources, so they have a higher control level of the robot. In this line, Python language provides a natural advantage as it allows to include external libraries to perform advanced processes, as computer vision ones, but also path planning, machine learning, or speech recognition. Regarding the documentation, the answers to the right question in Fig. 16 show that the students found it easy to use and adequate, even at this high educational level.

Analysis and discussion

The results obtained during six years of intensive use of the Robobo Project in real classrooms, directly through the questionnaires filled by students and teachers, as well as the indirect results extracted from them, allow us to state that the project responds affirmatively to the hypotheses raised in both RQs. Regarding RQ1, we can affirm that Robobo was shown to be an appropriate tool to support AI literacy in the long-term. It was applied at different educational levels to train core AI concepts of different difficulty with success. A major part of this achievement was due to the adapted resources and materials included in the Robobo project, which have been perceived as highly useful by students and teachers, as extracted from their positive answers to RQ2 too.

In secondary school, the Robobo project has been widely tested in European schools with novel students and teachers, with no previous experience on AI and with a basic background in digital education and programming. In this scope, the usability and simplicity of the project resources, as the RoboboSim, the Scratch blocks, or the documentation, have shown to be adequate for this initial level. Although the main part of the students’ work was carried out in simulation in this level, the ease of transfer to the real robot was also very engaging and motivating for students, that could try their solutions in the real world. It is especially interesting the positive feedback obtained from teachers in relation with the teaching units and the suitability of the robot to support AI topics learning. This is a very relevant result for the project team, as simplifying the advanced AI tools and methods commonly used in the field could result in inadequate materials, for being too simplistic or too complex for students without a technical background.

At university level, the students have also provided a positive response regarding the use of the Robobo Project as a tool to learn intelligent robotics in a more technical scope. The possibility of using Python is one of the main reasons behind such success. First, the robobo.py library was positively valuated by students due to the functionalities it provides, from high level perception functions to direct stream of sensorial information. Second, the Python compatibility with third party AI libraries allowed advanced students to integrate state of the art algorithms and methods in their programs easily. In terms of simulation, the RoboboSim was not perceived as too basic for technical development, as students could use the random mode in the worlds and enable the high realism degree to make the problem more challenging. In addition, for these students, the simulator was seen just an initial step towards developing the solution of the task in the real robot, which was the main goal on this educational level. University students also considered the documentation to be helpful to support their autonomous work and learning.

After this period of six years and all the educational experiences carried out using the Robobo robot for AI education, it is important to conclude this discussion by comparing it with the two main competitors in this scope: the Thymio and the Fable robots. Table 5 summarizes the 7 key features established in Section “Literature review” for an educational robot to support formal AI literacy. As it can be observed in the table, and as it has been shown throughout the paper, the Robobo robot accomplishes all of them, as they were part of the design specifications. In the case of the Thymio robot, it is limited in terms of advanced sensing (mainly camera), advanced actuation (LCD screen) and computational power, making this robot not adequate for learning about some key topics on AI like computer vision, natural interaction, or reinforcement learning. However, this platform is a great option for some topics, mainly in terms of price/quality ratio, but it must be backed with other type of robot to cope with AI education in a global frame. Regarding the Fable, it lacks a specific simulation software, which is an important drawback for class organization, rate of progress, and investment. The rest of features are supported, so it can be used for proper AI learning in real setups.

But the main advantage of Robobo for long-term education on AI, as compared to the other two robots, is on the existing teaching resources, point 7 on the table. Its resources are fully aligned with the official recommendations for AI literacy explained in this paper, which is not the case on the competitors. As a consequence, teachers can start using the robot in the short-term for their classes with confidence. In one hand, they have open access to reliable materials to be directly applied, and on the other, the potentiality of the available resources provides them with a long range of capabilities to develop new teaching units and materials.

In this realm, it must be highlighted that all Robobo Project users provided similar feedback when it comes to the future possibilities of the robot, showing that they perceive to be using just a small part of them. This is very relevant for the project, as it reinforces the core idea of combining a simple mobile base with a smartphone that is the element that is updated and supports new developments and improvements. Therefore, the future applications of the Robobo project are open, which is very important in the realm of AIEd.

Conclusions

The main contribution of this work has been the presentation of a reliable classroom tool to support AI literacy teaching at different educational levels. While policy makers are designing plans and curricula to introduce AI training in formal education, the researchers in the field of AIEd must carry out projects like the one shown here, in which specific tools for AI learning are tested in real classes with teachers and students, so useful conclusions can be obtained for the whole educational community, mainly for the mentioned policy makers. The social impact of this type of contribution is very relevant, as in education, piloting resources with students is a must.

The Robobo Project is based on the application of educational robots as optimal classroom resource to learn about the core AI topics under the intelligent agent approach, the one followed by the most relevant AIEd initiatives when it comes to AI literacy principles. Throughout this paper, it has been shown that a robot with adequate technical capabilities, together with software tools adapted to the educational level, and formal teaching units, can be successfully applied for AI training in the long-term in real courses. Teachers can use the robot in complete AI courses to face the five basic AI topics, or just for particular ones integrated into subjects like mathematics, informatics, or physics. The suitability of robots for the STEM approach is reinforced in this scope with their adaptation to AI literacy teaching, as AI is fitted also perfectly on STEM principles, increasing its applicability in many educational environments.

Finally, it must be clarified that, from the experience gained in the Robobo Project, we can conclude that not all types of educational robots are appropriate for AIEd. That is, the simple fact of developing challenges in the realm of autonomous robotics is not enough for students to learn about AI. Robots in this context must support specific AI features, like computer vision, speech recognition and production, tactile interaction, visual interfaces, high-computational power, and internet connection. In addition, we can also conclude that it is fundamental to develop materials that support the teacher, as AI is a new field for most of them, and having quality resources for them is the only way to ensure a proper advance in AIEd.

Data availability

The data obtained in the students’ and teachers’ surveys are available at: https://github.com/GII/robobo-education-paper/tree/main/Surveys . The teaching units developed in the Erasmus+ projects are available at: https://aiplus.udc.es/results/ . https://aim4vet.udc.es/modules/ . The teaching units belonging to university degrees can be accessed at https://github.com/GII/robobo-education-paper/tree/main .

Code availability

All the Robobo libraries and source code can be downloaded from: https://github.com/mintforpeople/robobo-programming/wiki . https://github.com/mintforpeople/robobo.py .

https://github.com/mintforpeople/robobo-programming/wiki/Blocks .

https://github.com/mintforpeople/robobo-programming/wiki/python-doc .

https://github.com/mintforpeople/robobo-programming/wiki/Unity .

https://github.com/mintforpeople/robobo-programming/wiki .

http://education.theroboboproject.com/en/scratch3/sample-projects .

AI at code.org (2023) Anyone can learn Computer Science. https://code.org/ai . Accessed 11 Oct 2024

AI+ (2019) AI+ Project Results. https://aiplus.udc.es/results/ . Accessed 11 Oct 2024

AI4K12 (2023) The Artificial Intelligence for K-12 initiative. https://ai4k12.org/ . Accessed 11 Oct 2024

Amsters R, Slaets P (2020) Turtlebot 3 as a Robotics Education Platform. In: Merdan M, Lepuschitz W, Koppensteiner G, et al. (eds) Robotics in Education. Springer International Publishing, Cham, pp 170–181

Ananiadou K, Claro M (2009) 21st Century Skills and Competences for New Millennium Learners in OECD Countries. https://doi.org/10.1787/218525261154

Anwar S, Bascou NA, Menekse M, Kardgar A (2019) A systematic review of studies on educational robotics. J. Pre-Coll. Eng. Educ. Res. (J.-PEER) 9:2

Google Scholar  

Aranda MR, Estrada Roca A, Margalef Martí M (2019) Idoneidad didáctica en educación infantil: matemáticas con robots Blue-Bot. EDMETIC 8:150–168. https://doi.org/10.21071/edmetic.v8i2.11589

Article   Google Scholar  

Bellas F, Guerreiro-Santalla S, Naya M, Duro RJ (2022) AI Curriculum for European High Schools: An Embedded Intelligence Approach. Int. J. Artif. Intell. Educ. https://doi.org/10.1007/s40593-022-00315-0

Bellas F, Naya M, Varela G, et al. (2017) The Robobo project: bringing educational robotics closer to real-world applications. In: International Conference on Robotics and Education RiE 2017. Springer, pp 226–237

Bezerra Junior JE, Queiroz PGG, de Lima RW (2018) A study of the publications of educational robotics: a systematic review of literature. IEEE Lat. Am. Trans. 16:1193–1199. https://doi.org/10.1109/TLA.2018.8362156

Brew M, Taylor S, Lam R, et al. (2023) Towards developing AI literacy: three student provocations on AI in higher education. Asian J. Distance Educ. 18:1–11

Castro E, Cecchi F, Valente M, et al. (2018) Can educational robotics introduce young children to robotics and how can we measure it. J. Comp. Assist. Learn. 34:970–977. https://doi.org/10.1111/jcal.12304

Chen X, Zou D, Xie H, et al. (2022) Two decades of artificial intelligence in education: contributors, collaborations, research topics, challenges, and future directions. Educ. Technol. Soc. 25:28–47

Chu S-T, Hwang G-J, Tu Y-F (2022) Artificial intelligence-based robots in education: a systematic review of selected SSCI publications. Computers Educ.: Artif. Intell. 3:100091. https://doi.org/10.1016/j.caeai.2022.100091

Darmawansah D, Hwang G-J, Chen M-RA, Liang J-C (2023) Trends and research foci of robotics-based STEM education: a systematic review from diverse angles based on the technology-based learning model. Int. J. STEM Educ. 10:12. https://doi.org/10.1186/s40594-023-00400-3

Dignum V, Penagos M, Pigmans K, Vosloo S (2021) Policy guidance on AI for children. Unicef. https://www.unicef.org/innocenti/reports/policy-guidance-ai-children . Accessed 11 Oct 2024

Esquivel-Barboza EA, Llamas LF, Vázquez P, et al. (2020) Adapting computer vision algorithms to smartphone-based robot for education. In: 2020 Fourth International Conference on Multimedia Computing, Networking and Applications (MCNA). pp 51–56

Farias G, Fabregas E, Peralta E, et al. (2019) Development of an easy-to-use multi-agent platform for teaching mobile robotics. IEEE Access 7:55885–55897. https://doi.org/10.1109/ACCESS.2019.2913916

Fontenla-Romero O, Bellas F, Sánchez-Maroño N, Becerra JA (2022) Learning by Doing in Higher Education: An Experience to Increase Self-learning and Motivation in First Courses. In: Gude Prego JJ, de la Puerta JG, García Bringas P, et al. (eds) 14th International Conference on Computational Intelligence in Security for Information Systems and 12th International Conference on European Transnational Educational (CISIS 2021 and ICEUTE 2021). Springer International Publishing, Cham, pp 336–345

Global Education Monitoring Report Team (2023) United Nations Educational, Scientific and Cultural Organization. https://www.unesco.org/gem-report/en/technology . Accessed 11 Oct 2024

Gomoll AS, Hmelo-Silver CE, Tolar E, et al. (2017) Moving Apart and Coming Together: Discourse, Engagement, and Deep Learning. J. Educ. Technol. Soc. 20:219–232

Heinerman J, Rango M, Eiben AE (2015) Evolution, Individual Learning, and Social Learning in a Swarm of Real Robots. In: 2015 IEEE Symposium Series on Computational Intelligence. pp 1055–1062

Hornberger M, Bewersdorff A, Nerdel C (2023) What do university students know about Artificial Intelligence? Development and validation of an AI literacy test. Computers Educ.: Artif. Intell. 5:100165. https://doi.org/10.1016/j.caeai.2023.100165

Huang C, Zhang Z, Mao B, Yao X (2023) An Overview of Artificial Intelligence Ethics. IEEE Trans. Artif. Intell. 4:799–819. https://doi.org/10.1109/TAI.2022.3194503

ISTE (2023) Artificial Intelligence in Education. https://www.iste.org/areas-of-focus/AI-in-education . Accessed 11 Oct 2024

Kivunja, C. (2015) Exploring the Pedagogical Meaning and Implications of the 4Cs “Super Skills” for the 21st Century through Bruner’s 5E Lenses of Knowledge Construction to Improve Pedagogies of the New Learning Paradigm. Creative Education, 6, 224–239. https://doi.org/10.4236/ce.2015.62021

Konijn EA, Hoorn JF (2020) Robot tutor and pupils’ educational ability: teaching the times tables. Computers Educ. 157:103970. https://doi.org/10.1016/j.compedu.2020.103970

Laupichler MC, Aster A, Schirch J, Raupach T (2022) Artificial intelligence literacy in higher and adult education: a scoping literature review. Computers Educ.: Artif. Intell. 3:100101. https://doi.org/10.1016/j.caeai.2022.100101

Lee I, Ali S, Zhang H, et al. (2021) Developing Middle School Students’ AI Literacy. In: Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. Association for Computing Machinery, New York, NY, USA, pp 191–197

Llamas LF, Paz-Lopez A, Prieto A, et al. (2020) Artificial Intelligence Teaching Through Embedded Systems: A Smartphone-Based Robot Approach. In: Silva MF, Luís Lima J, Reis LP, et al. (eds) Robot 2019: Fourth Iberian Robotics Conference. Springer International Publishing, Cham, pp 515–527

Machine Learning for Kids (2023) Machine Learning Projects. https://machinelearningforkids.co.uk/#!/worksheets . Accessed 11 Oct 2024

Miras K (2024) Learning Machines. https://studiegids.vu.nl/en/courses/2024-2025/XM_0061#/ . Accessed 11 Oct 2024

MIT (2024) Artificial Intelligence with MIT App Inventor. https://appinventor.mit.edu/explore/ai-with-mit-app-inventor . Accessed 11 Oct 2024

Mondada F, Bonani M, Riedo F, et al. (2017) Bringing robotics to formal education: the thymio open-source hardware robot. IEEE Robot. Autom. Mag. 24:77–85. https://doi.org/10.1109/MRA.2016.2636372

Murphy RR (2019) Introduction to AI robotics. MIT press

Mussati A, Giang C, Piatti A, Mondada F (2019) A Tangible Programming Language for the Educational Robot Thymio. In: 2019 10th International Conference on Information, Intelligence, Systems and Applications (IISA). pp 1–4

Naya-Varela M, Guerreiro-Santalla S, Baamonde T, Bellas F (2023) Robobo SmartCity: an autonomous driving model for computational intelligence learning through educational robotics. IEEE Trans. Learn. Technol. 16:543–559. https://doi.org/10.1109/TLT.2023.3244604

Ng DTK, Leung JKL, Chu SKW, Qiao MS (2021) Conceptualizing AI literacy: an exploratory review. Computers Educ.: Artif. Intell. 2:100041. https://doi.org/10.1016/j.caeai.2021.100041

Pacheco M (2024a) Fable: Education Components. https://teacherzone.shaperobotics.com/en-US/home . Accessed 11 Oct 2024

Pacheco M (2024b) Fable: STEAM lab components. https://shaperobotics.com/thinken/ . Accessed 11 Oct 2024

Pacheco M, Fogh R, Lund HH, Christensen DJ (2015) Fable II: Design of a modular robot for creative learning. In: 2015 IEEE International Conference on Robotics and Automation (ICRA). pp 6134–6139

Pacheco M, Moghadam M, Magnússon A, et al. (2013) Fable: Design of a modular robotic playware platform. In: 2013 IEEE International Conference on Robotics and Automation. pp 544–550

Poole DL, Mackworth AK (2010) Artificial Intelligence: foundations of computational agents. Cambridge University Press

Renda C, Prieto A, Bellas F (2024) Teaching Reinforcement Learning Fundamentals in Vocational Education and Training with RoboboSim. In: Marques L, Santos C, Lima JL, et al. (eds) Robot 2023: Sixth Iberian Robotics Conference. Springer Nature Switzerland, Cham, pp 526–538

Rétornaz P, Riedo F, Magnenat S, et al. (2013) Seamless multi-robot programming for the people: ASEBA and the wireless Thymio II robot. In: 2013 IEEE International Conference on Information and Automation (ICIA). pp 337–343

Russell S, Norvig P (2021) Artificial Intelligence: A Modern Approach, 4th US ed

Sáez-López J-M, Sevillano-García M-L, Vazquez-Cano E (2019) The effect of programming on primary school students’ mathematical and scientific understanding: educational use of mBot. Educ. Technol. Res. Dev. 67:1405–1425. https://doi.org/10.1007/s11423-019-09648-5

Seckel MJ, Salinas C, Font V, Sala-Sebastià G (2023) Guidelines to develop computational thinking using the Bee-bot robot from the literature. Educ. Inf. Technol. 28:16127–16151. https://doi.org/10.1007/s10639-023-11843-0

Shin J, Siegwart R, Magnenat S (2014) Visual programming language for Thymio II robot. In: Conference on interaction design and children (idc’14). ETH Zürich

Sullivan A, Bers MU (2019) Investigating the use of robotics to increase girls’ interest in engineering during early elementary school. Int. J. Technol. Des. Educ. 29:1033–1051. https://doi.org/10.1007/s10798-018-9483-y

The Robobo Project (2024) The next generation of educational robots. https://theroboboproject.com/en/ . Accessed 11 Oct 2024

The University of Helsinki (2024) Elements of AI. https://www.elementsofai.com . Accessed 11 Oct 2024

Thymio robot (2024) Helping people to discover and practice digital technologies. https://www.thymio.org/ . Accessed 11 Oct 2024

TIOBE Index (2024) TIOBE Index for June 2024. https://www.tiobe.com/tiobe-index/ . Accessed 11 Oct 2024

UNESCO (2022) K-12 AI curricula: a mapping of government-endorsed AI curricula. https://unesdoc.unesco.org/ark:/48223/pf0000380602 . Accessed 11 Oct 2024

UNESCO (2023) Guidance for generative AI in education and research. https://www.unesco.org/en/articles/guidance-generative-ai-education-and-research . Accessed 11 Oct 2024

Weiss T, Reichhuber S, Tomforde S (2023) From Simulated to Real Environments: Q-Learning for MAZE-Navigation of a TurtleBot. In: Rutkowski L, Scherer R, Korytkowski M, et al. (eds) Artificial Intelligence and Soft Computing. Springer Nature Switzerland, Cham, pp 192–203

Xu W, Ouyang F (2022) The application of AI technologies in STEM education: a systematic review from 2011 to 2021. Int. J. STEM Educ. 9:59. https://doi.org/10.1186/s40594-022-00377-5

Download references

Acknowledgements

MINT SL (Northius Group) is the manufacturer and owner of the Robobo robot, and the authors want to thank the company for its support to carry out the Robobo Project educational initiatives. CITIC, as a center accredited for excellence within the Galician University System and a member of the CIGUS Network, receives subsidies from the Department of Education, Science, Universities, and Vocational Training of the Xunta de Galicia. Additionally, it is co-financed by the EU through the FEDER Galicia 2021-27 operational program (Ref. ED431G 2023/01). Martin Naya-Varela is supported by the Xunta de Galicia with his grant ED481B.

Author information

Authors and affiliations.

Integrated Group for Engineering Research, CITIC Research Center, Universidade da Coruña, A Coruña, Spain

Francisco Bellas, Martin Naya-Varela, Alma Mallo & Alejandro Paz-Lopez

You can also search for this author in PubMed   Google Scholar

Contributions

FB contributed as the main responsible of the introduction section in terms of AI literacy and institutional initiatives. In addition, he was devoted with sections Research contribution, Theoretical basis, and Teaching methodology, focused on methodological aspects, and also in the formal definition of the AI topics to be faced. Finally, he also helped with the challenges on Table 2 . All these issues are related with his work as coordinator of the Erasmus+ projects mentioned in the text. MN was responsible of the educational robotics state of art (Section “Literature review”), as well as Sections “Robobo: the real robot”, and “Analysis and discussion”. He was one of the engineers behind the hardware aspects of the Robobo real robot, and he has been also coordinating the simulation aspects of the project. AP was devoted with the software design of the Robobo Project, so he was the main responsible of Section “Software and development tools”. The Robobo Framework and the development of all the IDEs and extensions was his main focus in the last year. AM was the main responsible of the project testing with students, so she was one directly involved in all the results shown in Section “Experimental results. Robobo use cases with students”. She also supported FB in the definition of the teaching units and the documentation explained in Section “Methods and Materials”, due to her field expertise on testing the units with students.

Corresponding author

Correspondence to Francisco Bellas .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethical approval

Name of the approval body: Ethics Committee for Research and Teaching – Universidade da Coruña ( https://www.udc.es/en/ceid/ ). The Standard Operating Procedures (SOPs) of CEID-UDC are explained in this document https://www.udc.es/export/sites/udc/ceid/_galeria_down/PROCEDEMENTOS-NORMALIZADOS-DE-TRABALLO-do-CEID-UDC.pdf_2063069299.pdf . They are based on the current legislation of the Spanish State, as well as on the basic principles for conducting research studies, with emphasis on research with human beings based on the protection of human rights and human dignity, reflected in the Helsinki Declaration and the Oviedo Convention, as well as in the regulations on the protection of personal data. Approval number or ID: CEID‐UDC record number 2021‐0035. Date of approval: 18/04/2022. Scope of approval: the approved project was called “Learning Artificial Intelligence in pre-university education”, and its scope was limited to the pre-university students that participated in the AI+ project.

Informed consent

It should be noted that all the images included in this paper and the data collected from students and teachers have the formal authorisation of those involved. On the one hand, through specific agreements with the partners in the framework of the Erasmus+ projects 2019-1-ES01-KA201-065742 and 2022-1-ES01-KA220-VET-000089813, and on the other hand, through the consent validated by the ethics committee of the University of Coruña, approved on 18/04/2022.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Bellas, F., Naya-Varela, M., Mallo, A. et al. Education in the AI era: a long-term classroom technology based on intelligent robotics. Humanit Soc Sci Commun 11 , 1425 (2024). https://doi.org/10.1057/s41599-024-03953-y

Download citation

Received : 26 March 2024

Accepted : 16 October 2024

Published : 25 October 2024

DOI : https://doi.org/10.1057/s41599-024-03953-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

SYSTEMATIC REVIEW article

Integration of computational thinking in initial teacher training for primary schools: a systematic review.

• 1 Escola de Ciências e Tecnologia, Universidade de Trás-os-Montes e Alto Douro, Vila Real, Portugal
• 2 Instituto Politécnico de Coimbra, Escola Superior de Educação de Coimbra, Coimbra, Portugal
• 3 Centro de Investigação em Didática e Tecnologia na Formação de Formadores, Universidade de Aveiro, Aveiro, Portugal
• 4 Instituto de Telecomunicações, Delegação da Covilhã, Covilhã, Portugal
• 5 inED – Centro de Investigação e Inovação em Educação, Instituto Politécnico de Coimbra, Coimbra, Portugal

Computational Thinking, a capacity based on the principles of computing, has been highlighted in the specialized literature as an essential skill for the 21st century, bringing significant benefits to the problem-solving process. In this way, norms for the integration of Computational Thinking in education have emerged in the educational curricula of several countries. For this integration to be successful, it is essential that the training given to pre-service teachers enables them to develop well-planned and structured interventions to promote the development of Computational Thinking. This article presents a systematic review of the literature that aims to investigate how the development of Computational Thinking has been integrated into teacher training. Eleven articles that corresponded to the selected research criteria were found, and the characteristics of their studies are analysed and presented in this article. The article concludes that it is necessary to invest in pre-service teacher training, highlighting the need for long-term and more comprehensive training covering not only the theoretical component but also the practical component, as well as reflection on practice.

1 Introduction

Computational Thinking (CT) is increasingly valued in the specialized literature as an essential ability in problem-solving, and it is considered an essential skill for the 21st century by some authors ( Angeli et al., 2016 ; Peracaula-Bosch et al., 2020 ). Although the term was first coined by Papert (1980) , it was Wing’s publication in 2006 that boosted the development of numerous studies and investigations on the integration of CT in teaching ( Ausiku and Matthee, 2021 ; Knie et al., 2022 ; Menolli and Neto, 2022 ). Although there has been a significant increase in studies on this topic, uncertainty remains in defining what CT is and how it can be used in teaching ( Macann and Carvalho, 2021 ; Peracaula-Bosch et al., 2020 ; Tsarava et al., 2022 ). Still, it is widely recognized in the scientific community that the development of CT brings significant benefits to the problem-solving process ( Çoban and Korkmaz, 2021 ).

CT is a capacity that is based on computing processes ( Wing, 2006 ). However, it is essential for everyone, not just for those who work with technology ( Knie et al., 2022 ), nor is its development exclusively dependent on the specific use of technologies ( El-Hamamsy et al., 2021 ). Although programming and robotics are often associated with the development of CT, the mere implementation of activities involving these resources does not guarantee the development of CT ( Peracaula-Bosch et al., 2020 ). For the development of CT to occur, it is necessary to develop well-structured tasks designed for this purpose ( Espadeiro, 2021 ; Salinas et al., 2024 ).

Over the last few years, countries such as Portugal ( Rodrigues et al., 2022 ), Thailand ( Pewkam and Chamrat, 2021 ), Ireland ( Butler and Leahy, 2021 ), New Zealand ( Macann and Carvalho, 2021 ) and Norway ( Kravik et al., 2022 ; Nordby et al., 2022 ) have begun introducing CT into their educational curricula. In this context, it is essential to investigate how initial teacher training has prepared pre-service teachers to integrate the development of CT into their practices, thus raising the research problem of this study: How has Computational Thinking been integrated into the training curricula of future primary school teachers at various universities over the last 10 years?

The main objective of this systematic review is to investigate and present how Computational Thinking has been integrated into the training curricula of future primary school teachers at different universities over the last 10 years. The following research questions guiding the study were formulated:

1. What types of training have been used to develop Computational Thinking during initial training for primary school teachers?

2. How has this training been structured?

3. How has it been monitored?

The preparation of this systematic review followed the principles of the PRISMA 2020 Declaration ( Page et al., 2021 ), with the inclusion and exclusion criteria for articles presented below.

2.1 Search strategy

The process of collecting articles for the systematic review took place in the second week of July 2023, using the databases SCOPUS from Elsevier, Web of Science from Clarivate and ERIC from the Institute of Education Sciences. In the SCOPUS database, articles’ titles, abstracts, or keywords were searched using the terms of the following search equation: “Computational Thinking” AND (“Teacher Training” OR “Pre-service Teach*”) AND (“Primary School” OR “Elementary School”). In the Web of Science and ERIC databases, the main collection of articles was searched using the same search equation in all sections of the articles.

2.2 Eligibility criteria

The following inclusion criteria were defined: (i) articles published between January 1, 2014, and July 13, 2023; (ii) articles that focus on the development of Computational Thinking in initial primary teacher training; (iii) articles that describe a training practice developed in initial teacher training; (iv) articles involving pre-service primary teachers.

The following exclusion criteria were applied: (i) duplicate articles; (ii) articles published before January 1, 2014; (iii) articles in the review phase; (iv) review articles or conference proceedings; (v) articles that did not focus on the development of Computational Thinking in initial primary teacher training; (vi) articles that did not describe a training practice developed in initial teacher training; (vii) articles that do not involve pre-service primary teachers.

The following steps were taken during the article selection process: (1) search in the select-ed databases; (2) exclusion of duplicate articles; (3) exclusion of articles outside the intend-ed time period; (4) exclusion of articles in the review phase; (5) exclusion of review articles or conference proceedings; (6) exclusion of articles that did not focus on the development of Computational Thinking in initial primary teacher training; (7) exclusion of articles that did not describe a training practice developed in initial teacher training; (8) exclusion of articles that do not involve pre-service primary teachers; (9) reading and evaluation of the articles included in the systematic review.

With the defined search equation, initially 63 articles were obtained from the three searched databases. After removing duplicate articles and articles still in review, two researchers (the first author and an external researcher) conducted an initial analysis independently and decided to exclude 31 articles as conference proceedings and 2 as review articles. Next, the remaining 18 articles were analyzed by the two researchers, with seven articles being eliminated for not providing information on the development of CT in teacher training or on a training practice developed in the training of future teachers. Figure 1 illustrates the literature search and the selection of eligible studies.

Figure 1 . Prisma flowchart ( Page et al., 2021 ).

2.3 Analysis of the quality of studies

The quality of the 11 selected articles was analyzed using The Mixed Methods Appraisal Tool (MMAT), in its 2018 version ( Hong et al., 2018 ). This instrument assesses whether studies meet the requirements depending on the design of the study being analyzed (Qualitative, Quantitative randomized controlled trials, Quantitative nonrandomized, Quantitative descriptive and Mixed methods). If the studies meet the requirement described in each item they receive 1 point, if it is not clear they receive 0.5 points, and if they do not comply they receive 0 points. At the end of the evaluation, articles that have 0 to 2 points are considered low quality, 3 to 4 points are medium quality, and 5 to 7 points are high quality. The first and last author each evaluated the 11 selected articles using this evaluation instrument. As no discrepancies arose, it was not necessary for any further researcher to intervene. Thus, Table 1 presents the evaluation of the 11 selected studies.

Table 1 . Evaluation of articles according to the mixed methods appraisal tool (MMAT).

2.4 Description of the articles under analysis

Table 2 presents the main characteristics of the 11 articles selected for this systematic review: author/s, year of publication, country, target population of the study, and study design. This research was focused on articles that described a training practice developed with pre-service teachers. However, studies that did not focus exclusively on reporting practice but described a practice implemented to develop CT in pre-service teachers in any section of the article were also included. For example, articles that aimed to investigate changes in pre-service teachers’ beliefs, interest, and confidence in integrating CT into their practices were included. Although a time limit of 10 years for the publication of studies had initially been defined, no article published more than 8 years ago was found in the search that was carried out.

Table 2 . Summary of the main characteristics of the articles.

2.5 Analyzing studies and extracting data

The procedure for analyzing the 11 studies and deciding on the data to extract was carried out by the first and last authors independently. There was only one case of divergence, which was resolved through discussion with the contribution of the second author. Thus, considering that the objective of this literature review is to investigate how CT has been integrated into training curricula at different universities, in an initial analysis the researchers decided to extract information regarding the publication year of the articles, the country where the training had been implemented, the target population of the training, and the study design (as shown in Table 1 ).

In a more in-depth analysis and considering the research questions, the researchers decided to identify and summarize information from the studies on the article titles, authors, intervention objectives, type of intervention, description of the training, monitoring instruments, results, conclusions, and suggestions. The summary of the included studies can be found in the Supplementary Material .

3 Results and discussion

Of the 63 articles that were obtained through the search equation in the three databases, and after applying the already-stated exclusion criteria, 11 studies were analyzed in this literature review.

Inter-rater reliability (IRR) is crucial for the validity of systematic reviews, as it assesses the degree of agreement among researchers in the selection and analysis of studies. IRR is often measured using Cohen’s kappa coefficient ( Ankul et al., 2023 ), which quantifies the consistency between two or more raters, ensuring the objectivity and robustness of the review’s results. To ensure consistency and objectivity in the initial selection of articles, Cohen’s kappa coefficient was calculated ( Field, 2018 ), which quantifies the level of agreement between two researchers (the first author and an external researcher). A Cohen’s kappa coefficient of 0.90 was obtained, indicating an almost perfect agreement ( Landis and Koch, 1977 ) between the evaluators in the study selection process. During the analysis of the 11 articles selected using The Mixed Methods Appraisal Tool (MMAT), there were no discrepancies between the two researchers (the first and the last author), resulting in a Cohen’s kappa coefficient of 1, indicating perfect agreement ( Landis and Koch, 1977 ) between the evaluators in the analysis of the studies. In the decision-making process regarding the data to be extracted for the analysis of the 11 studies, carried out independently by the first and last authors, a Cohen’s kappa coefficient of 0,86 was obtained, indicating an almost perfect agreement ( Landis and Koch, 1977 ) between the researchers at this stage. The discrepancies observed during this process were resolved by the two authors through discussion, with contributions from the second author.

3.1 Summary of the selected studies

The objective of this systematic review is to analyze how CT has been integrated into initial primary teacher training, namely what the characteristics of the developed training are.

In order to answer the first research question “What types of training have been used to develop Computational Thinking during initial training for primary school teachers?” we can state that, in the 11 analyzed articles, training given to pre-service teachers within the scope of CT is divided into two distinct typologies: (a) modules or training actions that are inserted into already-existing curricular units in initial teacher training courses (5 studies); (b) optional courses, not integrated into the formal curriculum of initial teacher training courses (6 studies). Within the modules that were integrated into curricular units of initial teacher training, we found that two of the practices ( Tankiz and Atman Uslu, 2023 ; Zha et al., 2020 ) covered the entire period of an academic semester, while the remaining three practices ( Kaya et al., 2019 ; Molina-Ayuso et al., 2022 ; Sáez-López et al., 2020 ) consisted of sessions that took place over periods of two to five weeks during the academic semester. Regarding the optional training offered to pre-service teachers, one of the studies ( Butler and Leahy, 2021 ) describes a specialization course that takes place over 3 years, and another study ( Peracaula-Bosch and Gonzalez-Martinez, 2022 ) mentions an optional subject of the initial teacher training course which lasts for the entire period of an academic semester, similarly to two other optional courses ( Park et al., 2015 ; Tripon, 2022 ). The remaining two studies ( Drot-Delange et al., 2021 ; Pewkam and Chamrat, 2021 ) present small training actions lasting one to 3 days. The review of the analyzed studies has shown that the training provided should be incorporated into the initial teacher education as an integral part of the curriculum, rather than as a module within a course or with an optional character ( Zha et al., 2020 ), and should be of long duration. Only through a long-term intervention approach is it possible to include a fundamental theoretical component before didactic approaches and provide the support that should be given to future teachers throughout the learning process and the development of intervention plans. By integrating the development of CT into primary school teachers’ initial training and not treating it as something optional or sporadic, is it possible to promote the development of CT skills for all students from the beginning of their schooling, not just those attending schools with greater training opportunities in this area ( Butler and Leahy, 2021 ; Salinas et al., 2024 ).

Regarding the second research question “How has this training been structured?” we analyzed whether both the theoretical and practical components were included, the way the practice was developed, and the type of content associated with the development of CT. All the described practices include a theoretical component in the offered training. On the other hand, only eight of the analyzed studies describe training that includes a practical component, which involves course colleagues and promotes collaborative work. Only two studies mentioned the integration of practice with primary school students in the developed training, with this practice being optional in one of them. As for the content that was covered throughout training, all studies describe practices that associate the development of CT with programming activities, four of the practices also integrate robotics throughout the training, and another three incorporate unplugged activities into the developed training. This review has demonstrated that the practical aspect is essential in the context of teacher education in CT, particularly to bridge the often-evident gaps between theory and practice ( Macann and Carvalho, 2021 ). Future teachers must understand the definition of CT, but it is equally important for them to understand how it can be developed in educational practice. The interventions analyzed, where future teachers developed intervention plans and implemented them with their peers during training, showed that this practice creates an environment conducive to sharing learning, exchanging experiences, and reflecting on the proposals presented, promoting collaborative work ( Pewkam and Chamrat, 2021 ). The analyzed studies demonstrate that future teachers face difficulties in lesson planning or in developing activities with students ( Peracaula-Bosch et al., 2020 ; Tankiz and Atman Uslu, 2023 ), often failing to reflect on the possible challenges that students may encounter when developing CT. Thus, it becomes essential to include tasks in this area in initial teacher training programs. The results of the studies demonstrate that as future teachers design specific interventions throughout their training to implement with students, they tend to consider and reflect on the potential difficulties that students may encounter ( Drot-Delange et al., 2021 ). Our analysis therefore highlights that future teachers should be encouraged to reflect on practice, methodological options, student learning, and their potential difficulties. Our review identifies the trend of associating the development of CT with programming activities; however, most often the training starts with unplugged activities, meaning activities that do not directly involve the use of technological devices, aligning with what is mentioned in the literature ( Bjursten et al., 2023 ). The analyzed studies report that initially teachers express concerns and apprehensions regarding CT training, associating it with programming activities for education. However, by the end of the training, a positive change in the attitudes of future teachers is observed, supporting the conclusions of Kaya et al. (2019) , which state that training can help increase motivation and self-efficacy perception of future teachers regarding this type of activity.

To answer the third research question “How has it been monitored?” we found that the most used data collection instruments in the 11 studies were pre- and post-tests. Nine of the analyzed studies chose to use pre- and post-tests to monitor the developed training, and four of these did not use any other data collection instrument other than the tests. Written productions by students, semi-structured interviews and questionnaires were mentioned as instruments used to monitor training in the same number of studies (three). Observation grids were also used in two of the analyzed studies. In our review, it became evident that, although five out of the 11 studies emphasize in their conclusions the importance of reflection in the training process ( Butler and Leahy, 2021 ; Drot-Delange et al., 2021 ; Peracaula-Bosch and Gonzalez-Martinez, 2022 ; Tankiz and Atman Uslu, 2023 ; Zha et al., 2020 ), only one study mentioned the use of this instrument to monitor the training given to pre-service teachers. We also highlighted that several studies reflect on initial teachers’ attitudes toward CT training (e.g., motivation, self-efficacy perception, confidence), however they do not present instruments that allow monitoring this aspect.

The results of the studies that were analyzed in this systematic review provide important aspects related to the development of CT in initial primary teacher training, particularly the diversity of used modalities, reflections on the possible difficulties students faced, changes in the attitude of pre-service teachers throughout training, and the positive impact of collaboration in the training process. The authors of the reviewed studies point out the lack of training offered for the development of CT as an urgent challenge that requires an immediate solution. The studies reveal that pre-service teachers often lack knowledge about CT content, resulting in low self-efficacy, interest, and confidence, reinforcing the need for training in CT. Through training, pre-service teachers acquired crucial skills for the correct implementation of educational strategies involving CT ( Tripon, 2022 ).

4 Conclusion

This systematic review highlights the scarcity of training that promotes the development of CT in initial primary teacher training, even though more and more countries have incorporated CT into their basic education curricula in recent years.

The results underline the importance of the practical component, which is an essential part of initial teacher training, highlighting that this practice often does not occur. Practice with primary school students is essential for pre-service teachers to understand the real difficulties students face and the real impact that their plans have on learning. The prior implementation of the teaching experience planned by pre-service teachers in the initial teacher training class, with course colleagues playing the role of primary school students, enables the promotion of collaborative learning and reflection on the practice, anticipating possible difficulties.

Individual reflection is highlighted as a fundamental component in pre-service teacher training in the results of this systematic review. It was observed that training is often short and optional, which contrasts with the speciality literature that refers to the need for longer lasting and integrated training within the official initial teacher training curriculum.

4.1 Limitations and future directions

The reduced number of studies found on this topic stands out as a limitation of the present systematic review. Although the development of CT is an increasingly prevalent theme in the literature, this review shows that the search for interventions conducted in the training of future teachers still yields few results. By applying rigorous inclusion criteria such as peer-reviewed publications in indexed journals to ensure the quality of the analyzed articles gray literature was not considered, further reducing the number of initially found articles.

Following the results obtained in this systematic review, several recommendations for future research in the development of CT can be enumerated. It is recommended that future training programs involve not only a theoretical component around CT but also include a strong emphasis on the development of lesson plans or activities with students and their implementation in practice. It is also suggested that these training programs be long-term and include different data collection instruments to monitor not only what future teachers are doing but also their attitudes toward the development of CT.

Data availability statement

The original contributions presented in the study are included in the article/ Supplementary material , further inquiries can be directed to the corresponding author.

Author contributions

RR: Conceptualization, Data curation, Formal analysis, Funding acquisition, Methodology, Writing – original draft. CC: Conceptualization, Formal analysis, Funding acquisition, Methodology, Supervision, Writing – review & editing. FM: Conceptualization, Formal analysis, Funding acquisition, Methodology, Supervision, Writing – review & editing.

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work was supported by National Funds through FCT – Fundação para a Ciência e a Tecnologia, I.P., under the project UIDB/50008/2020, and DOI identifier https://doi.org/10.54499/UIDB/50008/2020 (Instituto de Telecomunicações), UIDB/05198/2020, and DOI identifier https://doi.org/10.54499/UIDB/05198/2020 (Centro de Investigação e Inovação em Educação, inED), UIDB/00194/2020 (CIDTFF) and under the doctoral scholarship 2022.09720.BD.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2024.1330065/full#supplementary-material

Angeli, C., Voogt, J., Fluck, A., Webb, M., Cox, M., Malyn-Smith, J., et al. (2016). A K-6 computational thinking curriculum framework: implications for teacher knowledge. Educ. Technol. Soc. 19, 47–57.

Google Scholar

Ankul, S. S., Chandran, L., Anuragh, S., Kaliappan, I., Rushendran, R., and Vellapandian, C. (2023). A systematic review of the neuropathology and memory decline induced by monosodium glutamate in the Alzheimer’s disease-like animal model. Front. Pharmacol. 14:1283440. doi: 10.3389/fphar.2023.1283440

PubMed Abstract | Crossref Full Text | Google Scholar

Ausiku, M., and Matthee, M. (2021). Preparing primary school teachers for teaching computational thinking: a systematic review . Learning Technologies and Systems: 19th International Conference on Web-Based Learning, ICWL 2020, and 5th International Symposium on Emerging Technologies for Education, SETE 2020, Ningbo, China, October 22–24, 2020, Proceedings, No. 12511, pp. 202–213.

Bjursten, E. L., Nilsson, T., and Gumaelius, L. (2023). Computer programming in primary schools: Swedish technology teachers’ pedagogical strategies. Int. J. Technol. Des. Educ. 33, 1345–1368. doi: 10.1007/s10798-022-09786-7

Crossref Full Text | Google Scholar

Butler, D., and Leahy, M. (2021). Developing preservice teachers’ understanding of computational thinking: a constructionist approach. Br. J. Educ. Technol. 52, 1060–1077. doi: 10.1111/bjet.13090

Çoban, E., and Korkmaz, Ö. (2021). An alternative approach for measuring computational thinking: performance-based platform. Think. Skills Creat. 42:100929. doi: 10.1016/j.tsc.2021.100929

Drot-Delange, B., Parriaux, G., and Reffay, C. (2021). Futurs enseignants de l’école primaire: connaissances des stratégies d’enseignement, curriculaires et disciplinaires pour l’enseignement de la programmation. RDST 23, 55–76. doi: 10.4000/RDST.3685

El-Hamamsy, L., Bruno, B., Chessel-Lazzarotto, F., Chevalier, M., Roy, D., Zufferey, J. D., et al. (2021). The symbiotic relationship between educational robotics and computer science in formal education. Educ. Inf. Technol. 26, 5077–5107. doi: 10.1007/s10639-021-10494-3

Espadeiro, R. G. (2021). O Pensamento Computacional no currículo de Matemática. Educ. Mat. 162, 5–10.

Field, A. (2018). Discovering statistics using IBM SPSS statistics . 5th Edn. Thousand Oaks, CA: SAGE Publications Ltd.

Hong, Q. N., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., et al. (2018). The mixed methods appraisal tool (MMAT) version 2018 for information professionals and researchers. Educ. Inf. 34, 285–291. doi: 10.3233/EFI-180221

Kaya, E., Yesilyurt, E., Newley, A., and Deniz, H. (2019). Examining the impact of a computational thinking intervention on pre-service elementary science teachers’ computational thinking teaching efficacy beliefs, interest and confidence. J. Comput. Math. Sci. Teach. 38, 385–392.

Knie, L., Standl, B., and Schwarzer, S. (2022). First experiences of integrating computational thinking into a blended learning in-service training program for STEM teachers. Comput. Appl. Eng. Educ. 30, 1423–1439. doi: 10.1002/cae.22529

Kravik, R., Berg, T. K., and Siddiq, F. (2022). Teachers’ understanding of programming and computational thinking in primary education - a critical need for professional development. Acta Didactica Norden 16:23. doi: 10.5617/adno.9194

Landis, J. R., and Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics 33, 159–174. doi: 10.2307/2529310

Macann, V., and Carvalho, L. (2021). Teachers use of public makerspaces to support students’ development of digital technology competencies. N. Z. J. Educ. Stud. 56, 125–142. doi: 10.1007/s40841-020-00190-0

Menolli, A., and Neto, J. C. (2022). Computational thinking in computer science teacher training courses in Brazil: a survey and a research roadmap. Educ. Inf. Technol. 27, 2099–2135. doi: 10.1007/s10639-021-10667-0

Molina-Ayuso, Á., Adamuz-Povedano, N., Bracho-López, R., Torralbo-Rodríguez, M., Molina-Ayuso, A., Adamuz-Povedano, N., et al. (2022). Introduction to computational thinking with scratch for teacher training for Spanish primary school teachers in mathematics. Educ. Sci. 12:899. doi: 10.3390/educsci12120899

Nordby, S. K., Bjerke, A. H., and Mifsud, L. (2022). Primary mathematics teachers’ understanding of computational thinking. KI Kunstliche Intell. 36, 35–46. doi: 10.1007/s13218-021-00750-6

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., et al. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews, no. 372. Available at: https://www.bmj.com/content/372/bmj.n71

Papert, S. (1980). Mindstorms: Children, computers, and powerful ideas . New York: Basic Books.

Park, S.-Y., Song, K.-S., and Kim, S.-H. (2015). Cognitive load changes in pre-service teachers with computational thinking education. Int. J. Softw. Eng. Appl. 9, 169–178. doi: 10.14257/ijseia.2015.9.10.17

Peracaula-Bosch, M., Estebanell-Minguell, M., Couso, D., González-Martínez, J., and Gonzalez-Martinez, J. (2020). What do pre-service teachers know about computational thinking? Aloma 38, 75–86. doi: 10.51698/aloma.2020.38.1.75-86

Peracaula-Bosch, M., and Gonzalez-Martinez, J. (2022). Developing computational thinking among pre-service teachers. QWERTY 17, 28–44. doi: 10.30557/QW000049

Pewkam, W., and Chamrat, S. (2021). Pre-service teacher training program of STEM-based activities in computing science to develop computational thinking. Inf. Educ. 21, 311–329. doi: 10.15388/infedu.2022.09

Rodrigues, R., Fonseca, J., Costa, C., and Martins, F. (2022). “Artefactos Digitais, Aprendizagens e Conhecimento Didático - Contributos para Promover a Compreensão da Matemática” in Pensamento computacional: dimensões desenvolvidas numa intervenção no estágio pedagógico . eds. F. Martins, R. Pinto, and C. Costa (Coimbra, IPC-ESEC), 117–134.

Sáez-López, J. M., del Olmo-Muñoz, J., González-Calero, J. A., Cózar-Gutiérrez, R., Saez-Lopez, J. M., del Olmo-Munoz, J., et al. (2020). Exploring the effect of training in visual block programming for preservice teachers. Multimodal Technol. Interact. 4:65. doi: 10.3390/mti4030065

Salinas, C., Seckel, M. J., Breda, A., and Espinoza, C. (2024). Integrating computational thinking into mathematics class: curriculum opportunities and the use of the bee-bot. Int. J. Educ. Methodol. 2024, 137–149. doi: 10.12973/ijem.10.1.937

Tankiz, E., and Atman Uslu, N. (2023). Preparing pre-service teachers for computational thinking skills and its teaching: a convergent mixed-method study. Technol. Knowl. Learn. 28, 1515–1537. doi: 10.1007/s10758-022-09593-y

Tripon, C. (2022). Supporting future teachers to promote computational thinking skills in teaching STEM—A case study. Sustainability (Switzerland) 14:12663. doi: 10.3390/su141912663

Tsarava, K., Moeller, K., Román-González, M., Golle, J., Leifheit, L., Butz, M., et al. (2022). A cognitive definition of computational thinking in primary education. Comput. Educ. 179:104425. doi: 10.1016/j.compedu.2021.104425

Wing, J. M. (2006). Computational thinking. Commun. ACM 49, 33–35. doi: 10.1145/1118178.1118215

Zha, S., Jin, Y., Moore, P., and Gaston, J. (2020). Hopscotch into coding: introducing pre-service teachers computational thinking. Tech Trends 64, 17–28. doi: 10.1007/s11528-019-00423-0

Keywords: computational thinking, pre-service teachers, primary school, higher education, systematic review

Citation: Rodrigues RN, Costa C and Martins F (2024) Integration of computational thinking in initial teacher training for primary schools: a systematic review. Front. Educ . 9:1330065. doi: 10.3389/feduc.2024.1330065

Received: 29 November 2023; Accepted: 30 September 2024; Published: 10 October 2024.

Reviewed by:

Copyright © 2024 Rodrigues, Costa and Martins. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Rita Neves Rodrigues, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Problem or project-based computer-supported collaborative learning practices in computer education: A systematic review of SSCI articles published from 2014 to 2023

• Published: 26 October 2024

Cite this article

• Khoirudin Asfani   ORCID: orcid.org/0000-0002-8220-7632 1 &
• Hsiu-Ling Chen   ORCID: orcid.org/0000-0002-8951-7043 1  

Although the Computer-Supported Collaborative Learning (CSCL) practice has a trend in its application, limited information exists in the literature on CSCL studies with Problem/Project-Based Learning in Computer Education. Therefore, this study reviewed 37 Social Science Citation Indexed (SSCI) articles on Problem/Project-Based CSCL practice in Computer Education from 2014 to 2023. The review focused on the research methods, educational technologies, interactions, student group formation, and research outcomes. The findings revealed that the most used research method was quantitative, followed by the mixed-method approach, and most participants were undergraduates, followed by junior high school students. Most studies used interactive technologies in implementing CSCL, followed by web-based tools, Learning Management Systems (LMS), social media, and video conferencing. Six types of interaction were found among instructor, learner, content, and interface. The most applied student group formation technique was group attributes selection with the random selection method, followed by students’ self-selection and teachers’ selection, while few studies used member attributes based on students’ personal traits and students’ knowledge/competence. Regarding research outcomes, the students’ perceptions were the most observed dependent variable, followed by students’ knowledge gain, students’ group task performance, and students’ social interaction. Surprisingly, students’ critical thinking, computational thinking, and problem-solving skills were found to be relatively understudied. Further suggestions for future studies in implementing and evaluating CSCL practices for researchers, educators, and policymakers in Computer Education settings are provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Explore related subjects

• Digital Education and Educational Technology

Data availability

My manuscript has data included as electronic supplementary material.

Abedi, E. A. (2024). “We [teachers] first require basic technical [skills] training”: Investigating formal professional development pathways and knowledge needs of teachers for technology integration. Education and Information Technologies, 29 (10), 11793–11814. https://doi.org/10.1007/s10639-023-12344-w

Article   Google Scholar  

Almulla, M. A. (2020). The Effectiveness of the Project-Based Learning (PBL) Approach as a Way to Engage Students in Learning. SAGE Open , 10 (3) https://doi.org/10.1177/2158244020938702

Andersen, R., Mørch, A. I., & Litherland, K. T. (2022). Collaborative learning with block-based programming: Investigating human-centered artificial intelligence in education. Behaviour and Information Technology, 41 (9), 1830–1847. https://doi.org/10.1080/0144929X.2022.2083981

Angeli, C., & Valanides, N. (2020). Developing young children’s computational thinking with educational robotics: An interaction effect between gender and scaffolding strategy. Computers in Human Behavior, 105 , 10595. https://doi.org/10.1016/j.chb.2019.03.018

Berland, M., Davis, D., & Smith, C. P. (2015). AMOEBA: Designing for collaboration in computer science classrooms through live learning analytics. International Journal of Computer-Supported Collaborative Learning, 10 (4), 425–447. https://doi.org/10.1007/s11412-015-9217-z

Blumentfeld, P. C., Soloway, E., Marx, R. W., Krajcik, J. S., Guzdial, M., & Palincsar, A. (2011). Motivating project-based learning: Sustaining the doing, supporting the learning. Journal Educational Psychologist, 26 (3–4), 369–398.

Google Scholar  

Brundiers, K., & Wiek, A. (2013). Do We Teach What We Preach? An International Comparison of Problem- and Project-Based Learning Courses in Sustainability. Sustainability, 5 , 1725–1746. https://doi.org/10.3390/SU5041725

Çakiroğlu, Ü., & Erdemir, T. (2019). Online project based learning via cloud computing: Exploring roles of instructor and students. Interactive Learning Environments, 27 (4), 547–566. https://doi.org/10.1080/10494820.2018.1489855

Celepkolu, M., Galdo, A. C., & Boyer, K. E. (2022). Fostering Balanced Contributions Among Children Through Dialogue Visualization. IEEE Transactions on Learning Technologies, 15 (4), 454–466. https://doi.org/10.1109/TLT.2022.3193648

Chang, S. C., & Wongwatkit, C. (2024). Effects of a peer assessment-based scrum project learning system on computer programming’s learning motivation, collaboration, communication, critical thinking, and cognitive load. Education and Information Technologies, 29 (6), 7105–7128. https://doi.org/10.1007/s10639-023-12084-x

Chen, C. M., & Kuo, C. H. (2019). An optimized group formation scheme to promote collaborative problem-based learning. Computers & Education, 133 , 94–115. https://doi.org/10.1016/J.COMPEDU.2019.01.011

Chen, J., Wang, M., Kirschner, P. A., & Tsai, C. C. (2018). The Role of Collaboration, Computer Use, Learning Environments, and Supporting Strategies in CSCL: A Meta-Analysis. Review of Educational Research, 88 (6), 799–843. https://doi.org/10.3102/0034654318791584

Cheng, Y. P., Shen, P. D., Hung, M. L., Tsai, C. W., Lin, C. H., & Hsu, L. C. (2022). Applying Online Content-Based Knowledge Awareness and Team Learning to Develop Students’ Programming Skills, Reduce their Anxiety, and Regulate Cognitive Load in a Cloud Classroom. Universal Access in the Information Society, 21 (2), 557–572. https://doi.org/10.1007/s10209-020-00789-6

Chorfi, A., Hedjazi, D., Aouag, S., & Boubiche, D. (2020). Problem-based collaborative learning groupware to improve computer programming skills. Behaviour and Information Technology, 41 (1), 1–20. https://doi.org/10.1080/0144929X.2020.1795263

Chugh, R., Turnbull, D., Cowling, M. A., Vanderburg, R., & Vanderburg, M. A. (2023). Implementing educational technology in Higher Education Institutions: A review of technologies, stakeholder perceptions, frameworks and metrics. Education and Information Technologies, 28 (12), 16403–16429. https://doi.org/10.1007/s10639-023-11846-x

Chung, C. Y., Awad, N., & Hsiao, H. (2021). Collaborative programming problem-solving in augmented reality: Multimodal analysis of effectiveness and group collaboration. Australasian Journal of Educational Technology, 37 (5), 17–31. https://doi.org/10.14742/ajet.7059

Connerley, M. L., & Mael, F. A. (2001). The importance and invasiveness of student team selection criteria. Journal of Management Education, 25 (5), 471–494. https://doi.org/10.1177/105256290102500502

Cress, U. (2020). The richness of CSCL environments. In International Journal of Computer-Supported Collaborative Learning (Vol. 15, Issue 4, pp. 383–388). Springer. https://doi.org/10.1007/s11412-020-09335-1

Cruz, W. M., & Isotani, S. (2014). Group formation algorithms in collaborative learning contexts: A systematic mapping of the literature. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , 8658 LNCS , 199–214. https://doi.org/10.1007/978-3-319-10166-8_18/COVER

De Santo, A., Farah, J. C., Martinez, M. L., Moro, A., Bergram, K., Purohit, A. K., Felber, P., Gillet, D., & Holzer, A. (2022). Promoting Computational Thinking Skills in Non-Computer-Science Students: Gamifying Computational Notebooks to Increase Student Engagement. IEEE Transactions on Learning Technologies, 15 (3), 392–405. https://doi.org/10.1109/TLT.2022.3180588

Donaldson, J. P., Han, A., Yan, S., Lee, S., & Kao, S. (2024). Learning experience network analysis for design-based research. Information and Learning Science, 125 (1–2), 22–43. https://doi.org/10.1108/ILS-03-2023-0026/FULL/PDF

Du, X., Dai, M., Tang, H., Hung, J. L., Li, H., & Zheng, J. (2023). A multimodal analysis of college students’ collaborative problem solving in virtual experimentation activities: A perspective of cognitive load. Journal of Computing in Higher Education, 35 (2), 272–295. https://doi.org/10.1007/s12528-022-09311-8

Durak, H. Y. (2024). Feedforward- or feedback-based group regulation guidance in collaborative groups. Journal of Computer Assisted Learning, 40 (2), 410–436. https://doi.org/10.1111/jcal.12887

Finlay, O., Mugford, A., Bredin, S., Scott, A., Taunton, J., & Warburton, D. (2020). Managing Change and the Related Performance Dimensions of Multiple-Discipline Science and Medical Services Departments in High Performance Sports Organizations: A Systematic Review. Research & Investigations in Sports Medicine, 7 (2), 608–619.

Francese, R., Gravino, C., Risi, M., Scanniello, G., & Tortora, G. (2015). Using Project-Based-Learning in a mobile application development course—An experience report. Journal of Visual Languages & Computing, 31 , 196–205. https://doi.org/10.1016/J.JVLC.2015.10.019

Gökçe, S., & Yenmez, A. A. (2023). Ingenuity of scratch programming on reflective thinking towards problem solving and computational thinking. Education and Information Technologies, 28 (5), 5493–5517. https://doi.org/10.1007/s10639-022-11385-x

Hashim, S., Masek, A., Zahir, N. Z. M., & Khamis, N. (2023). The Efficacy of Students‟ Knowledge Construction Process in Computer-Supported Collaborative Learning (CSCL) Environment: A Malaysian View. International Journal of Information and Education Technology, 13 (9), 1452–1461. https://doi.org/10.18178/IJIET.2023.13.9.1949

Hernández-Sellés, N., Muñoz-Carril, P. C., & González-Sanmamed, M. (2020). Interaction in computer supported collaborative learning: An analysis of the implementation phase. International Journal of Educational Technology in Higher Education, 17 (1), 23. https://doi.org/10.1186/s41239-020-00202-5

Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? In Educational Psychology Review (Vol. 16, Issue 3, pp. 235–266). Springer. https://doi.org/10.1023/B:EDPR.0000034022.16470.f3

Hoadley, C., & Campos, F. C. (2022). Design-based research: What it is and why it matters to studying online learning. Educational Psychologist, 57 (3), 207–220. https://doi.org/10.1080/00461520.2022.2079128

Hong, Q. N., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O’Cathain, A., Rousseau, M.-C., Vedel, I., & Pluye, P. (2018). The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Education for Information, 34 , 285–291. https://doi.org/10.3233/EFI-180221

Hou, H. Y., Agrawal, S., & Lee, C. F. (2020). Computational thinking training with technology for non-information undergraduates. Thinking Skills and Creativity, 38 , 100720. https://doi.org/10.1016/j.tsc.2020.100720

Howard, C., Di Eugenio, B., Jordan, P., & Katz, S. (2017). Exploring Initiative as a Signal of Knowledge Co-Construction During Collaborative Problem Solving. Cognitive Science, 41 (6), 1422–1449. https://doi.org/10.1111/cogs.12415

Hsia, L. H., & Sung, H. Y. (2020). Effects of a mobile technology-supported peer assessment approach on students’ learning motivation and perceptions in a college flipped dance class. International Journal of Mobile Learning and Organisation, 14 (1), 99–113. https://doi.org/10.1504/IJMLO.2020.103892

Hsu, T. C., Abelson, H., Patton, E., Chen, S. C., & Chang, H. N. (2021). Self-efficacy and behavior patterns of learners using a real-time collaboration system developed for group programming. International Journal of Computer-Supported Collaborative Learning, 16 (4), 559–582. https://doi.org/10.1007/s11412-021-09357-3

Hsu, T. C., Chang, S. C., & Hung, Y. T. (2018). How to learn and how to teach computational thinking: Suggestions based on a review of the literature. Computers and Education, 126 , 296–310. https://doi.org/10.1016/j.compedu.2018.07.004

Hu, C., Wong, A. F. L., Sharpe, L., Crawford, L., Gopinathan, S., Khine, M. S., & Moo, S. N. (2002). Building a learning community via videoconferencing. International Conference on Computers in Education, 2002. Proceedings. , 1 , 411–412. https://doi.org/10.1109/CIE.2002.1185960

Hu, L. (2024). Programming and 21st century skill development in K-12 schools: A multidimensional meta-analysis. Journal of Computer Assisted Learning, 40 (2), 610–636. https://doi.org/10.1111/jcal.12904

Huang, T. C., Jeng, Y. L., Hsiao, K. L., & Tsai, B. R. (2017). SNS collaborative learning design: Enhancing critical thinking for human–computer interface design. Universal Access in the Information Society, 16 (2), 303–312. https://doi.org/10.1007/s10209-016-0458-z

Järvelä, S., Kirschner, P. A., Panadero, E., Malmberg, J., Phielix, C., Jaspers, J., Koivuniemi, M., & Järvenoja, H. (2015). Enhancing socially shared regulation in collaborative learning groups: Designing for CSCL regulation tools. Educational Technology Research and Development, 63 (1), 125–142. https://doi.org/10.1007/s11423-014-9358-1

Jeong, H., Hmelo-Silver, C. E., & Jo, K. (2019). Ten years of Computer-Supported Collaborative Learning: A meta-analysis of CSCL in STEM education during 2005–2014. Educational Research Review, 28 , 100284. https://doi.org/10.1016/J.EDUREV.2019.100284

Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational Technology Research and Development, 48 (4), 63–85. https://doi.org/10.1007/BF02300500

Kim, J., Leftwich, A., & Castner, D. (2024). Beyond teaching computational thinking: Exploring kindergarten teachers’ computational thinking and computer science curriculum design considerations. Education and Information Technologies , 1–37. https://doi.org/10.1007/s10639-023-12406-z

Kwon, K., Ottenbreit-Leftwich, A. T., Brush, T. A., Jeon, M., & Yan, G. (2021). Integration of problem-based learning in elementary computer science education: Effects on computational thinking and attitudes. Educational Technology Research and Development, 69 (5), 2761–2787. https://doi.org/10.1007/S11423-021-10034-3

Lan, H., Yu, X., Wang, Z., Wang, P., Sun, Y., Wang, Z., Su, R., Wang, L., Zhao, J., Hu, Y., Wu, S., Ren, M., Yang, K., Liu, X., & Chen, Y. (2023). How about the evidence assessment tools used in education and management systematic reviews? Frontiers in Medicine, 10 , 1160289. https://doi.org/10.3389/FMED.2023.1160289/PDF

Lee, S. J., Francom, G. M., & Nuatomue, J. (2022). Computer science education and K-12 students’ computational thinking: A systematic review. International Journal of Educational Research, 114 , 102008. https://doi.org/10.1016/j.ijer.2022.102008

Li, K., Rollins, J., & Yan, E. (2018). Web of Science use in published research and review papers 1997–2017: A selective, dynamic, cross-domain, content-based analysis. Scientometrics, 115 (1), 1–20. https://doi.org/10.1007/s11192-017-2622-5

Li, W., Huang, J. Y., Liu, C. Y., Tseng, J. C. R., & Wang, S. P. (2023a). A study on the relationship between student’ learning engagements and higher-order thinking skills in programming learning. Thinking Skills and Creativity, 49 , 101369. https://doi.org/10.1016/j.tsc.2023.101369

Li, W., Liu, C., & Tseng, J. C. R. (2024). Development of a metacognitive regulation-based collaborative programming system and its effects on students’ learning achievements, computational thinking tendency and group metacognition. British Journal of Educational Technology, 55 (1), 318–339. https://doi.org/10.1111/bjet.13358

Li, W., Liu, C. Y., & Tseng, J. C. R. (2023b). Effects of the interaction between metacognition teaching and students’ learning achievement on students’ computational thinking, critical thinking, and metacognition in collaborative programming learning. Education and Information Technologies, 28 (10), 12919–12943. https://doi.org/10.1007/s10639-023-11671-2

Li, Y., Schoenfeld, A. H., DiSessa, A. A., Graesser, A. C., Benson, L. C., English, L. D., & Duschl, R. A. (2020). Computational Thinking Is More about Thinking than Computing. In Journal for STEM Education Research (Vol. 3, Issue 1, pp. 1–18). Springer Nature. https://doi.org/10.1007/s41979-020-00030-2

Li, Y., Xiao, Y., Wang, K., Zhang, N., Pang, Y., Wang, R., Qi, C., Yuan, Z., Xu, J., Nite, S. B., & Star, J. R. (2022). A systematic review of high impact empirical studies in STEM education. In International Journal of STEM Education (Vol. 9, Issue 1, pp. 1–18). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1186/s40594-022-00389-1

Lin, P. C., Hou, H. T., & Chang, K. E. (2022). The development of a collaborative problem solving environment that integrates a scaffolding mind tool and simulation-based learning: An analysis of learners’ performance and their cognitive process in discussion. Interactive Learning Environments, 30 (7), 1273–1290. https://doi.org/10.1080/10494820.2020.1719163

Lin, Y. L., & Wang, W. T. (2024). Enhancing students’ online collaborative PBL learning performance in the context of coauthoring-based technologies: A case of wiki technologies. Education and Information Technologies, 29 (2), 2303–2328. https://doi.org/10.1007/s10639-023-11907-1

Lin, Y. S., Chen, S. Y., Tsai, C. W., & Lai, Y. H. (2021). Exploring Computational Thinking Skills Training Through Augmented Reality and AIoT Learning. Frontiers in Psychology, 12 , 640115. https://doi.org/10.3389/fpsyg.2021.640115

Lin, Y. T., Wu, C. C., & Chiu, C. F. (2018). The use of wiki in teaching programming: Effects upon achievement, attitudes, and collaborative programming behaviors. International Journal of Distance Education Technologies, 16 (3), 18–45. https://doi.org/10.4018/IJDET.2018070102

Liu, C. Y., Li, W., Huang, J. Y., Lei, L. Y., & Zhang, P. R. (2023). Collaborative programming based on social shared regulation: An approach to improving students’ programming achievements and group metacognition. Journal of Computer Assisted Learning, 39 (5), 1714–1731. https://doi.org/10.1111/jcal.12828

Lu, J., Lajoie, S. P., & Wiseman, J. (2010). Scaffolding problem-based learning with CSCL tools. International Journal of Computer-Supported Collaborative Learning, 5 (3), 283–298. https://doi.org/10.1007/s11412-010-9092-6

Mahenthiran, S., & Rouse, P. J. (2000). The impact of group selection on student performance and satisfaction. International Journal of Educational Management, 14 (6), 255–264. https://doi.org/10.1108/09513540010348043/FULL/XML

Maqtary, N., Mohsen, A., & Bechkoum, K. (2019). Group Formation Techniques in Computer-Supported Collaborative Learning: A Systematic Literature Review. Technology, Knowledge and Learning, 24 (2), 169–190. https://doi.org/10.1007/s10758-017-9332-1

Montiel, H., & Gomez-Zermeño, M. G. (2021). Educational challenges for computational thinking in k–12 education: A systematic literature review of “scratch” as an innovative programming tool. Computers, 10 (6), 69. https://doi.org/10.3390/computers10060069

Muñoz-Carril, P. C., Hernández-Sellés, N., Fuentes-Abeledo, E. J., & González-Sanmamed, M. (2021). Factors influencing students’ perceived impact of learning and satisfaction in Computer Supported Collaborative Learning. Computers and Education, 174 , 104310. https://doi.org/10.1016/j.compedu.2021.104310

Nguyen, V., Dang, H. H., Do, N. K., & Tran, D. T. (2016). Enhancing team collaboration through integrating social interactions in a Web-based development environment. Computer Applications in Engineering Education, 24 (4), 529–545. https://doi.org/10.1002/cae.21729

Noroozi, O., Weinberger, A., Biemans, H. J. A., Mulder, M., & Chizari, M. (2013). Facilitating argumentative knowledge construction through a transactive discussion script in CSCL. Computers & Education, 61 (1), 59–76. https://doi.org/10.1016/J.COMPEDU.2012.08.013

Ong, Q. K. L., & Annamalai, N. (2024). Technological pedagogical content knowledge for twenty-first century learning skills: The game changer for teachers of industrial revolution 5.0. Education and Information Technologies, 29 (2), 1939–1980. https://doi.org/10.1007/s10639-023-11852-z

Ouyang, F., Chen, Z., Cheng, M., Tang, Z., & Su, C. Y. (2021). Exploring the effect of three scaffoldings on the collaborative problem-solving processes in China’s higher education. International Journal of Educational Technology in Higher Education, 18 (1), 1–22. https://doi.org/10.1186/s41239-021-00273-y

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372 . https://doi.org/10.1136/BMJ.N71

Pellas, N. (2014). Exploring interrelationships among high school students’ engagement factors in introductory programming courses via a 3D multi-user serious game created in open sim. Journal of Universal Computer Science, 20 (12), 1608–1628. https://doi.org/10.3217/jucs-020-12-1608

Peramunugamage, A., Ratnayake, U. W., & Karunanayaka, S. P. (2023). Systematic review on mobile collaborative learning for engineering education. Journal of Computers in Education, 10 (1), 83. https://doi.org/10.1007/S40692-022-00223-1

Perez-Aranda, J., Medina-Claros, S., & Urrestarazu-Capellán, R. (2023). Effects of a collaborative and gamified online learning methodology on class and test emotions. Education and Information Technologies, 29 (2), 1823–1855. https://doi.org/10.1007/S10639-023-11879-2

Reis, A. C. B., Barbalho, S. C. M., & Zanette, A. C. D. (2017). A bibliometric and classification study of Project-based Learning in Engineering Education. Production, 27 , e20162258. https://doi.org/10.1590/0103-6513.225816

Rodríguez-Abad, C., Fernández-De-la-iglesia, J. D. C., Martínez-Santos, A. E., & Rodríguez-González, R. (2021). A systematic review of augmented reality in health sciences: A guide to decision-making in higher education. International Journal of Environmental Research and Public Health, 18 (8), 4262. https://doi.org/10.3390/IJERPH18084262/S1

Ruiz-Gallardo, J. R., González-Geraldo, J. L., & Castaño, S. (2016). What are our students doing? Workload, time allocation and time management in PBL instruction. A case study in Science Education. Teaching and Teacher Education, 53 , 51–62. https://doi.org/10.1016/J.TATE.2015.10.005

Sadeghi, H., & Kardan, A. A. (2016). Toward effective group formation in computer-supported collaborative learning. Interactive Learning Environments, 24 (3), 382–395. https://doi.org/10.1080/10494820.2013.851090

Sankaranarayanan, S., Kandimalla, S. R., Bogart, C. A., Charles Murray, R., Hilton, M., Sakr, M. F., & Rose, C. P. (2022). Collaborative Programming for Work-Relevant Learning: Comparing Programming Practice With Example-Based Reflection for Student Learning and Transfer Task Performance. IEEE Transactions on Learning Technologies, 15 (5), 594–604. https://doi.org/10.1109/TLT.2022.3169121

Sentance, S., & Csizmadia, A. (2017). Computing in the curriculum: Challenges and strategies from a teacher’s perspective. Education and Information Technologies, 22 (2), 469–495. https://doi.org/10.1007/s10639-016-9482-0

Serrano-Cámara, L. M., Paredes-Velasco, M., Alcover, C. M., & Velazquez-Iturbide, J. Á. (2014). An evaluation of students’ motivation in computer-supported collaborative learning of programming concepts. Computers in Human Behavior, 31 (1), 499–508. https://doi.org/10.1016/j.chb.2013.04.030

Serrano-Cámara, L. M., Paredes-Velasco, M., Velázquez-Iturbide, J. Á., Alcover, C. M., & Castellanos, M. E. (2016). MoCAS: A Mobile Collaborative Tool for Learning Scope of Identifiers in Programming Courses. International Journal of Engineering Education, 32 (2), 969–981.

Silva, L., Mendes, A., Gomes, A., & Fortes, G. (2023). Fostering regulatory processes using computational scaffolding. In International Journal of Computer-Supported Collaborative Learning (Vol. 18, Issue 1). Springer US. https://doi.org/10.1007/s11412-023-09388-y

Silva, L., Mendes, A. J., & Gomes, A. (2020). Computer-supported collaborative learning in programming education: A systematic literature review. IEEE Global Engineering Education Conference, EDUCON , 2020 - April , 1086–1095. https://doi.org/10.1109/EDUCON45650.2020.9125237

Splichal, J. M., Oshima, J., & Oshima, R. (2018). Regulation of collaboration in project-based learning mediated by CSCL scripting reflection. Computers & Education, 125 , 132–145. https://doi.org/10.1016/J.COMPEDU.2018.06.003

Staddon, R. V. (2020). Bringing technology to the mature classroom: Age differences in use and attitudes. International Journal of Educational Technology in Higher Education, 17 (1), 1–20. https://doi.org/10.1186/s41239-020-00184-4

Su, A. Y. S., Huang, C. S. J., Yang, S. J. H., Ding, T. J., & Hsieh, Y. Z. (2015). Effects of annotations and homework on learning achievement: An empirical study of scratch programming pedagogy. Educational Technology and Society, 18 (4), 331–343.

Su, A. Y. S., Yang, S. J. H., Hwang, W. Y., Huang, C. S. J., & Tern, M. Y. (2014). Investigating the role of computer-supported annotation in problem-solving-based teaching: An empirical study of a Scratch programming pedagogy. British Journal of Educational Technology, 45 (4), 647–665. https://doi.org/10.1111/bjet.12058

Su, J., & Yang, W. (2023). A systematic review of integrating computational thinking in early childhood education. Computers and Education Open, 4 , 100122. https://doi.org/10.1016/j.caeo.2023.100122

Sun, C., Shute, V. J., Stewart, A., Yonehiro, J., Duran, N., & D’Mello, S. (2020). Towards a generalized competency model of collaborative problem solving. Computers and Education, 143 ,. https://doi.org/10.1016/j.compedu.2019.103672

Theodorio, A. O., Mataka, T. W., & Shambare, B. (2024). Teacher educators’ use of mind mapping in the development of TPACK in a technology-rich learning environment. Education and Information Technologies , 1–20. https://doi.org/10.1007/s10639-024-12587-1

Toledo, C. (2005). A Five-Stage Model of Computer Technology Integration Into Teacher Education Curriculum. Contemporary Issues in Technology and Teacher Education, 5 , 177–191.

Toledo, J. A. J., Collazos, C. A., Ortega Cantero, M., & Redondo, M. Á. (2019). Collaborative strategy with augmented reality for the development of algorithmic thinking. Communications in Computer and Information Science, 847 , 70–82. https://doi.org/10.1007/978-3-030-05270-6_6

Tomczyk, Ł, & Walker, C. (2021). The emergency (crisis) e-learning as a challenge for teachers in Poland. Education and Information Technologies, 26 (6), 6847–6877. https://doi.org/10.1007/s10639-021-10539-7

Tsompanoudi, D., Satratzemi, M., & Xinogalos, S. (2016). Evaluating the Effects of Scripted Distributed Pair Programming on Student Performance and Participation. IEEE Transactions on Education, 59 (1), 24–31. https://doi.org/10.1109/TE.2015.2419192

Unal, E., & Cakir, H. (2021). The effect of technology-supported collaborative problem solving method on students’ achievement and engagement. Education and Information Technologies, 26 (4), 4127–4150. https://doi.org/10.1007/s10639-021-10463-w

UNESCO 2012 The International Standard Classification of Education (ISCED) 2011 UNESCO Institute for Statistics https://doi.org/10.1007/BF02207511

Vandenberg, J., Zakaria, Z., Tsan, J., Iwanski, A., Lynch, C., Boyer, K. E., & Wiebe, E. (2021). Prompting collaborative and exploratory discourse: An epistemic network analysis study. International Journal of Computer-Supported Collaborative Learning, 16 (3), 339–366. https://doi.org/10.1007/s11412-021-09349-3

Vosinakis, S., Anastassakis, G., & Koutsabasis, P. (2018). Teaching and learning logic programming in virtual worlds using interactive microworld representations. British Journal of Educational Technology, 49 (1), 30–44. https://doi.org/10.1111/bjet.12531

Wagino, W., Maksum, H., Purwanto, W., Krismadinata, K., Suhendar, S., & Koto, R. D. (2023). Exploring the Full Potential of Collaborative Learning and E-Learning Environments in Universities: A Systematic Review. TEM Journal, 12 (3), 1772–1785. https://doi.org/10.18421/TEM123-60

Waite, J., Curzon, P., Marsh, W., & Sentance, S. (2020). Difficulties with design: The challenges of teaching design in K-5 programming. Computers and Education, 150 , 103838. https://doi.org/10.1016/j.compedu.2020.103838

Wang, C. Y., Gao, B. L., & Chen, S. J. (2024). The effects of metacognitive scaffolding of project-based learning environments on students’ metacognitive ability and computational thinking. Education and Information Technologies, 29 (5), 5485–5508. https://doi.org/10.1007/s10639-023-12022-x

Wang, X. M., & Hwang, G. J. (2017). A problem posing-based practicing strategy for facilitating students’ computer programming skills in the team-based learning mode. Educational Technology Research and Development, 65 (6), 1655–1671. https://doi.org/10.1007/s11423-017-9551-0

Wang, Y. (2023). The role of computer supported project-based learning in students’ computational thinking and engagement in robotics courses. Thinking Skills and Creativity, 48 , 101269. https://doi.org/10.1016/j.tsc.2023.101269

Weber, A. M., & Greiff, S. (2023). ICT Skills in the Deployment of 21st Century Skills: A (Cognitive) Developmental Perspective through Early Childhood. Applied Sciences, 13 (7), 4615. https://doi.org/10.3390/APP13074615

Wendt, J. L., & Rockinson-Szapkiw, A. J. (2015). The Effect of Online Collaboration on Adolescent Sense of Community in Eighth-Grade Physical Science. Journal of Science Education and Technology, 24 (5), 671–683. https://doi.org/10.1007/s10956-015-9556-6

Wijnia, L. (2016). The Problem With Problems in Problem-Based Learning: Difference Between Problem Explaining Versus Problem Solving. Health Professions Education, 2 (2), 59–60. https://doi.org/10.1016/J.HPE.2016.09.004

Wong, G. K. W. (2024). Amplifying children’s computational problem-solving skills: A hybrid-based design for programming education. Education and Information Technologies, 29 (2), 1761–1793. https://doi.org/10.1007/s10639-023-11880-9

Wu, B., Hu, Y., Ruis, A. R., & Wang, M. (2019). Analysing computational thinking in collaborative programming: A quantitative ethnography approach. Journal of Computer Assisted Learning, 35 (3), 421–434. https://doi.org/10.1111/jcal.12348

Xing, W., Wadholm, B., & Goggins, S. (2014). Learning analytics in CSCL with a focus on assessment: An exploratory study of activity theory-informed cluster analysis. ACM International Conference Proceeding Series , 59–67. https://doi.org/10.1145/2567574.2567587

Xinogalos, S., Satratzemi, M., Chatzigeorgiou, A., & Tsompanoudi, D. (2019). Factors Affecting Students’ Performance in Distributed Pair Programming. Journal of Educational Computing Research, 57 (2), 513–44. https://doi.org/10.1177/0735633117749432

Xu, W., Wu, Y., & Ouyang, F. (2023). Multimodal learning analytics of collaborative patterns during pair programming in higher education. International Journal of Educational Technology in Higher Education, 20 (1), 8. https://doi.org/10.1186/s41239-022-00377-z

Yağcı, M. (2018a). Web-Mediated Problem-Based Learning and Computer Programming: Effects of Study Approach on Academic Achievement and Attitude. Journal of Educational Computing Research, 56 (2), 272–292. https://doi.org/10.1177/0735633117706908

Article   MathSciNet   Google Scholar  

Yağcı, M. (2018b). Web-mediated problem-based learning and computer programming: Effects of thinking style on academic achievement and attitude. Computer Applications in Engineering Education, 26 (6), 2012–2025. https://doi.org/10.1002/cae.21964

Yang, X. (2023). A Historical Review of Collaborative Learning and Cooperative Learning. TechTrends, 67 (4), 718–28. https://doi.org/10.1007/s11528-022-00823-9

Yilmaz, K. (2013). Comparison of quantitative and qualitative research traditions: Epistemological, theoretical, and methodological differences. European Journal of Education, 48 (2), 311–25.

Zabolotna, K., Malmberg, J., & Järvenoja, H. (2023). Examining the interplay of knowledge construction and group-level regulation in a computer-supported collaborative learning physics task. Computers in Human Behavior, 138 , 107494. https://doi.org/10.1016/J.CHB.2022.107494

Zhu, C. (2013). The effect of cultural and school factors on the implementation of CSCL. British Journal of Educational Technology, 44 (3), 484–501. https://doi.org/10.1111/J.1467-8535.2012.01333.X

Download references

Acknowledgements

This study was funded by the National Science and Technology Council, Taiwan [NSTC 113-2410-H-011 -003 -MY3], the “Empower Vocational Education Research Center” of National Taiwan University of Science and Technology (NTUST) from the Featured Areas Research Center Program within the framework of the Higher Education Sprout Project by the Ministry of Education (MOE) in Taiwan, and State University of Malang in Indonesia.

Author information

Authors and affiliations.

Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology, No.43, Section 4 Keelung Rd, Da’an District, Taipei City, 106, Taiwan

Khoirudin Asfani & Hsiu-Ling Chen

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Hsiu-Ling Chen .

Ethics declarations

Conflicts of interests.

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Asfani, K., Chen, HL. Problem or project-based computer-supported collaborative learning practices in computer education: A systematic review of SSCI articles published from 2014 to 2023. Educ Inf Technol (2024). https://doi.org/10.1007/s10639-024-13125-9

Download citation

Received : 07 June 2024

Accepted : 17 October 2024

Published : 26 October 2024

DOI : https://doi.org/10.1007/s10639-024-13125-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Computer-Supported Collaborative Learning
• Problem-Based Learning
• Project-Based Learning
• Problem-Solving Tasks
• Computer Education
• Systematic Review
• Find a journal
• Publish with us
• Track your research

• Essay Editor

Trend Mapping of Traditional Ceremonial Studies: Systematic Literature Review

Key takeaway.

This article provides a comprehensive overview of trends in traditional ceremonial studies through a systematic literature review, mapping study themes, identifying author networks, and uncovering gaps and novel aspects within the field. The analysis is based on 200 articles sourced from Google Scholar and Crossref databases from 2012 to 2022.

This article presents a systematic literature review of trends in the study of traditional ceremonies, analyzing 200 scholarly articles from Google Scholar and Crossref databases spanning the years 2012 to 2022. The study employed the Publish or Perish application for metadata gathering and the VOSviewer application for data visualization. The outcomes reveal three main findings: 1) Mapping clusters of study themes related to traditional ceremonies, 2) Identifying the network of authors contributing to the field of traditional ceremonies, and 3) Uncovering gaps and novel aspects within traditional ceremony studies. The article highlights the evolving nature of traditional ceremony research, which encompasses multidisciplinary sciences such as religion, communication, sociology, psychology, and anthropology. However, the study's scope is limited to social science perspectives, focusing on rituals.

'If you don't “pass” as cis, you don't exist'. The trans audience's reproofs of 'Cis Gaze'and transnormativity in TV series

This study explores how trans audiences perceive their portrayal in TV series and how these depictions affect their everyday lives. The research highlights the need for more diverse narratives and the inclusion of trans people in the...

Time is of the essence: The importance of considering biological rhythms in an increasingly polluted world

The article highlights the importance of considering biological rhythms in chemical pollution research, as they play a crucial role in shaping organisms' biology and ecology. Integrating biological rhythmicity into this research can ...

The impact of generative artificial intelligence on socioeconomic inequalities and policy making

Generative AI has the potential to both exacerbate and alleviate existing socioeconomic inequalities, impacting areas such as work, education, and healthcare. However, it may also lead to challenges of increased misinformation and er...

Digital innovation in entrepreneurial firms: a systematic literature review

This systematic literature review investigates the intersection of digital innovation and entrepreneurial firms, identifying six key topics: start-ups' collaboration networks, business-model innovation, digital platforms, digital ven...

The Catalan Digital Republic. A Theoretical Review

The article examines digital nationalism in Catalonia, focusing on the Catalan Digital Republic, a concept promoted by activists and politicians. It explores how Catalan nationalism challenges previous studies on digital nationalism ...