importance of recommendation in research paper

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Research paper
• How to Write Recommendations in Research | Examples & Tips

How to Write Recommendations in Research | Examples & Tips

Published on September 15, 2022 by Tegan George . Revised on July 18, 2023.

Recommendations in research are a crucial component of your discussion section and the conclusion of your thesis , dissertation , or research paper .

As you conduct your research and analyze the data you collected , perhaps there are ideas or results that don’t quite fit the scope of your research topic. Or, maybe your results suggest that there are further implications of your results or the causal relationships between previously-studied variables than covered in extant research.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What should recommendations look like, building your research recommendation, how should your recommendations be written, recommendation in research example, other interesting articles, frequently asked questions about recommendations.

Recommendations for future research should be:

• Concrete and specific
• Supported with a clear rationale
• Directly connected to your research

Overall, strive to highlight ways other researchers can reproduce or replicate your results to draw further conclusions, and suggest different directions that future research can take, if applicable.

Relatedly, when making these recommendations, avoid:

• Undermining your own work, but rather offer suggestions on how future studies can build upon it
• Suggesting recommendations actually needed to complete your argument, but rather ensure that your research stands alone on its own merits
• Using recommendations as a place for self-criticism, but rather as a natural extension point for your work

Prevent plagiarism. Run a free check.

There are many different ways to frame recommendations, but the easiest is perhaps to follow the formula of research question   conclusion  recommendation. Here’s an example.

Conclusion An important condition for controlling many social skills is mastering language. If children have a better command of language, they can express themselves better and are better able to understand their peers. Opportunities to practice social skills are thus dependent on the development of language skills.

As a rule of thumb, try to limit yourself to only the most relevant future recommendations: ones that stem directly from your work. While you can have multiple recommendations for each research conclusion, it is also acceptable to have one recommendation that is connected to more than one conclusion.

These recommendations should be targeted at your audience, specifically toward peers or colleagues in your field that work on similar subjects to your paper or dissertation topic . They can flow directly from any limitations you found while conducting your work, offering concrete and actionable possibilities for how future research can build on anything that your own work was unable to address at the time of your writing.

See below for a full research recommendation example that you can use as a template to write your own.

Scribbr Citation Checker New

The AI-powered Citation Checker helps you avoid common mistakes such as:

• Missing commas and periods
• Incorrect usage of “et al.”
• Ampersands (&) in narrative citations
• Missing reference entries

If you want to know more about AI for academic writing, AI tools, or research bias, make sure to check out some of our other articles with explanations and examples or go directly to our tools!

Research bias

• Survivorship bias
• Self-serving bias
• Availability heuristic
• Halo effect
• Hindsight bias
• Deep learning
• Generative AI
• Machine learning
• Reinforcement learning
• Supervised vs. unsupervised learning

 (AI) Tools

• Grammar Checker
• Paraphrasing Tool
• Text Summarizer
• AI Detector
• Plagiarism Checker
• Citation Generator

While it may be tempting to present new arguments or evidence in your thesis or disseration conclusion , especially if you have a particularly striking argument you’d like to finish your analysis with, you shouldn’t. Theses and dissertations follow a more formal structure than this.

All your findings and arguments should be presented in the body of the text (more specifically in the discussion section and results section .) The conclusion is meant to summarize and reflect on the evidence and arguments you have already presented, not introduce new ones.

The conclusion of your thesis or dissertation should include the following:

• A restatement of your research question
• A summary of your key arguments and/or results
• A short discussion of the implications of your research

For a stronger dissertation conclusion , avoid including:

• Important evidence or analysis that wasn’t mentioned in the discussion section and results section
• Generic concluding phrases (e.g. “In conclusion …”)
• Weak statements that undermine your argument (e.g., “There are good points on both sides of this issue.”)

Your conclusion should leave the reader with a strong, decisive impression of your work.

In a thesis or dissertation, the discussion is an in-depth exploration of the results, going into detail about the meaning of your findings and citing relevant sources to put them in context.

The conclusion is more shorter and more general: it concisely answers your main research question and makes recommendations based on your overall findings.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

George, T. (2023, July 18). How to Write Recommendations in Research | Examples & Tips. Scribbr. Retrieved June 18, 2024, from https://www.scribbr.com/dissertation/recommendations-in-research/

Is this article helpful?

Tegan George

Other students also liked, how to write a discussion section | tips & examples, how to write a thesis or dissertation conclusion, how to write a results section | tips & examples, what is your plagiarism score.

Research Recommendations – Guiding policy-makers for evidence-based decision making

Research recommendations play a crucial role in guiding scholars and researchers toward fruitful avenues of exploration. In an era marked by rapid technological advancements and an ever-expanding knowledge base, refining the process of generating research recommendations becomes imperative.

But, what is a research recommendation?

Research recommendations are suggestions or advice provided to researchers to guide their study on a specific topic . They are typically given by experts in the field. Research recommendations are more action-oriented and provide specific guidance for decision-makers, unlike implications that are broader and focus on the broader significance and consequences of the research findings. However, both are crucial components of a research study.

Difference Between Research Recommendations and Implication

Although research recommendations and implications are distinct components of a research study, they are closely related. The differences between them are as follows:

Types of Research Recommendations

Recommendations in research can take various forms, which are as follows:

These recommendations aim to assist researchers in navigating the vast landscape of academic knowledge.

Let us dive deeper to know about its key components and the steps to write an impactful research recommendation.

Key Components of Research Recommendations

The key components of research recommendations include defining the research question or objective, specifying research methods, outlining data collection and analysis processes, presenting results and conclusions, addressing limitations, and suggesting areas for future research. Here are some characteristics of research recommendations:

Research recommendations offer various advantages and play a crucial role in ensuring that research findings contribute to positive outcomes in various fields. However, they also have few limitations which highlights the significance of a well-crafted research recommendation in offering the promised advantages.

The importance of research recommendations ranges in various fields, influencing policy-making, program development, product development, marketing strategies, medical practice, and scientific research. Their purpose is to transfer knowledge from researchers to practitioners, policymakers, or stakeholders, facilitating informed decision-making and improving outcomes in different domains.

How to Write Research Recommendations?

Research recommendations can be generated through various means, including algorithmic approaches, expert opinions, or collaborative filtering techniques. Here is a step-wise guide to build your understanding on the development of research recommendations.

1. Understand the Research Question:

Understand the research question and objectives before writing recommendations. Also, ensure that your recommendations are relevant and directly address the goals of the study.

2. Review Existing Literature:

Familiarize yourself with relevant existing literature to help you identify gaps , and offer informed recommendations that contribute to the existing body of research.

3. Consider Research Methods:

Evaluate the appropriateness of different research methods in addressing the research question. Also, consider the nature of the data, the study design, and the specific objectives.

4. Identify Data Collection Techniques:

Gather dataset from diverse authentic sources. Include information such as keywords, abstracts, authors, publication dates, and citation metrics to provide a rich foundation for analysis.

5. Propose Data Analysis Methods:

Suggest appropriate data analysis methods based on the type of data collected. Consider whether statistical analysis, qualitative analysis, or a mixed-methods approach is most suitable.

6. Consider Limitations and Ethical Considerations:

Acknowledge any limitations and potential ethical considerations of the study. Furthermore, address these limitations or mitigate ethical concerns to ensure responsible research.

7. Justify Recommendations:

Explain how your recommendation contributes to addressing the research question or objective. Provide a strong rationale to help researchers understand the importance of following your suggestions.

8. Summarize Recommendations:

Provide a concise summary at the end of the report to emphasize how following these recommendations will contribute to the overall success of the research project.

By following these steps, you can create research recommendations that are actionable and contribute meaningfully to the success of the research project.

Download now to unlock some tips to improve your journey of writing research recommendations.

Example of a Research Recommendation

Here is an example of a research recommendation based on a hypothetical research to improve your understanding.

Research Recommendation: Enhancing Student Learning through Integrated Learning Platforms

Background:

The research study investigated the impact of an integrated learning platform on student learning outcomes in high school mathematics classes. The findings revealed a statistically significant improvement in student performance and engagement when compared to traditional teaching methods.

Recommendation:

In light of the research findings, it is recommended that educational institutions consider adopting and integrating the identified learning platform into their mathematics curriculum. The following specific recommendations are provided:

• Implementation of the Integrated Learning Platform:

Schools are encouraged to adopt the integrated learning platform in mathematics classrooms, ensuring proper training for teachers on its effective utilization.

• Professional Development for Educators:

Develop and implement professional programs to train educators in the effective use of the integrated learning platform to address any challenges teachers may face during the transition.

• Monitoring and Evaluation:

Establish a monitoring and evaluation system to track the impact of the integrated learning platform on student performance over time.

• Resource Allocation:

Allocate sufficient resources, both financial and technical, to support the widespread implementation of the integrated learning platform.

By implementing these recommendations, educational institutions can harness the potential of the integrated learning platform and enhance student learning experiences and academic achievements in mathematics.

This example covers the components of a research recommendation, providing specific actions based on the research findings, identifying the target audience, and outlining practical steps for implementation.

Using AI in Research Recommendation Writing

Enhancing research recommendations is an ongoing endeavor that requires the integration of cutting-edge technologies, collaborative efforts, and ethical considerations. By embracing data-driven approaches and leveraging advanced technologies, the research community can create more effective and personalized recommendation systems. However, it is accompanied by several limitations. Therefore, it is essential to approach the use of AI in research with a critical mindset, and complement its capabilities with human expertise and judgment.

Here are some limitations of integrating AI in writing research recommendation and some ways on how to counter them.

1. Data Bias

AI systems rely heavily on data for training. If the training data is biased or incomplete, the AI model may produce biased results or recommendations.

How to tackle: Audit regularly the model’s performance to identify any discrepancies and adjust the training data and algorithms accordingly.

2. Lack of Understanding of Context:

AI models may struggle to understand the nuanced context of a particular research problem. They may misinterpret information, leading to inaccurate recommendations.

How to tackle: Use AI to characterize research articles and topics. Employ them to extract features like keywords, authorship patterns and content-based details.

3. Ethical Considerations:

AI models might stereotype certain concepts or generate recommendations that could have negative consequences for certain individuals or groups.

How to tackle: Incorporate user feedback mechanisms to reduce redundancies. Establish an ethics review process for AI models in research recommendation writing.

4. Lack of Creativity and Intuition:

AI may struggle with tasks that require a deep understanding of the underlying principles or the ability to think outside the box.

How to tackle: Hybrid approaches can be employed by integrating AI in data analysis and identifying patterns for accelerating the data interpretation process.

5. Interpretability:

Many AI models, especially complex deep learning models, lack transparency on how the model arrived at a particular recommendation.

How to tackle: Implement models like decision trees or linear models. Provide clear explanation of the model architecture, training process, and decision-making criteria.

6. Dynamic Nature of Research:

Research fields are dynamic, and new information is constantly emerging. AI models may struggle to keep up with the rapidly changing landscape and may not be able to adapt to new developments.

How to tackle: Establish a feedback loop for continuous improvement. Regularly update the recommendation system based on user feedback and emerging research trends.

The integration of AI in research recommendation writing holds great promise for advancing knowledge and streamlining the research process. However, navigating these concerns is pivotal in ensuring the responsible deployment of these technologies. Researchers need to understand the use of responsible use of AI in research and must be aware of the ethical considerations.

Exploring research recommendations plays a critical role in shaping the trajectory of scientific inquiry. It serves as a compass, guiding researchers toward more robust methodologies, collaborative endeavors, and innovative approaches. Embracing these suggestions not only enhances the quality of individual studies but also contributes to the collective advancement of human understanding.

Frequently Asked Questions

The purpose of recommendations in research is to provide practical and actionable suggestions based on the study's findings, guiding future actions, policies, or interventions in a specific field or context. Recommendations bridges the gap between research outcomes and their real-world application.

To make a research recommendation, analyze your findings, identify key insights, and propose specific, evidence-based actions. Include the relevance of the recommendations to the study's objectives and provide practical steps for implementation.

Begin a recommendation by succinctly summarizing the key findings of the research. Clearly state the purpose of the recommendation and its intended impact. Use a direct and actionable language to convey the suggested course of action.

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

• Diversity and Inclusion
• Industry News

6 Reasons Why There is a Decline in Higher Education Enrollment: Action plan to overcome this crisis

Over the past decade, colleges and universities across the globe have witnessed a concerning trend…

• Reporting Research

Academic Essay Writing Made Simple: 4 types and tips

The pen is mightier than the sword, they say, and nowhere is this more evident…

Ensuring Academic Integrity and Transparency in Academic Research: A comprehensive checklist for researchers

Academic integrity is the foundation upon which the credibility and value of scientific findings are…

• Publishing News

Unified AI Guidelines Crucial as Academic Writing Embraces Generative Tools

As generative artificial intelligence (AI) tools like ChatGPT are advancing at an accelerating pace, their…

How to Effectively Cite a PDF (APA, MLA, AMA, and Chicago Style)

The pressure to “publish or perish” is a well-known reality for academics, striking fear into…

How to Optimize Your Research Process: A step-by-step guide

Digital Citations: A comprehensive guide to citing of websites in APA, MLA, and CMOS…

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

What would be most effective in reducing research misconduct?

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Dissertation
• How to Write Recommendations in Research | Examples & Tips

How to Write Recommendations in Research | Examples & Tips

Published on 15 September 2022 by Tegan George .

Recommendations in research are a crucial component of your discussion section and the conclusion of your thesis , dissertation , or research paper .

As you conduct your research and analyse the data you collected , perhaps there are ideas or results that don’t quite fit the scope of your research topic . Or, maybe your results suggest that there are further implications of your results or the causal relationships between previously-studied variables than covered in extant research.

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

What should recommendations look like, building your research recommendation, how should your recommendations be written, recommendation in research example, frequently asked questions about recommendations.

Recommendations for future research should be:

• Concrete and specific
• Supported with a clear rationale
• Directly connected to your research

Overall, strive to highlight ways other researchers can reproduce or replicate your results to draw further conclusions, and suggest different directions that future research can take, if applicable.

Relatedly, when making these recommendations, avoid:

• Undermining your own work, but rather offer suggestions on how future studies can build upon it
• Suggesting recommendations actually needed to complete your argument, but rather ensure that your research stands alone on its own merits
• Using recommendations as a place for self-criticism, but rather as a natural extension point for your work

Prevent plagiarism, run a free check.

There are many different ways to frame recommendations, but the easiest is perhaps to follow the formula of research question   conclusion  recommendation. Here’s an example.

Conclusion An important condition for controlling many social skills is mastering language. If children have a better command of language, they can express themselves better and are better able to understand their peers. Opportunities to practice social skills are thus dependent on the development of language skills.

As a rule of thumb, try to limit yourself to only the most relevant future recommendations: ones that stem directly from your work. While you can have multiple recommendations for each research conclusion, it is also acceptable to have one recommendation that is connected to more than one conclusion.

These recommendations should be targeted at your audience, specifically toward peers or colleagues in your field that work on similar topics to yours. They can flow directly from any limitations you found while conducting your work, offering concrete and actionable possibilities for how future research can build on anything that your own work was unable to address at the time of your writing.

See below for a full research recommendation example that you can use as a template to write your own.

The current study can be interpreted as a first step in the research on COPD speech characteristics. However, the results of this study should be treated with caution due to the small sample size and the lack of details regarding the participants’ characteristics.

Future research could further examine the differences in speech characteristics between exacerbated COPD patients, stable COPD patients, and healthy controls. It could also contribute to a deeper understanding of the acoustic measurements suitable for e-health measurements.

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Correct my document today

While it may be tempting to present new arguments or evidence in your thesis or disseration conclusion , especially if you have a particularly striking argument you’d like to finish your analysis with, you shouldn’t. Theses and dissertations follow a more formal structure than this.

All your findings and arguments should be presented in the body of the text (more specifically in the discussion section and results section .) The conclusion is meant to summarize and reflect on the evidence and arguments you have already presented, not introduce new ones.

The conclusion of your thesis or dissertation should include the following:

• A restatement of your research question
• A summary of your key arguments and/or results
• A short discussion of the implications of your research

For a stronger dissertation conclusion , avoid including:

• Generic concluding phrases (e.g. “In conclusion…”)
• Weak statements that undermine your argument (e.g. “There are good points on both sides of this issue.”)

Your conclusion should leave the reader with a strong, decisive impression of your work.

In a thesis or dissertation, the discussion is an in-depth exploration of the results, going into detail about the meaning of your findings and citing relevant sources to put them in context.

The conclusion is more shorter and more general: it concisely answers your main research question and makes recommendations based on your overall findings.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

George, T. (2022, September 15). How to Write Recommendations in Research | Examples & Tips. Scribbr. Retrieved 18 June 2024, from https://www.scribbr.co.uk/thesis-dissertation/research-recommendations/

Is this article helpful?

Tegan George

Other students also liked, how to write a discussion section | tips & examples, how to write a thesis or dissertation conclusion, how to write a results section | tips & examples.

Implications or Recommendations in Research: What's the Difference?

• Peer Review

High-quality research articles that get many citations contain both implications and recommendations. Implications are the impact your research makes, whereas recommendations are specific actions that can then be taken based on your findings, such as for more research or for policymaking.

Updated on August 23, 2022

That seems clear enough, but the two are commonly confused.

This confusion is especially true if you come from a so-called high-context culture in which information is often implied based on the situation, as in many Asian cultures. High-context cultures are different from low-context cultures where information is more direct and explicit (as in North America and many European cultures).

Let's set these two straight in a low-context way; i.e., we'll be specific and direct! This is the best way to be in English academic writing because you're writing for the world.

Implications and recommendations in a research article

The standard format of STEM research articles is what's called IMRaD:

• Introduction
• Discussion/conclusions

Some journals call for a separate conclusions section, while others have the conclusions as the last part of the discussion. You'll write these four (or five) sections in the same sequence, though, no matter the journal.

The discussion section is typically where you restate your results and how well they confirmed your hypotheses. Give readers the answer to the questions for which they're looking to you for an answer.

At this point, many researchers assume their paper is finished. After all, aren't the results the most important part? As you might have guessed, no, you're not quite done yet.

The discussion/conclusions section is where to say what happened and what should now happen

The discussion/conclusions section of every good scientific article should contain the implications and recommendations.

The implications, first of all, are the impact your results have on your specific field. A high-impact, highly cited article will also broaden the scope here and provide implications to other fields. This is what makes research cross-disciplinary.

Recommendations, however, are suggestions to improve your field based on your results.

These two aspects help the reader understand your broader content: How and why your work is important to the world. They also tell the reader what can be changed in the future based on your results.

These aspects are what editors are looking for when selecting papers for peer review.

Implications and recommendations are, thus, written at the end of the discussion section, and before the concluding paragraph. They help to “wrap up” your paper. Once your reader understands what you found, the next logical step is what those results mean and what should come next.

Then they can take the baton, in the form of your work, and run with it. That gets you cited and extends your impact!

The order of implications and recommendations also matters. Both are written after you've summarized your main findings in the discussion section. Then, those results are interpreted based on ongoing work in the field. After this, the implications are stated, followed by the recommendations.

Writing an academic research paper is a bit like running a race. Finish strong, with your most important conclusion (recommendation) at the end. Leave readers with an understanding of your work's importance. Avoid generic, obvious phrases like "more research is needed to fully address this issue." Be specific.

The main differences between implications and recommendations (table)

Now let's dig a bit deeper into actually how to write these parts.

What are implications?

Research implications tell us how and why your results are important for the field at large. They help answer the question of “what does it mean?” Implications tell us how your work contributes to your field and what it adds to it. They're used when you want to tell your peers why your research is important for ongoing theory, practice, policymaking, and for future research.

Crucially, your implications must be evidence-based. This means they must be derived from the results in the paper.

Implications are written after you've summarized your main findings in the discussion section. They come before the recommendations and before the concluding paragraph. There is no specific section dedicated to implications. They must be integrated into your discussion so that the reader understands why the results are meaningful and what they add to the field.

A good strategy is to separate your implications into types. Implications can be social, political, technological, related to policies, or others, depending on your topic. The most frequently used types are theoretical and practical. Theoretical implications relate to how your findings connect to other theories or ideas in your field, while practical implications are related to what we can do with the results.

Key features of implications

• State the impact your research makes
• Helps us understand why your results are important
• Must be evidence-based
• Written in the discussion, before recommendations
• Can be theoretical, practical, or other (social, political, etc.)

Examples of implications

Let's take a look at some examples of research results below with their implications.

The result : one study found that learning items over time improves memory more than cramming material in a bunch of information at once .

The implications : This result suggests memory is better when studying is spread out over time, which could be due to memory consolidation processes.

The result : an intervention study found that mindfulness helps improve mental health if you have anxiety.

The implications : This result has implications for the role of executive functions on anxiety.

The result : a study found that musical learning helps language learning in children .

The implications : these findings suggest that language and music may work together to aid development.

What are recommendations?

As noted above, explaining how your results contribute to the real world is an important part of a successful article.

Likewise, stating how your findings can be used to improve something in future research is equally important. This brings us to the recommendations.

Research recommendations are suggestions and solutions you give for certain situations based on your results. Once the reader understands what your results mean with the implications, the next question they need to know is "what's next?"

Recommendations are calls to action on ways certain things in the field can be improved in the future based on your results. Recommendations are used when you want to convey that something different should be done based on what your analyses revealed.

Similar to implications, recommendations are also evidence-based. This means that your recommendations to the field must be drawn directly from your results.

The goal of the recommendations is to make clear, specific, and realistic suggestions to future researchers before they conduct a similar experiment. No matter what area your research is in, there will always be further research to do. Try to think about what would be helpful for other researchers to know before starting their work.

Recommendations are also written in the discussion section. They come after the implications and before the concluding paragraphs. Similar to the implications, there is usually no specific section dedicated to the recommendations. However, depending on how many solutions you want to suggest to the field, they may be written as a subsection.

Key features of recommendations

• Statements about what can be done differently in the field based on your findings
• Must be realistic and specific
• Written in the discussion, after implications and before conclusions
• Related to both your field and, preferably, a wider context to the research

Examples of recommendations

Here are some research results and their recommendations.

A meta-analysis found that actively recalling material from your memory is better than simply re-reading it .

• The recommendation: Based on these findings, teachers and other educators should encourage students to practice active recall strategies.

A medical intervention found that daily exercise helps prevent cardiovascular disease .

• The recommendation: Based on these results, physicians are recommended to encourage patients to exercise and walk regularly. Also recommended is to encourage more walking through public health offices in communities.

A study found that many research articles do not contain the sample sizes needed to statistically confirm their findings .

The recommendation: To improve the current state of the field, researchers should consider doing power analysis based on their experiment's design.

What else is important about implications and recommendations?

When writing recommendations and implications, be careful not to overstate the impact of your results. It can be tempting for researchers to inflate the importance of their findings and make grandiose statements about what their work means.

Remember that implications and recommendations must be coming directly from your results. Therefore, they must be straightforward, realistic, and plausible.

Another good thing to remember is to make sure the implications and recommendations are stated clearly and separately. Do not attach them to the endings of other paragraphs just to add them in. Use similar example phrases as those listed in the table when starting your sentences to clearly indicate when it's an implication and when it's a recommendation.

When your peers, or brand-new readers, read your paper, they shouldn't have to hunt through your discussion to find the implications and recommendations. They should be clear, visible, and understandable on their own.

That'll get you cited more, and you'll make a greater contribution to your area of science while extending the life and impact of your work.

The AJE Team

See our "Privacy Policy"

Get science-backed answers as you write with Paperpal's Research feature

How to Write a Conclusion for Research Papers (with Examples)

The conclusion of a research paper is a crucial section that plays a significant role in the overall impact and effectiveness of your research paper. However, this is also the section that typically receives less attention compared to the introduction and the body of the paper. The conclusion serves to provide a concise summary of the key findings, their significance, their implications, and a sense of closure to the study. Discussing how can the findings be applied in real-world scenarios or inform policy, practice, or decision-making is especially valuable to practitioners and policymakers. The research paper conclusion also provides researchers with clear insights and valuable information for their own work, which they can then build on and contribute to the advancement of knowledge in the field.

The research paper conclusion should explain the significance of your findings within the broader context of your field. It restates how your results contribute to the existing body of knowledge and whether they confirm or challenge existing theories or hypotheses. Also, by identifying unanswered questions or areas requiring further investigation, your awareness of the broader research landscape can be demonstrated.

Remember to tailor the research paper conclusion to the specific needs and interests of your intended audience, which may include researchers, practitioners, policymakers, or a combination of these.

Table of Contents

What is a conclusion in a research paper, summarizing conclusion, editorial conclusion, externalizing conclusion, importance of a good research paper conclusion, how to write a conclusion for your research paper, research paper conclusion examples.

• How to write a research paper conclusion with Paperpal? 

Frequently Asked Questions

A conclusion in a research paper is the final section where you summarize and wrap up your research, presenting the key findings and insights derived from your study. The research paper conclusion is not the place to introduce new information or data that was not discussed in the main body of the paper. When working on how to conclude a research paper, remember to stick to summarizing and interpreting existing content. The research paper conclusion serves the following purposes: 1

• Warn readers of the possible consequences of not attending to the problem.
• Recommend specific course(s) of action.
• Restate key ideas to drive home the ultimate point of your research paper.
• Provide a “take-home” message that you want the readers to remember about your study.

Types of conclusions for research papers

In research papers, the conclusion provides closure to the reader. The type of research paper conclusion you choose depends on the nature of your study, your goals, and your target audience. I provide you with three common types of conclusions:

A summarizing conclusion is the most common type of conclusion in research papers. It involves summarizing the main points, reiterating the research question, and restating the significance of the findings. This common type of research paper conclusion is used across different disciplines.

An editorial conclusion is less common but can be used in research papers that are focused on proposing or advocating for a particular viewpoint or policy. It involves presenting a strong editorial or opinion based on the research findings and offering recommendations or calls to action.

An externalizing conclusion is a type of conclusion that extends the research beyond the scope of the paper by suggesting potential future research directions or discussing the broader implications of the findings. This type of conclusion is often used in more theoretical or exploratory research papers.

Align your conclusion’s tone with the rest of your research paper. Start Writing with Paperpal Now!  

The conclusion in a research paper serves several important purposes:

• Offers Implications and Recommendations : Your research paper conclusion is an excellent place to discuss the broader implications of your research and suggest potential areas for further study. It’s also an opportunity to offer practical recommendations based on your findings.
• Provides Closure : A good research paper conclusion provides a sense of closure to your paper. It should leave the reader with a feeling that they have reached the end of a well-structured and thought-provoking research project.
• Leaves a Lasting Impression : Writing a well-crafted research paper conclusion leaves a lasting impression on your readers. It’s your final opportunity to leave them with a new idea, a call to action, or a memorable quote.

Writing a strong conclusion for your research paper is essential to leave a lasting impression on your readers. Here’s a step-by-step process to help you create and know what to put in the conclusion of a research paper: 2

• Research Statement : Begin your research paper conclusion by restating your research statement. This reminds the reader of the main point you’ve been trying to prove throughout your paper. Keep it concise and clear.
• Key Points : Summarize the main arguments and key points you’ve made in your paper. Avoid introducing new information in the research paper conclusion. Instead, provide a concise overview of what you’ve discussed in the body of your paper.
• Address the Research Questions : If your research paper is based on specific research questions or hypotheses, briefly address whether you’ve answered them or achieved your research goals. Discuss the significance of your findings in this context.
• Significance : Highlight the importance of your research and its relevance in the broader context. Explain why your findings matter and how they contribute to the existing knowledge in your field.
• Implications : Explore the practical or theoretical implications of your research. How might your findings impact future research, policy, or real-world applications? Consider the “so what?” question.
• Future Research : Offer suggestions for future research in your area. What questions or aspects remain unanswered or warrant further investigation? This shows that your work opens the door for future exploration.
• Closing Thought : Conclude your research paper conclusion with a thought-provoking or memorable statement. This can leave a lasting impression on your readers and wrap up your paper effectively. Avoid introducing new information or arguments here.
• Proofread and Revise : Carefully proofread your conclusion for grammar, spelling, and clarity. Ensure that your ideas flow smoothly and that your conclusion is coherent and well-structured.

Write your research paper conclusion 2x faster with Paperpal. Try it now!

Remember that a well-crafted research paper conclusion is a reflection of the strength of your research and your ability to communicate its significance effectively. It should leave a lasting impression on your readers and tie together all the threads of your paper. Now you know how to start the conclusion of a research paper and what elements to include to make it impactful, let’s look at a research paper conclusion sample.

How to write a research paper conclusion with Paperpal?

A research paper conclusion is not just a summary of your study, but a synthesis of the key findings that ties the research together and places it in a broader context. A research paper conclusion should be concise, typically around one paragraph in length. However, some complex topics may require a longer conclusion to ensure the reader is left with a clear understanding of the study’s significance. Paperpal, an AI writing assistant trusted by over 800,000 academics globally, can help you write a well-structured conclusion for your research paper. 

• Sign Up or Log In: Create a new Paperpal account or login with your details.  
• Navigate to Features : Once logged in, head over to the features’ side navigation pane. Click on Templates and you’ll find a suite of generative AI features to help you write better, faster.  
• Generate an outline: Under Templates, select ‘Outlines’. Choose ‘Research article’ as your document type.  
• Select your section: Since you’re focusing on the conclusion, select this section when prompted.  
• Choose your field of study: Identifying your field of study allows Paperpal to provide more targeted suggestions, ensuring the relevance of your conclusion to your specific area of research. 
• Provide a brief description of your study: Enter details about your research topic and findings. This information helps Paperpal generate a tailored outline that aligns with your paper’s content. 
• Generate the conclusion outline: After entering all necessary details, click on ‘generate’. Paperpal will then create a structured outline for your conclusion, to help you start writing and build upon the outline.  
• Write your conclusion: Use the generated outline to build your conclusion. The outline serves as a guide, ensuring you cover all critical aspects of a strong conclusion, from summarizing key findings to highlighting the research’s implications. 
• Refine and enhance: Paperpal’s ‘Make Academic’ feature can be particularly useful in the final stages. Select any paragraph of your conclusion and use this feature to elevate the academic tone, ensuring your writing is aligned to the academic journal standards. 

By following these steps, Paperpal not only simplifies the process of writing a research paper conclusion but also ensures it is impactful, concise, and aligned with academic standards. Sign up with Paperpal today and write your research paper conclusion 2x faster .  

The research paper conclusion is a crucial part of your paper as it provides the final opportunity to leave a strong impression on your readers. In the research paper conclusion, summarize the main points of your research paper by restating your research statement, highlighting the most important findings, addressing the research questions or objectives, explaining the broader context of the study, discussing the significance of your findings, providing recommendations if applicable, and emphasizing the takeaway message. The main purpose of the conclusion is to remind the reader of the main point or argument of your paper and to provide a clear and concise summary of the key findings and their implications. All these elements should feature on your list of what to put in the conclusion of a research paper to create a strong final statement for your work.

A strong conclusion is a critical component of a research paper, as it provides an opportunity to wrap up your arguments, reiterate your main points, and leave a lasting impression on your readers. Here are the key elements of a strong research paper conclusion: 1. Conciseness : A research paper conclusion should be concise and to the point. It should not introduce new information or ideas that were not discussed in the body of the paper. 2. Summarization : The research paper conclusion should be comprehensive enough to give the reader a clear understanding of the research’s main contributions. 3 . Relevance : Ensure that the information included in the research paper conclusion is directly relevant to the research paper’s main topic and objectives; avoid unnecessary details. 4 . Connection to the Introduction : A well-structured research paper conclusion often revisits the key points made in the introduction and shows how the research has addressed the initial questions or objectives. 5. Emphasis : Highlight the significance and implications of your research. Why is your study important? What are the broader implications or applications of your findings? 6 . Call to Action : Include a call to action or a recommendation for future research or action based on your findings.

The length of a research paper conclusion can vary depending on several factors, including the overall length of the paper, the complexity of the research, and the specific journal requirements. While there is no strict rule for the length of a conclusion, but it’s generally advisable to keep it relatively short. A typical research paper conclusion might be around 5-10% of the paper’s total length. For example, if your paper is 10 pages long, the conclusion might be roughly half a page to one page in length.

In general, you do not need to include citations in the research paper conclusion. Citations are typically reserved for the body of the paper to support your arguments and provide evidence for your claims. However, there may be some exceptions to this rule: 1. If you are drawing a direct quote or paraphrasing a specific source in your research paper conclusion, you should include a citation to give proper credit to the original author. 2. If your conclusion refers to or discusses specific research, data, or sources that are crucial to the overall argument, citations can be included to reinforce your conclusion’s validity.

The conclusion of a research paper serves several important purposes: 1. Summarize the Key Points 2. Reinforce the Main Argument 3. Provide Closure 4. Offer Insights or Implications 5. Engage the Reader. 6. Reflect on Limitations

Remember that the primary purpose of the research paper conclusion is to leave a lasting impression on the reader, reinforcing the key points and providing closure to your research. It’s often the last part of the paper that the reader will see, so it should be strong and well-crafted.

• Makar, G., Foltz, C., Lendner, M., & Vaccaro, A. R. (2018). How to write effective discussion and conclusion sections. Clinical spine surgery, 31(8), 345-346.
• Bunton, D. (2005). The structure of PhD conclusion chapters.  Journal of English for academic purposes ,  4 (3), 207-224.

Paperpal is a comprehensive AI writing toolkit that helps students and researchers achieve 2x the writing in half the time. It leverages 21+ years of STM experience and insights from millions of research articles to provide in-depth academic writing, language editing, and submission readiness support to help you write better, faster.  

Get accurate academic translations, rewriting support, grammar checks, vocabulary suggestions, and generative AI assistance that delivers human precision at machine speed. Try for free or upgrade to Paperpal Prime starting at US$19 a month to access premium features, including consistency, plagiarism, and 30+ submission readiness checks to help you succeed.  

Experience the future of academic writing – Sign up to Paperpal and start writing for free!  

Related Reads:

• 5 Reasons for Rejection After Peer Review
• Ethical Research Practices For Research with Human Subjects

7 Ways to Improve Your Academic Writing Process

• Paraphrasing in Academic Writing: Answering Top Author Queries

Preflight For Editorial Desk: The Perfect Hybrid (AI + Human) Assistance Against Compromised Manuscripts

You may also like, how to write the first draft of a..., mla works cited page: format, template & examples, how to write a high-quality conference paper, academic editing: how to self-edit academic text with..., measuring academic success: definition & strategies for excellence, phd qualifying exam: tips for success , ai in education: it’s time to change the..., is it ethical to use ai-generated abstracts without..., what are journal guidelines on using generative ai..., quillbot review: features, pricing, and free alternatives.

How to write recommendations in a research paper

Many students put in a lot of effort and write a good report however they are not able to give proper recommendations. Recommendations in the research paper should be included in your research. As a researcher, you display a deep understanding of the topic of research. Therefore you should be able to give recommendations. Here are a few tips that will help you to give appropriate recommendations. 

Recommendations in the research paper should be the objective of the research. Therefore at least one of your objectives of the paper is to provide recommendations to the parties associated or the parties that will benefit from your research. For example, to encourage higher employee engagement HR department should make strategies that invest in the well-being of employees. Additionally, the HR department should also collect regular feedback through online surveys.

Recommendations in the research paper should come from your review and analysis For example It was observed that coaches interviewed were associated with the club were working with the club from the past 2-3 years only. This shows that the attrition rate of coaches is high and therefore clubs should work on reducing the turnover of coaches.

Recommendations in the research paper should also come from the data you have analysed. For example, the research found that people over 65 years of age are at greater risk of social isolation. Therefore, it is recommended that policies that are made for combating social isolation should target this specific group.

Recommendations in the research paper should also come from observation. For example, it is observed that Lenovo’s income is stable and gross revenue has displayed a negative turn. Therefore the company should analyse its marketing and branding strategy.

Recommendations in the research paper should be written in the order of priority. The most important recommendations for decision-makers should come first. However, if the recommendations are of equal importance then it should come in the sequence in which the topic is approached in the research. 

Recommendations in a research paper if associated with different categories then you should categorize them. For example, you have separate recommendations for policymakers, educators, and administrators then you can categorize the recommendations. 

Recommendations in the research paper should come purely from your research. For example, you have written research on the impact on HR strategies on motivation. However, nowhere you have discussed Reward and recognition. Then you should not give recommendations for using rewards and recognition measures to boost employee motivation.

The use of bullet points offers better clarity rather than using long paragraphs. For example this paragraph “ It is recommended  that Britannia Biscuit should launch and promote sugar-free options apart from the existing product range. Promotion efforts should be directed at creating a fresh and healthy image. A campaign that conveys a sense of health and vitality to the consumer while enjoying biscuit  is recommended” can be written as:

• The company should launch and promote sugar-free options
• The company should work towards creating s fresh and healthy image
• The company should run a campaign to convey its healthy image

The inclusion of an action plan along with recommendation adds more weightage to your recommendation. Recommendations should be clear and conscience and written using actionable words. Recommendations should display a solution-oriented approach and in some cases should highlight the scope for further research. 

Educational resources and simple solutions for your research journey

Do Research Paper Recommendations Work? [With Research Paper Example]

Most content-based websites contain recommendations. Whether you are using Amazon, Netflix, Spotify, or Google Scholar, most sites provide some kind of link to other recommended content. How effective are these links? And can we compare recommender tools from one site to another? Is it valuable for a publisher to add links with research paper recommendations?

One thing to make clear immediately is that academic research paper recommendation tools work in a very different way to other kinds of recommendation. For an e-commerce site, recommendations are typically calculated by tracking what people bought. If you buy an inkjet printer, the chances are you will want to buy printer ink next, so it makes sense to provide a recommender tool based on what users bought after buying the item searched for.

But this kind of recommendation is of little use for academic searching. Firstly, academics aren’t buying anything; we don’t know if the article links they found were useful or not. You could argue that citations from other articles are the recommendations section of a research paper, and is some measure of success: if several academics cited this article, it must be important. But that isn’t necessarily the criterion of relevance for academics: what a researcher frequently wants to know is if anyone has investigated (say) the effect of aspirin on diabetes patients, whether or not their work has yet been cited. The UNSILO research paper recommendations algorithm does not make any use of citations; it is completely content-based, without any usage tracking.

A simple contrast between recommendations is to compare how the recommender tool behaves when no relevant link is found. A tool such as Netflix is designed to provide the user with something new to watch, even if you search for a title Netflix doesn’t offer (Netflix never returns zero results from a search). In contrast, academics are not interested in finding hits that are not relevant; they want only research paper recommendations that work. Unfortunately, even Google Scholar returns hits if a meaningless combination of terms is added (if you’re looking for research paper recommendations for “aspirin metallurgy volcanoes trainers” on Google Scholar, you’ll surely get 30+ results). Despite its title, Google Scholar is fundamentally a string-matching tool just like its bigger parent, and it isn’t difficult to see its limitations.

The introduction of AI-based tools has transformed the provision of research paper recommendations, moving from string search to concept-based search. Let’s say I am interested in articles on diabetes and aspirin. However much of an expert I might be, I certainly don’t know how the researchers phrased their title: is it “diabetes” or “diabetic”, or is diabetes referred to more elliptically, as “impaired glucose tolerance”? A concept-based tool such as UNSILO, which is not reliant on a few keywords for each article, can find all the main variants of phrases and ensure a search produces comprehensive research paper recommendations.

If we do succeed in providing relevant research paper recommendations, how can we measure its success or otherwise? Major surveys of how academics search for and find content (e.g.,  How Readers Discover Content , a long-term study updated every three years) show that academics use search and discovery very extensively, despite (we may assume) being subject-matter experts in their chosen domain. In other words, checking citations or recommendations section of research papers, listening to presentations, and reading articles still leaves gaps for an academic, gaps that that they fill using search. The details of how they search may vary, but it’s clear that most if not all academics are involved in searching for new content as part of their academic work.

Perhaps the most common metric used by publishers is the click-through rate (CTR), the number of times a user on the page clicks on one of the links provided. Click-through rate here means the number of times people who open a content page such as this click on one of the links on the right. This simple measure enables different research paper recommendation tools to be compared, but also gives an idea of how widely used these tools are. Has anyone measured the use of these links? Yes, a 2014 research paper showed that ratings by humans correlated quite closely with CTR (perhaps not surprising):

Ratings in the user study correlated strongly with CTR. This indicates that explicit user satisfaction (ratings) is a good approximation of the acceptance rate of recommendations (CTR), and vice versa.  ( Beel and Langer, 2014 )

In other words, researchers only click on research paper recommendation links they judge to be most relevant. Interestingly, the same paper found that providing links via what they termed “inferred ground-truths” (such as citations) are generally flawed and far less valuable for evaluating research paper recommendation systems.

What kind of click-through rates are found in practice? From feedback via publishers, we hear that a typical research paper recommendation system may achieve at best a 6.5% CTR. Although this figure may seem low, anyone familiar with actual usage of websites will know that the statistics for interaction with a site beyond the first search are always very low. The click-through rate for display adverts on a webpage are typically 0.1% – and even that figure often comprises mainly bots, automated systems that click on all the links on a site on a regular basis.

Can click-through rates be compared? An article in Forbes ( Fou, 2020 ) points out that every website will have a different result for CTR, and the results are not comparable – the time a user spends on a hotel booking site will be much greater than the time taken by an academic to find one scholarly article. So comparisons are only meaningful within the same site, or with other websites that have a similar function.

Another metric of site engagement is page views per session. For instance, an average pages per session of 2 means that each user to the website visited two pages before leaving. While this metric will vary widely from one site to another, it is a good way of identifying trends over time on a single website, or of comparing two websites with a similar function (such as two academic publisher sites). Using a research paper recommendation tool typically results in a clear increase in pages per session. A typical value when using a research paper recommendation tool will be above 2.

Given the differences between sites, it would be invidious to single out specific products, but publishers report to us widely differing results when comparing research paper recommendation tools. There is a very clear difference between recommenders – variations of 2x or even 3x the usage of one tool compared with another are common. We are happy to provide access to UNSILO research paper recommendation tools for evaluation purposes, and we strongly recommend trialling two or more products in this way, ideally via an A/B test lasting over at least a month.

One conclusion is very clear: research paper recommendation tools work. Any recommender is good, but concept-based tools seem to provide a good experience for researchers that they use again and again. Recommended links are a vital part of the academic workflow, and they significantly increase engagement. After all, any publisher who misses out on a significant enhancement in site usage and benefit to users is not making the most of their content.

Related Posts

How Publishers Can Enhance Reader Engagement with R Discovery’s Article Recommendation System

Turabian Format: A Beginner’s Guide

• Privacy Policy

Home » References in Research – Types, Examples and Writing Guide

References in Research – Types, Examples and Writing Guide

Table of Contents

References in Research

Definition:

References in research are a list of sources that a researcher has consulted or cited while conducting their study. They are an essential component of any academic work, including research papers, theses, dissertations, and other scholarly publications.

Types of References

There are several types of references used in research, and the type of reference depends on the source of information being cited. The most common types of references include:

References to books typically include the author’s name, title of the book, publisher, publication date, and place of publication.

Example: Smith, J. (2018). The Art of Writing. Penguin Books.

Journal Articles

References to journal articles usually include the author’s name, title of the article, name of the journal, volume and issue number, page numbers, and publication date.

Example: Johnson, T. (2021). The Impact of Social Media on Mental Health. Journal of Psychology, 32(4), 87-94.

Web sources

References to web sources should include the author or organization responsible for the content, the title of the page, the URL, and the date accessed.

Example: World Health Organization. (2020). Coronavirus disease (COVID-19) advice for the public. Retrieved from https://www.who.int/emergencies/disease/novel-coronavirus-2019/advice-for-public

Conference Proceedings

References to conference proceedings should include the author’s name, title of the paper, name of the conference, location of the conference, date of the conference, and page numbers.

Example: Chen, S., & Li, J. (2019). The Future of AI in Education. Proceedings of the International Conference on Educational Technology, Beijing, China, July 15-17, pp. 67-78.

References to reports typically include the author or organization responsible for the report, title of the report, publication date, and publisher.

Example: United Nations. (2020). The Sustainable Development Goals Report. United Nations.

Formats of References

Some common Formates of References with their examples are as follows:

APA (American Psychological Association) Style

The APA (American Psychological Association) Style has specific guidelines for formatting references used in academic papers, articles, and books. Here are the different reference formats in APA style with examples:

Author, A. A. (Year of publication). Title of book. Publisher.

Example : Smith, J. K. (2005). The psychology of social interaction. Wiley-Blackwell.

Journal Article

Author, A. A., Author, B. B., & Author, C. C. (Year of publication). Title of article. Title of Journal, volume number(issue number), page numbers.

Example : Brown, L. M., Keating, J. G., & Jones, S. M. (2012). The role of social support in coping with stress among African American adolescents. Journal of Research on Adolescence, 22(1), 218-233.

Author, A. A. (Year of publication or last update). Title of page. Website name. URL.

Example : Centers for Disease Control and Prevention. (2020, December 11). COVID-19: How to protect yourself and others. https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html

Magazine article

Author, A. A. (Year, Month Day of publication). Title of article. Title of Magazine, volume number(issue number), page numbers.

Example : Smith, M. (2019, March 11). The power of positive thinking. Psychology Today, 52(3), 60-65.

Newspaper article:

Author, A. A. (Year, Month Day of publication). Title of article. Title of Newspaper, page numbers.

Example: Johnson, B. (2021, February 15). New study shows benefits of exercise on mental health. The New York Times, A8.

Edited book

Editor, E. E. (Ed.). (Year of publication). Title of book. Publisher.

Example : Thompson, J. P. (Ed.). (2014). Social work in the 21st century. Sage Publications.

Chapter in an edited book:

Author, A. A. (Year of publication). Title of chapter. In E. E. Editor (Ed.), Title of book (pp. page numbers). Publisher.

Example : Johnson, K. S. (2018). The future of social work: Challenges and opportunities. In J. P. Thompson (Ed.), Social work in the 21st century (pp. 105-118). Sage Publications.

MLA (Modern Language Association) Style

The MLA (Modern Language Association) Style is a widely used style for writing academic papers and essays in the humanities. Here are the different reference formats in MLA style:

Author’s Last name, First name. Title of Book. Publisher, Publication year.

Example : Smith, John. The Psychology of Social Interaction. Wiley-Blackwell, 2005.

Journal article

Author’s Last name, First name. “Title of Article.” Title of Journal, volume number, issue number, Publication year, page numbers.

Example : Brown, Laura M., et al. “The Role of Social Support in Coping with Stress among African American Adolescents.” Journal of Research on Adolescence, vol. 22, no. 1, 2012, pp. 218-233.

Author’s Last name, First name. “Title of Webpage.” Website Name, Publication date, URL.

Example : Centers for Disease Control and Prevention. “COVID-19: How to Protect Yourself and Others.” CDC, 11 Dec. 2020, https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html.

Author’s Last name, First name. “Title of Article.” Title of Magazine, Publication date, page numbers.

Example : Smith, Mary. “The Power of Positive Thinking.” Psychology Today, Mar. 2019, pp. 60-65.

Newspaper article

Author’s Last name, First name. “Title of Article.” Title of Newspaper, Publication date, page numbers.

Example : Johnson, Bob. “New Study Shows Benefits of Exercise on Mental Health.” The New York Times, 15 Feb. 2021, p. A8.

Editor’s Last name, First name, editor. Title of Book. Publisher, Publication year.

Example : Thompson, John P., editor. Social Work in the 21st Century. Sage Publications, 2014.

Chapter in an edited book

Author’s Last name, First name. “Title of Chapter.” Title of Book, edited by Editor’s First Name Last name, Publisher, Publication year, page numbers.

Example : Johnson, Karen S. “The Future of Social Work: Challenges and Opportunities.” Social Work in the 21st Century, edited by John P. Thompson, Sage Publications, 2014, pp. 105-118.

Chicago Manual of Style

The Chicago Manual of Style is a widely used style for writing academic papers, dissertations, and books in the humanities and social sciences. Here are the different reference formats in Chicago style:

Example : Smith, John K. The Psychology of Social Interaction. Wiley-Blackwell, 2005.

Author’s Last name, First name. “Title of Article.” Title of Journal volume number, no. issue number (Publication year): page numbers.

Example : Brown, Laura M., John G. Keating, and Sarah M. Jones. “The Role of Social Support in Coping with Stress among African American Adolescents.” Journal of Research on Adolescence 22, no. 1 (2012): 218-233.

Author’s Last name, First name. “Title of Webpage.” Website Name. Publication date. URL.

Example : Centers for Disease Control and Prevention. “COVID-19: How to Protect Yourself and Others.” CDC. December 11, 2020. https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html.

Author’s Last name, First name. “Title of Article.” Title of Magazine, Publication date.

Example : Smith, Mary. “The Power of Positive Thinking.” Psychology Today, March 2019.

Author’s Last name, First name. “Title of Article.” Title of Newspaper, Publication date.

Example : Johnson, Bob. “New Study Shows Benefits of Exercise on Mental Health.” The New York Times, February 15, 2021.

Example : Thompson, John P., ed. Social Work in the 21st Century. Sage Publications, 2014.

Author’s Last name, First name. “Title of Chapter.” In Title of Book, edited by Editor’s First Name Last Name, page numbers. Publisher, Publication year.

Example : Johnson, Karen S. “The Future of Social Work: Challenges and Opportunities.” In Social Work in the 21st Century, edited by John P. Thompson, 105-118. Sage Publications, 2014.

Harvard Style

The Harvard Style, also known as the Author-Date System, is a widely used style for writing academic papers and essays in the social sciences. Here are the different reference formats in Harvard Style:

Author’s Last name, First name. Year of publication. Title of Book. Place of publication: Publisher.

Example : Smith, John. 2005. The Psychology of Social Interaction. Oxford: Wiley-Blackwell.

Author’s Last name, First name. Year of publication. “Title of Article.” Title of Journal volume number (issue number): page numbers.

Example: Brown, Laura M., John G. Keating, and Sarah M. Jones. 2012. “The Role of Social Support in Coping with Stress among African American Adolescents.” Journal of Research on Adolescence 22 (1): 218-233.

Author’s Last name, First name. Year of publication. “Title of Webpage.” Website Name. URL. Accessed date.

Example : Centers for Disease Control and Prevention. 2020. “COVID-19: How to Protect Yourself and Others.” CDC. https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html. Accessed April 1, 2023.

Author’s Last name, First name. Year of publication. “Title of Article.” Title of Magazine, month and date of publication.

Example : Smith, Mary. 2019. “The Power of Positive Thinking.” Psychology Today, March 2019.

Author’s Last name, First name. Year of publication. “Title of Article.” Title of Newspaper, month and date of publication.

Example : Johnson, Bob. 2021. “New Study Shows Benefits of Exercise on Mental Health.” The New York Times, February 15, 2021.

Editor’s Last name, First name, ed. Year of publication. Title of Book. Place of publication: Publisher.

Example : Thompson, John P., ed. 2014. Social Work in the 21st Century. Thousand Oaks, CA: Sage Publications.

Author’s Last name, First name. Year of publication. “Title of Chapter.” In Title of Book, edited by Editor’s First Name Last Name, page numbers. Place of publication: Publisher.

Example : Johnson, Karen S. 2014. “The Future of Social Work: Challenges and Opportunities.” In Social Work in the 21st Century, edited by John P. Thompson, 105-118. Thousand Oaks, CA: Sage Publications.

Vancouver Style

The Vancouver Style, also known as the Uniform Requirements for Manuscripts Submitted to Biomedical Journals, is a widely used style for writing academic papers in the biomedical sciences. Here are the different reference formats in Vancouver Style:

Author’s Last name, First name. Title of Book. Edition number. Place of publication: Publisher; Year of publication.

Example : Smith, John K. The Psychology of Social Interaction. 2nd ed. Oxford: Wiley-Blackwell; 2005.

Author’s Last name, First name. Title of Article. Abbreviated Journal Title. Year of publication; volume number(issue number):page numbers.

Example : Brown LM, Keating JG, Jones SM. The Role of Social Support in Coping with Stress among African American Adolescents. J Res Adolesc. 2012;22(1):218-233.

Author’s Last name, First name. Title of Webpage. Website Name [Internet]. Publication date. [cited date]. Available from: URL.

Example : Centers for Disease Control and Prevention. COVID-19: How to Protect Yourself and Others [Internet]. 2020 Dec 11. [cited 2023 Apr 1]. Available from: https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html.

Author’s Last name, First name. Title of Article. Title of Magazine. Year of publication; month and day of publication:page numbers.

Example : Smith M. The Power of Positive Thinking. Psychology Today. 2019 Mar 1:32-35.

Author’s Last name, First name. Title of Article. Title of Newspaper. Year of publication; month and day of publication:page numbers.

Example : Johnson B. New Study Shows Benefits of Exercise on Mental Health. The New York Times. 2021 Feb 15:A4.

Editor’s Last name, First name, editor. Title of Book. Edition number. Place of publication: Publisher; Year of publication.

Example: Thompson JP, editor. Social Work in the 21st Century. 1st ed. Thousand Oaks, CA: Sage Publications; 2014.

Author’s Last name, First name. Title of Chapter. In: Editor’s Last name, First name, editor. Title of Book. Edition number. Place of publication: Publisher; Year of publication. page numbers.

Example : Johnson KS. The Future of Social Work: Challenges and Opportunities. In: Thompson JP, editor. Social Work in the 21st Century. 1st ed. Thousand Oaks, CA: Sage Publications; 2014. p. 105-118.

Turabian Style

Turabian style is a variation of the Chicago style used in academic writing, particularly in the fields of history and humanities. Here are the different reference formats in Turabian style:

Author’s Last name, First name. Title of Book. Place of publication: Publisher, Year of publication.

Example : Smith, John K. The Psychology of Social Interaction. Oxford: Wiley-Blackwell, 2005.

Author’s Last name, First name. “Title of Article.” Title of Journal volume number, no. issue number (Year of publication): page numbers.

Example : Brown, LM, Keating, JG, Jones, SM. “The Role of Social Support in Coping with Stress among African American Adolescents.” J Res Adolesc 22, no. 1 (2012): 218-233.

Author’s Last name, First name. “Title of Webpage.” Name of Website. Publication date. Accessed date. URL.

Example : Centers for Disease Control and Prevention. “COVID-19: How to Protect Yourself and Others.” CDC. December 11, 2020. Accessed April 1, 2023. https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html.

Author’s Last name, First name. “Title of Article.” Title of Magazine, Month Day, Year of publication, page numbers.

Example : Smith, M. “The Power of Positive Thinking.” Psychology Today, March 1, 2019, 32-35.

Author’s Last name, First name. “Title of Article.” Title of Newspaper, Month Day, Year of publication.

Example : Johnson, B. “New Study Shows Benefits of Exercise on Mental Health.” The New York Times, February 15, 2021.

Editor’s Last name, First name, ed. Title of Book. Place of publication: Publisher, Year of publication.

Example : Thompson, JP, ed. Social Work in the 21st Century. Thousand Oaks, CA: Sage Publications, 2014.

Author’s Last name, First name. “Title of Chapter.” In Title of Book, edited by Editor’s Last name, First name, page numbers. Place of publication: Publisher, Year of publication.

Example : Johnson, KS. “The Future of Social Work: Challenges and Opportunities.” In Social Work in the 21st Century, edited by Thompson, JP, 105-118. Thousand Oaks, CA: Sage Publications, 2014.

IEEE (Institute of Electrical and Electronics Engineers) Style

IEEE (Institute of Electrical and Electronics Engineers) style is commonly used in engineering, computer science, and other technical fields. Here are the different reference formats in IEEE style:

Author’s Last name, First name. Book Title. Place of Publication: Publisher, Year of publication.

Example : Oppenheim, A. V., & Schafer, R. W. Discrete-Time Signal Processing. Upper Saddle River, NJ: Prentice Hall, 2010.

Author’s Last name, First name. “Title of Article.” Abbreviated Journal Title, vol. number, no. issue number, pp. page numbers, Month year of publication.

Example: Shannon, C. E. “A Mathematical Theory of Communication.” Bell System Technical Journal, vol. 27, no. 3, pp. 379-423, July 1948.

Conference paper

Author’s Last name, First name. “Title of Paper.” In Title of Conference Proceedings, Place of Conference, Date of Conference, pp. page numbers, Year of publication.

Example: Gupta, S., & Kumar, P. “An Improved System of Linear Discriminant Analysis for Face Recognition.” In Proceedings of the 2011 International Conference on Computer Science and Network Technology, Harbin, China, Dec. 2011, pp. 144-147.

Author’s Last name, First name. “Title of Webpage.” Name of Website. Date of publication or last update. Accessed date. URL.

Example : National Aeronautics and Space Administration. “Apollo 11.” NASA. July 20, 1969. Accessed April 1, 2023. https://www.nasa.gov/mission_pages/apollo/apollo11.html.

Technical report

Author’s Last name, First name. “Title of Report.” Name of Institution or Organization, Report number, Year of publication.

Example : Smith, J. R. “Development of a New Solar Panel Technology.” National Renewable Energy Laboratory, NREL/TP-6A20-51645, 2011.

Author’s Last name, First name. “Title of Patent.” Patent number, Issue date.

Example : Suzuki, H. “Method of Producing Carbon Nanotubes.” US Patent 7,151,019, December 19, 2006.

Standard Title. Standard number, Publication date.

Example : IEEE Standard for Floating-Point Arithmetic. IEEE Std 754-2008, August 29, 2008

ACS (American Chemical Society) Style

ACS (American Chemical Society) style is commonly used in chemistry and related fields. Here are the different reference formats in ACS style:

Author’s Last name, First name; Author’s Last name, First name. Title of Article. Abbreviated Journal Title Year, Volume, Page Numbers.

Example : Wang, Y.; Zhao, X.; Cui, Y.; Ma, Y. Facile Preparation of Fe3O4/graphene Composites Using a Hydrothermal Method for High-Performance Lithium Ion Batteries. ACS Appl. Mater. Interfaces 2012, 4, 2715-2721.

Author’s Last name, First name. Book Title; Publisher: Place of Publication, Year of Publication.

Example : Carey, F. A. Organic Chemistry; McGraw-Hill: New York, 2008.

Author’s Last name, First name. Chapter Title. In Book Title; Editor’s Last name, First name, Ed.; Publisher: Place of Publication, Year of Publication; Volume number, Chapter number, Page Numbers.

Example : Grossman, R. B. Analytical Chemistry of Aerosols. In Aerosol Measurement: Principles, Techniques, and Applications; Baron, P. A.; Willeke, K., Eds.; Wiley-Interscience: New York, 2001; Chapter 10, pp 395-424.

Author’s Last name, First name. Title of Webpage. Website Name, URL (accessed date).

Example : National Institute of Standards and Technology. Atomic Spectra Database. https://www.nist.gov/pml/atomic-spectra-database (accessed April 1, 2023).

Author’s Last name, First name. Patent Number. Patent Date.

Example : Liu, Y.; Huang, H.; Chen, H.; Zhang, W. US Patent 9,999,999, December 31, 2022.

Author’s Last name, First name; Author’s Last name, First name. Title of Article. In Title of Conference Proceedings, Publisher: Place of Publication, Year of Publication; Volume Number, Page Numbers.

Example : Jia, H.; Xu, S.; Wu, Y.; Wu, Z.; Tang, Y.; Huang, X. Fast Adsorption of Organic Pollutants by Graphene Oxide. In Proceedings of the 15th International Conference on Environmental Science and Technology, American Chemical Society: Washington, DC, 2017; Volume 1, pp 223-228.

AMA (American Medical Association) Style

AMA (American Medical Association) style is commonly used in medical and scientific fields. Here are the different reference formats in AMA style:

Author’s Last name, First name. Article Title. Journal Abbreviation. Year; Volume(Issue):Page Numbers.

Example : Jones, R. A.; Smith, B. C. The Role of Vitamin D in Maintaining Bone Health. JAMA. 2019;321(17):1765-1773.

Author’s Last name, First name. Book Title. Edition number. Place of Publication: Publisher; Year.

Example : Guyton, A. C.; Hall, J. E. Textbook of Medical Physiology. 13th ed. Philadelphia, PA: Saunders; 2015.

Author’s Last name, First name. Chapter Title. In: Editor’s Last name, First name, ed. Book Title. Edition number. Place of Publication: Publisher; Year: Page Numbers.

Example: Rajakumar, K. Vitamin D and Bone Health. In: Holick, M. F., ed. Vitamin D: Physiology, Molecular Biology, and Clinical Applications. 2nd ed. New York, NY: Springer; 2010:211-222.

Author’s Last name, First name. Webpage Title. Website Name. URL. Published date. Updated date. Accessed date.

Example : National Cancer Institute. Breast Cancer Prevention (PDQ®)–Patient Version. National Cancer Institute. https://www.cancer.gov/types/breast/patient/breast-prevention-pdq. Published October 11, 2022. Accessed April 1, 2023.

Author’s Last name, First name. Conference presentation title. In: Conference Title; Conference Date; Place of Conference.

Example : Smith, J. R. Vitamin D and Bone Health: A Meta-Analysis. In: Proceedings of the Annual Meeting of the American Society for Bone and Mineral Research; September 20-23, 2022; San Diego, CA.

Thesis or dissertation

Author’s Last name, First name. Title of Thesis or Dissertation. Degree level [Doctoral dissertation or Master’s thesis]. University Name; Year.

Example : Wilson, S. A. The Effects of Vitamin D Supplementation on Bone Health in Postmenopausal Women [Doctoral dissertation]. University of California, Los Angeles; 2018.

ASCE (American Society of Civil Engineers) Style

The ASCE (American Society of Civil Engineers) style is commonly used in civil engineering fields. Here are the different reference formats in ASCE style:

Author’s Last name, First name. “Article Title.” Journal Title, volume number, issue number (year): page numbers. DOI or URL (if available).

Example : Smith, J. R. “Evaluation of the Effectiveness of Sustainable Drainage Systems in Urban Areas.” Journal of Environmental Engineering, vol. 146, no. 3 (2020): 04020010. https://doi.org/10.1061/(ASCE)EE.1943-7870.0001668.

Example : McCuen, R. H. Hydrologic Analysis and Design. 4th ed. Upper Saddle River, NJ: Pearson Education; 2013.

Author’s Last name, First name. “Chapter Title.” In: Editor’s Last name, First name, ed. Book Title. Edition number. Place of Publication: Publisher; Year: page numbers.

Example : Maidment, D. R. “Floodplain Management in the United States.” In: Shroder, J. F., ed. Treatise on Geomorphology. San Diego, CA: Academic Press; 2013: 447-460.

Author’s Last name, First name. “Paper Title.” In: Conference Title; Conference Date; Location. Place of Publication: Publisher; Year: page numbers.

Example: Smith, J. R. “Sustainable Drainage Systems for Urban Areas.” In: Proceedings of the ASCE International Conference on Sustainable Infrastructure; November 6-9, 2019; Los Angeles, CA. Reston, VA: American Society of Civil Engineers; 2019: 156-163.

Author’s Last name, First name. “Report Title.” Report number. Place of Publication: Publisher; Year.

Example : U.S. Army Corps of Engineers. “Hurricane Sandy Coastal Risk Reduction Program, New York and New Jersey.” Report No. P-15-001. Washington, DC: U.S. Army Corps of Engineers; 2015.

CSE (Council of Science Editors) Style

The CSE (Council of Science Editors) style is commonly used in the scientific and medical fields. Here are the different reference formats in CSE style:

Author’s Last name, First Initial. Middle Initial. “Article Title.” Journal Title. Year;Volume(Issue):Page numbers.

Example : Smith, J.R. “Evaluation of the Effectiveness of Sustainable Drainage Systems in Urban Areas.” Journal of Environmental Engineering. 2020;146(3):04020010.

Author’s Last name, First Initial. Middle Initial. Book Title. Edition number. Place of Publication: Publisher; Year.

Author’s Last name, First Initial. Middle Initial. “Chapter Title.” In: Editor’s Last name, First Initial. Middle Initial., ed. Book Title. Edition number. Place of Publication: Publisher; Year:Page numbers.

Author’s Last name, First Initial. Middle Initial. “Paper Title.” In: Conference Title; Conference Date; Location. Place of Publication: Publisher; Year.

Example : Smith, J.R. “Sustainable Drainage Systems for Urban Areas.” In: Proceedings of the ASCE International Conference on Sustainable Infrastructure; November 6-9, 2019; Los Angeles, CA. Reston, VA: American Society of Civil Engineers; 2019.

Author’s Last name, First Initial. Middle Initial. “Report Title.” Report number. Place of Publication: Publisher; Year.

Bluebook Style

The Bluebook style is commonly used in the legal field for citing legal documents and sources. Here are the different reference formats in Bluebook style:

Case citation

Case name, volume source page (Court year).

Example : Brown v. Board of Education, 347 U.S. 483 (1954).

Statute citation

Name of Act, volume source § section number (year).

Example : Clean Air Act, 42 U.S.C. § 7401 (1963).

Regulation citation

Name of regulation, volume source § section number (year).

Example: Clean Air Act, 40 C.F.R. § 52.01 (2019).

Book citation

Author’s Last name, First Initial. Middle Initial. Book Title. Edition number (if applicable). Place of Publication: Publisher; Year.

Example: Smith, J.R. Legal Writing and Analysis. 3rd ed. New York, NY: Aspen Publishers; 2015.

Journal article citation

Author’s Last name, First Initial. Middle Initial. “Article Title.” Journal Title. Volume number (year): first page-last page.

Example: Garcia, C. “The Right to Counsel: An International Comparison.” International Journal of Legal Information. 43 (2015): 63-94.

Website citation

Author’s Last name, First Initial. Middle Initial. “Page Title.” Website Title. URL (accessed month day, year).

Example : United Nations. “Universal Declaration of Human Rights.” United Nations. https://www.un.org/en/universal-declaration-human-rights/ (accessed January 3, 2023).

Oxford Style

The Oxford style, also known as the Oxford referencing system or the documentary-note citation system, is commonly used in the humanities, including literature, history, and philosophy. Here are the different reference formats in Oxford style:

Author’s Last name, First name. Book Title. Place of Publication: Publisher, Year of Publication.

Example : Smith, John. The Art of Writing. New York: Penguin, 2020.

Author’s Last name, First name. “Article Title.” Journal Title volume, no. issue (year): page range.

Example: Garcia, Carlos. “The Role of Ethics in Philosophy.” Philosophy Today 67, no. 3 (2019): 53-68.

Chapter in an edited book citation

Author’s Last name, First name. “Chapter Title.” In Book Title, edited by Editor’s Name, page range. Place of Publication: Publisher, Year of Publication.

Example : Lee, Mary. “Feminism in the 21st Century.” In The Oxford Handbook of Feminism, edited by Jane Smith, 51-69. Oxford: Oxford University Press, 2018.

Author’s Last name, First name. “Page Title.” Website Title. URL (accessed day month year).

Example : Jones, David. “The Importance of Learning Languages.” Oxford Language Center. https://www.oxfordlanguagecenter.com/importance-of-learning-languages/ (accessed 3 January 2023).

Dissertation or thesis citation

Author’s Last name, First name. “Title of Dissertation/Thesis.” PhD diss., University Name, Year of Publication.

Example : Brown, Susan. “The Art of Storytelling in American Literature.” PhD diss., University of Oxford, 2020.

Newspaper article citation

Author’s Last name, First name. “Article Title.” Newspaper Title, Month Day, Year.

Example : Robinson, Andrew. “New Developments in Climate Change Research.” The Guardian, September 15, 2022.

AAA (American Anthropological Association) Style

The American Anthropological Association (AAA) style is commonly used in anthropology research papers and journals. Here are the different reference formats in AAA style:

Author’s Last name, First name. Year of Publication. Book Title. Place of Publication: Publisher.

Example : Smith, John. 2019. The Anthropology of Food. New York: Routledge.

Author’s Last name, First name. Year of Publication. “Article Title.” Journal Title volume, no. issue: page range.

Example : Garcia, Carlos. 2021. “The Role of Ethics in Anthropology.” American Anthropologist 123, no. 2: 237-251.

Author’s Last name, First name. Year of Publication. “Chapter Title.” In Book Title, edited by Editor’s Name, page range. Place of Publication: Publisher.

Example: Lee, Mary. 2018. “Feminism in Anthropology.” In The Oxford Handbook of Feminism, edited by Jane Smith, 51-69. Oxford: Oxford University Press.

Author’s Last name, First name. Year of Publication. “Page Title.” Website Title. URL (accessed day month year).

Example : Jones, David. 2020. “The Importance of Learning Languages.” Oxford Language Center. https://www.oxfordlanguagecenter.com/importance-of-learning-languages/ (accessed January 3, 2023).

Author’s Last name, First name. Year of Publication. “Title of Dissertation/Thesis.” PhD diss., University Name.

Example : Brown, Susan. 2022. “The Art of Storytelling in Anthropology.” PhD diss., University of California, Berkeley.

Author’s Last name, First name. Year of Publication. “Article Title.” Newspaper Title, Month Day.

Example : Robinson, Andrew. 2021. “New Developments in Anthropology Research.” The Guardian, September 15.

AIP (American Institute of Physics) Style

The American Institute of Physics (AIP) style is commonly used in physics research papers and journals. Here are the different reference formats in AIP style:

Example : Johnson, S. D. 2021. “Quantum Computing and Information.” Journal of Applied Physics 129, no. 4: 043102.

Example : Feynman, Richard. 2018. The Feynman Lectures on Physics. New York: Basic Books.

Example : Jones, David. 2020. “The Future of Quantum Computing.” In The Handbook of Physics, edited by John Smith, 125-136. Oxford: Oxford University Press.

Conference proceedings citation

Author’s Last name, First name. Year of Publication. “Title of Paper.” Proceedings of Conference Name, date and location: page range. Place of Publication: Publisher.

Example : Chen, Wei. 2019. “The Applications of Nanotechnology in Solar Cells.” Proceedings of the 8th International Conference on Nanotechnology, July 15-17, Tokyo, Japan: 224-229. New York: AIP Publishing.

Example : American Institute of Physics. 2022. “About AIP Publishing.” AIP Publishing. https://publishing.aip.org/about-aip-publishing/ (accessed January 3, 2023).

Patent citation

Author’s Last name, First name. Year of Publication. Patent Number.

Example : Smith, John. 2018. US Patent 9,873,644.

References Writing Guide

Here are some general guidelines for writing references:

• Follow the citation style guidelines: Different disciplines and journals may require different citation styles (e.g., APA, MLA, Chicago). It is important to follow the specific guidelines for the citation style required.
• Include all necessary information : Each citation should include enough information for readers to locate the source. For example, a journal article citation should include the author(s), title of the article, journal title, volume number, issue number, page numbers, and publication year.
• Use proper formatting: Citation styles typically have specific formatting requirements for different types of sources. Make sure to follow the proper formatting for each citation.
• Order citations alphabetically: If listing multiple sources, they should be listed alphabetically by the author’s last name.
• Be consistent: Use the same citation style throughout the entire paper or project.
• Check for accuracy: Double-check all citations to ensure accuracy, including correct spelling of author names and publication information.
• Use reputable sources: When selecting sources to cite, choose reputable and authoritative sources. Avoid sources that are biased or unreliable.
• Include all sources: Make sure to include all sources used in the research, including those that were not directly quoted but still informed the work.
• Use online tools : There are online tools available (e.g., citation generators) that can help with formatting and organizing references.

Purpose of References in Research

References in research serve several purposes:

• To give credit to the original authors or sources of information used in the research. It is important to acknowledge the work of others and avoid plagiarism.
• To provide evidence for the claims made in the research. References can support the arguments, hypotheses, or conclusions presented in the research by citing relevant studies, data, or theories.
• To allow readers to find and verify the sources used in the research. References provide the necessary information for readers to locate and access the sources cited in the research, which allows them to evaluate the quality and reliability of the information presented.
• To situate the research within the broader context of the field. References can show how the research builds on or contributes to the existing body of knowledge, and can help readers to identify gaps in the literature that the research seeks to address.

Importance of References in Research

References play an important role in research for several reasons:

• Credibility : By citing authoritative sources, references lend credibility to the research and its claims. They provide evidence that the research is based on a sound foundation of knowledge and has been carefully researched.
• Avoidance of Plagiarism : References help researchers avoid plagiarism by giving credit to the original authors or sources of information. This is important for ethical reasons and also to avoid legal repercussions.
• Reproducibility : References allow others to reproduce the research by providing detailed information on the sources used. This is important for verification of the research and for others to build on the work.
• Context : References provide context for the research by situating it within the broader body of knowledge in the field. They help researchers to understand where their work fits in and how it builds on or contributes to existing knowledge.
• Evaluation : References provide a means for others to evaluate the research by allowing them to assess the quality and reliability of the sources used.

Advantages of References in Research

There are several advantages of including references in research:

• Acknowledgment of Sources: Including references gives credit to the authors or sources of information used in the research. This is important to acknowledge the original work and avoid plagiarism.
• Evidence and Support : References can provide evidence to support the arguments, hypotheses, or conclusions presented in the research. This can add credibility and strength to the research.
• Reproducibility : References provide the necessary information for others to reproduce the research. This is important for the verification of the research and for others to build on the work.
• Context : References can help to situate the research within the broader body of knowledge in the field. This helps researchers to understand where their work fits in and how it builds on or contributes to existing knowledge.
• Evaluation : Including references allows others to evaluate the research by providing a means to assess the quality and reliability of the sources used.
• Ongoing Conversation: References allow researchers to engage in ongoing conversations and debates within their fields. They can show how the research builds on or contributes to the existing body of knowledge.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Research Findings – Types Examples and Writing...

Ethical Considerations – Types, Examples and...

Table of Contents – Types, Formats, Examples

Appendices – Writing Guide, Types and Examples

Research Paper – Structure, Examples and Writing...

Future Research – Thesis Guide

• Jump to menu
• Student Home
• Accept your offer
• How to enrol
• Student ID card
• Set up your IT
• Orientation Week
• Fees & payment
• Academic calendar
• Special consideration
• Transcripts
• The Nucleus: Student Hub
• Referencing
• Essay writing
• Learning abroad & exchange
• Professional development & UNSW Advantage
• Employability
• Financial assistance
• International students
• Equitable learning
• Postgraduate research
• Health Service
• Events & activities
• Emergencies
• Volunteering
• Clubs and societies
• Accommodation
• Health services
• Sport and gym
• Arc student organisation
• Security on campus
• Maps of campus
• Careers portal
• Change password

Why is Referencing Important?

Citations are not used simply to avoid plagiarism; they have other important roles too..

Referencing allows you to acknowledge the contribution of other writers and researchers in your work. Any university assignments that draw on the ideas, words or research of other writers must contain citations.

Referencing is also a way to give credit to the writers from whom you have borrowed words and ideas. By citing the work of a particular scholar, you acknowledge and respect the intellectual property rights of that researcher. As a student or academic, you can draw on any of the millions of ideas, insights and arguments published by other writers, many of whom have spent years researching and writing. All you need to do is acknowledge their contribution to your assignment.

Referencing is a way to provide evidence to support the assertions and claims in your own assignments. By citing experts in your field, you are showing your marker that you are aware of the field in which you are operating. Your citations map the space of your discipline and allow you to navigate your way through your chosen field of study, in the same way that sailors steer by the stars.

References should always be accurate, allowing your readers to trace the sources of information you have used. The best way to make sure you reference accurately is to keep a record of all the sources you used when reading and researching for an assignment.

Citations also make your writing more persuasive. 

Exercise: Look at the two paragraphs below: which one seems more authoritative?

The paragraphs are identical, except for the absence of citations from paragraph 1.

The first paragraph may be just as interesting as the second, but within an academic context, a context that requires you to show from where you have taken ideas, the second has far more authority, it is more persuasive. It shows that the ideas you are discussing are matters that are important to your particular academic community.

What kind of information do I need to reference?

Printed books are not the only sources that require acknowledgement. ANY words, ideas or information taken from ANY source requires a reference.

Reference when you are using words or ideas from:

• books and journal articles
• newspapers and magazines
• pamphlets or brochures
• films, documentaries, television programs or advertisements
• websites or electronic resources
• letters, emails, online discussion forums
• personal interviews
• lecturers or tutors. (Not always necessary but check with your lecturer or tutor about their preferences before you draw on their ideas.)

You also need to reference when you reprint any diagrams, illustrations, charts or pictures.

No need to reference when you are:

• writing your own observations or experiment results, for example, a report on a field trip
• writing about your own experiences, for example, a reflective journal
• writing your own thoughts, comments or conclusions in an assignment
• evaluating or offering your own analysis
• using 'common knowledge' (facts that can be found in numerous places and are likely to be known by a lot of people) or folklore
• using generally accepted facts or information. This will vary in different disciplines of study. If in doubt, ask your tutor.
• Academic Integrity & Plagiarism
• Academic integrity online module
• What is plagiarism?
• Intellectual insecurity
• Poor time management
• Lack of a clear argument
• Lack of critical and analytical skills
• Inadequate research
• Poor notetaking
• Why is referencing important?
• Underdeveloped writing skills
• Avoiding plagiarism
• Other plagiarism resources

Study Hacks Workshops | All the hacks you need! 28 May – 25 Jul 2024

• Directories
• What are citations and why should I use them?
• When should I use a citation?
• Why are there so many citation styles?
• Which citation style should I use?
• Chicago Notes Style
• Chicago Author-Date Style
• AMA Style (medicine)
• Bluebook (law)
• Additional Citation Styles
• Built-in Citation Tools
• Quick Citation Generators
• Citation Management Software
• Start Your Research
• Research Guides
• University of Washington Libraries
• Library Guides
• UW Libraries
• Citing Sources

Citing Sources: What are citations and why should I use them?

What is a citation.

Citations are a way of giving credit when certain material in your work came from another source. It also gives your readers the information necessary to find that source again-- it provides an important roadmap to your research process. Whenever you use sources such as books, journals or websites in your research, you must give credit to the original author by citing the source. 

Why do researchers cite?

Scholarship is a conversation  and scholars use citations not only to  give credit  to original creators and thinkers, but also to  add strength and authority  to their own work.  By citing their sources, scholars are  placing their work in a specific context  to show where they “fit” within the larger conversation.  Citations are also a great way to  leave a trail  intended to help others who may want to explore the conversation or use the sources in their own work.

In short, citations

(1) give credit

(2) add strength and authority to your work

(3) place your work in a specific context

(4) leave a trail for other scholars

"Good citations should reveal your sources, not conceal them. They should honeslty reflect the research you conducted." (Lipson 4)

Lipson, Charles. "Why Cite?"  Cite Right: A Quick Guide to Citation Styles--MLA, APA, Chicago, the Sciences, Professions, and More . Chicago: U of Chicago, 2006. Print.

What does a citation look like?

Different subject disciplines call for citation information to be written in very specific order, capitalization, and punctuation. There are therefore many different style formats. Three popular citation formats are MLA Style (for humanities articles) and APA or Chicago (for social sciences articles).

MLA style (print journal article):  

Whisenant, Warren A. "How Women Have Fared as Interscholastic Athletic Administrators Since the Passage of Title IX." Sex Roles Vol. 49.3 (2003): 179-182.

APA style (print journal article):

Whisenant, W. A. (2003) How Women Have Fared as Interscholastic Athletic Administrators Since the Passage of Title IX. Sex Roles , 49 (3), 179-182.

Chicago style (print journal article):

Whisenant, Warren A. "How Women Have Fared as Interscholastic Athletic Administrators Since the Passage of Title IX." Sex Roles 49, no. 3 (2003): 179-182.

No matter which style you use, all citations require the same basic information:

• Author or Creator
• Container (e.g., Journal or magazine, website, edited book)
• Date of creation or publication
• Publisher 

You are most likely to have easy access to all of your citation information when you find it in the first place. Take note of this information up front, and it will be much easier to cite it effectively later.

• << Previous: Basics of Citing
• Next: When should I use a citation? >>
• Last Updated: May 1, 2024 12:48 PM
• URL: https://guides.lib.uw.edu/research/citations

• - Google Chrome

Intended for healthcare professionals

• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• ROBVALU: a tool for...

ROBVALU: a tool for assessing risk of bias in studies about people’s values, utilities, or importance of health outcomes

• Related content
• Peer review
• Samer G Karam , doctoral student 1 2 ,
• Yuan Zhang , assistant clinical professor 1 2 ,
• Hector Pardo-Hernandez , researcher 3 4 ,
• Uwe Siebert , professor 5 6 7 ,
• Laura Koopman , senior adviser 8 ,
• Jane Noyes , professor 9 ,
• Jean-Eric Tarride , professor 1 10 11 ,
• Adrienne L Stevens , manager 12 ,
• Vivian Welch , senior investigator 13 ,
• Zuleika Saz-Parkinson , project adviser 14 ,
• Brendalynn Ens , director (retired) 15 ,
• Tahira Devji , medical student 16 ,
• Feng Xie , professor 1 10 ,
• Glen Hazlewood , associate professor 17 18 ,
• Lawrence Mbuagbaw , associate professor 1 19 20 21 22 23 ,
• Pablo Alonso-Coello , senior researcher 3 4 24 ,
• Jan L Brozek , associate professor 1 2 ,
• Holger J Schünemann , professor 1 25
• 1 Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
• 2 Michael G DeGroote Cochrane Canada and McMaster GRADE Centres, McMaster University, Hamilton, ON, Canada
• 3 Iberoamerican Cochrane Centre, Sant Antoni Maria Claret, Barcelona, Spain
• 4 Centro de Investigación Biomédica en Red de Epidemiología y Salud Pública (CIBERESP), Madrid, Spain
• 5 Department of Public Health, Health Services Research and Health Technology Assessment, Institute of Public Health, Medical Decision Making and Health Technology Assessment, UMIT TIROL-University for Health Sciences and Technology, Hall in Tirol, Austria
• 6 Center for Health Decision Science and Departments of Epidemiology and Health Policy and Management, Harvard T H Chan School of Public Health, Boston, MA, USA
• 7 Institute for Technology Assessment and Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
• 8 Department of Specialist Medical Care, National Health Care Institute, Diemen, Netherlands
• 9 School of Medical and Health Sciences, Bangor University, Wales, UK
• 10 Centre for Health Economics and Policy Analysis, McMaster University Faculty of Health Sciences, Hamilton, ON, Canada
• 11 Programs for Assessment of Technologies in Health, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada
• 12 Centre for Immunisation Programmes, Public Health Agency of Canada, ON, Canada
• 13 Bruyère Research Institute and, School of Epidemiology and Public Health, University of Ottawa, Ottawa, ON, Canada
• 14 European Commission, Joint Research Centre, Ispra, Italy
• 15 Implementation Support and Knowledge Mobilisation, Canadian Agency for Drugs and Technologies in Health, Ottawa, ON, Canada
• 16 Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada.
• 17 Department of Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
• 18 Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
• 19 Department of Anaesthesia, McMaster University, Hamilton, ON, Canada
• 20 Department of Paediatrics, McMaster University, Hamilton, ON, Canada
• 21 Biostatistics Unit, Father Sean O’Sullivan Research Centre, St Joseph’s Healthcare, Hamilton, ON, Canada
• 22 Centre for Development of Best Practices in Health, Yaoundé Central Hospital, Yaoundé, Cameroon
• 23 Division of Epidemiology and Biostatistics, Department of Global Health, Stellenbosch University, Cape Town, South Africa
• 24 Institut de Recerca Sant Pau (IR SANT PAU), Sant Quintí, Barcelona, Spain
• 25 Clinical Epidemiology and Research Centre (CERC), Humanitas University and Humanitas Research Hospital, Via Rita Levi Montalcini 4, 20090 Pieve Emanuele, Milan, Italy
• Correspondence to: H J Schünemann schuneh{at}mcmaster.ca
• Accepted 9 April 2024

People’s values are an important driver in healthcare decision making. The certainty of an intervention’s effect on benefits and harms relies on two factors: the certainty in the measured effect on an outcome in terms of risk difference and the certainty in its value, also known as utility or importance. The GRADE (Grading of Recommendations, Assessment, Development, and Evaluations) working group has proposed a set of questions to assess the risk of bias in a body of evidence from studies investigating how people value outcomes. However, these questions do not address risk of bias in individual studies that, similar to risk-of-bias tools for other research studies, is required to evaluate such evidence. Thus, the Risk of Bias in studies of Values and Utilities (ROBVALU) tool was developed. ROBVALU has good psychometric properties and will be useful when assessing individual studies in measuring values, utilities, or the importance of outcomes. As such, ROBVALU can be used to assess risk of bias in studies included in systematic reviews and health guidelines. It also can support health research assessments, where the risk of bias of input variables determines the certainty in model outputs. These assessments include, for example, decision analysis and cost utility or cost effectiveness analysis for health technology assessment, health policy, and reimbursement decision making.

Healthcare decision making relies on evidence on the relative effectiveness, safety, and cost effectiveness of an intervention evaluated in appropriate studies. 1 2 Choosing between different interventions (such as preventive, diagnostic, or treatment strategies) depends on the importance or value that people place on specific health states or health outcomes. 2 Values have a major role at different levels of decision making, from the individual level to the healthcare system level. In this context, people’s values reflect the importance they place on outcomes of interest that result from decisions about using an intervention—for example, taking a certain test or starting a new treatment regimen. 2 We use the term “people” when talking about value because the term is inclusive to patients, healthcare providers, policy makers, and the general public.

Utility instruments are widely used to elicit the absolute value of a health outcome, and provide an index measure anchored on a scale with 1 reflecting perfect health and 0 reflecting being dead. 3 4 Various methods are used to establish values, including direct measures of utility, indirect measurements of utility, or qualitative research. 2 5 The visual analogue scale (VAS) is one of the simplest measures to elicit these values. People are asked to rate a health state on a VAS that is then converted to a utility value. 6 7 While the scale directly measures the importance of an outcome, concerns exist about how accurate and valid it might be. 2 Other direct measures such as the standard gamble and time trade-off require people to choose between their current health state and a treatment option that could result in perfect health or in immediate death. 4 8 Discrete choice experiments ask people to choose between two or more treatment options where the choices differ in terms of their attributes, that are defined by the investigators. 9 The relative importance of each attribute is then inferred by analysing the responses, assuming that patients choose the option with the highest value. 9 Indirect methods of measuring utility values include validated, health related, quality-of-life instruments, such as the EQ-5D and the Health Utilities Index. 10 The EQ-5D requires respondents to answer questions across five domains that are converted to a utility value using validated scoring systems. 11 12

Summary points

Assessing the risk of bias in individual studies is an essential step to determine overall certainty of evidence in a systematic review or health technology assessment and for guideline development

The Risk of Bias in Values and Utilities (ROBVALU) tool assesses risk of bias in quantitative studies of people’s values, utilities, or importance of outcomes

A sequential mixed methods approach was used to develop ROBVALU, initially based on signalling questions and subdomains developed by the GRADE working group to assess risk of bias; a modified Delphi approach was used for final refinement of the tool

ROBVALU covers four separate subdomains through which bias might be introduced; individual subdomain judgments inform the overall risk of bias of studies

ROBVALU has demonstrated high validity and reliability

General application of utility values in research

These utility values allow researchers to weigh the benefits and harms of an option and, thus, they also are important in health economics and health technology assessments. 3 13 For instance, in decision analysis, they are required to calculate quality adjusted life years. Confidence in studies that report on values needs to be ascertained for decision making in guideline recommendations, health technology assessments, or coverage decision. 14 For example, in a systematic review on people with chronic obstructive pulmonary disease, we found moderate certainty that patients value adverse events as important, but on average valued them as less important than symptom relief. 15 We also found moderate certainty that exacerbation and hospital admission owing to exacerbation are the outcomes that patients with chronic obstructive pulmonary disease rate as most important. In another example, a systematic review on patients’ values on venous thromboembolism, we found that people with cancer placed more importance on a reduction in new or recurrent venous thromboembolism than on a decrease in major or minor bleeding events. 16

The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) Evidence to Decision frameworks is a widely used approach in guidelines, health technology assessment, and other decisions. The frameworks require judgments about the certainty in how much people value the main outcomes: “Is there important uncertainty about . . . how much people value the main outcomes?” 17 18 A key determinant of certainty is internal validity—that is, how well individual studies were designed and conducted (ie, internal validity, which GRADE and Cochrane label as the risk-of-bias (ROB) domain).

Risk of bias

Similar to other study designs, threats to internal validity arising from the study design, conduct, analysis, and reporting of the study introduce ROB in research on utility values. 2 Poor study quality could result in indirectness which encompasses applicability and external validity, often as a result of PICO (patient/population, intervention, comparison, and outcomes) elements. Another quality issue is low sample size or no sample size calculation, which could result in imprecision. ROB assessment tools are developed to assess biases that result in threats of internal validity and would not measure indirectness and precision. Quality assessment tools and reporting checklists often include all factors of a study’s qualities and safeguards, but these tools differ from a ROB assessment tool that aims to present a ROB judgment for a study. A key factor that might introduce bias in values studies is the instrument used to measure utilities of the people in the study. Bias means that a value people place on an outcome in a research study (eg, a value of 0.5 for stroke) would be systematically different from the true value that people would place on that outcome. For example, the true unbiased value might be 0.3 and, thus, use of biased estimates would provide inaccurate answers in the modelling and health decision making context.

ROB assessment tools exist for many study designs, including the Cochrane Risk of Bias 2 (RoB 2) for randomised trials, 19 ROBINS-I for non-randomised studies of the effects of interventions, 20 and ROBINS-E for studies about exposures. 21 22 Critical appraisal tools to assess the quality of a study are also study design specific, such as the Newcastle-Ottawa scale and the Joanna Briggs Institute’s critical appraisal tool for cross sectional studies. 23 24 These tools are regularly used by researchers to assess the quality of individual studies or to assess ROB, but they were not developed for studies on utility values. These checklists invariably include questions specific to the study design, which would not always be appropriate to answer in studies about people’s values (eg, “Were there deviations from the intended intervention that arose because of the trial context?” or “Was the exposure measured in a valid and reliable way?”). 19 21 22 For studies on utility values, a major concern that is not adequately addressed by any commonly used ROB tool is the method used to elicit people’s values. The measurement instrument needs to be valid and reliable, be used appropriately, use valid health outcomes, and explore proper understanding of the instrument. No validated tool is available for the nuanced assessment of ROB in individual studies measuring utility values. 9 20 25 26

To properly implement evidence based decision making and formulate evidence based recommendations in clinical or public health guidelines, evaluation of ROB is crucial in studies of values, utilities, or importance of outcomes. However, owing to the absence of specialised and validated tools to assess ROB, this evaluation is rarely done. Thus, our goal was to develop, validate, and describe a pragmatic tool for studies measuring the value people place on health outcomes with appropriate guidance to apply it correctly.

Development of the ROBVALU tool and guidance

We used a sequential, mixed methods approach to develop ROBVALU and related guidance document (supplement S1), 27 starting with a qualitative approach and followed by a quantitative phase to assess the psychometric properties of the tool ( fig 1 ). In the qualitative phase, we first considered the ROB signalling questions (appendix table A1) and subdomains that we had developed for GRADE guidance to assess ROB about values across studies in a body of evidence. 2 For that GRADE guidance, we iteratively developed the subdomains and signalling questions starting with a list of 23 items identified as part of a systematic survey project. 26 The core research group reviewed the 23 items to identify any missing item that might be relevant for the single study ROBVALU tool. After thorough group discussions, a decision was made not to add any new items or subdomains to avoid complexity, thereby improving applicability, feasibility, and adoption of the tool.

Tool development process for the Risk of Bias in Values and Utilities (ROBVALU) tool. GRADE=Grading of Recommendations, Assessment, Development, and Evaluation

• Download figure
• Open in new tab
• Download powerpoint

We first structured a preliminary version of the tool and added simple considerations to help answer the signalling questions. These signalling questions were categorised into four subdomains: selection of participants into the study, completeness of data, measurement instrument, and data analysis. We used a 4 point, Likert-type scale (ie, yes, probably yes, probably no, no) to judge the individual items, to avoid a neutral option of a 5 point Likert scale when studies lack sufficient information to make a proper judgment. In each subdomain, the tool asked how important and how serious the ROB issue is. The core research group iteratively revised the tool and accompanying guidance document. An advisory group of experts provided feedback and suggested appropriate changes to establish face and content validity (supplement S2).

Participant testing

We used purposeful sampling to recruit 15 participants with experience in critical appraisal, systematic reviews, or guidelines for user testing and semi-structured interviews (supplement S3). The participants had a broad level of expertise, from masters level students to senior researchers with experience in health research ranging from six months to 30 years (appendix table A2). All users received the ROBVALU tool and the accompanying guidance document (supplement S1). We instructed the participants to complete three to four assessments and every sample study was assessed by four users independently, 11 studies in total were assessed (appendix table A3). Based on feedback received in the semi-structured interview after user testing, we iteratively revised and improved the guidance document throughout the project with a focus on the wordings, spelling, and grammatical structure of the guidance document. The ROBVALU tool demonstrated good psychometric properties with an overall intraclass correlation coefficient of 0.87 and the four subdomains showed good to excellent reliability ranging from 0.80 to 0.91 ( table 1 and supplement S4). We also calculated the inter-rater reliability of the global ROB judgment using the ROBVALU tool with Kendall’s W, which showed substantial agreement of 0.62 (supplement S4). We invited four expert participants in the field to provide a global judgment for ROB without using the ROBVALU, with each expert rating three to four studies. When we added expert participant responses of the global ROB judgment, the Kendall’s W dropped to 0.45, showing moderate agreement (supplement S4). However, only four global judgment responses were more than one level of seriousness higher or lower than the expert participant judgment (appendix table A4).

Reliability of the Risk of Bias in Values and Utilities (ROBVALU) tool. CI=confidence interval

• View inline

Modified Delphi process

Finally, following our protocol, we used purposeful sampling to invite 20 experts in values, utilities, health technology assessment, and health decision science to participate in a modified Delphi process for final refinement of the tool (supplement S5, fig S8). 28 29 30 We used our extensive network of global colleagues working in the field of study to identify and invite the expert panel. Ten voting members accepted the invite to participate in the Delphi panel, and four members of the working group participated as non-voting members. We shared the ROBVALU tool draft, guidance document, and results of our participant testing with the panel members.

The first round of the Delphi process involved an anonymous survey to determine the signalling questions to be included. The second round took place via recorded video conferences with the aim of identifying common themes and reaching consensus on simplifying and harmonising language across the tool. The third and final round of the Delphi process included an anonymous survey for final consensus on the wording of the signalling questions and the proposed methods for providing a global ROB judgment. We used Google forms to prepare the surveys; the first survey used a 7 point Likert scale (ie, strongly agree, agree, somewhat agree, neutral, somewhat agree, disagree, and strongly disagree) to rate each item, with 70% agreement set as the cut-off threshold to retain or remove a signalling question. The final survey used a 3 point scale (ie, agree, neutral, and disagree) with a 70% agreement set as the cut-off threshold to retain the signalling question.

We had a 100% response rate in the first round of the Delphi process, with 80-100% consensus to retain all signalling questions. We also collected feedback from open ended questions for suggested edits for the signalling questions (supplement S6). In the second round of the Delphi process, we presented the ROBVALU tool, psychometric properties, exploratory factor analysis, and results of the first round of the Delphi to the panel members. After deliberating on the tool’s properties, agreement was reached to edit some signalling questions to simplify the language or to harmonise the language across the tool, which resulted in minor changes only. We also discussed how to make a final judgment for ROB for a study.

We had a 100% response rate in the third and final Delphi round, with 80-100% consensus on the tool’s signalling questions, including those with minor adjustments to the wording. We also established a consensus of >70% that the overall ROB judgment should match the most severe ROB judgment on an item, unless appraisers can provide justifications to rate the overall ROB lower (eg, many concerns on many items) or higher (eg, concern seems not to have an important influence on overall ROB). For example, if multiple subdomains were rated as very serious, the final judgment could be rated as extremely serious (supplement S7).

Risk-of-bias subdomains

ROBVALU includes seven key signalling questions across four subdomains: selection of participants into the study, completeness of data, measurement instrument, and data analysis ( table 2 ).

Subdomains and considerations in the Risk of Bias in Values and Utilities (ROBVALU) tool

Selection of participants into the study

Precise research questions include a clear definition of the target population. The study population of any empirical study must be representative for this target population, and is therefore, a critical component because bias in the selection will lead to biased estimates of the values people place on outcomes in the target population. 2 When assessing selection bias, users should consider the study’s sampling strategy, in particular if the achieved sample population deviates from the intended sample population, 2 because this might lead to biased estimates for the study’s population of interest owing to threats to internal validity. If the achieved sample population does not deviate from the intended sample population but differs from the population researchers intend to extrapolate the results to, this difference will result in a lack of generalisability. We refer to this lack of generalisability as indirectness, which encompasses applicability and external validity. The ROBVALU tool is not intended to deal with indirectness, a different domain in assessing the certainty of a body of evidence according to GRADE, but we are developing a tool that is specific to indirectness separately.

Completeness of data

When judging completeness of data, reviewers need to consider the response rate of the study population, the attrition rate if follow-up was involved, and the differential responders compared with non-responders. 2 High response rates and low proportion of loss to follow-up are clearly preferable, and a high proportion of non-response or dropout rates could be problematic. 2 Participants providing responses could plausibly differ from those who do not, and researchers should consider that results coming only from those participants who responded or completed follow-up might be misleading. 2

Measurement instrument

Reliable and valid instruments should be used to measure the relative importance of outcomes in values, preferences, and utility studies. 2 Using unreliable or poorly validated instruments can result in biased measurements of the outcome. Similarly, utility values for specific health states based on instruments not sufficiently validated that are used as input parameters for decision analytical models can result in biased estimates, such as quality adjusted life years derived from state transition models. 31 32 Researchers conducting primary empirical studies should provide information regarding the measurement properties of their chosen instrument. 2

Researchers should also demonstrate that the instrument has been used correctly and in a consistent manner across all participants in a study. For example, if the standard gamble is to be administered by an interviewer, but a subset of participants used self-administration, this could result in biased utility estimates that could be due to systematic differences between the two groups. In addition, an optimal representation of the outcome or health state should be presented or described in a way that accurately reflects the attribute the researchers intended to measure. This information could include a detailed explanation of how the outcome defines the experience, the probability of the outcome, durations, and possible consequences. Finally, researchers should evaluate whether participants had a proper understanding of the instrument to complete the tasks.

Data analysis

Studies should explore heterogeneity in values when appropriate and present results for the different subgroups. The data analysis plan and exploration of heterogeneity should be outlined a priori before collection of data. A causal framework that helps delineate health state and outcome interactions with possible confounding factors will help make assumptions explicit. If heterogeneity is found, the evaluator needs to consider whether the adjustment, stratification, or model selection used in the study reporting on values was appropriate. 2 Adjusting for important confounding factors (such as age if it is associated with the intervention and influences the estimated values) or reporting values in a stratified manner reduces biased estimates of the value placed on an outcome. In addition, self-inflicted biases, including selection bias or immortal time bias should be controlled for appropriately using modern causal inference methods (eg, target trial emulation or g methods for time varying confounding). 33

ROBVALU tool application

The assessment of ROB in studies evaluating the value people place on outcomes follows seven steps:

Specify the research or review question.

Specify the outcome being assessed.

Identify the sampling frame, the response rate and/or attrition rate, the measurement instrument used, and the data analysis plan.

Answer the signalling questions of the four subdomains.

Make a judgment if the four subdomains have important ROB concerns.

Formulate a ROB judgment for the four subdomains.

Formulate an overall ROB judgment for the study outcome being assessed.

The ROBVALU tool (supplement S8) provides users with space to record vital information of the study being assessed, and signalling questions to all four subdomains that must be answered. We validated a 4 point Likert-type scale (yes, probably yes, probably no, no) to respond to the individual signalling questions (items). When rating individual signalling questions, we suggest following the flowchart in figure 2 for consistent answers between raters. In each subdomain, the tool asks to specify how important the ROB issue is on a 4 point Likert-type scale (yes, probably yes, probably no, no), and how serious the overall ROB issue is on a 4 point Likert-type scale (not serious, serious, very serious, extremely serious). Responses to the signalling questions should provide the basis for the subdomain level judgment, of how important and how serious the ROB issues are in the study. Raters should provide a rationale for the response as free text, to justify their judgments. We suggest that the final judgment for each subdomain inversely correlates with the signalling question judgment. For example, in the measurement instrument subdomain, if the answer to “Was the instrument administered in the intended way?” was “No,” then the answer to “Are there important risk of bias issues concerning the measurement instruments?” should be “Yes.” If raters believe that the lowest signalling question judgment does not reflect the overall subdomain judgment, they might choose not to deem the results of the study at ROB for that subdomain, but they are asked to provide explanations for why they would not do this.

Rating individual signalling questions in the Risk of Bias in Values and Utilities (ROBVALU) tool

The global ROB judgment for a study corresponds to the lowest subdomain judgment ( table 3 ), because any domain level bias will lower our confidence in the study results. If users do not believe that the lowest subdomain judgment reflects the global ROB judgment, they should provide a justification. For example, if a study has a low response rate resulting in very serious ROB domain judgment and the study results are comparable to better quality studies, a reviewer might consider that the subdomain judgment does not reflect the global ROB judgment. Box 1 presents an illustrative example of a completed assessment (supplement S9).

Response options for judgments on risk of bias at an overall study level, according to the Risk of Bias in Values and Utilities (ROBVALU) tool

Example application of the Risk of Bias in Values and Utilities (ROBVALU) tool to assess risk of bias in values assigned to exacerbation of chronic obstructive pulmonary disease 34

In a study of 65 men and women with chronic obstructive pulmonary disease, researchers assessed the utility value that participants placed on an exacerbation, at seven study sites in the US when they visited an outpatient clinic within 48 hours of symptom onset. 34 Eligible participants were at least 40 years old and were current or former smokers with a history of at least 10 pack years. Of 65 participants, 59 completed the study, three were lost to follow-up, and three were ineligible. Utility values were measured using the EQ-5D.

An assessment using the ROBVALU tool revealed the following (supplement 9):

Selection of participants into the study would likely lead to risk of bias.

Exacerbations that required hospital admission were considered severe and were excluded from this study and might importantly bias the estimates. Thus, the population was deemed to be probably not representative of the intended population.

Completeness of data was present:

Only three patients were lost to follow-up, which did not cause risk of bias.

Measurement instrument caused some concern about risk of bias:

It was not clear whether the instrument was used in a valid and reliable manner, but it was applied in the intended way using a valid representation of the outcome. Patients also appeared to show an understanding of the instrument that was used and did not encounter difficulties, but this was not reported.

Data analysis did not cause concern for risk of bias:

Adjustment, stratification, and model selection was appropriate based on a plan created a priori.

ROBVALU assessment

Overall risk of bias was deemed serious because of issues related to the selection of participants into the study and the way the measurement instrument was used.

We have developed and validated the ROBVALU tool, a new instrument to assess ROB in studies measuring the value, utility, or relative importance that people place on health outcome. We followed a sequential mixed methods approach, by first adapting the signalling questions from the GRADE guidance for judging ROB across studies. ROBVALU differs from existing GRADE guidance by specifically assessing ROB in individual studies as opposed to across studies. 2 We iteratively revised the tool with our core group and an advisory group. The final draft tool contains 15 items in four subdomains: selection of participants, completeness of data, measurement instrument, and data analysis. We conducted a validation exercise with 15 participants that showed good reliability. Additional refinement using a modified Delphi process established construct validity on the final content of the tool.

Strengths and limitations

Assessing ROB is an essential step to assess the overall certainty of the evidence in a systematic review or health technology assessment and to develop a guideline. This assessment has often relied on adapting ROB tools not specifically designed for this type of research. 26 However, the lack of validation could lead to unreliable certainty of the evidence assessments, both for single studies and for a body of evidence. By using ROBVALU, evaluators can incorporate the ROB assessment into their meta-analysis, such as performing a sensitivity analysis to evaluate how studies with higher ROB might affect the study’s conclusion or primary outcomes. An advantage of the ROBVALU tool is the use of standardised GRADE terminology and judgments to facilitate assessment when establishing the certainty of the evidence. The ROBVALU tool can also be used to assess ROB in all elicitation studies of values, utilities, and importance of outcomes that use discrete choice, ranking, indifference, and rating methods. 35 Finally, the tool can be used in individual studies that use indirect methods to elicit people’s preferences, such as quality of life and EQ-5D scores.

This study and the derived tool also has several limitations. The new tool focuses on assessing values quantitively. For any given intervention, there is usually qualitative literature exploring what patients want to achieve and what they value (or not) from interventions; this information could be important for decision making. While some of the signalling questions might be used for qualitative studies, other signalling questions will not apply. Further exploration with qualitative studies should be performed to assess how ROBVALU can be adapted for that particular use, or whether a different tool is required. Furthermore, an exploratory factor analysis showed that one item in the tool had relatively poor fit (Was a valid representation of the outcome (health state) used?), but this poor fit could be due to the relatively small sample size. However, we retained this item because of feedback from the Delphi panel, who deemed it important. External validation of ROBVALU’s reliability by different users and on different studies will help refine the guidance and the tool.

Future implications

ROBVALU allows researchers to appraise individual studies reporting utilities, values, or the importance of outcomes for risk of bias. For example, in health technology assessments, the certainty of input variables from an individual study determines the certainty of outputs from decision analytical models (eg, cost utility and cost effectiveness analyses). 32 36 ROBVALU should also help with evaluating ROB as part of a systematic review, health technology assessment, or formal health guideline, to develop recommendations and make judgments across the overall body of this type of evidence (eg, assessing overall certainty of the evidence when following the GRADE approach).

Ethics statements

Ethical approval.

This international study was designed and coordinated at McMaster University after approval by the Hamilton Integrated Research Ethics Board (project ID 5634), and interviews and meetings were conducted in person or over video conference. All participants provided informed consent.

Contributors: The authors are epidemiologists, statisticians, systematic reviewers, and health services researchers, many of whom are involved with methods research and GRADE. Development of ROBVALU was informed by GRADE guideline 19, previously published tools for assessing risk of bias in intervention studies, systematic reviews of available tools to assess risk of bias in values and preferences, and the authors’ experience of developing similar tools to assess risk of bias. All authors contributed to development of the ROBVALU tool and to writing associated guidance. SGK, YZ, JLB, and HJS designed the study and formed the core group. YZ, JLB, and HJS conceived of the project. HJS oversaw the project and is guarantor. SGK, YZ, TD, JLB, and HJS drafted the ROBVALU tool. JN, PAC, FX, and US formed the advisory group. SGK led working groups and conducted the semi-structured interviews. SGK and LM analysed the data. HP-H, GH, YZ, and PAC assessed studies. PAC, FX, BE, ZSP, VW, ALS, J-ET, JN, LK, and US participated in the Delphi process as voting members, and HJS, YZ, SGK, and JLB were non-voting members. SGK and HJS drafted the manuscript. YZ, JLB, and HJS obtained funding for the study. All authors reviewed and commented on drafts of the manuscript. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding: The study was funded by the Canadian Institutes of Health Research (grant 401310 to HJS and JLB). The funders had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report, or decision to submit the article for publication.

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: support from the Canadian Institutes of Health Research for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

Provenance and peer review: Not commissioned; externally peer reviewed.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/ .

• ↵ Boyd CM, Singh S, Varadhan R, et al. Methods for benefit and harm assessment in systematic reviews. 2012. https://www.ncbi.nlm.nih.gov/books/NBK115750/pdf/Bookshelf_NBK115750.pdf
• Alonso-Coello P ,
• Guyatt GH ,
• Pieterse AH ,
• Stiggelbout AM
• McDonough CM ,
• Tosteson AN
• Torrance GW ,
• Rashidi AA ,
• Bleichrodt H ,
• Johannesson M
• Bridges JF ,
• Hauber AB ,
• Marshall D ,
• Horsman J ,
• Furlong W ,
• Slaughter KB ,
• Bambhroliya AB ,
• Schünemann HJ ,
• Piggott T ,
• Morgan RL ,
• Etxeandia-Ikobaltzeta I ,
• Brundisini F ,
• GRADE Working Group
• Kallenbach M ,
• Meerpohl J ,
• Sterne JAC ,
• Savović J ,
• Sterne JA ,
• Hernán MA ,
• Reeves BC ,
• Thayer KA ,
• Santesso N ,
• Higgins JPT ,
• Rooney AA ,
• ↵ Wells GA, Shea B, O’Connell D, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. 2000.
• ↵ Joanna Briggs Institute. The Joanna Briggs Institute critical appraisal tools for use in JBI systematic reviews checklist for analytical cross sectional studies. 2017.
• Mokkink LB ,
• Terwee CB ,
• Patrick DL ,
• Yepes-Nuñez JJ ,
• Creswell JW ,
• ↵ Helmer -Hirschberg O. Analysis of the future: The Delphi method. Rand, 1967.
• Murphy MK ,
• Lamping DL ,
• Siebert U ,
• Bayoumi AM ,
• ISPOR-SMDM Modeling Good Research Practices Task Force
• Arvandi M ,
• Goossens LM ,
• Nivens MC ,
• Rutten-van Mölken MP
• Soekhai V ,
• Whichello C ,
• Levitan B ,
• Briggs AH ,

This paper is in the following e-collection/theme issue:

Published on 18.6.2024 in Vol 26 (2024)

Identification of Ethical Issues and Practice Recommendations Regarding the Use of Robotic Coaching Solutions for Older Adults: Narrative Review

Authors of this article:

• Cécilia Palmier 1, 2 * , MSc   ; 
• Anne-Sophie Rigaud 1, 2 * , Prof Dr Med   ; 
• Toshimi Ogawa 3 , PhD   ; 
• Rainer Wieching 4 , Prof Dr   ; 
• Sébastien Dacunha 1, 2 * , MSc   ; 
• Federico Barbarossa 5 , MEng   ; 
• Vera Stara 5 , PhD   ; 
• Roberta Bevilacqua 5 , MSc   ; 
• Maribel Pino 1, 2 * , PhD  

1 Maladie d’Alzheimer, Université de Paris, Paris, France

2 Service de Gériatrie 1 & 2, Hôpital Broca, Assistance Publique - Hôpitaux de Paris, Paris, France

3 Smart-Aging Research Center, Tohoku University, Sendai, Japan

4 Institute for New Media & Information Systems, University of Siegen, Siegen, Germany

5 Scientific Direction, Istituto Nazionale di Ricovero e Cura per Anziani, Ancona, Italy

*these authors contributed equally

Corresponding Author:

Anne-Sophie Rigaud, Prof Dr Med

Service de Gériatrie 1 & 2

Hôpital Broca

Assistance Publique - Hôpitaux de Paris

54 rue Pascal

Paris, 75013

Phone: 33 144083503

Fax:33 144083510

Email: [email protected]

Background: Technological advances in robotics, artificial intelligence, cognitive algorithms, and internet-based coaches have contributed to the development of devices capable of responding to some of the challenges resulting from demographic aging. Numerous studies have explored the use of robotic coaching solutions (RCSs) for supporting healthy behaviors in older adults and have shown their benefits regarding the quality of life and functional independence of older adults at home. However, the use of RCSs by individuals who are potentially vulnerable raises many ethical questions. Establishing an ethical framework to guide the development, use, and evaluation practices regarding RCSs for older adults seems highly pertinent.

Objective: The objective of this paper was to highlight the ethical issues related to the use of RCSs for health care purposes among older adults and draft recommendations for researchers and health care professionals interested in using RCSs for older adults.

Methods: We conducted a narrative review of the literature to identify publications including an analysis of the ethical dimension and recommendations regarding the use of RCSs for older adults. We used a qualitative analysis methodology inspired by a Health Technology Assessment model. We included all article types such as theoretical papers, research studies, and reviews dealing with ethical issues or recommendations for the implementation of these RCSs in a general population, particularly among older adults, in the health care sector and published after 2011 in either English or French. The review was performed between August and December 2021 using the PubMed, CINAHL, Embase, Scopus, Web of Science, IEEE Explore, SpringerLink, and PsycINFO databases. Selected publications were analyzed using the European Network of Health Technology Assessment Core Model (version 3.0) around 5 ethical topics: benefit-harm balance, autonomy, privacy, justice and equity, and legislation.

Results: In the 25 publications analyzed, the most cited ethical concerns were the risk of accidents, lack of reliability, loss of control, risk of deception, risk of social isolation, data confidentiality, and liability in case of safety problems. Recommendations included collecting the opinion of target users, collecting their consent, and training professionals in the use of RCSs. Proper data management, anonymization, and encryption appeared to be essential to protect RCS users’ personal data.

Conclusions: Our analysis supports the interest in using RCSs for older adults because of their potential contribution to individuals’ quality of life and well-being. This analysis highlights many ethical issues linked to the use of RCSs for health-related goals. Future studies should consider the organizational consequences of the implementation of RCSs and the influence of cultural and socioeconomic specificities of the context of experimentation. We suggest implementing a scalable ethical and regulatory framework to accompany the development and implementation of RCSs for various aspects related to the technology, individual, or legal aspects.

Introduction

Challenges associated to population aging.

Technological and medical advances have led to a demographic shift in the population, with the number of older adults constantly increasing. According to the United Nations [ 1 ], older adults (aged 60-65 years) will represent 16% of the world’s population in 2050. In addition, life expectancy is increasing, from 64.2 years in 1990 to 72.6 years in 2019, and is expected to reach 77.1 years in 2050 [ 1 ]. However, there is a wide diversity of health conditions among older adults. The health status of older adults is dependent on multiple factors, including nonmodifiable genetic factors and environmental factors, such as lifestyle [ 2 ]. Thus, older adults represent a very heterogeneous population with multiple and diverse needs and desires. With advancing age, the loss of functional independence; frailty; and other health diseases such as cardiovascular problems, cancers, osteoarthritis, osteoporosis, or major neurocognitive disorders may appear [ 3 - 5 ]. Among age-related conditions, major neurocognitive disorders (eg, Alzheimer disease) receive particular attention due to the increasing prevalence of these diseases [ 6 ].

The aging population is not only a public health issue but also a socioeconomic one. To face this challenge, it is important to develop preventive measures to support active and healthy aging and to preserve the independent functioning and quality of life of older adults. The adoption of healthy behaviors can help prevent or delay the onset of pathologies or treat them if detected early [ 7 ].

The Use of Technologies for Older Adults

Preventive health measures can be supported through new technologies, such as robotic coaching solutions (RCSs) that promote healthy aging among older adults [ 8 , 9 ]. RCSs have been defined as personalized systems that continuously monitor the activities and environment of the user and provide them with timely health-related advice and interventions [ 10 - 12 ]. These systems can help users define and achieve different health-oriented goals [ 12 ].

RCSs may encompass artificial intelligence (AI) technologies that can analyze user data, personalize coaching programs, and adapt recommendations based on each individual’s needs [ 1 , 13 - 19 ]. RCSs can involve robots equipped with sensors such as cameras, microphones, or motion sensors to collect real-time data about the user, AI, and programming that enables their interaction with users [ 20 , 21 ]. These technologies are often equipped with voice and visual recognition and learning capabilities [ 20 , 21 ]. They can benefit from advanced natural language processing techniques, which allow for understanding of the user’s input, facilitating natural and effective communication [ 22 ]. RCSs can offer guidance, support, and feedback based on preprogrammed information or real-time data analysis. These data can inform coaching strategies and allow RCSs to provide users with relevant feedback [ 8 ].

RCSs can also encompass a virtual agent, which refers to a computer program or an AI system that interacts with users in a manner that simulates human conversation [ 14 , 18 , 23 ]. A virtual agent is an animated character capable of adopting a social behavior mimicking that of humans to encourage the users to make changes in their habits [ 14 ]. Virtual agents might take the form of a chatbot, voice assistant, or other AI-driven communication system [ 14 ]. Biometric monitoring devices to track physiological data such as heart rate, sleep patterns, or stress levels can also be included in RCSs [ 8 , 20 , 21 ]. These data can contribute to the configuration of personalized coaching plans. RCSs can also encompass advanced data analytics that can process large data sets generated by users’ interactions and behaviors. This functionality helps in identifying patterns, trends, and areas for improvement in coaching strategies [ 24 ]. Integrating Internet-of-Things devices in RCSs can provide additional data points about a user’s environment, lifestyle, or habits, thus contributing to a personalized coaching approach [ 25 ].

Health-oriented RCSs could enable users to lead a healthy lifestyle, by identifying needs and goals and providing appropriate risk predictions and individualized recommendations [ 12 , 26 - 28 ]. There are RCSs dedicated to a particular domain, such as physical activity or motor rehabilitation [ 9 , 16 ]. Others may have the objective of promoting independent and healthy aging [ 29 ].

Promoting active and healthy aging can allow older adults to maintain their independence and continue to live at home [ 4 , 30 ], which is a wish of many [ 3 ]. This intervention could also help to reduce the need for assistance, usually provided by informal caregivers and health professionals [ 4 , 19 , 30 - 33 ]. Furthermore, RCSs could lead to a reduction in individual and collective health care expenses [ 4 , 32 , 34 ] by easing access to health and social care interventions to a wide population, including hard-to-reach (eg, geographically isolated) individuals. However, although the use of health-related RCSs could have many benefits, several ethical issues arise with their development and implementation in human environments [ 3 , 35 - 38 ].

An Ethical Framework for the Use of Technologies for Older Adults

For RCSs to contribute to active and healthy aging, it is important that all the stakeholders (engineers, geriatricians, psychologists, etc) involved in their design and implementation refer to an ethical framework [ 3 , 38 ]. It is also important to inform society (politicians and legal experts) about such an extension of technology in people’s lives (private, professional, medicosocial, and commercial context), so that we can create a legal framework for the use of these technologies. An analysis of the way in which ethical and legal dimensions have been addressed by studies, in the field of RCSs for health care, seems useful to support the key actors in their development and implementation. The growing interest in the ethical questions associated with the use of social and assistive robots is evidenced by the volume of literature reviews [ 3 , 12 , 18 , 31 , 32 , 37 , 39 - 51 ] on the topic.

Now, it appears appropriate to systematically examine this body of work, focusing on the ethical analysis, and provide an overview of the literature. Therefore, we performed a review of the literature on RCSs for older adults using the European Network of Health Technology Assessment (EUnetHTA Core Model; version 3.0) model [ 52 ] for analysis. This Health Technology Assessment (HTA) model makes it possible to assess the intended and unintended consequences of the use of a specific technology regarding multiple domains (eg, technological, ethical, clinical, and organizational), providing methods and concepts for this analysis [ 53 ]. Therefore, HTA is a process that informs decision-making about the introduction of new technologies such as RCSs in health care. It also seems necessary to issue guidelines for the development and implementation of health-oriented RCSs [ 54 ].

The objective of this study was to highlight the main ethical questions and corresponding recommendations linked to the use of RCSs for older adults for engineers, researchers, and health professionals in this field. For this purpose, we conducted a narrative literature review using the ethical dimension of the EUnetHTA Core Model to guide the analysis. To the best of our knowledge, such a study has not been conducted so far.

A thematic analysis of the literature was performed to identify publications that describe RCSs for supporting older adults in health care and prevention and those that address ethical issues and recommendations regarding their development and implementation. The methodology used for the narrative review was inspired by the study by Green et al [ 55 ].

Inclusion and Exclusion Criteria

The review encompassed papers focusing on all populations, with particular attention to older adults. It focused on the concept of RCSs for health, while also incorporating publications discussing other health technologies for older adults if the authors have delved into relevant ethical considerations for their development or implementation.

The context of the review revolved around the use of RCSs (or related technologies), especially for older adults, across diverse living environments such as homes, hospitals, and nursing homes. Publications addressing RCSs and related ethical issues within the health care domain were considered, whereas those focusing solely on technical aspects (eg, AI and deep learning) or those outside the health care domain were excluded.

Various types of publications, including theoretical papers, research studies, and reviews, were included if they offered ethical reflections or recommendations for RCS use in health care. These reflections and recommendations were expected to align with the topics and issues of the ethical dimension of the EUnetHTA Core Model.

All publications, regardless of language (English or French), were eligible if published after 2011. This time frame was chosen considering the technological advancements over the past decade, which may have influenced the evolution of ethical issues and recommendations in the field of remote care systems and related technologies. Textbox 1 summarizes the inclusion and exclusion criteria adopted for the selection of papers in this review.

Inclusion criteria

• Types of participants: all populations
• Interventions or phenomena of interest: RCSs or other technologies used in health care, if ethical issues are discussed
• Context: the use of RCSs in the health care sector
• Paper type: all paper types (theoretical papers, research studies, and reviews) that discuss ethical issues
• Language: English or French
• Date of publication: after 2011

Exclusion criteria

• Types of participants: not applicable
• Interventions or phenomena of interest: RCSs or all other types of technology outside the health care sector
• Context: the use of RCSs in non–health care sectors
• Paper type: papers about RCSs and other technologies that are not dealing with ethical issues
• Language: all other languages
• Date of publication: before 2011

Search Strategy and Study Selection

The review was conducted using the following keywords: “seniors,” “older adults,” “social robots,” “assistive robots,” “assistive technology,” “robots,” “virtual coach,” “e-coaching,” “coaching system,” “coaching device,” “ethics,” and “recommendations.”

The review was performed between August 2021 and December 2021 using the PubMed, CINAHL, Embase, Scopus, Web of Science, IEEE Explore, SpringerLink, and PsycINFO databases.

This search allowed us to find 4928 initial publications. Then, secondary research using references from other articles and the same inclusion criteria was conducted. This search allowed us to find 13 additional papers.

In total, 4943 papers were analyzed. The selection of the final publications was performed after reading the title and abstract first and, then, the full article. This selection process helped us to exclude irrelevant papers and duplicates ( Figure 1 ). In total, 0.51% (25/4943) of the papers were included in our review.

Data Analysis Criteria

The selected papers were analyzed using the ethical domain of the EUnetHTA Core Model [ 52 ]. Proper registration of the use of EUnetHTA Core Model for the purpose of this review was made on the HTA Core Model website [ 52 ].

The model was developed for the production and sharing of HTA information, allowing for the support of evidence-based decision-making in health care, but it can also be customized to other research needs. The EUnetHTA Core Model is composed of 9 domains, each including several topics. Each topic also includes different issues (ie, questions that should be considered for the evaluation of health technologies). Thus, the model is structured into 3 levels: domain (level 1), topic (level 2), and issue (level 3). The combination of a domain, topic, and issue is linked to an assessment element ID, which can be identified using a specific code for standardization purposes (B0001, B0002, etc).

The main EUnetHTA model domains include the following: (1) health and current use of the technology, (2) description and technical characteristics of the technology, (3) safety, (4) clinical effectiveness, (5) costs and economic evaluation, (6) ethical aspects, (7) organizational aspects, (8) patient and social aspects, and (9) legal aspects.

The ethical domain (level 1) in the EUnetHTA Core Model [ 52 ] includes 5 topics (level 2): “benefit-harm balance,” “autonomy,” “respect for people,” “justice and equity,” and “legislation.” Each of these topics includes several issues (level 3) [ 52 ].

In this study, 2 authors (CP and ASR) independently analyzed the 25 selected articles. First, they read the articles several times to improve familiarity with the ideas addressing the ethical aspects of RCSs. Then, in each publication (methods, results, and discussion sections), they identified segments of data that were relevant or captured an idea linked to the “ethical” domain of the model. A subsequent exploration of the coded data (sentences or set of statements) was performed to get a more precise classification at the topic level (level 2) and at the issue level (level 3). Then, the coding was performed using the HTA nomenclature. The 2 experts (CP and ASR) compared their results. In a few cases, the coding results showed a lack of consensus between the 2 coding authors, which was resolved through a subsequent discussion between them. Interrater correlation was not calculated.

A thematic analysis using the EUnetHTA framework for conducting a literature review has been described in other studies [ 56 , 57 ]. Furthermore, the use of EUnetHTA to perform an ethical analysis of health technologies has already been proposed [ 58 ]. The 25 selected articles were all coded using this methodology. Some authors have previously emphasized the possibility of overlapping issues between topics in the HTA analysis. They have suggested to assess the overlapping issues in the most relevant topic section [ 59 ].

This review was not registered, and a protocol for the review was not prepared.

Selected articles are presented in Multimedia Appendix 1 [ 3 , 12 , 18 , 31 , 32 , 37 - 51 , 60 - 64 ]. For each topic, we have presented our findings in terms of questions and recommendations according to the EUnetHTA Core Model, wherever possible.

Ethical Issues and Recommendations for the Use of New Technologies

This section aims to summarize the ethical analysis performed regarding the use of RCSs with older adults and to provide recommendations for ethical use of these devices. Table 1 presents a synthetic summary of the elements presented in this section.

Topic 1: Benefit-Harm Balance

RCSs should be developed according to the principles of beneficence (ie, to promote the interest of users) and nonmaleficence (ie, to avoid inflicting harm) [ 39 , 60 , 64 ].

What Are the Known and Estimated Benefits and Harms for Patients When Implementing or Not Implementing the Technology?

Risk of social isolation.

According to Sharkey and Sharkey [ 50 ], technological devices, when used appropriately, could benefit older adults by promoting social interaction and connection with their loved ones [ 4 , 31 , 40 ]. Broadbent et al [ 19 ] have discussed the potential of robots to reduce older adults’ social isolation. However, other authors reported the negative influence of the use of robotic devices on human contact [ 31 , 32 , 65 ]. The use of robots (eg, telepresence robots) to make some cost savings (eg, reducing travel costs and time spent on trips for family and professionals to visit older adults) would reduce face-to-face interactions [ 3 , 36 , 39 , 40 ]. Moreover, according to Körtner [ 47 ], the more people become accustomed to communicating with robots, the less they will be used to communicating with humans. The use of social robots could lead to a reduction of interactions with humans and thus to social isolation and emotional dependence [ 39 ]. However, the influence of technological devices, such as RCSs, on social isolation is still under debate, and the impact of technology would depend on the manner in which it is used.

To avoid exacerbating the users’ social isolation, Portacolone et al [ 38 ] advocate that social robots and similar technologies should be designed with the objective of fostering interactions with other humans, for instance, keeping users informed about the entertainment and socializing activities near their home, connecting them with their loved ones, and so on.

Risk of Deception

Another major risk for users is deception [ 39 , 64 , 66 ]. Portacolone et al [ 38 ] described 3 types of deception that people with neurocognitive disorders may face when interacting with social robotic systems but which may also apply to all users. The first type involves the user’s misconception of what is driving the technological device [ 51 ]. Users may be misled if they think that behind a medical chatbot, there is a real physician who communicates and reads their messages [ 44 ] or, alternatively, if they are not aware that, at some point, there are real humans guiding the technological device [ 38 ]. The second type refers to robotic devices programmed to express feelings or other types of affective communication, which may lead the user to believe that the system’s emotions are authentic. Related to this issue, Körtner [ 47 ] discussed how some older adults may fear that their social robot will forget them during their absence from home. The resemblance with the living in terms of affective behavior (eg, crying, laughing, or expressing concern) can make the user believe that there is a reciprocity between human and robot feelings [ 43 ]. The last type of deception is related to the inadequate interpretations that older adults may have regarding the nature of the robot, for example, thinking that an animal-shaped robot is a real animal or a pet [ 38 ]. Some current developments of social robots tend to make them resemble a living being, in terms of their verbal and nonverbal behaviors [ 34 , 60 ] or by highly anthropomorphizing their design [ 47 ], which may blur the boundary between the real and the artificial [ 45 , 60 ]. These design choices can also impact users’ dignity by infantilizing them as they are led to believe in something that is false [ 50 ].

However, according to some researchers [ 51 , 63 , 64 ], the notion of deception should be considered in terms of the gradation between what is morally acceptable and what is not. Deception would be morally acceptable when it aims to improve a person’s health or quality of life, for example, the use companion robots to calm a person experiencing behavioral disorders linked to dementia [ 51 ].

According to Danaher [ 43 ] and Vandemeulebroucke et al [ 40 ], to avoid deception, it is essential to be transparent to users about the design and operation of devices. As the information given to the participants is the basis for obtaining consent to use the technology, it is essential to offer them documents explaining how the device is built and its advantages and limitations in a clear manner adapted to the user’s knowledge and experience. It is also important to inform users on how to behave with technology [ 12 ]. Researchers should also answer users’ questions, pay attention to their feedback, and use it to improve the device and its documentation [ 60 ]. During experiments with RCSs, it is also important that researchers regularly remind participants of the nature of the technological device to reduce the risk of misinterpretation and to ensure that they still consent to participate in the study [ 38 ].

Biases of Algorithms

An autonomous device does not work without AI or algorithms that allow it to make decisions. However, these technologies are created by humans, and programming biases can be incorporated into them and lead to failures [ 44 ]. A technological device can, for instance, misread a situation and react accordingly, leading to a safety risk for the user [ 18 ]. Thus, it is essential that the researcher scrutinizes the algorithms used in RCSs before their implementation [ 44 ]. Fiske et al [ 44 ] also suggest providing the users with detailed explanations about the algorithms present in the technological device they are using.

What Are the Benefits and Harms of the Technology for Relatives, Other Patients, Organizations, Commercial Entities, Society, Etc?

At the society level, Boada et al [ 39 ] mentioned an ethical consideration related to the ecological impact of robotic devices in the current context of climate crisis and the lack of natural resources. The construction of RCSs requires raw materials, high energy consumption, and the management of their waste. Therefore, it is important for developers to design technologies that consume less energy and can be recycled.

Are There Any Unintended Consequences of the Technology and Its Application for Patients?

Technologies evolving very quickly.

For some older adults, technologies evolve very quickly, which makes it difficult for them to keep up with [ 62 ]. Denning et al [ 67 ] encourage designers to develop products that are intuitive to use or to offer users a simplified training. However, although some technologies are progressing quickly, technological limitations are still present, especially regarding social robotic systems, impacting their performance [ 68 ] and generating frustration among some users [ 69 ].

Unsuitability of Technology

The lack of experience with the technologies and the fact that the systems are not suitable to everyone can reduce the usability and acceptability of RCSs among older adults [ 3 , 60 , 62 ]. Frennert and Östlund [ 62 ] highlighted that some older adults were not confident in their ability to handle a robot because of previous complicated experience with technology. Peek et al [ 70 ] also reported that users had doubts about their ability to use the technology and feared that they would easily forget how to use it. They may also fear false alarms generated by monitoring technologies. For example, a person may decide to sit on the floor, but this behavior can be considered as a fall by the technology, and it could call for an ambulance to be sent to the person’s home in vain [ 70 ].

To promote acceptability and usability of RCSs, it is essential to develop them considering the capabilities, needs, and wishes of various users [ 31 , 47 ]. “User-centered design” approaches should be used for this purpose [ 71 ]. This methodology must be performed in a continuous manner to consider the development, new preferences, and experiences of the users. Technology assessment should also be conducted before deployment in ecological environments to improve the predictability of RCSs and decrease the risk of confusion and accidents [ 40 , 47 ].

Topic 2: Autonomy

According to Anderson and Kamphorst [ 42 ], the notion of autonomy implies the recognition of people, for instance, users of RCSs, as thinking individuals who have their own perspective on matters and are able to judge what is best for them.

Is the Technology Used for Individuals Who Are Especially Vulnerable?

Free and informed consent is a prerequisite for the involvement of an individual in research, regardless of the domain. This aspect is mentioned in numerous codes and declarations such as the Declaration of Helsinki (1964-2008) [ 72 ]. In the context of studies of the use of RCSs, this principle ensures that the person has freely chosen to use a device. However, some older adults, particularly those with cognitive disorders, may have difficulties in understanding and evaluating information related to RCSs and therefore in making appropriate choices [ 3 ]. Moreover, the person may not remember that the RCS is in their environment or how it works [ 38 , 44 ]. The question of how to ensure that the older adult has understood the purpose of RCS and that their choice of using the technology is based solely on their own decision and not that of a relative, caregiver, or institution has also been discussed [ 46 ].

Researchers in the field of RCS should adapt to the cognitive abilities of the populations they work with to facilitate communication and decision-making [ 46 ]. Thus, the observation of the person’s behavior is necessary to identify potential reservations regarding the use of RCSs. When the person is very vulnerable to respond, informed consent could be sought by proxy such as from children, spouse, or partner [ 46 , 64 ]. However, according to Diaz-Orueta et al [ 37 ], the final decision of using RCSs lies with the user. To prevent loss of capacity and to guard against any risk of inducement to participate, advance directives [ 46 , 64 ] or implementation of an advance power of attorney [ 46 ] can be proposed.

Does the Implementation or Use of the Technology Affect the Patient’s Capability and Possibility to Exercise Autonomy?

Dependence on the technology.

Although the main interest of RCSs for older adults is the maintenance of functional independence, it has been claimed that these devices could make people dependent on them. By replacing users in tasks that they can still perform, the use of RCSs could create new forms of vulnerability [ 3 , 31 , 39 , 41 , 51 ].

People could rely entirely on autonomous technological devices, such as RCSs, to guide their behaviors, goals, and actions [ 12 , 73 ]. A questioning of the authenticity of users’ actions has been mentioned by Anderson and Kamphorst [ 42 ]. Users might not feel responsible for the success of their actions if they feel they are completely driven by the guidance of the RCS. People could also develop emotional and psychological feelings toward the technology. This may have negative consequences for the individuals [ 38 , 49 ] and lead to new vulnerabilities [ 39 ].

Loss of Freedom

Another ethical issue relates to the conflict between the user’s safety, encouraged by the technology guidance, and a loss of freedom. The RCS could impose constraints on the user under the pretext that the user’s actions are not good for them [ 39 , 40 , 74 ]. Sharkey and Sharkey [ 50 ] explained that to promote home care, RCS could act as a supervisor (ie, programmed to ensure that no danger is present and, if there is a danger, to implement procedures to stop it and avoid it in the future). For instance, the RCS could prevent the person from eating fatty and high-caloric food because it is harmful to them. To protect users and ensure that they live in good health, individuals using RCSs could end up being deprived of certain actions or being under some type of “house arrest” [ 50 ].

One of the goals of using such RCSs is to support older adults’ independence; therefore, it is essential that developers and researchers in the field take measures to preserve the person’s autonomy [ 75 ]. Furthermore, RCS users must have the opportunity to evaluate and re-evaluate the role given to the device, to assess whether the system is reliable and whether it is serving their interests [ 12 , 42 ].

Creating a New Source of Authority

The use of RCSs could alter human relationships, for example, by creating tensions between older adults and their informal caregivers. Their use could also create some tensions with health care professionals by creating a new source of authority [ 12 ]. Monitoring older adults through RCSs can generate anger in the user, for example, when the device insists that the older adult should take a medication that they do not want to take [ 41 , 75 ].

Topic 3: Respect for Persons

Does the implementation or use of the technology affect human dignity.

Human dignity may be affected by the use of RCSs as these technologies may be perceived as “problem evocators” [ 41 ]. Some RCSs are used to compensate for impaired capacities. However, according to Körtner [ 47 ], their use can make older adults aware of their limitations and lead to negative feelings, anxiety, or exhaustion. RCS use can also lead to a form of stigmatization by making one’s own inabilities visible to others [ 3 , 70 ]. It is important to have positive communication regarding RCSs, to provide a less stigmatizing view of their use.

Does the Technology Invade the Sphere of Privacy of the Patient or User?

To continue living at home, users are increasingly willing to tolerate intrusion in their privacy [ 70 ], but they are not always aware of when and how they are being monitored by RCSs [ 61 ]. Portacolone et al [ 38 ] provided the example of an animal-shaped companion robot, for which the older adults can signal that they no longer wish to interact with it by putting the robot to sleep. However, the animal-shaped robot can record data even when it is sleeping, but users are not always aware of this information. Forgetfulness and the lack of understanding of the device can lead to the risk of manipulation and coercion [ 44 ]. The person who is vulnerable may forget that they are being monitored and reveal personal information [ 50 ].

Technological devices, such as RCSs, must remain under the control of the users [ 47 ]. Users should have the ability to define when and where the device is used—when it collects data—to maintain their privacy, especially in intimate or private care settings.

Security of Data

According to Portacolone et al [ 38 ], remote monitoring technologies are usually controlled by third parties, sometimes even operating in another country, which can lead to cultural biases during the interaction between the older adult and the RCS. This context involves the risk that the person controlling the device (third party) takes advantage of the older adult’s vulnerability to steal their personal information or exposes the user to financial abuse [ 38 ]. Older adults are not always aware or vigilant about the sharing and use of data, which may be personal and sensitive [ 73 ]. Furthermore, RCSs can be connected to internet services that collect, store, and transfer these sensitive data [ 47 ] for commercial use [ 49 , 61 ].

In addition, the use of technologies connected to digital networks involves the risk of hacking and unauthorized surveillance [ 34 , 51 ], which can make people vulnerable [ 62 ]. Denning et al [ 67 ] found that home robots could not only be remotely located and identified but also hacked and controlled. First, users may have either preconceived and erroneous ideas about the capabilities of the device or a lack of knowledge to evaluate the safety, especially regarding data protection [ 3 ]. Second, users do not always configure their technological device correctly or update them [ 67 ].

Encryption or security systems must be put in place to protect users’ personal data captured by the devices at every stage: during collection, storage, transmission, and processing [ 3 ]. Researchers must also give particular attention to data security. In Europe, for instance, researchers and technology providers are required to comply with the General Data Protection Regulation [ 40 , 76 ]. Data collection must be performed legally or approved by the local relevant ethical committees.

To address data security challenges, 3 principles are recommended by Ienca et al [ 46 ] when developing technological devices: transparency, legitimate purpose, and proportionality. Transparency refers to the fact that the user knows that the system is collecting data and has consented to it. The user must also have precise information about when and what type of data are recorded and who has access to them [ 47 ]. Legitimate purpose refers to the notion that the monitoring and collection of data is performed for a specific purpose, (ie, in the best interest of the user or, if applicable, a relative who has consented to it). Finally, the principle of proportionality refers to the fact that the data collected are not disproportionate to the user’s needs.

Topic 4: Justice and Equity

The consequences of the technology implementation on the distribution of health care resources was discussed in the literature.

How Does Implementation or Withdrawal of the Technology Affect the Distribution of Health Care Resources?

Societal pressure.

Socioeconomic issues are also linked to the development and use of RCSs can also be raised. Individual freedom may be hindered by the “incentive” of certain stakeholders or authorities to enforce the use of RCSs [ 37 ]. The use of RCSs and similar systems may also lead to a lesser involvement of relatives, caregivers, and institutions that provide care to older adults and to the reduction of care costs; these perceived economic benefits may pressurize older adults to consent to use these devices [ 40 , 46 ]. It is also possible that older adults may have to agree to use the technological device to receive other health care benefits (eg, aids and subsidies) [ 42 ].

Digital Divide

Different opportunities to access RCSs can result in digital divide, defined by the Organisation for Economic Cooperation and Development [ 77 ] as a gap between those who have access to information and communication technologies and those who do not. This difference can create educational, economic, social, and even health-related disparities among citizens. Some citizens would be able to use these devices and, therefore, could benefit from their advantages, whereas others will not be able to use them and will not enjoy their benefits. The use of technologies in the health care context, through public or private institutions, should be subject to previous authorization by independent ethical committees to ensure that the use of these devices will not harm users in any way.

Inequalities in Resources

Questions about justice, equity, and equality among all citizens also arise [ 12 , 40 , 46 ]. RCSs have relatively high costs [ 64 ] and can generate additional expenses such as an internet subscription [ 3 ] that only a part of the population can afford, and this may be owing to the lack of research allowing to measure the cost-to-benefit ratio of these technologies on health [ 32 ]. It is important to ensure the access to RCSs among different living areas (ie, urban and rural). Therefore, involving municipalities and neighborhood associations seems an interesting way of raising awareness about the opportunities offered by RCSs for older adults and reaching a wider range of people.

To promote justice, equity, and fair distribution, Ienca et al [ 46 ] and Wangmo et al [ 64 ] recommend reducing the development costs of RCSs by promoting an open dissemination of source codes. In addition, RCSs should be distributed in priority to those in greatest need; therefore, measures to ensure access to RCSs under fair conditions should be established [ 51 ]. Joachim [ 78 ] also suggests to cover some of the costs of these health care–oriented technologies through health insurance.

Recommendations have been published by researchers to improve equality of access to technologies, such as using open-source software, providing priority access for individuals with low income, or relying on certain collective financing systems such as retirement or health insurance [ 46 , 51 , 78 ]. Discussions must be conducted among developers, legislators, and private and public organizations to identify viable financing solutions that allow for fair distribution of RCSs.

Replacement of Professionals

Researchers have also reported fears expressed by older adults and caregivers about how the use of technological devices could eliminate care-related jobs or replace humans [ 17 , 34 , 48 , 61 ]. There are also concerns about the use of these technological tools to reduce health care costs by decreasing the number of available health care resources and services, thereby exacerbating social inequalities [ 44 ]. The introduction of health-oriented RCSs requires adapting the contexts of care practices, which may threaten their quality [ 39 ]. Their incorporation into the care work environment can be difficult because the devices are automated and some care situations are unpredictable [ 17 , 62 ]. Furthermore, the gestion of certain tasks by technological devices requires a restructuring of the roles and responsibilities of caregivers [ 39 ]. Fiske et al [ 44 ] highlight that there are currently no recommendations or training to enable health care professionals to adopt RCSs, even though these professionals are increasingly confronted with technological devices in their practice.

The incorporation of RCSs must always be accompanied by a discussion with concerned care professionals regarding the advantages and limits of the technology. Professionals must also be supported in the use of these devices through effective training. Structured training and supervision will contribute to the development of a controlled framework of practice around the use of RCSs and thus avoid potential abuse [ 44 ]. Moreover, to encourage their use among professionals, it is essential to clearly define the role of RCSs as an additional resource for professionals and not a replacement of human care services [ 44 ].

Topic 5: Legislation

The ethical challenges linked to the lack of existing legislations and regulations dedicated to the use of the technology were discussed in the literature.

Can the Use of the Technology Pose Ethical Challenges That Have Not Been Considered in the Existing Legislations and Regulations?

Safety of devices.

The use of RCSs by older adults can result in damage and harm to their environment [ 79 ], especially when the device is still at the prototype stage [ 47 ]. Safety risks linked to the use of RCSs (eg, malfunctioning of the technology and incorrect decisions made by the coaching system) arise when they share a common space with humans and interact with them [ 39 ]. The following questions must be considered: Who is responsible in case of an accident, and who pays for the damages [ 39 , 40 , 48 , 62 , 80 ]? Is it the designer, the device, or the user himself? Currently, the civil code favors the cascade system (ie, first, the liability falls on the designer of the product; then, on the developer; and finally, on the user who has not followed the rules of use) [ 74 ]. However, the more the machine becomes autonomous, the less the existing legal frameworks can answer these questions [ 80 ]. This is a key legal issue regarding the implementation of RCSs in real settings because the person responsible for damage to the user or the environment may incur legal or even penal proceedings.

Damage and prejudice can also be caused by a failure to share authority [ 45 , 49 , 60 ]. Who between the human and the technological device holds the power to make decisions and control a functionality [ 81 ]? According to Grinbaum et al [ 45 ], it is important to specify the circumstances in which the human must take control over the technological device (RCS) and those in which the device should decide autonomously. According to Riek and Howard [ 49 ], it is preferable that in certain cases, the technological device, although autonomous, requires a human validation of its actions to keep the user in control of the device. In addition, Bensoussan and Puigmal [ 80 ] suggested the idea that technological devices must have an emergency stop button, so that the human can switch off the technology at any time.

Regulation of Technology

Currently, there is a gray area between the capabilities of RCSs, the reality of the field, and the regulations in force [ 38 ]. To accompany the researcher during the whole process of development and diffusion of RCSs, an ethical framework should be established [ 18 , 60 ]. Specifically, this can be in the form of an ethical code of conduct illustrating the expectations to all the employees of a company [ 18 ]. The researcher must regularly inform themselves about the ethics to be consistent with the evolution of the regulatory framework [ 60 ]. However, according to Nevejans [ 82 ], these ethical recommendations have no legal value and cannot protect humans from the damage caused by new technologies. Thus, it is necessary to think about a new legal framework to protect the users of RCSs [ 37 ].

The use of technologies, such as RCSs, in the health care field has grown significantly in recent years [ 17 , 18 ]. RCSs are increasingly being used for older adults with the aim of promoting healthy behaviors, quality of life, and well-being. However, the use of RCSs also raises several ethical challenges regarding the cost-to-benefit balance of these new care practices, respect for the autonomy of users, respect for privacy, justice and equity linked to their access, or need for a suitable legal framework. Such challenges could be addressed by establishing relevant recommendations for the development and use of RCSs. Some guidelines regarding the use of robotic systems have been published [ 49 , 83 ]. Moreover, in April 2021, the European Commission unveiled the first legal framework about AI [ 84 ]. However, to the best of our knowledge, no recommendations have been proposed in this field directly linked to an analysis of the literature dealing specifically with these ethical issues and potential solutions to address them.

This narrative review identified 25 articles in which authors highlighted ethical issues and recommendations related to the use of RCSs and similar technologies. The use of the EUnetHTA Core Model for the analysis of these articles made it possible to classify the information retrieved in the publications according to 5 main ethical topics—“benefit-harm balance,” “autonomy,” “respect for persons,” “justice and equity,” and “legislation”—and to provide a detailed analysis of RCS-related ethical issues. Our review also aimed to identify recommendations for better development, diffusion, and use of RCSs by a population of older adults.

Technology devices, such as RCSs, are used with older adults to enable them to live independently; to enhance their quality of life and well-being; and, therefore, to cope with the increasing care demands for older populations. RCSs may be used to encourage a range of health-related goals: physical, cognitive, nutritional, social, and emotional domains. To be effective, RCSs must be able to motivate the user by providing highly personalized care programs [ 85 , 86 ]. However, studies have shown that not all potential target users are included in the development of these devices [ 37 , 87 , 88 ]. Therefore, RCSs design might fail to meet a wide range of users’ needs, capabilities, and wishes. Thus, it is essential to apply “user-centered design” approaches and involve target users with various sociodemographic characteristics and technology experience throughout the development process. A strong involvement of the intended users of these systems in their design process would also improve the quality of the information provided to potential users of RCSs regarding their operation, type of data collected, and potential benefits of the technology. In this way, the involvement of the users would improve the quality of the process of obtaining the consent required from older adults to use the technology.

Another ethical challenge related to the use of RCSs is the fact that their wide implementation for older adults’ care may affect the distribution of health care resources. For instance, it has been found that for some older adults and informal and formal caregivers, the use of RCSs could replace humans in many caregiving tasks, eventually leading to a suppression of jobs or to a degradation of the quality of health care services [ 17 , 34 , 48 , 61 ]. In this regard, the participation of a third person (professional, volunteer, or family member) as a “human coach” could be considered when implementing RCSs in the older adults’ environment. This “human coach” could help build a “chain of trust” by being an intermediary between the RCS and the user. On the one hand, the involvement of a real person in the use of the RCS could reduce the risk of replacement of human assistance by technological assistance. On the other hand, the “human coach” could help enhance the acceptability and usability of the device, while at the same time, reassuring the user and providing recommendations to the developers, so that the RCS is consistent with users’ needs and desires. However, the benefits of involving a “human coach” in the RCS service provision has yet to be evaluated by scientific studies.

According to some studies [ 3 , 39 , 41 , 51 , 65 ], the use of RCSs can have an impact on social relationships, reducing human contact and even altering social relationships by creating tension between older adults and their caregivers. Thus, it would be interesting to identify the repercussions and implications of these devices in older adults’ daily life and in the life of the members of their social environment through new studies. It also seems necessary to evaluate the organizational impact of the implementation of RCSs and to identify potential obstacles to their use in the care professionals’ work context.

Our analysis also confirmed that for RCSs to provide personalized health-related recommendations, the collection of sensitive data is necessary. Data collection in this context also raises several ethical issues. For instance, personal data can be exposed to hacking and misuse. Proper data management, anonymization, and encryption are essential to protect the personal data of RCS users [ 86 ]. In addition, researchers and developers in this field must evaluate RCSs before implementation to ensure that they do not cause physical or moral harm to users. Thus, it has been suggested that stakeholders refer to local and regional regulatory and safety standards to guide their development and use.

Finally, our analysis also discussed how legal and ethical frameworks regarding the use of RCSs need to be adapted to cope with the constant development of new technologies. So far, existing legal frameworks are not yet adequate to respond effectively to the question of liability in case of damage caused by RCSs, particularly because these devices are becoming increasingly autonomous [ 80 ]. The establishment of “operational ethics committees in digital sciences and technologies” could help in the development and conduct of projects in this area [ 60 ]. Guidelines should be established to identify the types of applications and technological devices that require regulatory review and approval [ 44 ]. Research projects and working groups involving users, researchers, and lawyers should be set up to further investigate the legal and ethical issues related to the use of RCSs.

Some countries and regions, such as Europe and Japan have initiated the work of structuring relevant legal and ethical frameworks; however, their orientations and measures may differ culturally [ 78 ]. Future studies in the area of RCSs could consider the influence of cultural and socioeconomic specificities of the contexts of experimentation (countries and regions) regarding the acceptance and use of RCSs by older adults and formal and informal caregivers and regarding the definition of ethical and legal frameworks governing their uses. Therefore, the use of validated and widely applied analysis frameworks, for example, the Western, Educated, Industrialized, Rich and Democratic framework [ 89 ], formulated to measure countries’ commonalities in their approaches to the interpretation of behavioral research findings (eg, regarding technology adoption) could be interesting. The Western, Educated, Industrialized, Rich and Democratic framework [ 89 ] could help not only to explore the differences among countries regarding the validation and adoption of new technologies for older adult care but also to seek greater cultural and demographic diversity in technology research.

This dimension of cross-cultural comparison has received particular attention in the framework of a current international research partnership between Europe and Japan, such as the EU-Japan Virtual Coach for Smart Ageing (e-VITA) project. This project aims to develop a cross-cultural RCS that can be tailored to the needs of healthy older adults to promote aging well. The e-VITA RCS will be made available to older adults in their homes, which raises many of the ethical questions discussed in this paper. Therefore, the study will require the researchers to set up procedures adapted not only to the users but also to the 2 cultures (European and Japanese), respecting the corresponding ethical and legal regulations. Thus, it would be interesting to perform an analysis of the ethical issues raised by users from different countries and cultures within the framework of the e-VITA project.

Limitations

A narrative review of the literature was conducted to provide a nonexhaustive synthesis of the various ethical concerns and recommendations when using RCSs for older adults. This review has some limitations. Only articles in French and English were included. Some articles indicating ethical concerns or recommendations may not have been included when this information was not mentioned in the keywords or abstract.

Conclusions

The use of RCSs in the context of health care, particularly with an older adult population, tends to show many benefits. RCSs have the potential to improve the quality of life of older adults and their independence. When used in an ethical and appropriate manner, RCSs can help improve older adults’ emotions and cognitive and physical abilities and promote social relationships. By helping older adults to continue living at home for as long as possible, the use of health-oriented RCSs could help to address some of the challenges resulting from demographic aging. However, the use of these new health care technologies involves some ethical concerns, with the most cited issues being not only the risk of accidents, lack of reliability, loss of control, risk of deception, and risk of social isolation but also the confidentiality of data and liability in case of safety problems.

Some recommendations have been made in the past regarding the use of social and assistive robotic technologies for older adults, such as considering the opinion of target users; collecting their consent; training the care professionals to use them; and ensuring proper data management, anonymization, and encryption. However, the integration of RCSs in current health practices and, particularly, in the private homes of older adults can be disruptive. It requires the establishment of scalable and adapted ethical and regulatory frameworks that follow the technology progress and the social and digital change of society Thus, studies are needed to identify new ethical concerns arising from the organizational impact of the implementation of RCSs in different contexts, especially in the homes of older adults. The influence of cultural and socioeconomic specificities of the contexts of experimentation (countries and regions) regarding the acceptance and use of RCSs by older adults and formal and informal caregivers is also an area of interest for future studies.

Acknowledgments

This paper is a part of the EU-Japan Virtual Coach for Smart Ageing (e-VITA) project, which aims to develop a robotic coaching system for older adults [ 90 ]. The authors thank the collaborators who made this project possible: European Commission and Assistance Publique–Hôpitaux de Paris (Délégation à la Recherche Clinique et à l’Innovation). This review was based on data collected within the e-Vita project, funded by the European Union H2020 Program (grant 101016453) and the Japanese Ministry of Internal Affairs and Communication (Ministry of Internal Affairs and Communication; grant JPJ000595).

Data Availability

Data sharing is not applicable to this paper as no data sets were generated or analyzed during this review.

Conflicts of Interest

None declared.

The selected relevant articles on ethical issues regarding the use of robotic coaching solutions for older adults.

• United Nations. Croissant à un rythme plus lent, la population mondiale devrait atteindre 9,7 milliards d'habitants en 2050 et pourrait atteindre près de 11 milliards vers 2100 : Rapport de l'ONU. United Nations, Population Division. United Nations; 2019. URL: https://population.un.org/wpp/Publications/Files/WPP2019_PressRelease_FR.pdf [accessed 2022-03-04]
• Rapport mondial sur le vieillissement et la santé. Organisation Mondiale de la Santé. 2016. URL: https://apps.who.int/iris/handle/10665/206556 [accessed 2022-03-04]
• Chung J, Demiris G, Thompson HJ. Ethical considerations regarding the use of smart home technologies for older adults: an integrative review. Annu Rev Nurs Res. 2016;34:155-181. [ CrossRef ] [ Medline ]
• Ganesan B, Gowda T, Al-Jumaily A, Fong KN, Meena SK, Tong RK. Ambient assisted living technologies for older adults with cognitive and physical impairments: a review. Eur Rev Med Pharmacol Sci. Dec 2019;23(23):10470-10481. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Palazzolo J, Baudu C, Quaderi A. Psychothérapies du Sujet Âgé: Prise en Charge des Pathologies du Vieillissement. 2nd edition. Amsterdam, The Netherlands. Elsevier Masson; 2016.
• Principaux repères de l'OMS sur la démence. World Health Organization. URL: https://www.who.int/fr/news-room/fact-sheets/detail/dementia [accessed 2022-03-04]
• Guilbaud A, Mailliez A, Boulanger É. Vieillissement: une approche globale, multidimensionnelle et préventive. Med Sci (Paris). Dec 09, 2020;36(12):1173-1180. [ FREE Full text ] [ CrossRef ] [ Medline ]
• op den Akker H, Klaassen R, op den Akker R, Jones VM, Hermens HJ. Proceedings of the 26th IEEE International Symposium on Computer-Based Medical Systems. 2013. Presented at: CBMS '13; June 20-22, 2013:546-547; Porto, Portugal. URL: https://ieeexplore.ieee.org/document/6627870 [ CrossRef ]
• Görer B, Salah AA, Akın HL. A robotic fitness coach for the elderly. In: Augusto JC, Wichert R, Collier R, Keyson D, Salah AA, Tan AH, editors. Ambient Intelligence: Lecture Notes in Computer Science. Volume 8309. Cham, Switzerland. Springer; 2013.
• Lete N, Beristain A, García-Alonso A. Survey on virtual coaching for older adults. Health Informatics J. Dec 01, 2020;26(4):3231-3249. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Siewiorek D, Smailagic A, Dey A. Architecture and applications of virtual coaches. Proc IEEE. Aug 2012;100(8):2472-2488. [ CrossRef ]
• Kamphorst BA. E-coaching systems: what they are, and what they aren't. Pers Ubiquit Comput. May 23, 2017;21(4):625-632. [ CrossRef ]
• Kyriazakos S, Schlieter H, Gand K, Caprino M, Corbo M, Tropea P, et al. A novel virtual coaching system based on personalized clinical pathways for rehabilitation of older adults-requirements and implementation plan of the vCare project. Front Digit Health. Oct 5, 2020;2:546562. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wrobel J, Pino M, Wargnier P, Rigaud AS. Robots et agents virtuels au service des personnes âgées : une revue de l’actualité en gérontechnologie. NPG Neurol Psychiatr Geriatr. Aug 2014;14(82):184-193. [ CrossRef ]
• Pérez PJ, Garcia-Zapirain B, Mendez-Zorrilla A. Caregiver and social assistant robot for rehabilitation and coaching for the elderly. Technol Health Care. 2015;23(3):351-357. [ CrossRef ] [ Medline ]
• Fasola J, Mataric MJ. A socially assistive robot exercise coach for the elderly. J Hum-Robot Interact. 2013;2(2):3-32. [ CrossRef ]
• Cresswell K, Cunningham-Burley S, Sheikh A. Health care robotics: qualitative exploration of key challenges and future directions. J Med Internet Res. Jul 04, 2018;20(7):e10410. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Winfield AF, Jirotka M. Ethical governance is essential to building trust in robotics and artificial intelligence systems. Philos Trans A Math Phys Eng Sci. Oct 15, 2018;376(2133):20180085. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Broadbent E, Stafford R, MacDonald B. Acceptance of healthcare robots for the older population: review and future directions. Int J of Soc Robotics. Oct 3, 2009;1(4):319-330. [ CrossRef ]
• Abou Allaban A, Wang M, Padır T. A systematic review of robotics research in support of in-home care for older adults. Information. Jan 30, 2020;11(2):75. [ CrossRef ]
• Cortellessa G, De Benedictis R, Fracasso F, Orlandini A, Umbrico A, Cesta A. AI and robotics to help older adults: revisiting projects in search of lessons learned. Paladyn J Behav Robot. 2021;12(1):356-378. [ CrossRef ]
• Lima MR, Wairagkar M, Gupta M, Rodriguez y Baena F, Barnaghi P, Sharp DJ, et al. Conversational affective social robots for ageing and dementia support. IEEE Trans Cogn Dev Syst. Dec 2022;14(4):1378-1397. [ CrossRef ]
• Sidner CL, Bickmore T, Nooraie B, Rich C, Ring L, Shayganfar M, et al. Creating new technologies for companionable agents to support isolated older adults. ACM Trans Interact Intell Syst. Jul 24, 2018;8(3):1-27. [ CrossRef ]
• Pepito JA, Locsin RC, Constantino RE. Caring for older persons in a technologically advanced nursing future. Health. 2019;11(05):439-463. [ CrossRef ]
• Pinheiro PR, Pinheiro PG, Filho RH, Barrozo JP, Rodrigues JJ, Pinheiro LI, et al. Integration of the mobile robot and internet of things to monitor older people. IEEE Access. 2020;8:138922-138933. [ CrossRef ]
• Chatterjee A, Gerdes MW, Martinez S. eHealth initiatives for the promotion of healthy lifestyle and allied implementation difficulties. In: Proceedings of the 2019 International Conference on Wireless and Mobile Computing, Networking and Communications. 2019. Presented at: WiMOB '19; October 21-23, 2019:1-8; Barcelona, Spain. URL: https://ieeexplore.ieee.org/document/8923324 [ CrossRef ]
• Yousuf H, Reintjens R, Slipszenko E, Blok S, Somsen GA, Tulevski II, et al. Effectiveness of web-based personalised e‑Coaching lifestyle interventions. Neth Heart J. Jan 28, 2019;27(1):24-29. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bevilacqua R, Casaccia S, Cortellessa G, Astell A, Lattanzio F, Corsonello A, et al. Coaching through technology: a systematic review into efficacy and effectiveness for the ageing population. Int J Environ Res Public Health. Aug 15, 2020;17(16):5930. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Justo R, Ben Letaifa L, Palmero C, Gonzalez-Fraile E, Torp Johansen A, Vázquez A, et al. Analysis of the interaction between elderly people and a simulated virtual coach. J Ambient Intell Human Comput. May 22, 2020;11(12):6125-6140. [ CrossRef ]
• Fiorini L, De Mul M, Fabbricotti I, Limosani R, Vitanza A, D'Onofrio G, et al. Assistive robots to improve the independent living of older persons: results from a needs study. Disabil Rehabil Assist Technol. Jan 22, 2021;16(1):92-102. [ CrossRef ] [ Medline ]
• Sriram V, Jenkinson C, Peters M. Informal carers' experience of assistive technology use in dementia care at home: a systematic review. BMC Geriatr. Jun 14, 2019;19(1):160. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pilotto A, Boi R, Petermans J. Technology in geriatrics. Age Ageing. Nov 01, 2018;47(6):771-774. [ CrossRef ] [ Medline ]
• Koceska N, Koceski S, Beomonte Zobel P, Trajkovik V, Garcia N. A telemedicine robot system for assisted and independent living. Sensors (Basel). Feb 18, 2019;19(4):834. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Broadbent E. Interactions with robots: the truths we reveal about ourselves. Annu Rev Psychol. Jan 03, 2017;68(1):627-652. [ CrossRef ] [ Medline ]
• Suwa S, Tsujimura M, Ide H, Kodate N, Ishimaru M, Shimamura A, et al. Home-care professionals’ ethical perceptions of the development and use of home-care robots for older adults in Japan. Int J Hum Comput Interact. Mar 13, 2020;36(14):1295-1303. [ CrossRef ]
• Nordgren A. How to respond to resistiveness towards assistive technologies among persons with dementia. Med Health Care Philos. Sep 6, 2018;21(3):411-421. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Diaz-Orueta U, Hopper L, Konstantinidis E. Shaping technologies for older adults with and without dementia: reflections on ethics and preferences. Health Informatics J. Dec 2020;26(4):3215-3230. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Portacolone E, Halpern J, Luxenberg J, Harrison KL, Covinsky KE. Ethical issues raised by the introduction of artificial companions to older adults with cognitive impairment: a call for interdisciplinary collaborations. J Alzheimers Dis. 2020;76(2):445-455. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Boada JP, Maestre BR, Genís CT. The ethical issues of social assistive robotics: a critical literature review. Technol Soc. Nov 2021;67:101726. [ CrossRef ]
• Vandemeulebroucke T, Dierckx de Casterlé B, Gastmans C. The use of care robots in aged care: a systematic review of argument-based ethics literature. Arch Gerontol Geriatr. Jan 2018;74:15-25. [ CrossRef ] [ Medline ]
• Zafrani O, Nimrod G. Towards a holistic approach to studying human-robot interaction in later life. Gerontologist. Jan 09, 2019;59(1):e26-e36. [ CrossRef ] [ Medline ]
• Anderson J, Kamphorst B. Ethics of e-coaching: implications of employing pervasive computing to promote healthy and sustainable lifestyles. In: Proceedings of the 2014 IEEE International Conference on Pervasive Computing and Communication Workshops. 2014. Presented at: PerComW '14; March 24-28, 2014:351-356; Budapest, Hungary. [ CrossRef ]
• Danaher J. Robot betrayal: a guide to the ethics of robotic deception. Ethics Inf Technol. Jan 04, 2020;22(2):117-128. [ CrossRef ]
• Fiske A, Henningsen P, Buyx A. Your robot therapist will see you now: ethical implications of embodied artificial intelligence in psychiatry, psychology, and psychotherapy. J Med Internet Res. May 09, 2019;21(5):e13216. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Grinbaum A, Chatila R, Devillers L, Ganascia JG, Tessier C, Dauchet M. Ethics in robotics research: CERNA mission and context. IEEE Robot Automat Mag. Sep 2017;24(3):139-145. [ CrossRef ]
• Ienca M, Jotterand F, Vică C, Elger B. Social and assistive robotics in dementia care: ethical recommendations for research and practice. Int J of Soc Robotics. Jun 22, 2016;8(4):565-573. [ CrossRef ]
• Körtner T. Ethical challenges in the use of social service robots for elderly people. Z Gerontol Geriatr. Jun 25, 2016;49(4):303-307. [ CrossRef ] [ Medline ]
• Operto F. Ethics in advanced robotics. IEEE Robot Automat Mag. Mar 2011;18(1):72-78. [ CrossRef ]
• Riek LD, Howard DA. A Code of Ethics for the Human-Robot Interaction Profession. In: Proceedings of 2014 Conference on We Robot. 2014. Presented at: WeRobot '14; April 4-5, 2014:1; Coral Gables, FL. URL: https:/​/robots.​law.miami.edu/​2014/​wp-content/​uploads/​2014/​03/​a-code-of-ethics-for-the-human-robot-interaction-profession-riek-howard.​pdf
• Sharkey A, Sharkey N. Granny and the robots: ethical issues in robot care for the elderly. Ethics Inf Technol. Jul 3, 2010;14(1):27-40. [ CrossRef ]
• Yew GC. Trust in and ethical design of carebots: the case for ethics of care. Int J Soc Robot. May 23, 2021;13(4):629-645. [ FREE Full text ] [ CrossRef ] [ Medline ]
• HTA Core Model ® version 3. EUnetHTA. URL: http://www.corehta.info/model/HTACoreModel3.0.pdf [accessed 2022-03-04]
• Health technology assessment process: fundamentals. EUPATI Toolbox. 2015. URL: https://toolbox.eupati.eu/resources/health-technology-assessment-process-fundamentals/ [accessed 2022-03-04]
• Tuli TB, Terefe TO, Rashid MM. Telepresence mobile robots design and control for social interaction. Int J Soc Robot. Jul 13, 2021;13(5):877-886. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Green BN, Johnson CD, Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J Chiropr Med. Sep 2006;5(3):101-117. [ CrossRef ]
• Isabet B, Pino M, Lewis M, Benveniste S, Rigaud AS. Social telepresence robots: a narrative review of experiments involving older adults before and during the COVID-19 pandemic. Int J Environ Res Public Health. Mar 30, 2021;18(7):3597. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Naudé B, Rigaud AS, Pino M. Video calls for older adults: a narrative review of experiments involving older adults in elderly care institutions. Front Public Health. Jan 14, 2021;9:751150. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Saarni S, Hofmann B, Lampe K, Lühmann D, Mäkelä M, Velasco-Garrido M, et al. Ethical analysis to improve decision-making on health technologies. Bull World Health Organ. Aug 01, 2008;86(8):617-623. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lampe K, Mäkelä M, Garrido MV, Anttila H, Autti-Rämö I, Hicks NJ, et al. The HTA core model: a novel method for producing and reporting health technology assessments. Int J Technol Assess Health Care. Dec 23, 2009;25(S2):9-20. [ CrossRef ]
• Collectif C. Éthique de la recherche en robotique. ALLISTENE. 2014. URL: http://cerna-ethics-allistene.org/digitalAssets/38/38704_Avis_robotique_livret.pdf [accessed 2024-04-02]
• Tisseron S. Introduction. In: Tisseron S, Tordo F, editors. Robots, de Nouveaux Partenaires de Soins Psychiques. Fairfield CA. Érès Publication; 2018.
• Frennert S, Östlund B. How do older people think and feel about robots in health- and elderly care? In: Proceedings of the 2018 INBOTS Conference on Inclusive Robotics for a Better Society. 2018. Presented at: INBOTS '18; October 16-18, 2018:167-174; Pisa, Italy. URL: https://link.springer.com/chapter/10.1007/978-3-030-24074-5_28 [ CrossRef ]
• van Maris A, Zook N, Caleb-Solly P, Studley M, Winfield A, Dogramadzi S. Designing ethical social robots-a longitudinal field study with older adults. Front Robot AI. Jan 24, 2020;7:1. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wangmo T, Lipps M, Kressig RW, Ienca M. Ethical concerns with the use of intelligent assistive technology: findings from a qualitative study with professional stakeholders. BMC Med Ethics. Dec 19, 2019;20(1):98. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Portet F, Vacher M, Rossato S. Les technologies de la parole et du TALN pour l’assistance à domicile des personnes âgées : un rapide tour d’horizon (quick tour of NLP and speech technologies for ambient assisted living). In: Proceedings of the 2012 Workshop on Interactions Langagières pour personnes Agées Dans les habitats Intelligents. 2012. Presented at: ILADI '12; June 4-8, 2012; Grenoble, France. URL: https://aclanthology.org/W12-1402.pdf
• Bradwell HL, Winnington R, Thill S, Jones RB. Ethical perceptions towards real-world use of companion robots with older people and people with dementia: survey opinions among younger adults. BMC Geriatr. Jul 14, 2020;20(1):244. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Denning T, Matuszek C, Koscher K, Smith JR, Kohno T. A spotlight on security and privacy risks with future household robots: attacks and lessons. In: Proceedings of the 11th international conference on Ubiquitous computing. 2009. Presented at: UbiComp '09; September 30-October 3, 2009:105-114; Orlando, FL. URL: https://dl.acm.org/doi/10.1145/1620545.1620564 [ CrossRef ]
• Papadopoulos I, Koulouglioti C, Lazzarino R, Ali S. Enablers and barriers to the implementation of socially assistive humanoid robots in health and social care: a systematic review. BMJ Open. Jan 09, 2020;10(1):e033096. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pripfl J, Kortner T, Batko-Klein D. Results of a real world trial with a mobile social service robot for older adults. In: Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction. 2016. Presented at: HRI '16; March 7-10, 2016:497-498; Christchurch, New Zealand. URL: https://ieeexplore.ieee.org/document/7451824 [ CrossRef ]
• Peek ST, Wouters EJ, van Hoof J, Luijkx KG, Boeije HR, Vrijhoef HJ. Factors influencing acceptance of technology for aging in place: a systematic review. Int J Med Inform. Apr 2014;83(4):235-248. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lespinet-Najib V, Roche A, Chibaudel Q. Santé et handicap: d'une conception centrée «utilisateur» à la conception universelle. Ann Mines Réal Ind. 2017;2:25-27. [ CrossRef ]
• Déclaration d’Helsinki de l’AMM – Principes éthiques applicables à la recherche médicale impliquant des êtres humains. L'Association Médicale Mondiale. URL: https:/​/www.​wma.net/​fr/​policies-post/​declaration-dhelsinki-de-lamm-principes-ethiques-applicables-a-la-recherche-medicale-impliquant-des-etres-humains/​ [accessed 2022-02-03]
• Adams S, Niezen M. Digital ‘solutions’ to unhealthy lifestyle ‘problems’: the construction of social and personal risks in the development of eCoaches. Health Risk Soc. Feb 06, 2016;17(7-8):530-546. [ CrossRef ]
• Saerens P. Le droit des robots, un droit de l’homme en devenir ? Commun Technol Dév. Jun 30, 2020;(8). [ CrossRef ]
• Jenkins S, Draper H. Care, monitoring, and companionship: views on care robots from older people and their carers. Int J of Soc Robotics. Sep 15, 2015;7(5):673-683. [ CrossRef ]
• Règlement (UE) 2016/679 du parlement européen et du Conseil du 27 avril 2016 relatif à la protection des personnes physiques à l'égard du traitement des données à caractère personnel et à la libre circulation de ces données, et abrogeant la directive 95/46/CE (règlement général sur la protection des données) (Texte présentant de l'intérêt pour l'EEE). Le Parlement Européen Et Le Conseil De L'union Européenne. 2016. URL: http://data.europa.eu/eli/reg/2016/679/oj/fra [accessed 2022-02-03]
• Understanding the digital divide. OECD Digital Economy Papers. 2001. URL: https://econpapers.repec.org/paper/oecstiaab/49-en.htm [accessed 2024-04-02]
• Joachim C. Silver economy, robotique et droit - Comparaison franco-japonaise. Centre d’études et de Coopération Juridique Interdisciplinaire – Université de Poitier. Jan 2020. URL: https://hal.science/hal-03147937/document [accessed 2024-04-05]
• Kernaghan K. The rights and wrongs of robotics: ethics and robots in public organizations. Can Public Adm. Dec 12, 2014;57(4):485-506. [ CrossRef ]
• Bensoussan A, Puigmal L. Le droit des robots ? Quelle est l'autonomie de décision d'une machine ? Quelle protection mérite-t-elle ? Arch Philos Droit. 2017;59(1):165-174. [ CrossRef ]
• Tessier C. Robots autonomy: some technical issues. In: Lawless WF, Mittu R, Sofge D, Russell S, editors. Autonomy and Artificial Intelligence: A Threat or Savior? Cham, Switzerland. Springer; 2017:179-194.
• Nevejans N. Comment protéger l'homme face aux robots ? Arch Phil Droit. 2017;59(1):131-163. [ CrossRef ]
• Tambornino L, Lanzerath D, Rodrigues R, Wright D. SIENNA D4.3: survey of REC approaches and codes for artificial intelligence and robotics. Zenodo. Aug 19, 2019. URL: https://zenodo.org/records/4067990 [accessed 2024-04-04]
• Ethics guidelines for trustworthy AI. Shaping Europe's Digital Future. URL: https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai [accessed 2022-07-15]
• Kamali ME, Angelini L, Caon M, Carrino F, Rocke C, Guye S, et al. Virtual coaches for older adults’ wellbeing: a systematic review. IEEE Access. 2020;8:101884-101902. [ CrossRef ]
• Banos O, Nugent C. E-coaching for health. Computer. Mar 2018;51(3):12-15. [ CrossRef ]
• Gustafson DH, McTavish F, Gustafson DH, Mahoney JE, Johnson RA, Lee JD, et al. The effect of an information and communication technology (ICT) on older adults’ quality of life: study protocol for a randomized control trial. Trials. Apr 25, 2015;16(1):3-8. [ CrossRef ]
• Sakaguchi-Tang DK, Cunningham JL, Roldan W, Yip J, Kientz JA. Co-design with older adults: examining and reflecting on collaboration with aging communities. Proc ACM Hum Comput Interact. Oct 18, 2021;5(CSCW2):1-28. [ CrossRef ]
• Henrich J, Heine SJ, Norenzayan A. The weirdest people in the world? Behav Brain Sci. Jun 15, 2010;33(2-3):61-83. [ CrossRef ]
• EU-Japan virtual coach for smart ageing. e-VITA Consortium. URL: https://www.e-vita.coach/ [accessed 2024-04-05]

Abbreviations

Edited by A Mavragani; submitted 12.04.23; peer-reviewed by J Sedlakova, S Liu; comments to author 20.08.23; revised version received 22.12.23; accepted 12.03.24; published 18.06.24.

©Cécilia Palmier, Anne-Sophie Rigaud, Toshimi Ogawa, Rainer Wieching, Sébastien Dacunha, Federico Barbarossa, Vera Stara, Roberta Bevilacqua, Maribel Pino. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 18.06.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

• How to Write a Letter of Recommendation [+ Free Template]

Published: June 13, 2024

I’ve been helping people create letters of recommendation for jobs they want, internships, promotions, and more, for over ten years. I’ve learned during this time that if you’re not selling yourself effectively, you won’t make a lasting impression.

A recommendation letter differs from a resume or a cover letter because it comes from someone who knows you well, so it should feel more personal.

Read further to discover how to write a letter of recommendation that will help you land the job or opportunity you’ve been dreaming of.

Table of Contents

What is a letter of recommendation?

Do i write my own letter of recommendation, free letter of recommendation template, letter of recommendation samples to inspire you.

• Tips for Creating Effective Letters of Recommendations
• How to Ask For a Letter of Recommendation

A letter of recommendation is a one to two-page description of your merits from someone who has a particular insight into your character, work ethic, projects you’ve completed, and more.

Typically, a letter of recommendation is written by someone who is an authority figure to you, such as a past employer or manager. This person should be able to recommend your professional work or academic experience.

22 Job Seeking Templates

Download this bundle of 22 expertly-crafted templates for cover letters, resignation notices, and resumes.

• Resignation Letter Templates
• Cover Letter Templates
• Resume Templates

Download Free

All fields are required.

You're all set!

Click this link to access this resource at any time.

You might be thinking, “Wait, don’t I have my (boss, colleague, or friend) write a letter of recommendation for me? Why must I read this blog to create a letter of recommendation?”

You can, of course, ask them to write a letter of recommendation for you from scratch. But don’t be surprised if it takes them a really long time to write it. Even then, it may not meet your expectations.

There’s nothing wrong with giving the person you're asking an outline, a list of your past achievements, or even a draft of a letter of recommendation.

In fact, it’s expected that you will give the writer an idea of what you want them to include in the letter of recommendation while still allowing them the creative freedom to add their spin.

They can adjust whatever the letter is to fit what they feel comfortable signing their name to, and you will save them a significant amount of time — meaning you get a better letter of recommendation faster.

How to Write a Letter of Recommendation

Whether you’re writing a letter of recommendation for a professional or academic opportunity, the basic elements are the same.

Start by including the date and recipient’s information, introducing the writer, describing the applicant and their performance, and signing off with the writer’s contact information.

A long relationship with the candidate or a deep familiarity with their work is an important element of writing a letter of recommendation.

When ideating which details of your professional relationship to include, ask yourself which projects they worked with you on, what strengths you admire in them, specific moments they came through for you, and what you’ll miss about working with them when they leave.

Remember, a letter of recommendation is more than just a list of their professional experience, that’s what a resume is for, as the writer you can give a hiring manager insight into the personality of the candidate and if they’d be a good fit for the role.

Check out this free letter of recommendation template to get started.

Letter of Recommendation Format

Including a date is important for establishing the validity of a letter of recommendation.

Use the specific date that corresponds with the candidate's last day at an organization or even some time after they worked with the writer of the letter of recommendation.

If you’re asking for a letter of recommendation from a coworker or boss while still employed, be sure to tread lightly as most employers won’t be thrilled to recommend you to a job when you’re leaving.

If you do trust that they are ok recommending you, despite leaving the company, go ahead and put that day’s date.

2. Recipient's Information

Write out the name, position, and company of the person the letter of recommendation is going to. Or, if you’re not sure what companies you will be applying your letter of recommendation to, feel free to keep this section generic so you can fill it out later as opportunities arise.

3. Introductions

Introduce the writer of the letter of recommendation. Remember, use the first person (whether you’re the person writing the letter of recommendation, or the receiver creating a draft).

Go ahead and list their areas of expertise, education, current title, and anything else applicable. If the writer is a previous manager you’ll want to list their position, how long they’ve been at the company and their education. You should also say how long they’ve worked with or known you.

4. Performance and Qualifications

Use this section to talk about the commitment of the letter's requester in your organization. You

can also mention their most notable traits, skills, and abilities through adjectives.

This section is the bulk of your letter and the most important part. Remember, your boss/coworker/friend can put their own spin on what you say in the letter, you’re just giving them an idea of what you’d like included.

Ask yourself these questions when writing this section:

• How can I tell the story of my accomplishments?
• What personal details need to be included?
• What motivates me?
• What challenges have I overcome?
• What are my most relevant skills?
• Why do I want to work at this company or apply to this school?
• What makes me a good fit for this role?

Here’s an example of what answering these questions might look like:

“Jane Doe became my employee in 2016 after transferring from the Sales department. She is extremely motivated by sales-centric goals, employee satisfaction, and choosing tactics that deliver a high return on investment.

In my time working with Jane Doe, I’ve watched her tackle challenging projects, such as when our startup was bought out by a bigger brand.

She made sure each member of her team transitioned seamlessly while also still meeting her quarterly goals, an accomplishment that only one other team at the company achieved during this time.

Her background in sales has made her a friendly team player, a wise financial decision-maker, and an influential leader. She would be an excellent fit for any role that needs someone who is going to meet hard-to-reach targets, lead a team to excellence, and maintain organization.

In my time working with Jane Doe, I’ve often used her as my go-to-person because I know she is both reliable and hard-working.”

Don’t forget to write this section in the first person, and don’t be afraid to really sell yourself and your achievements!

If you’re not comfortable with being this specific, here’s an example that leaves space for the writer to put in their own thoughts.

“[NAME] became my employee in [YEAR] after transferring from [DEPARTMENT]. She is extremely motivated [ENTER PERSONAL QUALITIES].

In my time working with [NAME], I’ve watched her tackle challenging projects, such as [PAST PROJECT(S)]. [SENTENCE ABOUT WAS DONE WELL].

Their background in [ENTER BACKGROUND] has made them [FAVORABLE PERSONAL QUALITIES]. They would be an excellent fit for any role that needs [DESCRIPTION OF ROLE THEY’RE APPLYING FOR].

In my time working with [NAME], [DESCRIPTION OF HOW WE’VE WORKED TOGETHER IN PAST].”

5. Contact Information

Finally, you can close this letter wishing the applicant luck in their new professional stage. Most importantly, provide detailed contact information, as interviewers will need to confirm the information provided in this document.

How long should a letter of recommendation be?

Like a cover letter or a resume, a letter of recommendation should be about one page long. I’ve often erred on the side of shorter than longer because you really can say everything you need to in one page.

If you’re having a hard time whittling your letter of recommendation down to one page, consider asking a friend with writing experience to edit it down to the most important details, or even using AI tools to help you.

Writing your letters of recommendation from scratch can be time-consuming and difficult. Download your free recommendation template (pictured below) here as a Google Docs or Microsoft Word file.

Don't forget to share this post!

Related articles.

Get Past the ATS With These Resume Templates [+ Tips]

Is it Worth Using ChatGPT to Write Your Resume? Let’s Find Out

How to Write a Respectable Resignation Letter [+ Samples & Templates]

Quiet Quitting vs. Setting Healthy Boundaries: Where's The Line?

The Best Fonts for Your Resume in 2024, According to HubSpot Recruiters

How to Answer ‘What Makes You Unique?’ & Stand Out in Your Next Interview

Best Personal Website from Marketers, Creators, and Other Business Professionals Who’ll Inspire You

#OpenToWork: Are LinkedIn Photo Frames Actually Helpful?

What is a Letter of Intent? How to Write One for a Job [+ Examples]

The Best 30-60-90 Day Plan for Your New Job [Template + Example]

20 AI-driven tools, 60+ ways to use ChatGPT, resume-building tips, and more.

Marketing software that helps you drive revenue, save time and resources, and measure and optimize your investments — all on one easy-to-use platform

Defining the Role of Authors and Contributors

Page Contents

• Why Authorship Matters
• Who Is an Author?
• Non-Author Contributors
• Artificial Intelligence (AI)-Assisted Technology

1. Why Authorship Matters

Authorship confers credit and has important academic, social, and financial implications. Authorship also implies responsibility and accountability for published work. The following recommendations are intended to ensure that contributors who have made substantive intellectual contributions to a paper are given credit as authors, but also that contributors credited as authors understand their role in taking responsibility and being accountable for what is published.

Editors should be aware of the practice of excluding local researchers from low-income and middle-income countries (LMICs) from authorship when data are from LMICs. Inclusion of local authors adds to fairness, context, and implications of the research. Lack of inclusion of local investigators as authors should prompt questioning and may lead to rejection.

Because authorship does not communicate what contributions qualified an individual to be an author, some journals now request and publish information about the contributions of each person named as having participated in a submitted study, at least for original research. Editors are strongly encouraged to develop and implement a contributorship policy. Such policies remove much of the ambiguity surrounding contributions, but leave unresolved the question of the quantity and quality of contribution that qualify an individual for authorship. The ICMJE has thus developed criteria for authorship that can be used by all journals, including those that distinguish authors from other contributors.

2. Who Is an Author?

The ICMJE recommends that authorship be based on the following 4 criteria:

• Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND
• Drafting the work or reviewing it critically for important intellectual content; AND
• Final approval of the version to be published; AND
• Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

In addition to being accountable for the parts of the work done, an author should be able to identify which co-authors are responsible for specific other parts of the work. In addition, authors should have confidence in the integrity of the contributions of their co-authors.

All those designated as authors should meet all four criteria for authorship, and all who meet the four criteria should be identified as authors. Those who do not meet all four criteria should be acknowledged—see Section II.A.3 below. These authorship criteria are intended to reserve the status of authorship for those who deserve credit and can take responsibility for the work. The criteria are not intended for use as a means to disqualify colleagues from authorship who otherwise meet authorship criteria by denying them the opportunity to meet criterion #s 2 or 3. Therefore, all individuals who meet the first criterion should have the opportunity to participate in the review, drafting, and final approval of the manuscript.

The individuals who conduct the work are responsible for identifying who meets these criteria and ideally should do so when planning the work, making modifications as appropriate as the work progresses. We encourage collaboration and co-authorship with colleagues in the locations where the research is conducted. It is the collective responsibility of the authors, not the journal to which the work is submitted, to determine that all people named as authors meet all four criteria; it is not the role of journal editors to determine who qualifies or does not qualify for authorship or to arbitrate authorship conflicts. If agreement cannot be reached about who qualifies for authorship, the institution(s) where the work was performed, not the journal editor, should be asked to investigate. The criteria used to determine the order in which authors are listed on the byline may vary, and are to be decided collectively by the author group and not by editors. If authors request removal or addition of an author after manuscript submission or publication, journal editors should seek an explanation and signed statement of agreement for the requested change from all listed authors and from the author to be removed or added.

The corresponding author is the one individual who takes primary responsibility for communication with the journal during the manuscript submission, peer-review, and publication process. The corresponding author typically ensures that all the journal’s administrative requirements, such as providing details of authorship, ethics committee approval, clinical trial registration documentation, and disclosures of relationships and activities are properly completed and reported, although these duties may be delegated to one or more co-authors. The corresponding author should be available throughout the submission and peer-review process to respond to editorial queries in a timely way, and should be available after publication to respond to critiques of the work and cooperate with any requests from the journal for data or additional information should questions about the paper arise after publication. Although the corresponding author has primary responsibility for correspondence with the journal, the ICMJE recommends that editors send copies of all correspondence to all listed authors.

When a large multi-author group has conducted the work, the group ideally should decide who will be an author before the work is started and confirm who is an author before submitting the manuscript for publication. All members of the group named as authors should meet all four criteria for authorship, including approval of the final manuscript, and they should be able to take public responsibility for the work and should have full confidence in the accuracy and integrity of the work of other group authors. They will also be expected as individuals to complete disclosure forms.

Some large multi-author groups designate authorship by a group name, with or without the names of individuals. When submitting a manuscript authored by a group, the corresponding author should specify the group name if one exists, and clearly identify the group members who can take credit and responsibility for the work as authors. The byline of the article identifies who is directly responsible for the manuscript, and MEDLINE lists as authors whichever names appear on the byline. If the byline includes a group name, MEDLINE will list the names of individual group members who are authors or who are collaborators, sometimes called non-author contributors, if there is a note associated with the byline clearly stating that the individual names are elsewhere in the paper and whether those names are authors or collaborators.

3. Non-Author Contributors

Contributors who meet fewer than all 4 of the above criteria for authorship should not be listed as authors, but they should be acknowledged. Examples of activities that alone (without other contributions) do not qualify a contributor for authorship are acquisition of funding; general supervision of a research group or general administrative support; and writing assistance, technical editing, language editing, and proofreading. Those whose contributions do not justify authorship may be acknowledged individually or together as a group under a single heading (e.g. "Clinical Investigators" or "Participating Investigators"), and their contributions should be specified (e.g., "served as scientific advisors," "critically reviewed the study proposal," "collected data," "provided and cared for study patients," "participated in writing or technical editing of the manuscript").

Because acknowledgment may imply endorsement by acknowledged individuals of a study’s data and conclusions, editors are advised to require that the corresponding author obtain written permission to be acknowledged from all acknowledged individuals.

Use of AI for writing assistance should be reported in the acknowledgment section.

4. Artificial Intelligence (AI)-Assisted Technology

At submission, the journal should require authors to disclose whether they used artificial intelligence (AI)-assisted technologies (such as Large Language Models [LLMs], chatbots, or image creators) in the production of submitted work. Authors who use such technology should describe, in both the cover letter and the submitted work in the appropriate section if applicable, how they used it. For example, if AI was used for writing assistance, describe this in the acknowledgment section (see Section II.A.3). If AI was used for data collection, analysis, or figure generation, authors should describe this use in the methods (see Section IV.A.3.d). Chatbots (such as ChatGPT) should not be listed as authors because they cannot be responsible for the accuracy, integrity, and originality of the work, and these responsibilities are required for authorship (see Section II.A.1). Therefore, humans are responsible for any submitted material that included the use of AI-assisted technologies. Authors should carefully review and edit the result because AI can generate authoritative-sounding output that can be incorrect, incomplete, or biased. Authors should not list AI and AI-assisted technologies as an author or co-author, nor cite AI as an author. Authors should be able to assert that there is no plagiarism in their paper, including in text and images produced by the AI. Humans must ensure there is appropriate attribution of all quoted material, including full citations.

Next: Disclosure of Financial and Non-Financial Relationships and Activities, and Conflicts of Interest

Keep up-to-date Request to receive an E-mail when the Recommendations are updated.

Subscribe to Changes

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Crit Care Med (Targu Mures)
• v.4(1); 2018 Jan

The Importance of Referencing

Ario santini.

1 University of Medicine and Pharmacy of Tîrgu Mureș, Tîrgu Mureș, Romania

2 Hon Fellow, University of Edinburgh, Edinburgh, United Kingdom

It is imperative that authors of medical papers give fitting credit to the sources that they quote in their manuscripts. Frequently this is not the case, and a wrong, inappropriate or unsuitable citation often blights the quality of a paper. Many concerns can proceed from questions relating to improper citations. These matters may be directed at a personal level or may have a bearing on the general research community. They can range from doubt being levied at the individual’s academic ability, to his/her academic honesty, or even the suggestion of potential plagiarism. Meticulous referencing of sources will eliminate all of these concerns.

The stamp of a good research worker is attention to detail at all levels of his/her research. Attention to detail cultivates good habits and the detail required in referencing and preparing a bibliography focuses attention on the whole research procedure. It aids scientific thought and analysis and makes for better research reporting.

Good referencing includes attention to detail such as correct page numbering, the spelling of author’s names and the accuracy of relevant facts that will be stated in the paper. Attention to referencing not only makes you a better researcher but enhances your reputation amongst editors, reviewers and readers.

An editor or reader may interpret poor referencing as a sign of intellectual laziness, unclear thinking, and inaccurate writing. When you cite sources correctly, you leave no question in your readers’ minds regarding your point. By citing precisely and accurately, the phrase “It is said” need ever be used in your academic manuscript.An excellent bibliography shows off your scientific knowledge. An extensive bibliography is the hallmark of a widely read and well-informed clinician or scientist.

Errors of omission frequently are the cause of a reviewer not accepting a manuscript for publication. By not citing a critical or well-known paper extant in the scientific literature, authors can be accused of not having undertaken an extensive literature review or having inadequate knowledge of their subject. It may even lead a reviewer to consider that the work is, in part, plagiarised.

Reviewers look at the bibliography for several reasons. Attention to detail, with regard to spelling and punctuation, as stated above, is a primary concern. Moreover, the citation method, be it Harvard or Vancouver or other, must strictly follow the instructions to authors as prescribed by the journal in question. If the most recent reference is more than five years or so, this may indicate that a full up to date review of the literature has not been undertaken. A preponderance of references by one author(s) may suggest self- promotion, self- plagiarism, or the selective citation of papers whose conclusion are in accord with those of the author.

Plagiarism is a prominent problem encountered in the academic process and is one of the most common causes of compromising the academic integrity of the author. Sources must be cited in an appropriate form. Copying, using, or the misuse of other people’s ideas, words or concepts, without proper referencing is prohibited. It is not enough to change a few words in a phrase from the source material into “own words”. Changing the word-order of a sentence is unacceptable, as is the use of synonyms. Referencing and references, signify the quality of the work, detail the primary sources and are indicative of the extent of information on the subject. Proper referencing removes any question of plagiarism.

Every submitted academic manuscript is scrutinised several times before it is accepted for publication. The process starts with editors, progresses via reviewers and continues even after acceptance when editorial assistants will locate sources in the bibliography and ensure that the citations are accurate.

Your manuscript is much more likely to survive the rigours of the review process with minimal criticism and positive feedback if all references are contemporaneous, refer to a primary source and have been accurately and correctly cited.

Additional Reading

Reference Examples

More than 100 reference examples and their corresponding in-text citations are presented in the seventh edition Publication Manual . Examples of the most common works that writers cite are provided on this page; additional examples are available in the Publication Manual .

To find the reference example you need, first select a category (e.g., periodicals) and then choose the appropriate type of work (e.g., journal article ) and follow the relevant example.

When selecting a category, use the webpages and websites category only when a work does not fit better within another category. For example, a report from a government website would use the reports category, whereas a page on a government website that is not a report or other work would use the webpages and websites category.

Also note that print and electronic references are largely the same. For example, to cite both print books and ebooks, use the books and reference works category and then choose the appropriate type of work (i.e., book ) and follow the relevant example (e.g., whole authored book ).

Examples on these pages illustrate the details of reference formats. We make every attempt to show examples that are in keeping with APA Style’s guiding principles of inclusivity and bias-free language. These examples are presented out of context only to demonstrate formatting issues (e.g., which elements to italicize, where punctuation is needed, placement of parentheses). References, including these examples, are not inherently endorsements for the ideas or content of the works themselves. An author may cite a work to support a statement or an idea, to critique that work, or for many other reasons. For more examples, see our sample papers .

Reference examples are covered in the seventh edition APA Style manuals in the Publication Manual Chapter 10 and the Concise Guide Chapter 10

Related handouts

• Common Reference Examples Guide (PDF, 147KB)
• Reference Quick Guide (PDF, 225KB)

Textual Works

Textual works are covered in Sections 10.1–10.8 of the Publication Manual . The most common categories and examples are presented here. For the reviews of other works category, see Section 10.7.

• Journal Article References
• Magazine Article References
• Newspaper Article References
• Blog Post and Blog Comment References
• UpToDate Article References
• Book/Ebook References
• Diagnostic Manual References
• Children’s Book or Other Illustrated Book References
• Classroom Course Pack Material References
• Religious Work References
• Chapter in an Edited Book/Ebook References
• Dictionary Entry References
• Wikipedia Entry References
• Report by a Government Agency References
• Report with Individual Authors References
• Brochure References
• Ethics Code References
• Fact Sheet References
• ISO Standard References
• Press Release References
• White Paper References
• Conference Presentation References
• Conference Proceeding References
• Published Dissertation or Thesis References
• Unpublished Dissertation or Thesis References
• ERIC Database References
• Preprint Article References

Data and Assessments

Data sets are covered in Section 10.9 of the Publication Manual . For the software and tests categories, see Sections 10.10 and 10.11.

• Data Set References
• Toolbox References

Audiovisual Media

Audiovisual media are covered in Sections 10.12–10.14 of the Publication Manual . The most common examples are presented together here. In the manual, these examples and more are separated into categories for audiovisual, audio, and visual media.

• Artwork References
• Clip Art or Stock Image References
• Film and Television References
• Musical Score References
• Online Course or MOOC References
• Podcast References
• PowerPoint Slide or Lecture Note References
• Radio Broadcast References
• TED Talk References
• Transcript of an Audiovisual Work References
• YouTube Video References

Online Media

Online media are covered in Sections 10.15 and 10.16 of the Publication Manual . Please note that blog posts are part of the periodicals category.

• Facebook References
• Instagram References
• LinkedIn References
• Online Forum (e.g., Reddit) References
• TikTok References
• X References
• Webpage on a Website References
• Clinical Practice References
• Open Educational Resource References
• Whole Website References

Invest in news coverage you can trust.

Donate to PBS News Hour by June 30 !

Ian N. Mills, The Conversation Ian N. Mills, The Conversation

Leave your feedback

• Copy URL https://www.pbs.org/newshour/arts/the-crosby-schoyen-codex-one-of-the-oldest-surviving-books-just-sold-for-3-9-million-heres-what-to-know

The Crosby-Schøyen Codex, one of the oldest surviving books, just sold for $3.9 million. Here’s what to know

An important piece of early Christian history, the Crosby-Schøyen Codex, sold for nearly $3.9 million on June 11, 2024 , at Christie’s in London. This codex is a mid-fourth century book from Egypt containing a combination of biblical and other early Christian texts.

The Crosby-Schøyen Codex was discovered alongside more than 20 other codices near Dishna, Egypt, in 1952. These manuscripts are collectively known as “the Dishna Papers” or “the Bodmer Papyri,” after the Swiss collector Martin Bodmer.

READ MORE: Archaeologists find records of Pompeii survivors that reveal how they rebuilt their lives

Though often overshadowed by other 20th century discoveries, this trove of ancient manuscripts represents one of the most significant finds for understanding the history of early Christianity. As a scholar of early Christian reading practices , I consider the Dishna Papers an invaluable witness to the formation of the Christian Bible. This ancient library shows how, before the consolidation of the Bible, early Christians read canonical and non-canonical scriptures – as well as pagan classics – side by side.

An overshadowed discovery

The middle decades of the 20th century were exciting years for scholars of early Christianity.

In 1945, a collection of 13 ancient codices was discovered near Nag Hammadi, Egypt. These contained dozens of otherwise unknown works, mostly associated with minority and marginalized forms of early Christianity. With titles like “The Gospel of Thomas” and “The Secret Revelation of John,” this cache of non-canonical scriptures captured the public’s imagination and inspired a bestseller .

The very next year, Bedouin shepherds discovered ancient Hebrew scrolls hidden in a cave at Qumran on the northwestern shore of the Dead Sea.

The “ Dead Sea Scrolls ” found in this and a dozen subsequently discovered caves constituted a massive library of Jewish texts, including biblical works and hitherto unknown texts with remarkable parallels to the writings of the New Testament. This find was celebrated in news stories , documentaries and other publications as among the greatest discoveries of the 20th century.

At the very same time, the Dishna Papers were discovered, smuggled out of Egypt and sold to European collectors with considerably less fanfare. No headline hailed the discovery of the Dishna Papers. Instead, pieces of this collection were sold to the highest bidders, scattering the ancient library across the globe.

A view of one of the oldest books in existence, the Crosby-Schoyen Codex, in New York on April 2, 2024. Photo by Andrew Hofstetter/ Reuters


The Dishna Papers

Though less exotic than Nag Hammadi or Qumran, the contents of the Crosby-Schøyen Codex and the 20-some additional codices discovered near Dishna have proved every bit as important for our understanding of early Christianity.

Two manuscripts of the canonical gospels, Luke and John, belonging to this ancient library predate almost every other surviving copy of these gospels. Scholars used these new manuscripts to revise the text of the New Testament .

For instance, the vast majority of manuscripts of the Gospel of John describe Jesus as “the only-begotten Son” (1:18). But the early manuscripts discovered at Dishna read “the only-begotten God.” Here and elsewhere, English translations of the Bible were changed to reflect the contents of the Dishna Papers.

But the library discovered near Dishna did not consist entirely of texts that ended up in the Christian Bible. Scriptures that were not included in the Christian canon, like Paul’s “ Third Letter to the Corinthians ” and “ The Shepherd of Hermas ,” were also found among the Dishna Papers.

One codex from Dishna contains the “ Acts of Paul ,” an extra-Biblical account of Paul’s travels and martyrdom. Another contains the “ Infancy Gospel of James ,” a non-canonical story about the life of Mary, Jesus’ mother. The discoveries at Dishna provide evidence that these writings, though unfamiliar to modern readers of the Bible, spent centuries on the periphery of Christian scripture.

A person shows the Crosby-Schoyen Codex. Photo by Andrew Hofstetter/ Reuters

The Dishna Papers included a few additional literary texts. One codex in this mostly Christian library contains several comedies by the Hellenistic playwright Menander. Another codex binds together a chapter of Thucydides’ “History of the Peloponnesian War” with a Greek version of the biblical Book of Daniel.

Evidently, the owner of this Christian library had no aversion to the arts and sciences of pre-Christian Hellenism. In this library, pagan classics and Christian scripture stood side by side.

But whose library was this?

The Crosby-Schøyen Codex actually supplies several important clues to the origin of the Dishna Papers with which it was found.

Thanks to recent radiocarbon dating of this codex and the contents of a closely related manuscript, the Crosby-Schøyen Codex can be dated with some measure of confidence to the middle of the fourth century – roughly 325 to 350 C.E.

The Crosby-Schøyen Codex itself contains five texts in Sahidic Coptic, a dialect of the ancient Egyptian language. Three texts are Biblical: Jonah, Second Maccabees 5:27-7:41, and 1 Peter. The rest of the codex contains part of a well-known Easter homily and a brief otherwise unknown exhortation.

These texts, argue scholars Albert Pietersma and Susan Comstock, may have been collected into a single codex for use as an Easter lectionary. A lectionary is a collection of readings used in Christian worship services. Such lectionaries were used in Pachomian monasteries, like the one located only a few miles west of Dishna.

This monastery was established in the mid-330s by Pachomius, the reputed founder of communal monasticism . His Pachomian Rule , by which the monks would have ordered their communal life, makes frequent reference to the public and private use of books. Pachomius’ monasteries even taught illiterate monks to read.

It seems likely that this eclectic library of canonical and non-canonical scriptures, early Christian writings and pagan classics belonged to these book-loving monks in central Egypt. One of the Pachomian rules allowed monks to borrow books from the monastic library for up to one week.

This article was first published in The Conversation on May 22, 2024.

Ian N. Mills is a visiting assistant professor of classics and religious studies at Hamilton College

Support Provided By: Learn more

Support PBS News:

Educate your inbox

Subscribe to Here’s the Deal, our politics newsletter for analysis you won’t find anywhere else.

Thank you. Please check your inbox to confirm.