social work research concept

Social Work Research: Concept, Scope

Last Updated on December 30, 2022 by Team TSW

An effort to create new knowledge or to upgrade existing knowledge either through observation, available facts, evidences or any other method, is research. We often make our mind or take decision, based on our observation about certain objects or phenomena. During whole process we remain unaware of our biases, we do not question them and we attribute our observations entirely to the object being observed. Though it is still possible to arrive at right decision on the basis of wrong reasons or vice versa. This whole thing questions the process of observation. Was the observation error-free? While observing are we aware of our limitations? Every method of observation has certain limitations. Important thing here is to take biases, the errors and limitations into consideration. Social work research is the application of research methods in the field of social work.

Table of Contents

Purpose of social work research is to produce new knowledge or to increase already available knowledge in the field of social work. Social work research gives new dimensions to social work techniques and methods and provides new ways to deal with problems. Social work research attempts to highlight insights about what intervention or treatment is actually helpful in practice and bring the best result. It also throw light on what hinder the attainment of desired goal. It also look for answers to problems faced by practitioners.

Relevance of Research in Social Work

Social work research tries to find answers to questions faced by practitioners and to make existing intervention more effective. The problems are not only professional but personal too. Overall aim is to make existing social work methods and techniques better and more effective.

In social work research, we study the problems from the point of view of professional social work. The designing of research problems, data collection and its interpretation will have to be attempted in a manner as would be useful to professional social work. The process should add new knowledge to social work theory and practice and also to enhance the outcome of professional social workers.

Limitation of Scientific Research Method in Social Work

Social work primarily deals with human behaviour, which is by and large complex and dynamic in nature. This means that different humans tend to behave differently under the same circumstances. One person can be happy in given circumstances and the second may be sad and at the same time others may remain indifferent. So it can easily be deduced that data collected for humans is subjective in nature and means very little for scientific research. Therefore one can not investigate human behaviour under guided conditions as in natural science. This creates many problems for researchers.

Social work will never realize the objective of research as completely as natural science does, but still social work does not completely diminish the importance of scientific research methods.

Social Work is a diverse profession and work almost at all level of social system. Possible broad research areas could be:-

Community health.
Community mental health.
Child welfare.
Women welfare.
Youth welfare.
Juvenile delinquency.
Crime and correction.
Aged welfare.
Poverty alleviation.
Management of Social Welfare Department and Organization.
Disaster Management.
Industrial Social Work.

These are the areas which are very frequently studied by Social workers. Researchers might focus on individuals, families, groups, community or broad social systems.

Facts, events, and evidence help acquire reliable knowledge about various aspects of human behaviour. To get that knowledge, the method of science is still the most commonly used method. Objectivity, replication, prediction and verifiability are the characteristics of scientific approach, which keeps the researchers at bay from their personal biases, beliefs, perceptions, values, attitudes and emotions. With all the limitations and characteristics, scientific research is the best method to arrive at generalization in the field of social work.

Qualitative Research

Qualitative data — facts that cannot be measured or expressed in terms of mere numbers or counts — offer rich insights into individuals, groups and societies. It can be collected via interviews and observations.

What attitudes do students have toward the reduced-price school lunch program?
What strategies do individuals use to moderate their weekly digital media consumption?
What factors made community members more or less likely to access a specific medical service last year?

Qualitative research can thereby provide a textured view of social contexts and systems that may not have been possible with quantitative methods. Plus, it may even suggest new lines of inquiry for social work research.

Mixed Methods Research

Combining quantitative and qualitative methods into a single study is known as mixed methods research. This form of research has gained popularity in the study of social sciences, according to a 2019 report in the academic journal Theory and Society. Since quantitative and qualitative methods answer different questions, merging them into a single study can balance the limitations of each and potentially produce more in-depth findings.

However, mixed methods research is not without its drawbacks. Combining research methods increases the complexity of a study and generally requires a higher level of expertise to collect, analyze and interpret the data. It also requires a greater level of effort, time and often money.

The Importance of Research Design

Data-driven practice plays an essential role in social work. Unlike philanthropists and altruistic volunteers, social workers are obligated to operate from a scientific knowledge base.

To know whether their programs are effective, social workers must conduct research to determine results, aggregate those results into comprehensible data, analyze and interpret their findings, and use evidence to justify next steps.

Employing the proper design ensures that any evidence obtained during research enables social workers to reliably answer their research questions.

Research Methods in Social Work

The various social work research methods have specific benefits and limitations determined by context. Common research methods include surveys, program evaluations, needs assessments, randomized controlled trials, descriptive studies and single-system designs.

Surveys involve a hypothesis and a series of questions in order to test that hypothesis. Social work researchers will send out a survey, receive responses, aggregate the results, analyze the data, and form conclusions based on trends.

Surveys are one of the most common research methods social workers use — and for good reason. They tend to be relatively simple and are usually affordable. However, surveys generally require large participant groups, and self-reports from survey respondents are not always reliable.

Program Evaluations

Social workers ally with all sorts of programs: after-school programs, government initiatives, nonprofit projects and private programs, for example.

Crucially, social workers must evaluate a program’s effectiveness in order to determine whether the program is meeting its goals and what improvements can be made to better serve the program’s target population.

Evidence-based programming helps everyone save money and time, and comparing programs with one another can help social workers make decisions about how to structure new initiatives. Evaluating programs becomes complicated, however, when programs have multiple goal metrics, some of which may be vague or difficult to assess (e.g., “we aim to promote the well-being of our community”).

Needs Assessments

Social workers use needs assessments to identify services and necessities that a population lacks access to.

Common social work populations that researchers may perform needs assessments on include:

People in a specific income group
Everyone in a specific geographic region
A specific ethnic group
People in a specific age group

In the field, a social worker may use a combination of methods (e.g., surveys and descriptive studies) to learn more about a specific population or program. Social workers look for gaps between the actual context and a population’s or individual’s “wants” or desires.

For example, a social worker could conduct a needs assessment with an individual with cancer trying to navigate the complex medical-industrial system. The social worker may ask the client questions about the number of hours they spend scheduling doctor’s appointments, commuting and managing their many medications. After learning more about the specific client needs, the social worker can identify opportunities for improvements in an updated care plan.

In policy and program development, social workers conduct needs assessments to determine where and how to effect change on a much larger scale. Integral to social work at all levels, needs assessments reveal crucial information about a population’s needs to researchers, policymakers and other stakeholders. Needs assessments may fall short, however, in revealing the root causes of those needs (e.g., structural racism).

Randomized Controlled Trials

Randomized controlled trials are studies in which a randomly selected group is subjected to a variable (e.g., a specific stimulus or treatment) and a control group is not. Social workers then measure and compare the results of the randomized group with the control group in order to glean insights about the effectiveness of a particular intervention or treatment.

Randomized controlled trials are easily reproducible and highly measurable. They’re useful when results are easily quantifiable. However, this method is less helpful when results are not easily quantifiable (i.e., when rich data such as narratives and on-the-ground observations are needed).

Descriptive Studies

Descriptive studies immerse the researcher in another context or culture to study specific participant practices or ways of living. Descriptive studies, including descriptive ethnographic studies, may overlap with and include other research methods:

Informant interviews
Census data
Observation

By using descriptive studies, researchers may glean a richer, deeper understanding of a nuanced culture or group on-site. The main limitations of this research method are that it tends to be time-consuming and expensive.

Single-System Designs

Unlike most medical studies, which involve testing a drug or treatment on two groups — an experimental group that receives the drug/treatment and a control group that does not — single-system designs allow researchers to study just one group (e.g., an individual or family).

Single-system designs typically entail studying a single group over a long period of time and may involve assessing the group’s response to multiple variables.

For example, consider a study on how media consumption affects a person’s mood. One way to test a hypothesis that consuming media correlates with low mood would be to observe two groups: a control group (no media) and an experimental group (two hours of media per day). When employing a single-system design, however, researchers would observe a single participant as they watch two hours of media per day for one week and then four hours per day of media the next week.

These designs allow researchers to test multiple variables over a longer period of time. However, similar to descriptive studies, single-system designs can be fairly time-consuming and costly.

Learn More About Social Work Research Methods

Social workers have the opportunity to improve the social environment by advocating for the vulnerable — including children, older adults and people with disabilities — and facilitating and developing resources and programs.

Learn more about how you can earn your Master of Social Work online at Virginia Commonwealth University . The highest-ranking school of social work in Virginia, VCU has a wide range of courses online. That means students can earn their degrees with the flexibility of learning at home. Learn more about how you can take your career in social work further with VCU.

From M.S.W. to LCSW: Understanding Your Career Path as a Social Worker

How Palliative Care Social Workers Support Patients With Terminal Illnesses

How to Become a Social Worker in Health Care

Gov.uk, Mixed Methods Study

MVS Open Press, Foundations of Social Work Research

Open Social Work Education, Scientific Inquiry in Social Work

Open Social Work, Graduate Research Methods in Social Work: A Project-Based Approach

Routledge, Research for Social Workers: An Introduction to Methods

SAGE Publications, Research Methods for Social Work: A Problem-Based Approach

Theory and Society, Mixed Methods Research: What It Is and What It Could Be

READY TO GET STARTED WITH OUR ONLINE M.S.W. PROGRAM FORMAT?

Bachelor’s degree is required.

VCU Program Helper

This AI chatbot provides automated responses, which may not always be accurate. By continuing with this conversation, you agree that the contents of this chat session may be transcribed and retained. You also consent that this chat session and your interactions, including cookie usage, are subject to our privacy policy .

Subject List
Take a Tour
For Authors
Subscriber Services
Publications
African American Studies
African Studies
American Literature
Anthropology
Architecture Planning and Preservation
Art History
Atlantic History
Biblical Studies
British and Irish Literature
Childhood Studies
Chinese Studies
Cinema and Media Studies
Communication
Criminology
Environmental Science
Evolutionary Biology
International Law
International Relations
Islamic Studies
Jewish Studies
Latin American Studies
Latino Studies
Linguistics
Literary and Critical Theory
Medieval Studies
Military History
Political Science
Public Health
Renaissance and Reformation

Social Work

Urban Studies
Victorian Literature
Browse All Subjects

How to Subscribe

Free Trials

In This Article Expand or collapse the "in this article" section Social Work Research Methods

Introduction.

History of Social Work Research Methods
Feasibility Issues Influencing the Research Process
Measurement Methods
Existing Scales
Group Experimental and Quasi-Experimental Designs for Evaluating Outcome
Single-System Designs for Evaluating Outcome
Program Evaluation
Surveys and Sampling
Introductory Statistics Texts
Advanced Aspects of Inferential Statistics
Qualitative Research Methods
Qualitative Data Analysis
Historical Research Methods
Meta-Analysis and Systematic Reviews
Research Ethics
Culturally Competent Research Methods
Teaching Social Work Research Methods

Other Subject Areas

Forthcoming articles expand or collapse the "forthcoming articles" section.

Abolitionist Perspectives in Social Work
Randomized Controlled Trials in Social Work
Social Work Practice with Transgender and Gender Expansive Youth
Find more forthcoming articles...
Export Citations
Share This Facebook LinkedIn Twitter

Social Work Research Methods by Allen Rubin LAST REVIEWED: 14 December 2009 LAST MODIFIED: 14 December 2009 DOI: 10.1093/obo/9780195389678-0008

Social work research means conducting an investigation in accordance with the scientific method. The aim of social work research is to build the social work knowledge base in order to solve practical problems in social work practice or social policy. Investigating phenomena in accordance with the scientific method requires maximal adherence to empirical principles, such as basing conclusions on observations that have been gathered in a systematic, comprehensive, and objective fashion. The resources in this entry discuss how to do that as well as how to utilize and teach research methods in social work. Other professions and disciplines commonly produce applied research that can guide social policy or social work practice. Yet no commonly accepted distinction exists at this time between social work research methods and research methods in allied fields relevant to social work. Consequently useful references pertaining to research methods in allied fields that can be applied to social work research are included in this entry.

This section includes basic textbooks that are used in courses on social work research methods. Considerable variation exists between textbooks on the broad topic of social work research methods. Some are comprehensive and delve into topics deeply and at a more advanced level than others. That variation is due in part to the different needs of instructors at the undergraduate and graduate levels of social work education. Most instructors at the undergraduate level prefer shorter and relatively simplified texts; however, some instructors teaching introductory master’s courses on research prefer such texts too. The texts in this section that might best fit their preferences are by Yegidis and Weinbach 2009 and Rubin and Babbie 2007 . The remaining books might fit the needs of instructors at both levels who prefer a more comprehensive and deeper coverage of research methods. Among them Rubin and Babbie 2008 is perhaps the most extensive and is often used at the doctoral level as well as the master’s and undergraduate levels. Also extensive are Drake and Jonson-Reid 2007 , Grinnell and Unrau 2007 , Kreuger and Neuman 2006 , and Thyer 2001 . What distinguishes Drake and Jonson-Reid 2007 is its heavy inclusion of statistical and Statistical Package for the Social Sciences (SPSS) content integrated with each chapter. Grinnell and Unrau 2007 and Thyer 2001 are unique in that they are edited volumes with different authors for each chapter. Kreuger and Neuman 2006 takes Neuman’s social sciences research text and adapts it to social work. The Practitioner’s Guide to Using Research for Evidence-based Practice ( Rubin 2007 ) emphasizes the critical appraisal of research, covering basic research methods content in a relatively simplified format for instructors who want to teach research methods as part of the evidence-based practice process instead of with the aim of teaching students how to produce research.

Drake, Brett, and Melissa Jonson-Reid. 2007. Social work research methods: From conceptualization to dissemination . Boston: Allyn and Bacon.

This introductory text is distinguished by its use of many evidence-based practice examples and its heavy coverage of statistical and computer analysis of data.

Grinnell, Richard M., and Yvonne A. Unrau, eds. 2007. Social work research and evaluation: Quantitative and qualitative approaches . 8th ed. New York: Oxford Univ. Press.

Contains chapters written by different authors, each focusing on a comprehensive range of social work research topics.

Kreuger, Larry W., and W. Lawrence Neuman. 2006. Social work research methods: Qualitative and quantitative applications . Boston: Pearson, Allyn, and Bacon.

An adaptation to social work of Neuman's social sciences research methods text. Its framework emphasizes comparing quantitative and qualitative approaches. Despite its title, quantitative methods receive more attention than qualitative methods, although it does contain considerable qualitative content.

Rubin, Allen. 2007. Practitioner’s guide to using research for evidence-based practice . Hoboken, NJ: Wiley.

This text focuses on understanding quantitative and qualitative research methods and designs for the purpose of appraising research as part of the evidence-based practice process. It also includes chapters on instruments for assessment and monitoring practice outcomes. It can be used at the graduate or undergraduate level.

Rubin, Allen, and Earl R. Babbie. 2007. Essential research methods for social work . Belmont, CA: Thomson Brooks Cole.

This is a shorter and less advanced version of Rubin and Babbie 2008 . It can be used for research methods courses at the undergraduate or master's levels of social work education.

Rubin, Allen, and Earl R. Babbie. Research Methods for Social Work . 6th ed. Belmont, CA: Thomson Brooks Cole, 2008.

This comprehensive text focuses on producing quantitative and qualitative research as well as utilizing such research as part of the evidence-based practice process. It is widely used for teaching research methods courses at the undergraduate, master’s, and doctoral levels of social work education.

Thyer, Bruce A., ed. 2001 The handbook of social work research methods . Thousand Oaks, CA: Sage.

This comprehensive compendium includes twenty-nine chapters written by esteemed leaders in social work research. It covers quantitative and qualitative methods as well as general issues.

Yegidis, Bonnie L., and Robert W. Weinbach. 2009. Research methods for social workers . 6th ed. Boston: Allyn and Bacon.

This introductory paperback text covers a broad range of social work research methods and does so in a briefer fashion than most lengthier, hardcover introductory research methods texts.

Users without a subscription are not able to see the full content on this page. Please subscribe or login .

Oxford Bibliographies Online is available by subscription and perpetual access to institutions. For more information or to contact an Oxford Sales Representative click here .

About Social Work »
Meet the Editorial Board »
Adolescent Depression
Adolescent Pregnancy
Adolescents
Adoption Home Study Assessments
Adult Protective Services in the United States
African Americans
Aging out of foster care
Aging, Physical Health and
Alcohol and Drug Abuse Problems
Alcohol and Drug Problems, Prevention of Adolescent and Yo...
Alcohol Problems: Practice Interventions
Alcohol Use Disorder
Alzheimer's Disease and Other Dementias
Anti-Oppressive Practice
Asian Americans
Asian-American Youth
Autism Spectrum Disorders
Baccalaureate Social Workers
Behavioral Health
Behavioral Social Work Practice
Bereavement Practice
Bisexuality
Brief Therapies in Social Work: Task-Centered Model and So...
Bullying and Social Work Intervention
Canadian Social Welfare, History of
Case Management in Mental Health in the United States
Central American Migration to the United States
Child Maltreatment Prevention
Child Neglect and Emotional Maltreatment
Child Poverty
Child Sexual Abuse
Child Welfare
Child Welfare and Child Protection in Europe, History of
Child Welfare and Parents with Intellectual and/or Develop...
Child Welfare Effectiveness
Child Welfare, Immigration and
Child Welfare Practice with LGBTQ Youth and Families
Children of Incarcerated Parents
Christianity and Social Work
Chronic Illness
Clinical Social Work Practice with Adult Lesbians
Clinical Social Work Practice with Males
Cognitive Behavior Therapies with Diverse and Stressed Pop...
Cognitive Processing Therapy
Cognitive-Behavioral Therapy
Community Development
Community Policing
Community-Needs Assessment
Comparative Social Work
Computational Social Welfare: Applying Data Science in Soc...
Conflict Resolution
Council on Social Work Education
Counseling Female Offenders
Criminal Justice
Crisis Interventions
Cultural Competence and Ethnic Sensitive Practice
Culture, Ethnicity, Substance Use, and Substance Use Disor...
Dementia Care
Dementia Care, Ethical Aspects of
Depression and Cancer
Development and Infancy (Birth to Age Three)
Differential Response in Child Welfare
Digital Storytelling for Social Work Interventions
Direct Practice in Social Work
Disabilities
Disability and Disability Culture
Domestic Violence Among Immigrants
Early Pregnancy and Parenthood Among Child Welfare–Involve...
Eating Disorders
Ecological Framework
Elder Mistreatment
End-of-Life Decisions
Epigenetics for Social Workers
Ethical Issues in Social Work and Technology
Ethics and Values in Social Work
European Institutions and Social Work
European Union, Justice and Home Affairs in the
Evidence-based Social Work Practice: Issues, Controversies...
Families with Gay, Lesbian, or Bisexual Parents
Family Caregiving
Family Group Conferencing
Family Policy
Family Services
Family Therapy
Family Violence
Fathering Among Families Served By Child Welfare
Fetal Alcohol Spectrum Disorders
Field Education
Financial Literacy and Social Work
Financing Health-Care Delivery in the United States
Forensic Social Work
Foster Care
Foster care and siblings
Gender, Violence, and Trauma in Immigration Detention in t...
Generalist Practice and Advanced Generalist Practice
Grounded Theory
Group Work across Populations, Challenges, and Settings
Group Work, Research, Best Practices, and Evidence-based
Harm Reduction
Health Care Reform
Health Disparities
Health Social Work
History of Social Work and Social Welfare, 1900–1950
History of Social Work and Social Welfare, 1950-1980
History of Social Work and Social Welfare, pre-1900
History of Social Work from 1980-2014
History of Social Work in China
History of Social Work in Northern Ireland
History of Social Work in the Republic of Ireland
History of Social Work in the United Kingdom
HIV/AIDS and Children
HIV/AIDS Prevention with Adolescents
Homelessness
Homelessness: Ending Homelessness as a Grand Challenge
Homelessness Outside the United States
Human Needs
Human Trafficking, Victims of
Immigrant Integration in the United States
Immigrant Policy in the United States
Immigrants and Refugees
Immigrants and Refugees: Evidence-based Social Work Practi...
Immigration and Health Disparities
Immigration and Intimate Partner Violence
Immigration and Poverty
Immigration and Spirituality
Immigration and Substance Use
Immigration and Trauma
Impaired Professionals
Indigenous Peoples
Individual Placement and Support (IPS) Supported Employmen...
In-home Child Welfare Services
Intergenerational Transmission of Maltreatment
International Human Trafficking
International Social Welfare
International Social Work
International Social Work and Education
International Social Work and Social Welfare in Southern A...
Internet and Video Game Addiction
Interpersonal Psychotherapy
Intervention with Traumatized Populations
Intimate-Partner Violence
Juvenile Justice
Kinship Care
Korean Americans
Latinos and Latinas
Law, Social Work and the
LGBTQ Populations and Social Work
Mainland European Social Work, History of
Major Depressive Disorder
Management and Administration in Social Work
Maternal Mental Health
Medical Illness
Men: Health and Mental Health Care
Mental Health
Mental Health Diagnosis and the Addictive Substance Disord...
Mental Health Needs of Older People, Assessing the
Mental Health Services from 1990 to 2023
Mental Illness: Children
Mental Illness: Elders
Microskills
Middle East and North Africa, International Social Work an...
Military Social Work
Mixed Methods Research
Moral distress and injury in social work
Motivational Interviewing
Multiculturalism
Native Americans
Native Hawaiians and Pacific Islanders
Neighborhood Social Cohesion
Neuroscience and Social Work
Nicotine Dependence
Organizational Development and Change
Pain Management
Palliative Care
Palliative Care: Evolution and Scope of Practice
Pandemics and Social Work
Parent Training
Personalization
Person-in-Environment
Philosophy of Science and Social Work
Physical Disabilities
Podcasts and Social Work
Police Social Work
Political Social Work in the United States
Positive Youth Development
Postsecondary Education Experiences and Attainment Among Y...
Post-Traumatic Stress Disorder (PTSD)
Practice Interventions and Aging
Practice Interventions with Adolescents
Practice Research
Primary Prevention in the 21st Century
Productive Engagement of Older Adults
Profession, Social Work
Program Development and Grant Writing
Promoting Smart Decarceration as a Grand Challenge
Psychiatric Rehabilitation
Psychoanalysis and Psychodynamic Theory
Psychoeducation
Psychometrics
Psychopathology and Social Work Practice
Psychopharmacology and Social Work Practice
Psychosocial Framework
Psychosocial Intervention with Women
Psychotherapy and Social Work
Race and Racism
Readmission Policies in Europe
Redefining Police Interactions with People Experiencing Me...
Refugee Children, Unaccompanied Immigrant and
Rehabilitation
Religiously Affiliated Agencies
Reproductive Health
Restorative Justice
Risk Assessment in Child Protection Services
Risk Management in Social Work
Rural Social Work in China
Rural Social Work Practice
School Social Work
School Violence
School-Based Delinquency Prevention
Services and Programs for Pregnant and Parenting Youth
Severe and Persistent Mental Illness: Adults
Sexual and Gender Minority Immigrants, Refugees, and Asylu...
Sexual Assault
Single-System Research Designs
Social and Economic Impact of US Immigration Policies on U...
Social Development
Social Insurance and Social Justice
Social Justice and Social Work
Social Movements
Social Planning
Social Policy
Social Policy in Denmark
Social Security in the United States (OASDHI)
Social Work and Islam
Social Work and Social Welfare in East, West, and Central ...
Social Work and Social Welfare in Europe
Social Work Education and Research
Social Work Leadership
Social Work Luminaries: Luminaries Contributing to the Cla...
Social Work Luminaries: Luminaries contributing to the fou...
Social Work Luminaries: Luminaries Who Contributed to Soci...
Social Work Practice, Rare and Orphan Diseases and
Social Work Regulation
Social Work Research Methods
Social Work with Interpreters
Solution-Focused Therapy
Strategic Planning
Strengths Perspective
Strengths-Based Models in Social Work
Supplemental Security Income
Survey Research
Sustainability: Creating Social Responses to a Changing En...
Syrian Refugees in Turkey
Task-Centered Practice
Technology Adoption in Social Work Education
Technology, Human Relationships, and Human Interaction
Technology in Social Work
Terminal Illness
The Impact of Systemic Racism on Latinxs’ Experiences with...
Transdisciplinary Science
Translational Science and Social Work
Transnational Perspectives in Social Work
Transtheoretical Model of Change
Trauma-Informed Care
Triangulation
Tribal child welfare practice in the United States
United States, History of Social Welfare in the
Universal Basic Income
Veteran Services
Vicarious Trauma and Resilience in Social Work Practice wi...
Vicarious Trauma Redefining PTSD
Victim Services
Virtual Reality and Social Work
Welfare State Reform in France
Welfare State Theory
Women and Macro Social Work Practice
Women's Health Care
Work and Family in the German Welfare State
Workforce Development of Social Workers Pre- and Post-Empl...
Working with Non-Voluntary and Mandated Clients
Young and Adolescent Lesbians
Youth at Risk
Youth Services
Privacy Policy
Cookie Policy
Legal Notice
Accessibility

[185.80.151.9]
185.80.151.9

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

2.2 Paradigms, theories, and how they shape a researcher’s approach

Learning objectives.

Define paradigm, and describe the significance of paradigms
Identify and describe the four predominant paradigms found in the social sciences
Define theory
Describe the role that theory plays in social work research

The terms paradigm and theory are often used interchangeably in social science, although social scientists do not always agree whether these are identical or distinct concepts. This text makes a clear distinction between the two ideas because thinking about each concept as analytically distinct provides a useful framework for understanding the connections between research methods and social scientific ways of thinking.

Paradigms in social science

For our purposes, we’ll define paradigm as a way of viewing the world (or “analytic lens” akin to a set of glasses) and a framework from which to understand the human experience (Kuhn, 1962). It can be difficult to fully grasp the idea of paradigmatic assumptions because we are very ingrained in our own, personal everyday way of thinking. For example, let’s look at people’s views on abortion. To some, abortion is a medical procedure that should be undertaken at the discretion of each individual woman. To others, abortion is murder and members of society should collectively have the right to decide when, if at all, abortion should be undertaken. Chances are, if you have an opinion about this topic, you are pretty certain about the veracity of your perspective. Then again, the person who sits next to you in class may have a very different opinion and yet be equally confident about the truth of their perspective. Who is correct?

You are each operating under a set of assumptions about the way the world does—or at least should—work. Perhaps your assumptions come from your political perspective, which helps shape your view on a variety of social issues, or perhaps your assumptions are based on what you learned from your parents or in church. In any case, there is a paradigm that shapes your stance on the issue. Those paradigms are a set of assumptions. Your classmate might assume that life begins at conception and the fetus’ life should be at the center of moral analysis. Conversely, you may assume that life begins when the fetus is viable outside the womb and that a mother’s choice is more important than a fetus’s life. There is no way to scientifically test when life begins, whose interests are more important, or the value of choice. They are merely philosophical assumptions or beliefs. Thus, a pro-life paradigm may rest in part on a belief in divine morality and fetal rights. A pro-choice paradigm may rest on a mother’s self-determination and a belief that the positive consequences of abortion outweigh the negative ones. These beliefs and assumptions influence how we think about any aspect of the issue.

In Chapter 1, we discussed the various ways that we know what we know. Paradigms are a way of framing what we know, what we can know, and how we can know it. In social science, there are several predominant paradigms, each with its own unique ontological and epistemological perspective. Recall that ontology is the study of what is real, and epistemology is the study of how we come to know what is real. Let’s look at four of the most common social scientific paradigms that might guide you as you begin to think about conducting research.

The first paradigm we’ll consider, called positivism, is the framework that likely comes to mind for many of you when you think of science. Positivism is guided by the principles of objectivity, knowability, and deductive logic. Deductive logic is discussed in more detail in next section of this chapter. The positivist framework operates from the assumption that society can and should be studied empirically and scientifically. Positivism also calls for a value-free science, one in which researchers aim to abandon their biases and values in a quest for objective, empirical, and knowable truth.

Another predominant paradigm in social work is social constructionism . Peter Berger and Thomas Luckman (1966) are credited by many for having developed this perspective in sociology. While positivists seek “the truth,” the social constructionist framework posits that “truth” varies. Truth is different based on who you ask, and people change their definitions of truth all the time based on their interactions with other people. This is because we, according to this paradigm, create reality ourselves (as opposed to it simply existing and us working to discover it) through our interactions and our interpretations of those interactions. Key to the social constructionist perspective is the idea that social context and interaction frame our realities.

Researchers operating within this framework take keen interest in how people come to socially agree, or disagree, about what is real and true. Consideration of how meanings of different hand gestures vary across different regions of the world aptly demonstrates that meanings are constructed socially and collectively. Think about what it means to you when you see a person raise their middle finger. In the United States, people probably understand that person isn’t very happy (nor is the person to whom the finger is being directed). In some societies, it is another gesture, such as the thumbs up gesture, that raises eyebrows. While the thumbs up gesture may have a particular meaning in North American culture, that meaning is not shared across cultures (Wong, 2007). So, what is the “truth” of the middle finger or thumbs up? It depends on what the person giving it intended, how the person receiving it interpreted it, and the social context in which the action occurred.

It would be a mistake to think of the social constructionist perspective as only individualistic. While individuals may construct their own realities, groups—from a small one such as a married couple to large ones such as nations—often agree on notions of what is true and what “is.” In other words, the meanings that we construct have power beyond the individual people who create them. Therefore, the ways that people and communities work to create and change such meanings is of as much interest to social constructionists as how they were created in the first place.

A third paradigm is the critical paradigm. At its core, the critical paradigm is focused on power, inequality, and social change. Although some rather diverse perspectives are included here, the critical paradigm, in general, includes ideas developed by early social theorists, such as Max Horkheimer (Calhoun, Gerteis, Moody, Pfaff, & Virk, 2007), and later works developed by feminist scholars, such as Nancy Fraser (1989). Unlike the positivist paradigm, the critical paradigm posits that social science can never be truly objective or value-free. Further, this paradigm operates from the perspective that scientific investigation should be conducted with the express goal of social change in mind. Researchers in the critical paradigm might start with the knowledge that systems are biased against, for example, women or ethnic minorities. Moreover, their research projects are designed not only to collect data, but also change the participants in the research as well as the systems being studied. The critical paradigm not only studies power imbalances but seeks to change those power imbalances.

Finally, postmodernism is a paradigm that challenges almost every way of knowing that many social scientists take for granted (Best & Kellner, 1991). While positivists claim that there is an objective, knowable truth, postmodernists would say that there is not. While social constructionists may argue that truth is in the eye of the beholder (or in the eye of the group that agrees on it), postmodernists may claim that we can never really know such truth because, in the studying and reporting of others’ truths, the researcher stamps their own truth on the investigation. Finally, while the critical paradigm may argue that power, inequality, and change shape reality and truth, a postmodernist may in turn ask whose power, whose inequality, whose change, whose reality, and whose truth. As you might imagine, the postmodernist paradigm poses quite a challenge for researchers. How do you study something that may or may not be real or that is only real in your current and unique experience of it? This fascinating question is worth pondering as you begin to think about conducting your own research. Part of the value of the postmodern paradigm is its emphasis on the limitations of human knowledge. Table 2.1 summarizes each of the paradigms discussed here.

Table 2.1 Four social science paradigms

Positivism	Objectivity, knowability, and deductive logic	Society can and should be studied empirically and scientifically.
Social Constructionism	Truth as varying, socially constructed, and ever-changing	Reality is created collectively. Social context and interaction frame our realities.
Critical	Power, inequality, and social change	Social science can never be truly value-free and should be conducted with the express goal of social change in mind.
Postmodernism	Inherent problems with previous paradigms.	Truth is always bound within historical and cultural context. There are no universally true explanations.

Let’s work through an example. If we are examining a problem like substance abuse, what would a social scientific investigation look like in each paradigm? A positivist study may focus on precisely measuring substance abuse and finding out the key causes of substance abuse during adolescence. Forgoing the objectivity of precisely measuring substance abuse, social constructionist study might focus on how people who abuse substances understand their lives and relationships with various drugs of abuse. In so doing, it seeks out the subjective truth of each participant in the study. A study from the critical paradigm would investigate how people who have substance abuse problems are an oppressed group in society and seek to liberate them from external sources of oppression, like punitive drug laws, and internal sources of oppression, like internalized fear and shame. A postmodern study may involve one person’s self-reported journey into substance abuse and changes that occurred in their self-perception that accompanied their transition from recreational to problematic drug use. These examples should illustrate how one topic can be investigated across each paradigm.

Social science theories

Much like paradigms, theories provide a way of looking at the world and of understanding human interaction. Paradigms are grounded in big assumptions about the world—what is real, how do we create knowledge—whereas theories describe more specific phenomena. A common definition for theory in social work is “a systematic set of interrelated statements intended to explain some aspect of social life” (Rubin & Babbie, 2017, p. 615). At their core, theories can be used to provide explanations of any number or variety of phenomena. They help us answer the “why” questions we often have about the patterns we observe in social life. Theories also often help us answer our “how” questions. While paradigms may point us in a particular direction with respect to our “why” questions, theories more specifically map out the explanation, or the “how,” behind the “why.”

Introductory social work textbooks introduce students to the major theories in social work—conflict theory, symbolic interactionism, social exchange theory, and systems theory. As social workers study longer, they are introduced to more specific theories in their area of focus, as well as perspectives and models (e.g., the strengths perspective), which provide more practice-focused approaches to understanding social work.

As you may recall from a class on social work theory, systems theorists view all parts of society as interconnected and focus on the relationships, boundaries, and flows of energy between these systems and subsystems (Schriver, 2011). Conflict theorists are interested in questions of power and who wins and who loses based on the way that society is organized. Symbolic interactionists focus on how meaning is created and negotiated through meaningful (i.e., symbolic) interactions. Finally, social exchange theorists examine how human beings base their behavior on a rational calculation of rewards and costs.

Just as researchers might examine the same topic from different levels of inquiry or paradigms, they could also investigate the same topic from different theoretical perspectives. In this case, even their research questions could be the same, but the way they make sense of whatever phenomenon it is they are investigating will be shaped in large part by theory. Table 2.2 summarizes the major points of focus for four major theories and outlines how a researcher might approach the study of the same topic, in this case the study of substance abuse, from each of the perspectives.

Table 2.2 Four social work theories as related to the study of substance abuse

Systems	Interrelations between parts of society; how parts work together	How a lack of employment opportunities might impact rates of substance abuse in an area
Conflict	Who wins and who loses based on the way that society is organized	How the War on Drugs has impacted minority communities
Symbolic interactionism	How meaning is created and negotiated though interactions	How people’s self-definitions as “addicts” helps or hurts their ability to remain sober
Utility theory	How behavior is influenced by costs and rewards	Whether increased distribution of anti-overdose medications makes overdose more or less likely

Within each area of specialization in social work, there are many other theories that aim to explain more specific types of interactions. For example, within the study of sexual harassment, different theories posit different explanations for why harassment occurs. One theory, first developed by criminologists, is called routine activities theory. It posits that sexual harassment is most likely to occur when a workplace lacks unified groups and when potentially vulnerable targets and motivated offenders are both present (DeCoster, Estes, & Mueller, 1999). Other theories of sexual harassment, called relational theories, suggest that a person’s relationships, such as their marriages or friendships, are the key to understanding why and how workplace sexual harassment occurs and how people will respond to it when it does occur (Morgan, 1999). Relational theories focus on the power that different social relationships provide (e.g., married people who have supportive partners at home might be more likely than those who lack support at home to report sexual harassment when it occurs). Finally, feminist theories of sexual harassment take a different stance. These theories posit that the way our current gender system is organized, where those who are the most masculine have the most power, best explains why and how workplace sexual harassment occurs (MacKinnon, 1979). As you might imagine, which theory a researcher applies to examine the topic of sexual harassment will shape the questions the researcher asks about harassment. It will also shape the explanations the researcher provides for why harassment occurs.

For an undergraduate student beginning their study of a new topic, it may be intimidating to learn that there are so many theories beyond what you’ve learned in your theory classes. What’s worse is that there is no central database of different theories on your topic. However, as you review the literature in your topic area, you will learn more about the theories that scientists have created to explain how your topic works in the real world. In addition to peer-reviewed journal articles, another good source of theories is a book about your topic. Books often contain works of theoretical and philosophical importance that are beyond the scope of an academic journal.

Paradigm and theory in social work

Theories, paradigms, levels of analysis, and the order in which one proceeds in the research process all play an important role in shaping what we ask about the social world, how we ask it, and in some cases, even what we are likely to find. A micro-level study of gangs will look much different than a macro-level study of gangs. In some cases, you could apply multiple levels of analysis to your investigation, but doing so isn’t always practical or feasible. Therefore, understanding the different levels of analysis and being aware of which level you happen to be employing is crucial. One’s theoretical perspective will also shape a study. In particular, the theory invoked will likely shape not only the way a question about a topic is asked but also which topic gets investigated in the first place. Further, if you find yourself especially committed to one theory over another, it may limit the kinds of questions you pose. As a result, you may miss other possible explanations.

The limitations of paradigms and theories do not mean that social science is fundamentally biased. At the same time, we can never claim to be entirely value free. Social constructionists and postmodernists might point out that bias is always a part of research to at least some degree. Our job as researchers is to recognize and address our biases as part of the research process, if an imperfect part. We all use our own approaches, be they theories, levels of analysis, or temporal processes, to frame and conduct our work. Understanding those frames and approaches is crucial not only for successfully embarking upon and completing any research-based investigation, but also for responsibly reading and understanding others’ work.

Spotlight on UTA School of Social Work

Catherine labrenz connects social theory and child welfare research.

When Catherine LaBrenz, an assistant professor at the University of Texas at Arlington’s School of Social Work was a child welfare practitioner, she noticed that several children who had reunified with their biological parents from the foster care system were re-entering care because of continued exposure to child maltreatment. As she observed the challenging behaviors these children often presented, she wondered how the agency might better support families to prevent children from re-entering foster care after permanence. In her doctoral studies, she used her practice experience to form a research project with the goal of better understanding how agencies could better support families post-reunification.

From a critical paradigm, Dr. LaBrenz approached this question with the understanding that families that come into contact with child welfare systems often experience disadvantage and are subjected to unequal power distributions when accessing services, going to court, and participating in case decision-making (LaBrenz & Fong, 2016). Furthermore, the goal of this research was to change some of the aspects of the child welfare system, particularly within the practitioner’s agency, to better support families.

To better understand why some families may be more at-risk for multiple entries into foster care, Dr. LaBrenz began with an extensive literature review that identified diverse theories that explained factors at the child, family, and system- level that could impact post-permanence success. Figure 2.1 displays the micro-, meso-, and macro-level theories that she and her research team identified and decided to explore further.

This figure displays a three-level model of theories: At the top Child - Attachment, beneath that Family - family systems theory, and at the bottom System - systems theory and critical race theory

At the child-level, Attachment theory posits that consistent, stable nurturing during infancy impacts children’s ability to form relationships with others throughout their life (Ainsworth, Blehar, Waters, & Wall, 1978; Bowlby, 1969). At the family-level, Family systems theory posits that family interactions impact functioning among all members of a family unit (Broderick 1971). At the macro-level, Critical race theory (Delgado & Stefancic, 2001) can help understand racial disparities in child welfare systems. Moreover, Systems theory (Bronfenbrenner, 1986) can help examine interactions among the micro-, meso- and macro-levels to assess diverse systems that impact families involved in child welfare services.

In the next step of the project, national datasets were used to examine child-, family-, and system- factors that impacted rates of successful reunification, or reunification with no future re-entries into foster care. Then, a systematic review of the literature was conducted to determine what evidence existed for interventions to increase rates of successful reunification. Finally, a different national dataset was used to examine how effective diverse interventions were for specific groups of families, such as those with infants and toddlers.

Figure 2.2 displays the principal findings from the research project and connects each main finding to one of the theoretical frameworks.

A figure displaying Catherine LaBrenz' findings by 4 different social theories: Attachment Theory, Family Systems Theory, Systems Theory, and Critical Race Theory

The first part of the research project found parents who felt unable to cope with their parental role, and families with previous attachment disruptions, to have higher rates of re-entry into foster care. This connects with Attachment theory, in that families with more instability and inconsistency in caregiving felt less able to fulfill their parental roles, which in turn led to further disruption in the child’s attachment.

With regards to family-level theories, Dr. LaBrenz found that family-level risk and protective factors were more predictive of re-entry to foster care than child- or agency-level factors. The systematic review also found that interventions that targeted parents, such as Family Drug Treatment Courts, led to better outcomes for children and families. This aligns with Family systems theory in that family-centered interventions and targeting the entire family leads to better family functioning and fewer re-entries into foster care.

In parallel, the systematic review concluded that interventions that integrated multiple systems, such as child welfare and substance use, increased the likelihood of successful reunification. This supports Systems theory, in that multiple systems can be engaged to provide ongoing support for families in child welfare systems (Trucco, 2012). Furthermore, the results from the analyses of the national datasets found that rates of re-entry into foster care for African American and Latino families varied significantly by state. Thus, racial and ethnic disparities remained in some, but not all, state child welfare systems.

Overall, the findings from the research project supported Attachment theory, Family systems theory, Systems theory, and Critical race theory as guiding explanations for why some children and families experience foster care re-entry while others do not. Dr. LaBrenz was able to present these findings and connect them to direct implications for practices and policies that could support attachment, multi-system collaborations, and family-centered practices.

Key Takeaways

Paradigms shape our everyday view of the world.
Researchers use theory to help frame their research questions and to help them make sense of the answers to those questions.
Applying the four key theories of social work is a good start, but you will likely have to look for more specific theories about your topic.
Critical paradigm- a paradigm in social science research focused on power, inequality, and social change
Paradigm- a way of viewing the world and a framework from which to understand the human experience
Positivism- a paradigm guided by the principles of objectivity, knowability, and deductive logic
Postmodernism- a paradigm focused on the historical and contextual embeddedness of scientific knowledge and a skepticism towards certainty and grand explanations in social science
Social constructionism- a paradigm based on the idea that social context and interaction frame our realities
Theory- “a systematic set of interrelated statements intended to explain some aspect of social life” (Rubin & Babbie, 2017, p. 615)

Image attributions

point mold and cloud mold by tasaikensuke CC-0

why by GDJ CC-0

Foundations of Social Work Research Copyright © 2020 by Rebecca L. Mauldin is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Scientific Inquiry in Social Work

(9 reviews)

Matthew DeCarlo, Radford University

ISBN 13: 9781975033729

Publisher: Open Social Work Education

Language: English

Formats Available

Conditions of use.

Learn more about reviews.

Reviewed by Shannon Blajeski, Assistant Professor, Portland State University on 3/10/23

Comprehensiveness rating: 5 see less

This book provides an introduction to research and inquiry in social work with an applied focus geared for the MSW student. The text covers 16 chapters, including several dedicated to understanding how to begin the research process, a chapter on ethics, and then eight chapters dedicated to research methods. The subchapters (1-5 per chapter) are concise and focused while also being tied to current knowledge and events so as to hold the reader's attention. It is comprehensive, but some of the later chapters covering research methods as well as the final chapter seem a bit scant and could be expanded. The glossary at the end of each chapter is helpful as is the index that is always accessible from the left-hand drop-down menu.

Content Accuracy rating: 4

The author pulls in relevant current and recent public events to illustrate important points about social research throughout the book. Each sub-chapter reads as accurate. I did not come across any inaccuracies in the text, however I would recommend a change in the title of Chapter 15 as "real world research" certainly encompasses more than program evaluation, single-subject designs, and action research.

Relevance/Longevity rating: 5

Another major strength of this book is that it adds currency to engage the reader while also maintaining its relevance to research methods. None of the current events/recent events that are described seem dated nor will they fade from relevance in a number of years. In addition, the concise nature of the modules should make them easy to update when needed to maintain relevancy in future editions.

Clarity rating: 5

Clarity is a major strength of this textbook. As described in the interface section, this book is written to be clear and concise, without unnecessary extra text that detracts from the concise content provided in each chapter. Any lengthy excerpts are also very engaging which lends itself to a clear presentation of content for the reader.

Consistency rating: 5

The text and content seems to be presented consistently throughout the book. Terminology and frameworks are balanced with real-world examples and current events.

Modularity rating: 5

The chapters of this textbook are appropriately spaced and easily digestible, particularly for readers with time constraints. Each chapter contains 3-5 sub-chapters that build upon each other in a scaffolding style. This makes it simple for the instructor to assign each chapter (sometimes two) per weekly session as well as add in additional assigned readings to complement the text.

Organization/Structure/Flow rating: 5

The overall organization of the chapters flow well. The book begins with a typical introduction to research aimed at social work practitioners or new students in social work. It then moves into a set of chapters on beginning a research project, reviewing literature, and asking research questions, followed by a chapter on ethics. Next, the text transitions to three chapters covering constructs, measurement, and sampling, followed by five chapters covering research methods, and a closing chapter on dissemination of research. This is one of the more logically-organized research methods texts that I have used as an instructor.

Interface rating: 5

The modular chapters are easy to navigate and the interface of each chapter follows a standard presentation style with the reading followed by a short vocabulary glossary and references. This presentation lends itself to a familiarity for students that helps them become more efficient with completing reading assignments, even in short bursts of time. This is particularly important for online and returning learners who may juggle their assignment time with family and work obligations.

Grammatical Errors rating: 5

No grammatical errors were noted.

Cultural Relevance rating: 4

At first glance at the table of contents, the book doesn't seem to be overtly committed to cultural representation, however, upon reading the chapters, it becomes clear that the author does try to represent and reference marginalized groups (e.g., women, individuals with disabilities, racial/ethnic/gender intersectionality) within the examples used. I also am very appreciative that the bottom of each introduction page for each chapter contains content trigger warnings for any possible topics that could be upsetting, e.g., substance abuse, violence.

As the author likely knows, social work students are eager to engage in learning that is current and relevant to their social causes. This book is written in a way that engages a non-researcher social worker into reading about research by weaving research information into topics that they might find compelling. It also does this in a concise way where short bits of pertinent information are presented, making the text accessible without needing to sustain long periods of attention. This is particularly important for online and returning learners who may need to sit with their readings in short bursts due to attending school while juggling work and family obligations.

Reviewed by Lynn Goerdt, Associate Professor, University of Wisconsin - Superior on 9/17/21

Text appears to be comprehensive in covering steps for typical SWK research class, taking students from the introduction of the purpose and importance of research to how to design and analyze research. Author covers the multitude of ways that social workers engage in research as way of building knowledge and ways that social work practitioners conduct research to evaluate their practice, including outcome evaluation, single subject design, and action research. I particularly appreciated the last section on reporting research, which should be very practical.

Overall, content appears mostly accurate which few errors. Definitions and citations are mostly thorough and clear. Author does cite Wikipedia in at least one occasion which could be credible, depending on the source of the Wikipedia content. There were a few references within the text to comic or stories but the referenced material was not always apparent.

Relevance/Longevity rating: 4

The content of Scientific Inquiry for Social Work is relevant, as the field of social work research methods does not appear to change quickly, although there are innovations. The author referenced examples which appear to be recent and likely relatable to interests of current students. Primary area of innovation is in using technology for the collection and analysis of data, which could be expanded, particularly using social media for soliciting research participants.

Style is personable and content appears to be accessible, which is a unique attribute for a research textbook. Author uses first person in many instances, particularly in the beginning to present the content as relatable.

Format appears to be consistent in format and relative length. Each chapter includes learning objectives, content advisory (if applicable), key takeaways and glossary. Author uses color and text boxes to draw attention to these sections.

Modularity rating: 4

Text is divided into modules which could easily be assigned and reviewed in a class. The text modules could also be re-structured if desired to fit curricular uniqueness’s. Author uses images to illuminate the concepts of the module or chapter, but they often take about 1/3 of the page, which extends the size of the textbook quite a bit. Unclear if benefit of images outweighs additional cost if PDF version is printed.

Textbook is organized in a very logical and clear fashion. Each section appears to be approximately 6-10 pages in length which seems to be an optimal length for student attention and comprehension.

Interface rating: 4

There were some distortions of the text (size and visibility) but they were a fairly minor distraction and did not appear to reduce access to the content. Otherwise text was easy to navigate.

Grammatical Errors rating: 4

No grammatical errors were noted but hyperlinks to outside content were referenced but not always visible which occasionally resulted in an awkward read. Specific link may be in resources section of each chapter but occasionally they were also included in the text.

I did not recognize any text which was culturally insensitive or offensive. Images used which depicted people, appeared to represent diverse experiences, cultures, settings and persons. Did notice image depicting homelessness appeared to be stereotypical person sleeping on sidewalk, which can perpetuate a common perception of homelessness. Would encourage author to consider images representing a wider range of experiences of a social phenomena. Content advisories are used for each section, which is not necessarily cultural relevance but is respectful and recognizes the diversity of experiences and triggers that the readers may have.

Overall, I was very impressed and encouraged with the well organized content and thoughtful flow of this important textbook for social work students and instructors. The length and readability of each chapter would likely be appreciated by instructors as well as students, increasing the extent that the learning outcomes would be achieved. Teaching research is very challenging because the content and application can feel very intimidating. The author also has provided access to supplemental resources such as presentations and assignments.

Reviewed by elaine gatewood, Adjunct Faculty, Bridgewater State University on 6/15/21

The book provides concrete and clear information on using research as consumers, It provides a comprehensive review of each step to take to develop a research project from beginning to completion, with examples. read more

Content Accuracy rating: 5

From my perspective, content is highly accurate in the field of learning research method and unbiased. It's all there!

The content is highly relevant and up-to-date in the field. The book is written and arranged in a way that its easy to follow along with adding updates.

The book is written in clear and concise. The book provides appropriate context for any jargon/technical terminology used along with examples which readers are able to follow along and understand.

The contents of the book flow quite well. The framework in the book is consistent.

The text appears easily adaptable for readers and the author also provides accompanying PowerPoint presentations; these are a good foundation tools for readers to use and implement.

Organization/Structure/Flow rating: 4

The contents of the book flow very well. Readers would be able to put into practice the key reading strategies shared in the book ) because its organization is laid out nicely

Interface rating: 3

The interface is generally good, but I was only able to download the .pdf. This may present issues for some student readers.

There are no grammatical errors.

The text was culturally relevant and provided diverse research and practice examples. The text could have benefited from sexamples of intersectional and anti-oppressive lenses for students to consider in their practice.

This text is a comprehensive introduction to research that can be easily adapted for a BSW/MSW research course.

Reviewed by Taylor Hall, Assistant Professor, Bridgewater State University on 6/30/20

This text is more comprehensive than the text I currently use in my Research Methods in Social Work course, which students have to pay for. This text not only covers both qualitative and quantitative research methods, but also all parts of the research process from thinking about research ideas to questions all the way to evaluation after social work programs/policies have been employed.

Not much to say here- with research methods, things are black and white; it is or isn't. This content is accurate. I also like to way the content is explained in light of social work values and ethics. This is something our students can struggle with, and this is helpful in terms of showing why social work needs to pay attention to research.

There are upcoming changes to CSWE's competencies, therefore lots of text materials are going to need to be updated soon. Otherwise, case examples are pertinent and timely.

Clarity rating: 4

I think that research methods for social workers is a difficult field of study. Many go into the field to be clinicians, and few understand (off the bat) the importance of understanding methods of research. I think this textbook makes it clear to me, but hard to rate a 5 as I know from a student's perspective, lots of the terminology is so new.

Appears to be so- I was able to follow, seems consistent.

Yes- and I think this is a strong point of this text. This was easy to follow and read, and I could see myself easily divvying up different sections for students to work on in groups.

Yes- makes sense to me and the way I teach this course. I like the 30,000 ft view then honing in on specific types of research, all along the way explaining the different pieces of the research process and in writing a research paper.

I sometimes struggle with online platforms versus in person texts to read, and this OER is visually appealing- there is not too much text on the pages, it is spaced in a way that makes it easier to read. Colors are used well to highlight pertinent information.

Not something I found in this text.

Cultural Relevance rating: 3

This is a place where I feel the text could use some work. A nod to past wrongdoings in research methods on oppressed groups, and more of a discussion on social work's role in social justice with an eye towards righting the wrongs of the past. Updated language re: person first language, more diverse examples, etc.

This is a very useful text, and I am going to recommend my department check it out for future use, especially as many of our students are first gen and working class and would love to save money on textbooks where possible.

Reviewed by Olubunmi Oyewuwo-Gassikia, Assistant Professor, Northeastern Illinois University on 5/5/20

This text is an appropriate and comprehensive introduction to research methods for BSW students. It guides the reader through each stage of the research project, including identifying a research question, conducting and writing a literature review, research ethics, theory, research design, methodology, sampling, and dissemination. The author explains complex concepts - such as paradigms, epistemology, and ontology - in clear, simple terms and through the use of practical, social work examples for the reader. I especially appreciated the balanced attention to quantitative and qualitative methods, including the explanation of data collection and basic analysis techniques for both. The text could benefit from the inclusion of an explanation of research design notations.

The text is accurate and unbiased. Additionally, the author effectively incorporates referenced sources, including sources one can use for further learning.

The content is relevant and timely. The author incorporates real, recent research examples that reflects the applicability of research at each level of social practice (micro, meso, and macro) throughout the text. The text will benefit from regular updates in research examples.

The text is written in a clear, approachable manner. The chapters are a reasonable length without sacrificing appropriate depth into the subject manner.

The text is consistent throughout. The author is effective in reintroducing previously explained terms from previous chapters.

The text appears easily adaptable. The instructions provided by the author on how to adapt the text for one's course are helpful to one who would like to use the text but not in its entirety. The author also provides accompanying PowerPoint presentations; these are a good foundation but will likely require tailoring based on the teaching style of the instructor.

Generally, the text flows well. However, chapter 5 (Ethics) should come earlier, preferably before chapter 3 (Reviewing & Evaluating the Literature). It is important that students understand research ethics as ethical concerns are an important aspect of evaluating the quality of research studies. Chapter 15 (Real-World Research) should also come earlier in the text, most suitably before or after chapter 7 (Design and Causality).

The interface is generally good, but I was only able to download the .pdf. The setup of the .pdf is difficult to navigate, especially if one wants to jump from chapter to chapter. This may present issues for the student reader.

The text was culturally relevant and provided diverse research and practice examples. The text could have benefited from more critical research examples, such as examples of research studies that incorporate intersectional and anti-oppressive lenses.

This text is a comprehensive introduction to research that can be easily adapted for a BSW level research course.

Reviewed by Smita Dewan, Assistant Professor, New York City College of Technology, Department of Human Services on 12/6/19

Comprehensiveness rating: 4 see less

This is a very good introductory research methodology textbook for undergraduate students of social work or human services. For students who might be intimidated by social research, the text provides assurance that by learning basic concepts of research methodology, students will be better scholars and social work or human service practitioners. The content and flow of the text book supports a basic assignment of most research methodology courses which is to develop a research proposal or a research project. Each stage of research is explained well with many examples from social work practice that has the potential to keep the student engaged.

The glossary at the end of each chapter is very comprehensive but does not include the page number/s where the content is located. The glossary at the end of the book also lacks page numbers which might make it cumbersome for students seeking a quick reference.

The content is accurate and unbiased. Suggested exercises and prompts for students to engage in critical thinking and to identify biases in research that informs practice may help students understand the complexities of social research.

Content is up-to-date and concepts of research methodology presented is unlikely to be obsolete in the coming years. However, recent trends in research such as data mining, using algorithms for social policy and practice implications, privacy concerns, role of social media are topics that could be considered for inclusion in the forthcoming editions.

Content is presented very clearly for undergraduate students. Key takeaways and glossary for each section of the chapter is very useful for students.

Presentation of content, format and organization is consistent throughout the book.

Subsections within each chapter is very helpful for the students who might be assigned readings just in parts for the class.

Students would benefit from reading about research ethics right after the introductory chapter. I would also move Chapter 8 to right after the literature review which might inform creating and refining the research question. Content on evaluation research could also be moved up to follow the chapter on experimental designs. Regardless of the organization, the course instructors can assign chapters according to the course requirements.

PDF version of the book is very easy to use especially as students can save a copy on their computers and do not have to be online. Charts and tables are well presented but some of the images/photographs do not necessarily serve to enhance learning. Image attributions could be provided at the end of the chapter instead of being listed under the glossary. Students might also find it useful to be able to highlight the content and make annotations. This requires that students sign-in. Students should be able to highlight and annotate a downloaded version through Adobe Reader.

I did not find any grammatical errors.

Cultural Relevance rating: 5

Content is not insensitive or offensive in any way. Supporting examples in chapters are very diverse. Students would benefit from some examples of international research (both positive and negative examples) of protection of human subjects.

Reviewed by Jill Hoffman, Assistant Professor, Portland State University on 10/29/19

This text includes 16 chapters that cover content related to the process of conducting research. From identifying a topic and reviewing the literature, to formulating a question, designing a study, and disseminating findings, the text includes research basics that most other introductory social work research texts include. Content on ethics, theory, and to a lesser extent evaluation, single-subject design, and action research are also included. There is a glossary at the end of the text that includes information on the location of the terms. There is a practice behaviors index, but not an index in the traditional sense. If using the text electronically, search functions make it easy to find necessary information despite not having an index. If using a printed version, this would be more difficult. The text includes examples to illustrate concepts that are relevant to settings in which social workers might work. As most other introductory social work research texts, this book appears to come from a mainly positivist view. I would have appreciated more of a discussion related to power, privilege, and oppression and the role these play in the research topics that get studied and who benefits, along with anti-oppressive research. Related to evaluations, a quick mention of logic models would be helpful.

The information appears to be accurate and error free. The language in the text seems to emphasize "right/wrong" choices/decisions instead of highlighting the complexities of research and practice. Using gender-neutral pronouns would also make the language more inclusive.

Content appears to be up-to-date and relevant. Any updating would be straightforward to carry out. I found at least one link that did not work (e.g., NREPP) so if you use this text it will be important to check and make sure things are updated.

The content is clearly written, using examples to illustrate various concepts. I appreciated prompts for questions throughout each chapter in order to engage students in the content. Key terms are bolded, which helps to easily identify important points.

Information is presented in a consistent manner throughout the text.

Each chapter is divided into subsections that help with readability. It is easy to pick and choose various pieces of the text for your course if you're not using the entire thing.

There are many ways you can organize a social work research text. Personally, I prefer to talk about ethics and theory early on, so that students have this as a framework as they read about other's studies and design their own. In the case of this text, I'd put those two chapters right after chapter 1. As others have suggested, I'd also move up the content on research questions, perhaps after chapter 4.

In the online version, no significant interface issues arose. The only thing that would be helpful is to have chapter titles clearly presented when navigating through the text in the online version. For example, when you click through to a new chapter, the title simply says "6.0 Chapter introduction." In order to see the chapter title you have to click into the contents tab. Not a huge issue but could help with navigating the online version. In the pdf version, the links in the table of contents allowed me to navigate through to various sections. I did notice that some of the external links were not complete (e.g., on page 290, the URL is linked as "http://baby-").

Cultural representation in the text is similar to many other introductory social work research texts. There's more of an emphasis on white, western, cis-gendered individuals, particularly in the images. In examples, it appeared that only male/female pronouns were used.

Reviewed by Monica Roth Day, Associate Professor, Social Work, Metropolitan State University (Saint Paul, Minnesota) on 12/26/18

The book provides concrete and clear information on using research as consumers, then developing research as producers of knowledge. It provides a comprehensive review of each step to take to develop a research project from beginning to completion, with appropriate examples. More specific social work links would be helpful as students learn more about the field and the uses of research.

The book is accurate and communicates information and largely without bias. Numerous examples are provided from varied sources, which are then used to discuss potential for bias in research. The addition of critical race theory concepts would add to this discussion, to ground students in the importance of understanding implicit bias as researchers and ways to develop their own awareness.

The book is highly relevant. It provides historical and current examples of research which communicate concepts using accessible language that is current to social work. The text is written so that updates should be easy. Links need to be updated on a regular basis.

The book is accessible for students at it uses common language to communicate concepts while helping students build their research vocabulary. Terminology is communicate both within the text and in glossaries, and technical terms are minimally used.

The book is consistent in its use of terminology and framework. It follows a pattern of development, from consuming research to producing research. The steps are predictable and walk students through appropriate actions to take.

The book is easily readable. Each chapter is divided in sections that are easy to navigate and understand. Pictures and tables are used to support text.

Chapters are in logical order and follow a common pattern.

When reading the book online, the text was largely free of interface issues. As a PDF, there were issues with formatting. Be aware that students who may wish to download the book into a Kindle or other book reader may experience issues.

The text was grammatically correct with no misspellings.

While the book is culturally relevant, it lacks the application of critical race theory. While students will learn about bias in research, critical race theory would ground students in the importance of understanding implicit bias as researchers and ways to develop their own awareness. It would also help students understand why the background of researchers is important in relation to the ways of knowing.

Reviewed by Jennifer Wareham, Associate Professor, Wayne State University on 11/30/18

The book provides a comprehensive introduction to research methods from the perspective of the discipline of Social Work. The book borrows heavily from Amy Blackstone’s Principles of Sociological Inquiry – Qualitative and Quantitative Methods open textbook. The book is divided into 16 chapters, covering: differences in reasoning and scientific thought, starting a research project, writing a literature review, ethics in social science research, how theory relates to research, research design, causality, measurement, sampling, survey research, experimental design, qualitative interviews and focus groups, evaluation research, and reporting research. Some of the more advanced concepts and topics are only covered at superficial level, which limits the intended population of readers to high school students, undergraduate students, or those with no background in research methods. Since the book is geared toward Social Work undergraduate students, the chapters and content address methodologies commonly used in this field, but ignore methodologies that may be more popular in other social science fields. For example, the material on qualitative methods is narrow and focuses on commonly used qualitative methods in Social Work. In addition, the chapter on evaluation is limited to a general overview of evaluation research, which could be improved with more in-depth discussion of different types of evaluation (e.g., needs assessment, evaluability assessment, process evaluation, impact/outcomes evaluation) and real-world examples of different types of evaluation implemented in Social Work. Overall, the author provides examples that are easy for practitioners in Social Work to understand, which are also easily relatable for students in similar disciplines such as criminal justice. The book provides a glossary of key terms. There is no index; however, users can search for terms using the find (Ctrl-F) function in the PDF version of the book.

Overall, the content inside this book is accurate, error-free, and unbiased. However, the content is limited to the Social Work perspective, which may be considered somewhat biased or inaccurate from the perspective of others in different disciplines.

The book describes classic examples used in most texts on social science research methods. It also includes contemporary and relevant examples. Some of the content (such as web addresses and contemporary news pieces) will need to be updated every few years. The text is written and arranged in such a way that any necessary updates should be relatively easy and straightforward to implement.

The book is written in clear and accessible prose. The book provides appropriate context for any jargon/technical terminology used. Readers from any social science discipline should be able to understand the content and context of the material presented in the book.

The framework and use of terminology in the book are consistent.

This book is highly modular. The author has even improved upon the modularity of the book from Blackstone’s open text (which serves as the basis of the present text). Each chapter is divided into short, related subsections. The design of the chapters and their subsections make it easy to divide the material into units of study across a semester or quarter of instruction.

Generally, the book is organized in a similar manner as other texts on social science research methods. However, the organization could be improved slightly. Chapters 2 through 4 describe the process of beginning a research project and conducting a literature review. Chapter 8 describes refining a research question. This chapter could be moved to follow the Chapter 4. Chapter 12 describes experimental design, while Chapter 15 provides a description and examples of evaluation research. Since evaluation research tends to rely on experimental and quasi-experimental design, this chapter should follow the experimental design chapter.

For the online version of the book, there were no interface issues. The images and charts were clear and readable. The hyperlinks to sources mentioned in the text worked. The Contents menu allowed for easy and quick access to any section of the book. For the PDF version of the book, there were interface issues. The images and charts were clear and readable. However, the URLs and hyperlinks were not active in the PDF version. Furthermore, the PDF version was not bookmarked, which made it more difficult to access specific sections of the book.

I did not find grammatical errors in the book.

Overall, the cultural relevance and sensitivity were consistent with other social science research methods texts. The author does a good job of using both female and male pronouns in the prose. While there are pictures of people of color, there could be more. Most of the pictures are of white people. Also, the context is generally U.S.-centric.

Chapter 1: Introduction to research
Chapter 2: Beginning a research project
Chapter 3: Reading and evaluating literature
Chapter 4: Conducting a literature review
Chapter 5: Ethics in social work research
Chapter 6: Linking methods with theory
Chapter 7: Design and causality
Chapter 8: Creating and refining a research question
Chapter 9: Defining and measuring concepts
Chapter 10: Sampling
Chapter 11: Survey research
Chapter 12: Experimental design
Chapter 13: Interviews and focus groups
Chapter 14: Unobtrusive research: Qualitative and quantitative approaches
Chapter 15: Real-world research: Evaluation, single-subjects, and action research
Chapter 16: Reporting and reading research

Ancillary Material

Open Social Work Education

About the Book

As an introductory textbook for social work students studying research methods, this book guides students through the process of creating a research project. Students will learn how to discover a researchable topic that is interesting to them, examine scholarly literature, formulate a proper research question, design a quantitative or qualitative study to answer their question, carry out the design, interpret quantitative or qualitative results, and disseminate their findings to a variety of audiences. Examples are drawn from the author's practice and research experience, as well as topical articles from the literature.

There are ancillary materials available for this book.

About the Contributors

Matt DeCarlo earned his PhD in social work at Virginia Commonwealth University and is an Assistant Professor of Social Work at Radford University. He earned an MSW from George Mason University in 2010 and a BA in Psychology from the College of William and Mary in 2007. His research interests include open educational resources, self-directed Medicaid supports, and basic income. Matt is an Open Textbook Network Campus Leader for Radford University. He is the founder of Open Social Work Education, a non-profit collaborative advancing OER in social work education.

Contribute to this Page

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

17 5.2 Conceptualization

Learning objectives.

Define concept
Identify why defining our concepts is important
Describe how conceptualization works in quantitative and qualitative research
Define dimensions in terms of social scientific measurement
Apply reification to conceptualization

In this section, we’ll take a look at one of the first steps in the measurement process, which is conceptualization. This has to do with defining our terms as clearly as possible and also not taking ourselves too seriously in the process. Our definitions mean only what we say they mean—nothing more and nothing less. Let’s talk first about how to define our terms, and then we’ll examine not taking ourselves (or our terms, rather) too seriously.

Concepts and conceptualization

So far, the word concept has come up quite a bit, and it would behoove us to make sure we have a shared understanding of that term. A concept is the notion or image that we conjure up when we think of some cluster of related observations or ideas. For example, masculinity is a concept. What do you think of when you hear that word? Presumably, you imagine some set of behaviors and perhaps even a particular style of self-presentation. Of course, we can’t necessarily assume that everyone conjures up the same set of ideas or images when they hear the word masculinity . In fact, there are many possible ways to define the term. And while some definitions may be more common or have more support than others, there isn’t one true, always-correct-in-all-settings definition. What counts as masculine may shift over time, from culture to culture, and even from individual to individual (Kimmel, 2008). This is why defining our concepts is so important.

You might be asking yourself why you should bother defining a term for which there is no single, correct definition. Believe it or not, this is true for any concept you might measure in a research study—there is never a single, always-correct definition. When we conduct empirical research, our terms mean only what we say they mean. There’s a New Yorker cartoon that aptly represents this idea. It depicts a young George Washington holding an axe and standing near a freshly chopped cherry tree. Young George is looking up at a frowning adult who is standing over him, arms crossed. The caption depicts George explaining, “It all depends on how you define ‘chop.’” Young George Washington gets the idea—whether he actually chopped down the cherry tree depends on whether we have a shared understanding of the term chop .

Without a shared understanding of this term, our understandings of what George has just done may differ. Likewise, without understanding how a researcher has defined her key concepts, it would be nearly impossible to understand the meaning of that researcher’s findings and conclusions. Thus, any decision we make based on findings from empirical research should be made based on full knowledge not only of how the research was designed, but also of how its concepts were defined and measured.

So, how do we define our concepts? This is part of the process of measurement, and this portion of the process is called conceptualization. The answer depends on how we plan to approach our research. We will begin with quantitative conceptualization and then discuss qualitative conceptualization.

In quantitative research, conceptualization involves writing out clear, concise definitions for our key concepts. Sticking with the previously mentioned example of masculinity, think about what comes to mind when you read that term. How do you know masculinity when you see it? Does it have something to do with men? With social norms? If so, perhaps we could define masculinity as the social norms that men are expected to follow. That seems like a reasonable start, and at this early stage of conceptualization, brainstorming about the images conjured up by concepts and playing around with possible definitions is appropriate. However, this is just the first step.

It would make sense as well to consult other previous research and theory to understand if other scholars have already defined the concepts we’re interested in. This doesn’t necessarily mean we must use their definitions, but understanding how concepts have been defined in the past will give us an idea about how our conceptualizations compare with the predominant ones out there. Understanding prior definitions of our key concepts will also help us decide whether we plan to challenge those conceptualizations or rely on them for our own work. Finally, working on conceptualization is likely to help in the process of refining your research question to one that is specific and clear in what it asks.

If we turn to the literature on masculinity, we will surely come across work by Michael Kimmel, one of the preeminent masculinity scholars in the United States. After consulting Kimmel’s prior work (2000; 2008), we might tweak our initial definition of masculinity just a bit. Rather than defining masculinity as “the social norms that men are expected to follow,” perhaps instead we’ll define it as “the social roles, behaviors, and meanings prescribed for men in any given society at any one time” (Kimmel & Aronson, 2004, p. 503). Our revised definition is both more precise and more complex. Rather than simply addressing one aspect of men’s lives (norms), our new definition addresses three aspects: roles, behaviors, and meanings. It also implies that roles, behaviors, and meanings may vary across societies and over time. To be clear, we’ll also have to specify the particular society and time period we’re investigating as we conceptualize masculinity.

As you can see, conceptualization isn’t quite as simple as merely applying any random definition that we come up with to a term. Sure, it may involve some initial brainstorming, but conceptualization goes beyond that. Once we’ve brainstormed a bit about the images a particular word conjures up for us, we should also consult prior work to understand how others define the term in question. And after we’ve identified a clear definition that we’re happy with, we should make sure that every term used in our definition will make sense to others. Are there terms used within our definition that also need to be defined? If so, our conceptualization is not yet complete. And there is yet another aspect of conceptualization to consider—concept dimensions. We’ll consider that aspect along with an additional word of caution about conceptualization in the next subsection.

Conceptualization in qualitative research

Conceptualization in qualitative research proceeds a bit differently than in quantitative research. Because qualitative researchers are interested in the understandings and experiences of their participants, it is less important for the researcher to find one fixed definition for a concept before starting to interview or interact with participants. The researcher’s job is to accurately and completely represent how their participants understand a concept, not to test their own definition of that concept.

If you were conducting qualitative research on masculinity, you would likely consult previous literature like Kimmel’s work mentioned above. From your literature review, you may come up with a working definition for the terms you plan to use in your study, which can change over the course of the investigation. However, the definition that matters is the definition that your participants share during data collection. A working definition is merely a place to start, and researchers should take care not to think it is the only or best definition out there.

In qualitative inquiry, your participants are the experts (sound familiar, social workers?) on the concepts that arise during the research study. Your job as the researcher is to accurately and reliably collect and interpret their understanding of the concepts they describe while answering your questions. Conceptualization of qualitative concepts is likely to change over the course of qualitative inquiry, as you learn more information from your participants. Indeed, getting participants to comment on, extend, or challenge the definitions and understandings of other participants is a hallmark of qualitative research. This is the opposite of quantitative research, in which definitions must be completely set in stone before the inquiry can begin.

A word of caution about conceptualization

Whether you have chosen qualitative or quantitative methods, you should have a clear definition for the term masculinity and make sure that the terms we use in our definition are equally clear—and then we’re done, right? Not so fast. If you’ve ever met more than one man in your life, you’ve probably noticed that they are not all exactly the same, even if they live in the same society and at the same historical time period. This could mean there are dimensions of masculinity. In terms of social scientific measurement, concepts can be said to have multiple dimensions when there are multiple elements that make up a single concept. With respect to the term masculinity , dimensions could be regional (is masculinity defined differently in different regions of the same country?), age-based (is masculinity defined differently for men of different ages?), or perhaps power-based (does masculinity differ based on membership to privileged groups?). In any of these cases, the concept of masculinity would be considered to have multiple dimensions. While it isn’t necessarily required to spell out every possible dimension of the concepts you wish to measure, it may be important to do so depending on the goals of your research. The point here is to be aware that some concepts have dimensions and to think about whether and when dimensions may be relevant to the concepts you intend to investigate.

Before we move on to the additional steps involved in the measurement process, it would be wise to remind ourselves not to take our definitions too seriously. Conceptualization must be open to revisions, even radical revisions, as scientific knowledge progresses. Although that we should consult prior scholarly definitions of our concepts, it would be wrong to assume that just because prior definitions exist that they are more real than the definitions we create (or, likewise, that our own made-up definitions are any more real than any other definition). It would also be wrong to assume that just because definitions exist for some concept that the concept itself exists beyond some abstract idea in our heads. This idea, assuming that our abstract concepts exist in some concrete, tangible way, is known as reification .

To better understand reification, take a moment to think about the concept of social structure. This concept is central to critical thinking. When social scientists talk about social structure, they are talking about an abstract concept. Social structures shape our ways of being in the world and of interacting with one another, but they do not exist in any concrete or tangible way. A social structure isn’t the same thing as other sorts of structures, such as buildings or bridges. Sure, both types of structures are important to how we live our everyday lives, but one we can touch, and the other is just an idea that shapes our way of living.

Here’s another way of thinking about reification: Think about the term family . If you were interested in studying this concept, we’ve learned that it would be good to consult prior theory and research to understand how the term has been conceptualized by others. But we should also question past conceptualizations. Think, for example, about how different the definition of family was 50 years ago. Because researchers from that time period conceptualized family using now outdated social norms, social scientists from 50 years ago created research projects based on what we consider now to be a very limited and problematic notion of what family means. Their definitions of family were as real to them as our definitions are to us today. If researchers never challenged the definitions of terms like family, our scientific knowledge would be filled with the prejudices and blind spots from years ago. It makes sense to come to some social agreement about what various concepts mean. Without that agreement, it would be difficult to navigate through everyday living. But at the same time, we should not forget that we have assigned those definitions, they are imperfect and subject to change as a result of critical inquiry.

Key Takeaways

Conceptualization is a process that involves coming up with clear, concise definitions.
Conceptualization in quantitative research comes from the researcher’s ideas or the literature.
Qualitative researchers conceptualize by creating working definitions which will be revised based on what participants say.
Some concepts have multiple elements or dimensions.
Researchers should acknowledge the limitations of their definitions for concepts.
Concept- notion or image that we conjure up when we think of some cluster of related observations or ideas
Conceptualization- writing out clear, concise definitions for our key concepts, particularly in quantitative research
Multi-dimensional concepts- concepts that are comprised of multiple elements
Reification- assuming that abstract concepts exist in some concrete, tangible way

Image attributions

thought by TeroVesalainen CC-0

mindmap by TeroVesalainen CC-0

Share This Book

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Module 2 Chapter 1: The Nature of Social Work Research Questions

The search for empirical evidence typically begins with a question or hypothesis. The nature of the questions asked determine many features of the studies that lead to answers: the study approach, design, measurement, participant selection, data collection, data analysis, and reporting of results. Not just any type of question will do, however:

“When the question is poorly formulated, the design, analysis, sample size calculations, and presentation of results may not be optimal. The gap between research and clinical practice could be bridged by a clear, complete, and informative research question” (Mayo, Asano, & Barbic, 2013, 513).

The topic concerning the nature of social work research questions has two parts: what constitutes a research question, and what makes it a social work question. We begin this chapter by examining a general model for understanding where different types of questions fit into the larger picture of knowledge building explored in Module 1. We then look at research questions and social work questions separately. Finally, we reassemble them to identify strong social work research questions.

In this chapter, you will learn:

4 types of social work research for knowledge building,
characteristics of research questions,
characteristics of social work research questions.

Translational Science

The concept of translational science addresses the application of basic science discoveries and knowledge to routine professional practice. In medicine, the concept is sometimes described as “bench to trench,” meaning that it takes what is learned at the laboratory “bench” to practitioners’ work in the real-world, or “in the trenches.” This way of thinking is about applied science—research aimed at eventual applications to create or support change. Figure 1-1 assembles the various pieces of the translational science knowledge building enterprise:

Figure 1-1. Overview of translational science elements

Basic Research . Federal policy defines basic research as systematic study that is directed toward understanding the fundamental aspects of phenomena without specific applications in mind (adapted from 32 CFR 272.3). Basic research efforts are those designed to describe something or answer questions about its nature. Basic research in social and behavioral science addresses questions of at least two major types: epidemiology and etiology questions.

Epidemiology questions. Questions about the nature of a population, problem, or social phenomenon are often answered through epidemiological methods. Epidemiology is the branch of science (common in public health) for understanding how a problem or phenomenon is distributed in a population. Epidemiologists also ask and address questions related to the nature of relationships between problems or phenomena—such as the relationship between opioid misuse and infectious disease epidemics (NAS, 2018). One feature offered by epidemiological research is a picture of trends over time. Consider, for example, epidemiology data from the Centers for Disease Control and Prevention (the CDC) regarding trends in suicide rates in the state of Ohio over a four-year period (see Figure 1-2, created from data presented by CDC WONDER database).

Figure 1-2. Graph reflecting Ohio trend in suicide rate, 2012-2016

Since the upward trend is of concern, social workers might pursue additional questions to examine possible causes of the observed increases, as well as what the increase might mean to the expanded need for supportive services to families and friends of these individuals. The epidemiological data can help tease out some of these more nuanced answers. For example, epidemiology also tells us that firearms were the recorded cause in 46.9% of known suicide deaths among individuals aged 15-24 years across the nation during 2016 (CDC, WONDER database). Not only do we now know the numbers of suicide deaths in this age group, we know something about a relevant factor that might be addressed through preventive intervention and policy responses.

Epidemiology also addresses questions about the size and characteristics of a population being impacted by a problem or the scope of a problem. For example, a social worker might have a question about the “shape” of a problem defined as sexual violence victimization. Data from the United States’ 2010-2012 National Intimate Partner and Sexual Violence Survey (NISVS) indicated that over 36% of woman (1 in 3) and 17% of men (1 in 6) have experienced sexual violence involving physical contact at some point in their lives; the numbers vary by state, from 29.5% to 47.5% for women and 10.4% to 29.3% for men (Smith et al., 2017).

In developing informed responses to a problem, it helps to know for whom it is a problem. Practitioners, program administrators, and policy decision makers may not be aware that the problem of sexual violence is so prevalent, or that men are victimized at worrisome rates, as well as women. It is also helpful to know how the problem of interest might interface with other problems. For example, the interface between perpetrating sexual assault and alcohol use was examined in a study of college men (Testa & Cleveland, 2017). The study investigators determined that frequently attending parties and bars was associated with a greater probability of perpetrating sexual assault. Thus, epidemiological research helps answer questions about the scope and magnitude of a problem, as well as how it relates to other issues or factors, which can then inform next steps in research to address the problem.

Etiology questions. Etiology research tests theories and hypotheses about the origins and natural course of a problem or phenomenon. This includes answering questions about factors that influence the appearance or course of a problem—these may be factors that mediate or moderate the phenomenon’s development or progression (e.g., demographic characteristics, co-occurring problems, or other environmental processes). To continue with our intimate partner violence example, multiple theories are presented in the literature concerning the etiology of intimate partner violence perpetration—theories also exist concerning the etiology of being the target of intimate partner violence (Begun, 2003). Perpetration theories include:

personality/character traits
biological/hereditary/genetic predisposition
social learning/behavior modeling
social skills
self-esteem
cultural norms (Begun, 2003, p. 642).

Evidence supporting each of these theories exists, to some degree; each theory leads to the development of a different type of prevention or intervention response. The “best” interventions will be informed by theories with the strongest evidence or will integrate elements from multiple evidence-supported theories.

Etiology research is often about understanding the mechanisms underlying the phenomena of interest. The questions are “how” questions—how does this happen (or not)? For example, scientists asked the question: how do opioid medications (used to manage pain) act on neurons compared to opioids that naturally occur in the brain (Stoeber et al., 2018)? They discovered that opioid medications used to treat pain bind to receptors inside n erve cells, which is a quite different mechanism than the conventional wisdom that they behave the same way that naturally occurring (endogenous) opioids do—binding only on the surface of nerve cells. Understanding this mechanism opens new options for developing pain relievers that are less- or non-addicting than current opioid medicines like morphine and oxycodone. Once these mechanisms of change are understood, interventions can be developed, then tested through intervention research approaches.

Intervention Research. Interventions are designed around identified needs: epidemiology research helps to support intervention design by identify the needs. Epidemiology research also helps identify theories concerning the causes and factors affecting social work problems. Intervention development is further supported by later theory-testing and etiology research. However, developing an intervention is not sufficient: interventions need to be tested and evaluated to ensure that they are (1) safe, (2) effective, and (3) cost-efficient to deliver. This is where intervention research comes into play. Consider the example of Motivational Interviewing (MI) approaches to addressing client ambivalence about engaging in a behavior change effort. Early research concerning MI addressed questions about its effectiveness. For example, a meta-analytic review reported that “MI should be considered as a treatment for adolescent substance abuse” because the evidence demonstrated small, but significant effect sizes, and that the treatment gains were retained over time (Jensen et al., 2011). Subsequently, when its safety and effectiveness were consistently demonstrated through this kind of evidence, investigators assessed MI as cost-efficient or cost-effective. For example, MI combined with providing feedback was demonstrated to be cost-effective in reducing drinking among college students who engaged in heavy drinking behavior (Cowell et al., 2012).

Intervention research not only is concerned with the outcomes of delivering an intervention, but may also address the mechanisms of change through which an intervention has its effects—not only what changes happen, but how they happen. For example, investigators are exploring how psychotherapy works, moving beyond demonstrating that it works (Ardito & Rabellino, 2011; Kazdin, 2007; Wampold, 2015). One mechanism that has garnered attention is the role of therapeutic alliance—the relationships, bonds, and interactions that occur in the context of treatment—on treatment outcomes.

Therapeutic alliance is one common factor identified across numerous types of effective psychotherapeutic approaches (Wampold, 2015). Authors summarizing a number of studies about therapeutic alliance and its positive relationship to treatment outcomes concluded that the quality of therapeutic alliance may be a more powerful predictor of positive outcome than is the nature or type of intervention delivered (Ardito & Rabellino, 2011). However, it is important to determine the extent to which (a) therapeutic alliance enhances clients’ symptom improvement, (b) gradual improvements in symptoms lead to enhanced therapeutic alliance, or (c) the relationship between therapeutic alliance and symptom improvement are iterative—they go back and forth, influencing each other over time (Kazdin, 2007).

Implementation Science . Social work and other disciplines have produced a great deal of evidence about “what works” for intervening around a great number of social work problems. Unfortunately, many best practices with this kind of evidence support are slow to become common practices. Implementation science is about understanding facilitators and barriers to these evidence-supported interventions becoming adopted into routine practice: characteristics of the interventions themselves, conditions and processes operating in the organizations where interventions are implemented, and factors external to these organizations all influence practitioners’ adoption of evidence supported interventions.

Even under optimal internal organizational conditions, implementation can be undermined by changes in organizations’ external environments, such as fluctuations in funding, adjustments in contracting practices, new technology, new legislation, changes in clinical practice guidelines and recommendations, or other environmental shifts” (Birken, et al, 2017).

Research for/about Research . In addition, social work investigators engage in research that is specifically about scientific methodology. This is where advances in measurement, participant recruitment and retention, and data analysis emerge. The results of these kinds of research studies are used to improve the research in basic, intervention, and implementation research. Later in the course you will see some of these products in action as we learn about best practices in research and evaluation methodology. Here are a few examples related to measurement methods:

Concept mapping to assess community needs of sexual minority youth (Davis, Saltzburg, & Locke, 2010)
Field methodologies for measuring college student drinking in natural environments (Clapp et al., 2007)
Intergenerational contact measurement (Jarrott, Weaver, Bowen, & Wang, 2018)
Perceived Social Competence Scale-II (Anderson-Butcher et al., 2016)
Safe-At-Home Instrument to measure readiness to change intimate partner violence behavior (Begun et al., 2003; 2008; Sielski, Begun, & Hamel, 2015)
Teamwork Scale for Youth (Lower, Newman, & Anderson-Butcher, 2016)

And, here are a few examples related to involving participants in research studies:

Conducting safe research with at risk populations (Kyriakakis, Waller, Kagotho, & Edmond, 2015)
Recruitment strategies for non-treatment samples in addiction studies (Subbaraman et al., 2015)
Variations in recruitment results across Internet platforms (Shao et al., 2015)

Take a moment to complete the following activity.

Research Questions

In this section, we take a closer look at research questions and their relationship to the types of research conducted by investigators. It may be easier to understand research questions by first ruling out what are not research questions. In that spirit, let’s begin with examples of questions where applying research methods will not help to find answers:

Trauma informed education. The first issue with this example is obvious: it is not worded as a question. The second is critically important: this is a general topic, it is not a research question. This topic is too vague and broad making it impossible to determine what answers would look like or how to approach finding answers.
How is my client feeling about what just happened? This type of question about an individual is best answered by asking clinical questions of that individual, within the context of the therapeutic relationship, not by consulting research literature or conducting a systematic research study.
Will my community come together in protest of a police-involved shooting incident? This type of question may best be answered by waiting to see what the future brings. Research might offer a guess based on data from how other communities behaved in the past but cannot predict how groups in individual situations will behave. A better research question might be: What factors predict community protest in response to police-involved shooting incidents?
Should I order salad or soup to go with my sandwich? This type of question is not of general interest, making it a poor choice as a research question. The question might be reframed as a general interest question: Is it healthier to provide salad or soup along with a sandwich? The answer to that researchable question might inform a personal decision.
Why divorce is bad for children. There are two problems with this example. First, it is a statement, not a question, despite starting with the word “why.” Second, this question starts out with a biased assumption—that divorce is bad for children. Research questions should support unbiased investigation, leading to evidence and answers representative of what exists rather than what someone sets out wanting to prove is the case. A better research question might be: How does divorce affect children?

Tuning back to our first example of what is not a research question, consider several possible school social work research questions related to that general topic:

To what extent do elementary school personnel feel prepared to engage in trauma informed education with their students?
What are the barriers and facilitators of integrating trauma informed education in middle school?
Does integrating trauma informed education result in lower rates of suicidal ideation among high school students?

Is there a relationship between parent satisfaction and the implementation of trauma informed education in their children’s schools?

Does implementing trauma informed education in middle schools affect the rate of student discipline referrals?

What is the difference between these research questions and the earlier “not research” questions? First, research questions are specific. This is an important distinction between identifying a topic of interest (e.g., trauma informed education) and asking a researchable question. For example, the question “How does divorce affect children?” is not a good research question because it remains too broad. Instead, investigators might focus their research questions on one or two specific effects of interest, such as emotional or mental health, academic performance, sibling relationships, aggression, gender role, or dating relationship outcomes.

Image of a family with a tear seperating a father from a mother with children

Related to a question being “researchable” is its feasibility for study. Being able to research a question requires that appropriate data can be collected with integrity. For example, it may not be feasible to study what would happen if every child was raised by two parents, because (a) it is impossible to study every child and (2) this reality cannot ethically be manipulated to systematically explore it. No one can ethically conduct a study whereby children are randomly assigned by study investigators to the compared conditions of being raised by two parents versus being raised by one or no parents. Instead, we settle for observing what has occurred naturally in different families.

Second, “good” research questions are relevant to knowledge building. For this reason, the question about what to eat was not a good research question—it is not relevant to others’ knowledge development. Relevance is in the “eye of the beholder,” however. A social work researcher may not see the relevance of using a 4-item stimulus array versus a 6-item stimulus array in testing children’s memory, but this may be an important research question for a cognitive psychology researcher. It may, eventually, have implications for assessment measures used in social work practice.

Third, is the issue of bias built into research questions. Remembering that investigators are a product of their own developmental and social contexts, what they choose to study and how they choose to study it are socially constructed. An important aspect at the heart of social work research relates to a question’s cultural appropriateness and acceptability. To demonstrate this point, consider an era (during the 1950s to early 1970s) when research questions were asked about the negative effects on child development of single-parent, black family households compared to two-parent, white family households in America. This “majority comparison” frame of reference is not culturally appropriate or culturally competent. Today, in social work, we adopt a strengths perspective, and avoid making comparisons of groups against a majority model. For example, we might ask questions like: What are the facilitators and barriers of children’s positive development as identified by single parents of diverse racial/ethnic backgrounds? What strengths do African American parents bring to the experience of single-parenting and how does it shape their children’s development? What are the similar and different experiences of single-parenting experienced by families of different racial/ethnic composition?

Research Questions versus Research Hypotheses . You have now seen examples of “good” research questions. Take, for example, the last one we listed about trauma informed education:

Based on a review of literature, practice experience, previous research efforts, and the school’s interests, an investigator may be prepared to be even more specific about the research question (see Figure 1-3). Assume that these sources led the investigator to believe that implementing the trauma informed education approach will have the effect of reducing the rate of disciplinary referrals. The investigator may then propose to test the following hypothesis:

Implementing trauma informed education in middle schools will result in a reduction in the number of student discipline referrals.

The research hypothesis is a clear statement that can be tested with quantitative data and will either be rejected or not, depending on the evidence. Research hypotheses are predictions about study results—what the investigator expects the results will show. The prediction, or hypothesis, is based on theory and/or other evidence. A study hypothesis is, by definition, quantifiable—the answer lies in numerical data, which is why we do not generally see hypotheses in qualitative, descriptive research reports.

Hypotheses are also specific to one question at a time. Thus, an investigator would need to state and test a second hypothesis to answer the question:

The stated hypothesis might be:

Parent satisfaction is higher in middle schools where trauma informed education is implemented.

Figure 1-3. Increasing specificity from research topic to question to hypothesis

Social Work Questions

It is difficult to find a simple way to characterize social work research. The National Institutes of Health (NIH) described social work research in the following way:

Historically, social work research has focused on studies of the individual, family, group, community, policy and/or organizational level, focusing across the lifespan on prevention, intervention, treatment, aftercare and rehabilitation of acute and chronic conditions, including the effects of policy on social work practice (OBSSR, 2003, p. 5) .

For all the breadth expressed in this statement, it reflects only how social work research relates to the health arena—it does not indicate many other domains and service delivery systems of social work influence:

physical, mental, and behavioral health
substance misuse/addiction and other addictive behaviors
income/poverty
criminal justice
child and family welfare
housing and food security/insecurity
environmental social work
intimate partner, family, and community violence
and others.

In addition to breadth of topic, social work research is characterized by its biopsychosocial nature. This means that social work researchers not only pursue questions relating to biological, psychological, and social context factors, but also questions relating to their intersections and interactions. Related to this observation is that social work not only addresses questions related to the multiple social system levels, social work also addresses the ways multiple levels intersect and interact (i.e., those levels represented in the NIH statement about individuals, families, groups, communities, organizations, and policy).

It is worth noting that research need not be conducted by social workers to be relevant to social work–many disciplines and professions contribute to the knowledge base which informs social work practice (medicine, nursing, education, occupational therapy, psychology, sociology, criminal justice, political science, economics, and more). Authors of one social work research textbook summarize the relevance issue in the following statement:

“To social workers, a relevant research question is one whose answers will have an impact on policies, theories, or practices related to the social work profession” (Grinnell & Unrau, 2014, p. 46).

Social Work Research Questions and Specific Aims

The kinds of questions that help inform social work practice and policy are relevant to understanding social work problems, diverse populations, social phenomena, or interventions. Most social work research questions can be divided into two general categories: background questions and foreground questions . The major distinction between these two categories relates to the specific aims that emerge in relation to the research questions.

Background Questions. This type of question is answerable with a fact or set of facts. Background questions are generally simple in structure, and they direct a straightforward search for evidence. This type of question can usually be formulated using the classic 5 question words: who, what, when, where, or why. Here are a few examples of social work background questions related to the topic of fetal alcohol exposure:

Who is at greatest risk of fetal alcohol exposure?
What are the developmental consequences of fetal alcohol exposure?
When in gestation is the risk of fetal alcohol exposure greatest?
Where do women get information about the hazards of drinking during pregnancy?
Why is fetal alcohol exposure (FAE) presented as a spectrum disorder, different from fetal alcohol syndrome (FAS)?

These kinds of questions direct a social worker to review literature about human development, human behavior, the distribution of the problem across populations, and factors that determine the nature of a specific social work problem like fetal exposure to alcohol. Where the necessary knowledge is lacking, investigators aim to explore or describe the phenomenon of interest. Many background questions can be answered by epidemiology or etiology evidence.

Image of glasses of wine on the left and an outline of a woman with a baby inside of her on the right

Foreground Questions. This type of question is more complex than the typical background question. Foreground questions typically are concerned with making specific choices by comparing or evaluating options. These types of questions required more specialized evidence and may lead to searching different types of resources than would be helpful for answering background questions. Foreground questions are dealt with in greater detail in our second course, SWK 3402 which is about understanding social work interventions. A quick foreground question example related to the fetal exposure to alcohol topic might be:

Which is the best tool for screening pregnant women for alcohol use with the aim of reducing fetal exposure, the T-ACE, TWEAK, or AUDIT?

This type of question leads the social worker to search for evidence that compares different approaches. These kinds of evidence are usually found in comparative reviews, or require the practitioner to conduct a review of literature, locating individual efficacy and effectiveness studies. Where knowledge is found to be lacking, investigators aim to experiment with different approaches or interventions.

Three Question Types and Their Associated Research Aims

Important distinctions exist related to different types of background questions. Consider three general categories of questions that social workers might ask about populations, problems, and social phenomena: exploratory, descriptive, and explanatory. The different types of questions matter because the nature of the research questions determines the specific aims and most appropriate research approaches investigators apply in answering them.

Exploratory Research Questions. Social workers may find themselves facing a new, emerging problem where there is little previously developed knowledge available—so little, in fact, that it is premature to begin asking any more complex questions about causes or developing testable theories. Exploratory research questions open the door to beginning understanding and are basic; answers would help build the foundation of knowledge for asking more complex descriptive and explanatory questions. For example, in the early days of recognition that HIV/AIDS was emerging as a significant public health problem, it was premature to jump to questions about how to treat or prevent the problem. Not enough was known about the nature and scope of the problem, for whom it was a problem, how the problem was transmitted, factors associated with risk for exposure, what factors influenced the transition from HIV exposure to AIDS as a disease state, and what issues or problems might co-occur along with either HIV exposure or AIDS. In terms of a knowledge evolution process, a certain degree of exploration had to occur before intervention strategies for prevention and treatment could be developed, tested, and implemented.

In 1981, medical providers, public health officials, and the Centers for Disease Control and Prevention (CDC) began to circulate and publish observations about a disproportionate, unexpectedly high incidence rate of an unusual pneumonia and Kaposi’s sarcoma appearing in New York City and San Francisco/California among homosexual men (Curran, & Jaffe, 2011). As a result, a task force was formed and charged with conducting an epidemiologic investigation of this outbreak; “Within 6 months, it was clear that a new, highly concentrated epidemic of life threatening illness was occurring in the United States” (Curran & Jaffe, 2011, p. 65). The newly recognized disease was named for its symptoms: acquired immune deficiency syndrome, or AIDS. Exploratory research into the social networks of 90 living patients in 10 different cities indicated that 40 had a sexual contact link with another member of the 90-patient group (Auerbach, Darrow, Jaffe, & Curran, 1984). Additionally, cases were identified among persons who had received blood products related to their having hemophilia, persons engaged in needle sharing during substance use, women who had sexual contact with a patient, and infants born to exposed women. Combined, these pieces of information led to an understanding that the causal infectious factor (eventually named the human immunodeficiency virus, HIV) was transmitted by sexual contact, blood, and placental connection. This, in turn, led to knowledge building activities to develop both preventive and treatment strategies which could be implemented and studied. Social justice concerns relate to the slow rate at which sufficient resources were committed for evolving to the point of effective solutions for saving lives among those at risk or already affected by a heavily stigmatized problem.

The exploratory research approaches utilized in the early HIV/AIDS studies were both qualitative and quantitative in nature. Qualitative studies included in-depth interviews with identified patients—anthropological and public health interviews about many aspects of their living, work, and recreational environments, as well as many types of behavior. Quantitative studies included comparisons between homosexually active men with and without the diseases of concern. In addition, social network study methods combined qualitative and quantitative approaches. These examples of early exploratory research supported next steps in knowledge building to get us to where we are today. “Today, someone diagnosed with HIV and treated before the disease is far advanced can live nearly as long as someone who does not have HIV” (hiv.gov). While HIV infection cannot (yet) be “cured,” it can be controlled and managed as a chronic condition.

Descriptive Research Questions. Social workers often ask for descriptions about specific populations, problems, processes, or phenomena. Descriptive research questions might be expressed in terms of searching to create a profile of a group or population, create categories or types (typology) to describe elements of a population, document facts that confirm or contradict existing beliefs about a topic or issue, describe a process, or identify steps/stages in a sequential process (Grinnell & Unrau, 2014). Investigators may elect to approach the descriptive question using qualitative methods that result in a rich, deep description of certain individuals’ experiences or perceptions (Yegidis, Weinbach, & Meyers, 2018). Or, the descriptive question might lead investigators to apply quantitative methods, assigning numeric values, measuring variables that describe a population, process, or situation of interest. In descriptive research, investigators do not manipulate or experiment with the variables; investigators seek to describe what naturally occurs (Yegidis, Weinbach, & Meyers, 2018). As a result of studies answering descriptive questions, tentative theories and hypotheses may be generated.

Here are several examples of descriptive questions.

How do incarcerated women feel about the option of medication-assisted treatment for substance use disorders?
What barriers to engaging in substance misuse treatment do previously incarcerated persons experience during community reentry?
How often do emerging adults engage in binge drinking in different drinking contexts (e.g., bars, parties, sporting events, at home)?
What percent of incarcerated adults experience a substance use disorder?
What is the magnitude of racial/ethnic disparities in access to treatment for substance use disorders?
Who provides supervision or coordination of services for aging adults with intellectual or other developmental disabilities?
What is the nature of the debt load among students in doctoral social work programs?

Image of a prison cell from outside of the bars

An example of descriptive research, derived from a descriptive question, is represented in an article where investigators addressed the question: How is the topic of media violence and aggression reported in print media (Martins et al., 2013)? This question led the investigators to conduct a qualitative content analysis, resulting in a description showing a shift in tone where earlier articles (prior to 2000) emphasized the link as a point of concern and later articles (since 2000) assumed a more neutral stance.

Correlational Research Questions. One important type of descriptive question asks about relationships that might exist between variables—looking to see if variable x and variable y are associated or correlated with each other. This is an example of a correlational research question; it does not indicate whether “x” causes “y” or “y” causes “x”, only whether these two are related. Consider again the topic of exposure to violence in the media and its relationship to aggression. A descriptive question asked about the existence of a relationship between exposure to media violence ( variable x ) and children’s expression of aggression ( variable y ). Investigators reported one study of school-aged children, examining the relationship between exposure to three types of media violence (television, video games, and movies/videos) and three types of aggression (verbal, relational, and physical; Gentile, Coyne, & Walsh, 2011). The study investigators reported that media violence exposure was, indeed, correlated with all three types of aggressive behavior (and less prosocial behavior, too).

For a positive correlation (the blue line), as the value of the “x” variable increases, so does the value of the “y” variable (see Figure 1-4 for a general demonstration). An example might be as age or grade in school increases (“x”), so does the number of preadolescent, adolescent, and emerging adults who have used alcohol (“y”). For a negative correlation (the orange line), as the value of the “x” variable increases, the value of the “y” variable decreases. An example might be as the number of weeks individuals are in treatment for depression symptoms (“x”), the reported depression symptoms decreases (“y”). The neutral of non-correlation line (grey) means that the two variables, “x” and “y” do not have an association with each other. For example, number of years of teachers’ education (“x”) might be unrelated to the number of students dropping out of high school (“y”).

Figure 1-4. Depicting positive, negative, and neutral correlation lines

Descriptive correlational studies are sometimes called comparison studies because the descriptive question is answered by comparing groups that differ on one of the variables (low versus high media violence exposure) to see how they might differ on the other variable (aggressive behavior).

Explanatory Research Questions. To inform the design of evidence-informed interventions, social workers need answers to questions about the nature of the relationships between potentially influential factors or variables. An explanatory research question might be mapped as: Does variable x cause, lead to or prevent changes in variable y (Grinnell & Unrau, 2014)? These types of questions often test theory related to etiology.

Comparative research might provide information about a relationship between variables. For example, the difference in outcomes between persons experiencing a substance use disorder and have been incarcerated compared to others with the same problem but have not been incarcerated may be related to their employability and ability to generate a living-wage income for themselves and their families. However, to develop evidence-informed interventions, social workers need to know that variables are not only related, but that one variable actually plays a causal role in relation to the other. Imagine, for example, that evidence demonstrated a significant relationship between adolescent self-esteem and school performance. Social workers might spend a great deal of effort developing interventions to boost self-esteem in hopes of having a positive impact on school performance. However, what if self-esteem comes from strong school performance? The self-esteem intervention efforts will not likely have the desired effect on school performance. Just because research demonstrates a significant relationship between two variables does not mean that the research has demonstrated a causal relationship between those variables. Investigators need to be cautious about the extent to which their study designs can support drawing conclusions about causality; anyone reviewing research reports also needs to be alert to where causal conclusions are properly and improperly drawn.

Person at desk with stack of books and papers

The questions that drive intervention and evaluation research studies are explanatory in nature: does the intervention ( x ) have a significant impact on outcomes of interest ( y )? Another type of explanatory question related to intervention research concerns the mechanisms of change. In other words, not only might social workers be interested to find out what outcomes or changes can be attributed to an intervention, they may also be interested to learn how the intervention causes those changes or outcomes.

$Cartoon of confusing math with man pointing at center that says "Then a Miracle Occurs" and caption below stating "I think you should be more explicit here in step two"$

Chapter Summary

In this chapter, you learned about different aspects of the knowledge building process and where different types of research questions might fit into the big picture. No single research study covers the entire spectrum; each study contributes a piece of the puzzle as a whole. Research questions come in many different forms and several different types. What is important to recall as we move through the remainder of the course is that the decisions investigators make about research approaches, designs, and procedures all start with the nature of the question being asked. And, the questions being asked are influenced by multiple factors, including what is previously known and remains unknown, the culture and context of the questioners, and what theories they have about what is to be studied. That leads us to the next chapter.

Share This Book

11 Important Social Work Theories and Methods

View all blog posts under News and Articles

Social work plays an important role in the mental and emotional health of individuals and communities. Social workers serve clients who are in crisis, who may have addiction disorders, who are in unhealthy relationships, or who are housing or food insecure. While social work is a thoroughly modern profession, its roots date to the 19th century, with the passage of laws aimed at improving the condition of poor and vulnerable communities.

Social work theories and methods are crucial to the modern practice of social work. Accredited Master of Social Work (MSW) programs teach these important theories to prepare students for a career as a licensed social worker.

The following 11 social work theories and methods are some of the most important principles in the field today:

1. Psychosocial Theory

Psychosocial theory, which Erik Erikson developed in the 1950s, is the main principle of social work. Also referred to as person-in-environment (PIE) theory, psychosocial theory posits that a person develops a personality in stages, based on environment and relationships with family and community.

As children, teens and adults, humans go through successive stages, acquiring autonomy, initiative, identity, creativity and a capacity for intimacy. However, at every stage, the chance exists that people will instead develop a capacity for mistrust, shame, guilt, isolation and despair. For example, in the identity vs. role confusion stage, teens undergo conflict as they work out their identity in relation to the expectations of their parents, teachers and community.

2. Attachment Theory

Attachment theory is one of the best-known theories that provide a framework for social workers to understand human behavior. Attachment theory holds that babies have innate behaviors whose purpose is to ensure that caregivers meet their needs. These behaviors include crying, making eye contact, clinging and smiling. The development of healthy attachments lets a child be secure enough to meet the world with confidence. However, when the attachment is inconsistent or broken, children develop maladaptive behaviors that impact development.

3. Systems Theory

Systems theory provides a framework for understanding why a person behaves in a certain way. Social workers can investigate all the factors that impact or have been impacted by a client, and by understanding all these systems, they can put together a picture of what drives a client’s behavior and choices.

For example, systems theory provides an understanding of adolescent risk-taking behavior. Researchers who studied Swiss teenagers reported in 2021 that teenage boys were more likely to engage in risky behavior because their drive for sensation seeking outpaced their drive for self-regulation.

4. Behavioral Theory

Behavioral theory, or behaviorism, holds that people learn behaviors through conditioning. A person performs an action that’s reinforced through a natural consequence or a negative consequence. Social workers often use behavioral therapy techniques to treat patients. For example, therapists may use conditioning techniques to help clients modify undesirable behaviors. Behavioral theory is often used in conjunction with cognitive components to form cognitive behavioral therapy treatments.

5. Cognitive Theory

Cognitive theory holds that emotional responses come from thought processes. Social workers can use cognitive theory to help patients identify the thoughts that trigger a certain behavior. They can help patients reframe these thought processes to overcome negative behaviors. Cognitive theory and the associated social cognitive theory can be used to help patients overcome phobias, such as social phobia.

6. Cognitive Behavioral Theory

Social workers use cognitive behavioral methods to help clients reframe limiting or negative behaviors. They guide individuals through steps to understand their behavior, including the thought processes leading up to it. Social workers may use exposure therapy, meditation, journaling or other tools to help clients overcome anxiety and fears. Clients with depression, obsessive-compulsive disorder (OCD) and post-traumatic stress disorder (PTSD) respond well to cognitive behavioral methods.

7. Motivational Theory

What pushes a person to act? Many types of motivational theories seek to answer that question. One of the most famous is Maslow’s hierarchy of needs: This theory states that only when the most pressing needs (food, shelter, safety) have been met can people seek higher goals (love, learning, art). One example of motivational theory in practice is motivational interviewing. In this technique, a social worker guides and empowers clients to manage change. The technique is collaborative and respectful and can be applied in a variety of settings.

8. Empowerment Theory

Empowerment theory is a central tenet of the National Association of Social Workers (NASW) Code of Ethics, as part of the profession’s commitment to social justice. Empowerment theory holds that social workers must support clients and their communities in building connections, fighting injustice and creating grassroots organizations. Empowerment theory, like conflict theory, aims to change society rather than provide a treatment model for individuals.

9. Task-Centered Model

The combination of social work theories and methods provides a powerful toolkit for social workers. Based on the theories presented here, social workers have numerous methods for working with patients.

For example, social workers may use a task-centered model to help their clients develop problem-solving skills. The goal of a task-centered practice is to help individuals achieve autonomy. The social worker guides clients through the five stages of problem-solving: (1) defining the problem, (2) brainstorming ideas and running through scenarios, (3) choosing a solution, (4) applying the solution, and (5) analyzing how well the solution worked.

The task-centered model can seem simplistic, but as social workers and their clients often discover, learning to be effective problem-solvers is harder than it looks.

10. Crisis Intervention

Social workers have been at the forefront of the COVID-19 pandemic, just as they are during other natural disasters. Social workers employ crisis intervention techniques and methodologies to treat and stabilize the mental and emotional health of people in crisis. They work in the community, in hospitals and in other healthcare facilities. Social workers treat clients suffering from illness and grief. They also mobilize community responses and help already marginalized individuals and communities receive resources and treatment.

11. Narrative Method

Narrative methods recognize that we all tell stories about ourselves and others. Social workers use narrative therapy to help clients define their stories and identities. For example, this narrative technique can help change an individual’s self-perception as a criminal to someone worthy of redemption. The narrative method centers clients as the experts in their own life and avoids blame. It focuses on helping clients change behaviors that’ve injured them in the past.

Make a Difference: Explore the MSW Program at the University of Nevada, Reno

Social work is an evidence-based profession with a long history of research and publication in human psychology. Social workers have a vast toolkit with which to treat individuals and help heal communities. If you’re drawn to a career that uses proven theories to help those in need, learn more about the online MSW program at the University of Nevada, Reno. With a curriculum grounded in these social work theories and methods, it offers graduates an excellent foundation for a future in social work.

ONLINE MASTER OF SOCIAL WORK

Northwest Commission on Colleges and Universities (NWCCU)
Council on Social Work Education (CSWE) Accreditation

The social work education programs provided by the University of Nevada, Reno School of Social Work are accredited at the baccalaureate and master’s levels by the Council on Social Work Education (CSWE). This indicates to the public and to potential employers that graduates meet the high professional standards established by CSWE in its Educational Policy and Accreditation Standards (EPAS). Please refer to www.cswe.org for a complete list of Educational Policy and Accreditation Standards. The university’s MSW program has been accredited by CSWE since 1991.

Recognized as a National Tier 1 University by U.S. News & World Report
Ranks among the top 150 national universities in research and development
R1 Carnegie Classification research institution

1.	Which program are you interested in applying to?

2.	Complete the form to download your brochure.

Skip to primary navigation
Skip to main content
Skip to primary sidebar

Best Social Work Degrees & Career Options

Social Work Research Method

Social Work research methods include surveys, ethnographic descriptions, studies, randomized trials, and needs tests. What makes one data point stronger than another? Ask any researcher, social work domain or otherwise. They’ll refer to the importance of avoiding the many forms of researcher bias, clearly expressing the problem, and using the properly structured methodology, among other essential steps.

Strategies for collecting reliable data may vary slightly depending on the nature of the study, but the underlying concept is this: nothing outside the natural environment can influence the data. If the researcher controls the data, then the correlation between the hypothesis and the results is weakened significantly.

So, how can researchers in the social work field strive for the highest standards of objectivity in their findings? The answer starts with the proper methodology. The structure of each study needs to conform to the environment in which the hypothesis was formed and not the other way around. For example, if a researcher wants to test how participants respond to different management styles in the workplace, they would do best not to remove the person from their work environment – even if the researcher simulated one of their own.

In some scenarios, observing participants in their environments like the above example is appropriate. Maybe a simple survey would indeed work when it comes to perceptions and behaviors that aren’t dependent on the environment. However, researchers in the social work field have to rely on several research methods to observe and collect the data as it exists in its natural domain. Let’s summarize some of the most common structures, starting with the already mentioned survey method. Research methods include, sage research methods, qualitative methods, methods map, research methods in social workers research project, social workers evidence based practice

Especially when researchers have access to large participant groups, surveys are a simple, affordable, and reliable method. The structure is simple: participants answer a series of questions designed to test the researcher’s hypothesis. If the researcher wants to evaluate the effects of digital media on certain perceptions, for example, they could ask participants to express their thoughts on popular events and people. Then, researchers send out the surveys, aggregate the results and form their conclusions based on the trends within the data.

As with many of the following research methods, it is not the implementation of the survey method itself that can be tricky but knowing when and how to use it properly. If the topic(s) being covered by the survey can’t be addressed with simple questions and answers, researchers need to opt for more open-ended data collection techniques. If respondents feel embarrassed or incriminated by answering truthfully, they may skew the results – observational methods would prevent this issue in many cases. Research methods include, quantitative and qualitative methods, sage research methods, methods map, evidence based practice in social work

Ethnographic Description

Like a probe sent deep into a planet’s surface to collect data unobtainable from the surface, ethnographic studies seek to immerse the researcher in another culture for more significant insights into any number of behaviors and beliefs. Contrary to surveys, ethnographic research methods are generally more time-consuming and costly. A researcher may travel across the world to live within a culture for weeks, months, or longer, adopting that culture’s practices to enrich their understanding. Then, they bring all of their data back home, where they use it to help other groups merge with members of the researched group.

Ethnographic research methods models can overlap with others. As mentioned, the researcher will generally travel to an area and immerse themselves in the culture. This can include:

Informant interviews in social work research
Surveys and census data in social work research
Observation in social work research
Participation in social work research

Case Studies

Popular in the business world, case studies are also well-suited to research methods and efforts in social work. Simply put, a case study is an example – a real-life scenario that provides a testing ground for a hypothesis. Researchers can examine data from an ethnographic study, for example, even if the survey had a completely different objective, research methods in qualitative research and evidence based practice social work to demonstrate certain behaviors’ social or individual impact (or lack thereof). Though everyday events can be justified as case studies, researchers are often hard-pressed to prove that no extraneous variables affect the data since real-life scenarios don’t occur in controlled environments.

Case studies are helpful in many scenarios, but they address a specific theory. Therefore, they can be used throughout the literature review and research methods phases to accomplish the following objectives:

Practically demonstrate a theory
Call for more research methods
Debunk a hypothesis
Test research methods and their findings in the real world
Uncover new social work research methods and variables affecting the hypothesis

Single-System Design

Experiments in the medical field especially tend to follow a model that compares the results (of a drug, treatment, etc.) across two groups: the control group, which doesn’t receive treatment, and social workers which does. In a single-system design, however, there is only one group. Often, this “group” is just one person. Moreover, the person or group is generally studied to assess their response to different variables over a long period.

With no control group, though, how do researchers gauge the effects of any particular variable? By manipulating the variable itself, not the audience. Single-system designs test the products of different independent variables. The experimenter applies the dependent variable and the result of these changes, and the theory being tested.

Let’s say that a researcher in the social work domain wants to determine the effect of digital media consumption on antisocial behaviors. Instead of setting up a control group (no digital media consumption) and an experimental group (two hours of digital media consumption per day per participant) to test their hypothesis, the researcher will change the nature of the digital media consumption for a single participant, recording the results of each change.

Program Evaluation

This particular vein of social work research methods is highly relevant to social workers, who often ally with programs of all kinds as a way of increasing access to vital resources for their clientele. The government or private investors may fund a program. Regardless, nobody supports a project unless they think it will be successful. Key concepts allow everyone to assess a program’s fitness across multiple dynamics.

These social work research methods requires a comprehensive look at recent findings to prove the effectiveness of a particular program. Even after a program has launched, key concepts can help to refine things for greater efficiency. The following list of questions will help to define the purpose and applications of a program evaluation:

Will this program work?
How much will the program cost per participant?
How can we expand the program?
Is there a better way to serve the program’s population?
Are there any disadvantages for program participants?

Needs Assessment

Needs assessments are also fundamental to the sociological perspective because they seek to identify deficiencies in specific populations. Of course, one does not define a population only by region, income level, or ethnicity. However, these three factors comprise the majority of cases.

These research methods are integral to social work at all levels. A social worker in the field, for example, can use needs assessments to identify opportunities for improvement with an individual client. Conversely, researchers, program planners, and executive-level social work professionals can apply needs assessments to entire communities to affect change on a larger scale. In either case, needs assessments are part of the planning process when conducting social work research methods, creating resources, or developing a care plan for one person.

Types of research methods

quantitative and qualitative methods
qualitative research
qualitative methods
sage research methods

Randomized Trials

Finally, the randomized trial is one of the purest and most broadly applied experimental models. Randomization, in this case, refers to how you select participants to be part of the control or experimental groups. Furthermore, you experiment with a formulaic, easily reproducible, and highly measurable fashion. First, the randomly assigned experimental group is subjected to the variable. It may be a treatment or a specific stimulus. Then, the randomly assigned group is not in social work. Next, the response of both groups are measured and compared, and when applicable, a new variable is tested in the same fashion.

These social work research methods model is most appropriate when responses are easily quantifiable in both social work and the medical field. Comparing subjective responses between two groups yields less actionable and prominent information than, for example, a measurable change in blood pressure.

To reiterate, no research model is objectively “better” than the other; each has its application. Properly selecting and applying a model (or a combination of models) requires researchers to comprehensively evaluate the subject’s environment, the nature of the data (subjective, objective, or both?), the hypothesis, and so forth. Nevertheless, the proper social work research methods can introduce precious findings that hold up against future inquiries when used correctly. Methods include sage research methods, quantitative methods, program evaluations in such research.

5 Important Crises Faced by Social Workers
What Are Different Types of Community Based Interventions?
What Do Social Workers Need To Know About Trauma?

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Part 2: Conceptualizing your research project

9. Writing your research question

Chapter outline.

Empirical vs. ethical questions (4 minute read)
Characteristics of a good research question (4 minute read)
Quantitative research questions (7 minute read)
Qualitative research questions (3 minute read)
Evaluating and updating your research questions (4 minute read)

Content warning: examples in this chapter include references to sexual violence, sexism, substance use disorders, homelessness, domestic violence, the child welfare system, cissexism and heterosexism, and truancy and school discipline.

9.1 Empirical vs. ethical questions

Learning objectives.

Learners will be able to…

Define empirical questions and provide an example
Define ethical questions and provide an example

Writing a good research question is an art and a science. It is a science because you have to make sure it is clear, concise, and well-developed. It is an art because often your language needs “wordsmithing” to perfect and clarify the meaning. This is an exciting part of the research process; however, it can also be one of the most stressful.

Creating a good research question begins by identifying a topic you are interested in studying. At this point, you already have a working question. You’ve been applying it to the exercises in each chapter, and after reading more about your topic in the scholarly literature, you’ve probably gone back and revised your working question a few times. We’re going to continue that process in more detail in this chapter. Keep in mind that writing research questions is an iterative process, with revisions happening week after week until you are ready to start your project.

Empirical vs. ethical questions

When it comes to research questions, social science is best equipped to answer empirical questions —those that can be answered by real experience in the real world—as opposed to ethical questions —questions about which people have moral opinions and that may not be answerable in reference to the real world. While social workers have explicit ethical obligations (e.g., service, social justice), research projects ask empirical questions to help actualize and support the work of upholding those ethical principles.

In order to help you better understand the difference between ethical and empirical questions, let’s consider a topic about which people have moral opinions. How about SpongeBob SquarePants? [1] In early 2005, members of the conservative Christian group Focus on the Family (2005) [2] denounced this seemingly innocuous cartoon character as “morally offensive” because they perceived his character to be one that promotes a “pro-gay agenda.” Focus on the Family supported their claim that SpongeBob is immoral by citing his appearance in a children’s video designed to promote tolerance of all family forms (BBC News, 2005). [3] They also cited SpongeBob’s regular hand-holding with his male sidekick Patrick as further evidence of his immorality.

So, can we now conclude that SpongeBob SquarePants is immoral? Not so fast. While your mother or a newspaper or television reporter may provide an answer, a social science researcher cannot. Questions of morality are ethical, not empirical. Of course, this doesn’t mean that social science researchers cannot study opinions about or social meanings surrounding SpongeBob SquarePants (Carter, 2010). [4] We study humans after all, and as you will discover in the following chapters of this textbook, we are trained to utilize a variety of scientific data-collection techniques to understand patterns of human beliefs and behaviors. Using these techniques, we could find out how many people in the United States find SpongeBob morally reprehensible, but we could never learn, empirically, whether SpongeBob is in fact morally reprehensible.

Let’s consider an example from a recent MSW research class I taught. A student group wanted to research the penalties for sexual assault. Their original research question was: “How can prison sentences for sexual assault be so much lower than the penalty for drug possession?” Outside of the research context, that is a darn good question! It speaks to how the War on Drugs and the patriarchy have distorted the criminal justice system towards policing of drug crimes over gender-based violence.

Unfortunately, it is an ethical question, not an empirical one. To answer that question, you would have to draw on philosophy and morality, answering what it is about human nature and society that allows such unjust outcomes. However, you could not answer that question by gathering data about people in the real world. If I asked people that question, they would likely give me their opinions about drugs, gender-based violence, and the criminal justice system. But I wouldn’t get the real answer about why our society tolerates such an imbalance in punishment.

As the students worked on the project through the semester, they continued to focus on the topic of sexual assault in the criminal justice system. Their research question became more empirical because they read more empirical articles about their topic. One option that they considered was to evaluate intervention programs for perpetrators of sexual assault to see if they reduced the likelihood of committing sexual assault again. Another option they considered was seeing if counties or states with higher than average jail sentences for sexual assault perpetrators had lower rates of re-offense for sexual assault. These projects addressed the ethical question of punishing perpetrators of sexual violence but did so in a way that gathered and analyzed empirical real-world data. Our job as social work researchers is to gather social facts about social work issues, not to judge or determine morality.

Key Takeaways

Empirical questions are distinct from ethical questions.
There are usually a number of ethical questions and a number of empirical questions that could be asked about any single topic.
While social workers may research topics about which people have moral opinions, a researcher’s job is to gather and analyze empirical data.
Take a look at your working question. Make sure you have an empirical question, not an ethical one. To perform this check, describe how you could find an answer to your question by conducting a study, like a survey or focus group, with real people.

9.2 Characteristics of a good research question

Identify and explain the key features of a good research question
Explain why it is important for social workers to be focused and clear with the language they use in their research questions

Now that you’ve made sure your working question is empirical, you need to revise that working question into a formal research question. So, what makes a good research question? First, it is generally written in the form of a question. To say that your research question is “the opioid epidemic” or “animal assisted therapy” or “oppression” would not be correct. You need to frame your topic as a question, not a statement. A good research question is also one that is well-focused. A well-focused question helps you tune out irrelevant information and not try to answer everything about the world all at once. You could be the most eloquent writer in your class, or even in the world, but if the research question about which you are writing is unclear, your work will ultimately lack direction.

In addition to being written in the form of a question and being well-focused, a good research question is one that cannot be answered with a simple yes or no. For example, if your interest is in gender norms, you could ask, “Does gender affect a person’s performance of household tasks?” but you will have nothing left to say once you discover your yes or no answer. Instead, why not ask, about the relationship between gender and household tasks. Alternatively, maybe we are interested in how or to what extent gender affects a person’s contributions to housework in a marriage? By tweaking your question in this small way, you suddenly have a much more fascinating question and more to say as you attempt to answer it.

A good research question should also have more than one plausible answer. In the example above, the student who studied the relationship between gender and household tasks had a specific interest in the impact of gender, but she also knew that preferences might be impacted by other factors. For example, she knew from her own experience that her more traditional and socially conservative friends were more likely to see household tasks as part of the female domain, and were less likely to expect their male partners to contribute to those tasks. Thinking through the possible relationships between gender, culture, and household tasks led that student to realize that there were many plausible answers to her questions about how gender affects a person’s contribution to household tasks. Because gender doesn’t exist in a vacuum, she wisely felt that she needed to consider other characteristics that work together with gender to shape people’s behaviors, likes, and dislikes. By doing this, the student considered the third feature of a good research question–she thought about relationships between several concepts. While she began with an interest in a single concept—household tasks—by asking herself what other concepts (such as gender or political orientation) might be related to her original interest, she was able to form a question that considered the relationships among those concepts.

This student had one final component to consider. Social work research questions must contain a target population. Her study would be very different if she were to conduct it on older adults or immigrants who just arrived in a new country. The target population is the group of people whose needs your study addresses. Maybe the student noticed issues with household tasks as part of her social work practice with first-generation immigrants, and so she made it her target population. Maybe she wants to address the needs of another community. Whatever the case, the target population should be chosen while keeping in mind social work’s responsibility to work on behalf of marginalized and oppressed groups.

In sum, a good research question generally has the following features:

It is written in the form of a question
It is clearly written
It cannot be answered with “yes” or “no”
It has more than one plausible answer
It considers relationships among multiple variables
It is specific and clear about the concepts it addresses
It includes a target population
A poorly focused research question can lead to the demise of an otherwise well-executed study.
Research questions should be clearly worded, consider relationships between multiple variables, have more than one plausible answer, and address the needs of a target population.

Okay, it’s time to write out your first draft of a research question.

Once you’ve done so, take a look at the checklist in this chapter and see if your research question meets the criteria to be a good one.

Brainstorm whether your research question might be better suited to quantitative or qualitative methods.

Describe why your question fits better with quantitative or qualitative methods.
Provide an alternative research question that fits with the other type of research method.

9.3 Quantitative research questions

Describe how research questions for exploratory, descriptive, and explanatory quantitative questions differ and how to phrase them
Identify the differences between and provide examples of strong and weak explanatory research questions

Quantitative descriptive questions

The type of research you are conducting will impact the research question that you ask. Probably the easiest questions to think of are quantitative descriptive questions. For example, “What is the average student debt load of MSW students?” is a descriptive question—and an important one. We aren’t trying to build a causal relationship here. We’re simply trying to describe how much debt MSW students carry. Quantitative descriptive questions like this one are helpful in social work practice as part of community scans, in which human service agencies survey the various needs of the community they serve. If the scan reveals that the community requires more services related to housing, child care, or day treatment for people with disabilities, a nonprofit office can use the community scan to create new programs that meet a defined community need.

Quantitative descriptive questions will often ask for percentage, count the number of instances of a phenomenon, or determine an average. Descriptive questions may only include one variable, such as ours about student debt load, or they may include multiple variables. Because these are descriptive questions, our purpose is not to investigate causal relationships between variables. To do that, we need to use a quantitative explanatory question.

Quantitative explanatory questions

Most studies you read in the academic literature will be quantitative and explanatory. Why is that? If you recall from Chapter 2 , explanatory research tries to build nomothetic causal relationships. They are generalizable across space and time, so they are applicable to a wide audience. The editorial board of a journal wants to make sure their content will be useful to as many people as possible, so it’s not surprising that quantitative research dominates the academic literature.

Structurally, quantitative explanatory questions must contain an independent variable and dependent variable. Questions should ask about the relationship between these variables. The standard format I was taught in graduate school for an explanatory quantitative research question is: “What is the relationship between [independent variable] and [dependent variable] for [target population]?” You should play with the wording for your research question, revising that standard format to match what you really want to know about your topic.

Let’s take a look at a few more examples of possible research questions and consider the relative strengths and weaknesses of each. Table 9.1 does just that. While reading the table, keep in mind that I have only noted what I view to be the most relevant strengths and weaknesses of each question. Certainly each question may have additional strengths and weaknesses not noted in the table. Each of these questions is drawn from student projects in my research methods classes and reflects the work of many students on their research question over many weeks.

Table 9.1 Sample research questions: Strengths and weaknesses

What are the internal and external effects/problems associated with children witnessing domestic violence?	Written as a question	Not clearly focused	How does witnessing domestic violence impact a child’s romantic relationships in adulthood?
	Considers relationships among multiple concepts	Not specific and clear about the concepts it addresses
	Contains a population
What causes foster children who are transitioning to adulthood to become homeless, jobless, pregnant, unhealthy, etc.?	Considers relationships among multiple concepts	Concepts are not specific and clear	What is the relationship between sexual orientation or gender identity and homelessness for late adolescents in foster care?
	Contains a population
	Not written as a yes/no question
How does income inequality predict ambivalence in the Stereo Content Model using major U.S. cities as target populations?	Written as a question	Unclear wording	How does income inequality affect ambivalence in high-density urban areas?
	Considers relationships among multiple concepts	Population is unclear
Why are mental health rates higher in white foster children than African Americans and other races?	Written as a question	Concepts are not clear	How does race impact rates of mental health diagnosis for children in foster care?
	Not written as a yes/no question	Does not contain a target population

Making it more specific

A good research question should also be specific and clear about the concepts it addresses. A student investigating gender and household tasks knows what they mean by “household tasks.” You likely also have an impression of what “household tasks” means. But are your definition and the student’s definition the same? A participant in their study may think that managing finances and performing home maintenance are household tasks, but the researcher may be interested in other tasks like childcare or cleaning. The only way to ensure your study stays focused and clear is to be specific about what you mean by a concept. The student in our example could pick a specific household task that was interesting to them or that the literature indicated was important—for example, childcare. Or, the student could have a broader view of household tasks, one that encompasses childcare, food preparation, financial management, home repair, and care for relatives. Any option is probably okay, as long as the researcher is clear on what they mean by “household tasks.” Clarifying these distinctions is important as we look ahead to specifying how your variables will be measured in Chapter 11 .

Table 9.2 contains some “watch words” that indicate you may need to be more specific about the concepts in your research question.

Table 9.2 “Watch words” in explanatory research questions

Factors, Causes, Effects, Outcomes	What causes or effects are you interested in? What causes and effects are important, based on the literature in your topic area? Try to choose one or a handful you consider to be the most important.
Effective, Effectiveness, Useful, Efficient	Effective at doing what? Effectiveness is meaningless on its own. What outcome should the program or intervention have? Reduced symptoms of a mental health issue? Better socialization?
Etc., and so forth	Don’t assume that your reader understands what you mean by “and so forth.” Remember that focusing on two or a small handful concepts is necessary. Your study cannot address everything about a social problem, though the results will likely have implications on other aspects of the social world.

It can be challenging to be this specific in social work research, particularly when you are just starting out your project and still reading the literature. If you’ve only read one or two articles on your topic, it can be hard to know what you are interested in studying. Broad questions like “What are the causes of chronic homelessness, and what can be done to prevent it?” are common at the beginning stages of a research project as working questions. However, moving from working questions to research questions in your research proposal requires that you examine the literature on the topic and refine your question over time to be more specific and clear. Perhaps you want to study the effect of a specific anti-homelessness program that you found in the literature. Maybe there is a particular model to fighting homelessness, like Housing First or transitional housing, that you want to investigate further. You may want to focus on a potential cause of homelessness such as LGBTQ+ discrimination that you find interesting or relevant to your practice. As you can see, the possibilities for making your question more specific are almost infinite.

Quantitative exploratory questions

In exploratory research, the researcher doesn’t quite know the lay of the land yet. If someone is proposing to conduct an exploratory quantitative project, the watch words highlighted in Table 9.2 are not problematic at all. In fact, questions such as “What factors influence the removal of children in child welfare cases?” are good because they will explore a variety of factors or causes. In this question, the independent variable is less clearly written, but the dependent variable, family preservation outcomes, is quite clearly written. The inverse can also be true. If we were to ask, “What outcomes are associated with family preservation services in child welfare?”, we would have a clear independent variable, family preservation services, but an unclear dependent variable, outcomes. Because we are only conducting exploratory research on a topic, we may not have an idea of what concepts may comprise our “outcomes” or “factors.” Only after interacting with our participants will we be able to understand which concepts are important.

Remember that exploratory research is appropriate only when the researcher does not know much about topic because there is very little scholarly research. In our examples above, there is extensive literature on the outcomes in family reunification programs and risk factors for child removal in child welfare. Make sure you’ve done a thorough literature review to ensure there is little relevant research to guide you towards a more explanatory question.

Descriptive quantitative research questions are helpful for community scans but cannot investigate causal relationships between variables.
Explanatory quantitative research questions must include an independent and dependent variable.
Exploratory quantitative research questions should only be considered when there is very little previous research on your topic.
Identify the type of research you are engaged in (descriptive, explanatory, or exploratory).
Create a quantitative research question for your project that matches with the type of research you are engaged in.

Preferably, you should be creating an explanatory research question for quantitative research.

9.4 Qualitative research questions

List the key terms associated with qualitative research questions
Distinguish between qualitative and quantitative research questions

Qualitative research questions differ from quantitative research questions. Because qualitative research questions seek to explore or describe phenomena, not provide a neat nomothetic explanation, they are often more general and openly worded. They may include only one concept, though many include more than one. Instead of asking how one variable causes changes in another, we are instead trying to understand the experiences , understandings , and meanings that people have about the concepts in our research question. These keywords often make an appearance in qualitative research questions.

Let’s work through an example from our last section. In Table 9.1, a student asked, “What is the relationship between sexual orientation or gender identity and homelessness for late adolescents in foster care?” In this question, it is pretty clear that the student believes that adolescents in foster care who identify as LGBTQ+ may be at greater risk for homelessness. This is a nomothetic causal relationship—LGBTQ+ status causes changes in homelessness.

However, what if the student were less interested in predicting homelessness based on LGBTQ+ status and more interested in understanding the stories of foster care youth who identify as LGBTQ+ and may be at risk for homelessness? In that case, the researcher would be building an idiographic causal explanation . The youths whom the researcher interviews may share stories of how their foster families, caseworkers, and others treated them. They may share stories about how they thought of their own sexuality or gender identity and how it changed over time. They may have different ideas about what it means to transition out of foster care.

Because qualitative questions usually center on idiographic causal relationships, they look different than quantitative questions. Table 9.3 below takes the final research questions from Table 9.1 and adapts them for qualitative research. The guidelines for research questions previously described in this chapter still apply, but there are some new elements to qualitative research questions that are not present in quantitative questions.

Qualitative research questions often ask about lived experience, personal experience, understanding, meaning, and stories.
Qualitative research questions may be more general and less specific.
Qualitative research questions may also contain only one variable, rather than asking about relationships between multiple variables.

Table 9.3 Quantitative vs. qualitative research questions

How does witnessing domestic violence impact a child’s romantic relationships in adulthood?	How do people who witness domestic violence understand its effects on their current relationships?
What is the relationship between sexual orientation or gender identity and homelessness for late adolescents in foster care?	What is the experience of identifying as LGBTQ+ in the foster care system?
How does income inequality affect ambivalence in high-density urban areas?	What does racial ambivalence mean to residents of an urban neighborhood with high income inequality?
How does race impact rates of mental health diagnosis for children in foster care?	How do African-Americans experience seeking help for mental health concerns?

Qualitative research questions have one final feature that distinguishes them from quantitative research questions: they can change over the course of a study. Qualitative research is a reflexive process, one in which the researcher adapts their approach based on what participants say and do. The researcher must constantly evaluate whether their question is important and relevant to the participants. As the researcher gains information from participants, it is normal for the focus of the inquiry to shift.

For example, a qualitative researcher may want to study how a new truancy rule impacts youth at risk of expulsion. However, after interviewing some of the youth in their community, a researcher might find that the rule is actually irrelevant to their behavior and thoughts. Instead, their participants will direct the discussion to their frustration with the school administrators or the lack of job opportunities in the area. This is a natural part of qualitative research, and it is normal for research questions and hypothesis to evolve based on information gleaned from participants.

However, this reflexivity and openness unacceptable in quantitative research for good reasons. Researchers using quantitative methods are testing a hypothesis, and if they could revise that hypothesis to match what they found, they could never be wrong! Indeed, an important component of open science and reproducability is the preregistration of a researcher’s hypotheses and data analysis plan in a central repository that can be verified and replicated by reviewers and other researchers. This interactive graphic from 538 shows how an unscrupulous research could come up with a hypothesis and theoretical explanation after collecting data by hunting for a combination of factors that results in a statistically significant relationship. This is an excellent example of how the positivist assumptions behind quantitative research and intepretivist assumptions behind qualitative research result in different approaches to social science.

Qualitative research questions often contain words or phrases like “lived experience,” “personal experience,” “understanding,” “meaning,” and “stories.”
Qualitative research questions can change and evolve over the course of the study.
Using the guidance in this chapter, write a qualitative research question. You may want to use some of the keywords mentioned above.

9.5 Evaluating and updating your research questions

Evaluate the feasibility and importance of your research questions
Begin to match your research questions to specific designs that determine what the participants in your study will do

Feasibility and importance

As you are getting ready to finalize your research question and move into designing your research study, it is important to check whether your research question is feasible for you to answer and what importance your results will have in the community, among your participants, and in the scientific literature

Key questions to consider when evaluating your question’s feasibility include:

Do you have access to the data you need?
Will you be able to get consent from stakeholders, gatekeepers, and others?
Does your project pose risk to individuals through direct harm, dual relationships, or breaches in confidentiality? (see Chapter 6 for more ethical considerations)
Are you competent enough to complete the study?
Do you have the resources and time needed to carry out the project?

Key questions to consider when evaluating the importance of your question include:

Can we answer your research question simply by looking at the literature on your topic?
How does your question add something new to the scholarly literature? (raises a new issue, addresses a controversy, studies a new population, etc.)
How will your target population benefit, once you answer your research question?
How will the community, social work practice, and the broader social world benefit, once you answer your research question?
Using the questions above, check whether you think your project is feasible for you to complete, given the constrains that student projects face.
Realistically, explore the potential impact of your project on the community and in the scientific literature. Make sure your question cannot be answered by simply reading more about your topic.

Matching your research question and study design

This chapter described how to create a good quantitative and qualitative research question. In Parts 3 and 4 of this textbook, we will detail some of the basic designs like surveys and interviews that social scientists use to answer their research questions. But which design should you choose?

As with most things, it all depends on your research question. If your research question involves, for example, testing a new intervention, you will likely want to use an experimental design. On the other hand, if you want to know the lived experience of people in a public housing building, you probably want to use an interview or focus group design.

We will learn more about each one of these designs in the remainder of this textbook. We will also learn about using data that already exists, studying an individual client inside clinical practice, and evaluating programs, which are other examples of designs. Below is a list of designs we will cover in this textbook:

Surveys: online, phone, mail, in-person
Experiments: classic, pre-experiments, quasi-experiments
Interviews: in-person or via phone or videoconference
Focus groups: in-person or via videoconference
Content analysis of existing data
Secondary data analysis of another researcher’s data
Program evaluation

The design of your research study determines what you and your participants will do. In an experiment, for example, the researcher will introduce a stimulus or treatment to participants and measure their responses. In contrast, a content analysis may not have participants at all, and the researcher may simply read the marketing materials for a corporation or look at a politician’s speeches to conduct the data analysis for the study.

I imagine that a content analysis probably seems easier to accomplish than an experiment. However, as a researcher, you have to choose a research design that makes sense for your question and that is feasible to complete with the resources you have. All research projects require some resources to accomplish. Make sure your design is one you can carry out with the resources (time, money, staff, etc.) that you have.

There are so many different designs that exist in the social science literature that it would be impossible to include them all in this textbook. The purpose of the subsequent chapters is to help you understand the basic designs upon which these more advanced designs are built. As you learn more about research design, you will likely find yourself revising your research question to make sure it fits with the design. At the same time, your research question as it exists now should influence the design you end up choosing. There is no set order in which these should happen. Instead, your research project should be guided by whether you can feasibly carry it out and contribute new and important knowledge to the world.

Research questions must be feasible and important.
Research questions must match study design.
Based on what you know about designs like surveys, experiments, and interviews, describe how you might use one of them to answer your research question.
You may want to refer back to Chapter 2 which discusses how to get raw data about your topic and the common designs used in student research projects.

Media Attributions

patrick-starfish-2062906_1920 © Inspired Images is licensed under a CC0 (Creative Commons Zero) license
financial-2860753_1920 © David Schwarzenberg is licensed under a CC0 (Creative Commons Zero) license
target-group-3460039_1920 © Gerd Altmann is licensed under a CC0 (Creative Commons Zero) license
Not familiar with SpongeBob SquarePants? You can learn more about him on Nickelodeon’s site dedicated to all things SpongeBob: http://www.nick.com/spongebob-squarepants/ ↵
Focus on the Family. (2005, January 26). Focus on SpongeBob. Christianity Today . Retrieved from http://www.christianitytoday.com/ct/2005/januaryweb-only/34.0c.html ↵
BBC News. (2005, January 20). US right attacks SpongeBob video. Retrieved from: http://news.bbc.co.uk/2/hi/americas/4190699.stm ↵
In fact, an MA thesis examines representations of gender and relationships in the cartoon: Carter, A. C. (2010). Constructing gender and relationships in “SpongeBob SquarePants”: Who lives in a pineapple under the sea . MA thesis, Department of Communication, University of South Alabama, Mobile, AL. ↵

research questions that can be answered by systematically observing the real world

unsuitable research questions which are not answerable by systematic observation of the real world but instead rely on moral or philosophical opinions

the group of people whose needs your study addresses

attempts to explain or describe your phenomenon exhaustively, based on the subjective understandings of your participants

"Assuming that the null hypothesis is true and the study is repeated an infinite number times by drawing random samples from the same populations(s), less than 5% of these results will be more extreme than the current result" (Cassidy et al., 2019, p. 233).

whether you can practically and ethically complete the research project you propose

the impact your study will have on participants, communities, scientific knowledge, and social justice

Graduate research methods in social work Copyright © 2021 by Matthew DeCarlo, Cory Cummings, Kate Agnelli is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Methods in Social Work and its concept

Social work is a profession that helps individuals, groups, and communities to improve their social and emotional well-being. There are five main methods of social work: social casework, social group work, community organization, social welfare administration, and research.

Definition of Social work

Bohem (1959) : Social work seeks to enhance the social functioning of the individuals, singly and in groups, by activities focused upon their social relationships that constitute the interaction between man and his environment. These activities can be the provision of individual and social resources and prevention of social dysfunction.

Methods of Social Work:

The social worker believes in the capacity of the individual and also recognizes individual differences. The individual’s self-determination is given importance. He should be understood from both domestic and cultural points of view. Social work is a combination of “idealism and realism”. To a social worker, an individual is important but society is equally important. The individual is greatly molded by social circumstances. But, ultimately the individual must bear the responsibility for his or her conduct and behavior. The worker has to solve the problem on account of which the client is disturbed.

Hence, professional social work with selected knowledge and the set of social work values has to be transformed into a professional service. A social worker has to establish a positive relationship with the clients. She should know how to interview and write reports. He or she should be able to diagnose i.e., find out the cause for the problem and finally should work out a treatment plan. An Assessment of the problem, planning for its solution, implementing the plan, and evaluating the outcome are the four major steps involved in social work. The social worker’s keen interest in helping the client, alone will not solve the problem.

The methods of social work will help his/her to understand ways of helping people. Social work methods are:

Primary methods (direct helping method)

1) Social casework

2) Social group work.

3) Community organization.

Secondary methods (Auxiliary methods)

4) Social work research.

5) Social welfare administration.

6) Social Action

These six social work methods are systematic and planned ways of helping people.

Social casework deals with individual problems- individual in the total environment or as a part of it. An individual is involved in the problem as he is unable to deal with it on his own, because of reasons beyond his control. His anxiety sometimes temporarily makes him incapable of solving it. In any case, his social functioning is disturbed. The caseworker gets information regarding the client’s total environment, finds out the causes, prepares a treatment plan and with a professional relationship tries to bring about a change in the perception and attitudes of the client.

Social group work is a social work service in which a professionally qualified person helps individuals through group experience so as to help them move towards improved relationships and social functioning. In group work individuals are important and they are helped to improve their social relationships, with flexible programs, giving importance to the personality development of the individual in group functioning and relationships. The group is the medium and through it and in it, individuals are helped to make necessary changes and adjustments.

Community Organisation is another method of social work. Being made up of groups, a community means organized systems of relationships but in reality, no community is perfectly organized. Community Organisation is a process by which a systematic attempt is made to improve relationships in a community. Identifying the problems, finding out resources for solving community problems, developing social relationships, and necessary programmes to realize the objectives of the community are all involved in community organization. In this way, the community can become self-reliant and develop a co-operative attitude among its members.

Social Welfare Administration is a process through which social work services both private and public, are organized and administered. Developing programmes, mobilizing resources, involving selection and recruitment of personnel, proper organization, coordination, providing skillful and sympathetic leadership, guidance and supervision of the staff, dealing with financing and budgeting of the programmes and evaluation are, some of the functions of a social worker in administration.

Social work research is a systematic investigation for finding out new facts, test old hypotheses, verify existing theories, and discover causal relationships of the problems in which the social worker is interested. In order to scientifically initiate any kind of social work program, a systematic study of the given situation is necessary, through social work research and surveys.

Social action aims at bringing about desirable changes to ensure social progress. Creating awareness about social problems, mobilizing resources, encouraging different ‘sections of people to raise their voice against undesirable practices, and also creating pressure to bring about the legislation are some of the activities of the social workers using the method of social action. It seeks to achieve a proper balance between community needs and solutions mainly through individual and group initiatives and self-help activities

thanks alot we can cooporate to conduct studies in future if you like.

Thanks a lot

“Why me?”: Qualitative research on why patients ask, what they mean, how they answer and what factors and processes are involved

Klitzman, Robert

Patients often ask, “why me?” but questions arise regarding what this statement means, how, when and why patients ask, how they answer and why. Interviews were conducted as part of several qualitative research studies exploring how patients view and cope with various conditions, including HIV, cancer, Huntington’s disease and infertility. A secondary qualitative analysis was performed. Many patients ask, “why me?” but this statement emerges as having varying meanings, and entailing complex psychosocial processes. Patients commonly recognize that this question may lack a clear answer and that asking it is irrational, but they ask nonetheless, given the roles of unknown factors and chance in disease causation, psychological stresses of illness and lack of definitive answers. Patients may focus on different aspects of the question – e.g., on possible causes of illness (Why me? – whether God or randomness is involved) and/or on whether they are being singled out and/or punished (Why me vs. someone else?). Patients frequently undergo dynamic processes, confronting this question at various points, and arriving at different answers, looking for explanations that have narrative coherence for them, and make sense to them emotionally. Social contexts can affect these processes, with friends, family, providers or others rejecting or accepting patients’ responses to this question (e.g., beliefs about whether the patient is being punished and/or these questions are worth asking). Anger, depression, despair and/or resistance to notions about the roles of randomness or chaos can also shape these processes. While prior studies have each operationalized “why me?” in differing ways, focusing on varying aspects of it, the concept emerges here as highly multidimensional, involving complex processes and often affected by social contexts. These data, the first to examine key aspects and meanings of the phrase, “why me?” have critical implications for future practice, research and education.

Medical ethics
Hospital care

thumnail for Klitzman - 2023 - “Why me” Qualitative research on why patients as.pdf

Also Published In

More about this work.

DOI Copy DOI to clipboard

share this!

August 12, 2024

This article has been reviewed according to Science X's editorial process and policies . Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

trusted source

One way social work researchers can better understand community needs—and move the field forward

by Matt Shipman, North Carolina State University

Researchers are calling on the social work community to begin incorporating a methodology called "discrete choice experiments" (DCEs) into their research, to better understand the needs and preferences of key stakeholders. This technique is well established in other fields but is rarely used in social work.

The paper, " How to Use Discrete Choice Experiments to Capture Stakeholder Preferences in Social Work Research ," is published in the Journal of the Society for Social Work and Research .

"Social workers need to engage with a wide variety of stakeholders, from policy makers to the people who use social services ," says Alan Ellis, an associate professor of social work at North Carolina State University and corresponding author of a paper introducing social work researchers to the DCE methodology.

"But social work, as a research discipline, has not identified a standard technique for eliciting the preferences of those stakeholders—even though this is a critical issue," Ellis says.

"Although traditional survey methods can be used to evaluate stakeholder perspectives, the DCE is one of several methodologies that were specifically designed to assess the degree to which people prioritize one thing over another. In this paper, we propose that social work researchers adopt DCEs as a robust tool for capturing stakeholder preferences on any number of issues."

In a DCE, researchers ask participants to complete a series of choice tasks: hypothetical situations in which each participant is presented with alternative scenarios and selects one or more.

"For example, social work researchers may want to know how parents and other caregivers prioritize different aspects of mental health treatment when choosing services for their children," Ellis says. "A DCE can explore this question by presenting scenarios that include different types of mental health care providers, treatment methods, costs, locations and so on. Caregivers' stated choices in these scenarios can provide a lot of information about their priorities."

DCEs were first developed by marketing researchers and are now widely used in fields ranging from transportation to health care.

"We know that DCEs effectively capture preferences on a wide variety of subjects," Ellis says. "We simply want to begin using them more consistently to address issues that are important to stakeholders in social work.

"From a pure research standpoint, having a better understanding of stakeholder needs and preferences can move the field forward by helping us develop better research questions and better studies," says Ellis. "Beyond that, having a better understanding of our clients' preferences and goals will make us better social workers. Adopting DCEs can strengthen the link between social work research and practice—and ground our research , policy, and practice in the values that are important to the people we serve.

"I'm optimistic that DCEs could help us collaborate with stakeholders to effect positive change."

The paper was co-authored by Qiana Cryer-Coupet of Georgia State University, Bridget Weller of Wayne State University, Kirsten Howard and Rakhee Raghunandan of the University of Sydney, and Kathleen Thomas of the University of North Carolina at Chapel Hill.

Provided by North Carolina State University

Explore further

Feedback to editors

Saturday Citations: Citizen scientists observe fast thing; controlling rat populations; clearing nanoplastic from water

13 hours ago

New AI tool captures how proteins behave in context

19 hours ago

Scientists discover phenomenon impacting Earth's radiation belts

Geophysicists find link between seismic waves called PKP precursors and strange anomalies in Earth's mantle

New twist on synthesis technique promises sustainable manufacturing

Researchers discover smarter way to recycle polyurethane

Aug 16, 2024

DNA study challenges thinking on ancestry of people in Japan

A visionary approach: How a team developed accessible maps for colorblind scientists

New tool simplifies cell tracking data analysis

How some states help residents avoid costly debt during hard times

Relevant physicsforums posts, cover songs versus the original track, which ones are better, why are abba so popular.

12 hours ago

Today's Fusion Music: T Square, Cassiopeia, Rei & Kanade Sato

16 hours ago

Favorite songs (cont.)

Talent worthy of wider recognition, history of railroad safety - spotlight on current derailments.

More from Art, Music, History, and Linguistics

Working the quads for better eHealth: Combining four areas of stakeholder relationships

Aug 8, 2024

Study outlines factors that help engage nonresident fathers in child welfare efforts

Aug 9, 2021

Social workers debunk the myths about how they help patients heal

Mar 29, 2024

Behavioral and computational study shows that social preferences can be inferred from decision speed alone

Jun 20, 2024

Study highlights complexity of public responses to corporate crises

Feb 12, 2020

Study identifies ways to better help children experiencing homelessness

Feb 6, 2024

Recommended for you

Singing from memory unlocks a surprisingly common musical superpower

Aug 15, 2024

Study suggests five-second break can defuse an argument between coupled partners

Aug 14, 2024

Findings suggest empowering women is key to both sustainable energy and gender justice

Aug 13, 2024

Exploring the evolution of social norms with a supercomputer

Aug 9, 2024

Study shows people associate kindness with religious belief

Let us know if there is a problem with our content.

Use this form if you have come across a typo, inaccuracy or would like to send an edit request for the content on this page. For general inquiries, please use our contact form . For general feedback, use the public comments section below (please adhere to guidelines ).

Please select the most appropriate category to facilitate processing of your request

Thank you for taking time to provide your feedback to the editors.

Your feedback is important to us. However, we do not guarantee individual replies due to the high volume of messages.

E-mail the story

Your email address is used only to let the recipient know who sent the email. Neither your address nor the recipient's address will be used for any other purpose. The information you enter will appear in your e-mail message and is not retained by Phys.org in any form.

Newsletter sign up

Get weekly and/or daily updates delivered to your inbox. You can unsubscribe at any time and we'll never share your details to third parties.

More information Privacy policy

Donate and enjoy an ad-free experience

We keep our content available to everyone. Consider supporting Science X's mission by getting a premium account.

E-mail newsletter

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

View all journals
Explore content
About the journal
Publish with us
Sign up for alerts
Open access
Published: 13 August 2024

FEW questions, many answers: using machine learning to assess how students connect food–energy–water (FEW) concepts

Emily A. Royse 1 ,
Amanda D. Manzanares 2 ,
Heqiao Wang 3 ,
Kevin C. Haudek 4 ,
Caterina Belle Azzarello 2 ,
Lydia R. Horne 5 ,
Daniel L. Druckenbrod 6 ,
Megan Shiroda 7 ,
Sol R. Adams 8 ,
Ennea Fairchild 9 ,
Shirley Vincent 10 ,
Steven W. Anderson 11 &
Chelsie Romulo ORCID: orcid.org/0000-0003-1612-1969 12

Humanities and Social Sciences Communications volume 11 , Article number: 1033 ( 2024 ) Cite this article

145 Accesses

Metrics details

Environmental studies

There is growing support and interest in postsecondary interdisciplinary environmental education, which integrates concepts and disciplines in addition to providing varied perspectives. There is a need to assess student learning in these programs as well as rigorous evaluation of educational practices, especially of complex synthesis concepts. This work tests a text classification machine learning model as a tool to assess student systems thinking capabilities using two questions anchored by the Food-Energy-Water (FEW) Nexus phenomena by answering two questions (1) Can machine learning models be used to identify instructor-determined important concepts in student responses? (2) What do college students know about the interconnections between food, energy, and water, and how have students assimilated systems thinking into their constructed responses about FEW? Reported here is a broad range of model performances across 26 text classification models associated with two different assessment items, with model accuracy ranging from 0.755 to 0.992. Expert-like responses were infrequent in our dataset compared to responses providing simpler, incomplete explanations of the systems presented in the question. For those students moving from describing individual effects to multiple effects, their reasoning about the mechanism behind the system indicates advanced systems thinking ability. Specifically, students exhibit higher expertise in explaining changing water usage than discussing trade-offs for such changing usage. This research represents one of the first attempts to assess the links between foundational, discipline-specific concepts and systems thinking ability. These text classification approaches to scoring student FEW Nexus Constructed Responses (CR) indicate how these approaches can be used, in addition to several future research priorities for interdisciplinary, practice-based education research. Development of further complex question items using machine learning would allow evaluation of the relationship between foundational concept understanding and integration of those concepts as well as a more nuanced understanding of student comprehension of complex interdisciplinary concepts.

Methods for measuring career readiness of high school students: based on multidimensional item response theory and text mining

A longitudinal Q-study to assess changes in students’ perceptions at the time of pandemic

A meta-analysis to gauge the impact of pedagogies employed in mixed-ability high school biology classrooms

Introduction.

Many global problems are considered “wicked” in that they integrate complex systems that are often studied in distinct disciplines (Balint et al., 2011 ). To solve these 21st-century socio-ecological problems, students must instead learn cross-cutting concepts across disciplines within interdisciplinary programs. The Next Generation Science Standards (NGSS) identify crosscutting concepts as a framework to link different science disciplines, providing a means for students to link knowledge across fields to establish a cogent, scientifically-based way of interpreting the world (National Research Council, 2012 ; NGSS Lead States, 2013 ). In environmental programs within higher education, recent efforts are defining the key disciplinary ideas, concepts, practices, and skills embedded in complex meaningful learning and implementing new curricula with interdisciplinary frameworks ( Global Council for Science and the Environment , n.d. ; Vincent et al., 2013 ). Frameworks that link concepts across disciplines can include the Sustainable Development Goals (SDGs; Education for Sustainable Development), Resilience Thinking, the United Nations Principles for Responsible Management (UN-PRME), and the Food-Energy-Water Nexus (FEW Nexus) (Leah Filho et al., 2001 ; Martins et al., 2022 ). Interdisciplinary approaches to curricula and course design rely on content mastery and skill development to understand systems interactions and higher-order thinking. With this shift toward higher-level learning across interdisciplinary environmental and sustainability (IES) programs, we need new assessments that elicit complex student thinking and can be used to identify and categorize different levels of understanding, not just memorization of facts (Laverty et al., 2016 ; J. W. Pellegrino et al., 2013 ; Underwood et al., 2018 ).

Assessing crosscutting, interdisciplinary learning is challenging, and often constructed responses (CR) (i.e., open-ended questions) are used for assessing interdisciplinary connections because student thinking and reasoning are more explicit compared to multiple choice type questions; however, these CR assessment items are challenging and time-consuming to design and grade. One rapidly developing tool with the potential to support this kind of assessment is text classification models, which are machine learning (ML) algorithms and statistical models that learn from and analyze data patterns. Due to the challenges of assessing interdisciplinary learning, IES programs provide a useful context for education research on the application of these types of ML for studying complex CR assessment items. Further, technology such as ML may help us evaluate these complex formative assessments and provide an opportunity to improve science teaching and learning (Harrison et al., 2023 ). Often in science assessment, each individual model is specifically developed for each question and response set in an iterative process using human coding and model development and selection methods, making this process potentially very time-intensive (Brew and Leacock, 2013 ). However, once a model is constructed, it can be used to score many responses very quickly, thus addressing the labor and time-intensive aspects of evaluated CR questions to allow for big data research using those specific questions and associated models. While this trade-off between model development and model use is an important consideration, the process of model development itself can be aided by several considerations, which may speed development time and improve the validity of the final model (Rupp, 2018 ). Thus a well designed process for developing and evaluating both questions and models is essential, although iterations throughout the process will always be necessary.

Here, we report on a process of using human-scored responses to construct ML-based text classification models for assessing CR questions focused on the food–energy–water (FEW) Nexus. As part of this focus on the model development process, we address two research questions (1) Can machine learning models be used to identify instructor-determined important concepts in student responses? (2) What do our students know about the interconnections between food, energy, and water, and how have students assimilated “systems thinking” into their constructed responses about FEW?

Systems thinking as an example of cross-cutting concepts

Systems thinking involves understanding the interdisciplinary connections and relationships between associated components within a system, rather than simply focusing on discrete concepts (Meadows, 2008 ). The Global Council for Science and the Environment’s (GCSE) draft proposal for key competencies in sustainability higher education identifies systems thinking as a core skill and includes increasing complexity across scales in their definition as the foundation for strategic solution development and future thinking (Brundiers et al., 2023 ). This level of understanding typically falls on the higher levels of Bloom’s Taxonomy of knowledge that include categories such as “apply”, “analyze” and “evaluate” (Bloom and Krathwohl, 1956 ; Krathwohl, 2002 ). Systems thinking is a key competency in STEM education, both in discipline-specific and interdisciplinary reasoning (Blatti et al., 2019 ; Hmelo-Silver et al., 2007 ; Mambrey et al., 2020 ; Momsen et al., 2022 ; Ravi et al., 2021 ; Redman et al., 2021 ; Redman and Wiek, 2021 ), and is recognized as a core competency by the National Science Foundation (NSF), the National Academies of Sciences, Engineering, and Medicine (National Science Foundation, 2020 ), and the US Next Generation Science Standards for K12 education (NGSS Lead States, 2013 ). Systems thinking was recently identified as a key competency by IES educators in higher education (Vincent et al., 2013 ), where understanding complex natural and social systems is applied and evaluated using systems thinking (Clark and Wallace, 2015 ; Varela-Losada et al., 2016 ).

While fostering systems thinking remains challenging, many potential strategies exist to help anchor student learning. Assessment of systems thinking is challenging and typically is approached from the context of the subject matter (see Randle and Stroink, 2018 ; Grohs et al., 2018 ; Gray et al., 2019 ; Bustamante et al., 2021 ; Liu, 2023 ; Dugan et al., 2022 and references within), which means there is not one agreed upon definition or assessment for systems thinking. For example, Soltis and McNeil ( 2022 ) have developed a systems thinking concept inventory specific to Earth Science, but valid and reliable approaches for measuring learning gains associated with systems thinking more broadly or in other applications are currently lacking. However, within the field of interdisciplinary environmental programs, there is a widely accepted definition of systems thinking from Wiek et al. ( 2016 ) for complex problem-solving for sustainability and commonly accepted concepts associated with systems thinking from Redman and Wiek ( 2021 ) (Box 1 ) and the 2021 NAS report on Strengthening Sustainability Education. Assessing systems thinking can be thus understood in the context of how it is integrated within a particular concept or set of concepts.

The FEW Nexus provides a concrete concept integration framework for developing the skill of systems thinking that applies across many interdisciplinary environmental programs as it connects complex environmental processes, management, policy, and socioeconomics of FEW resources (Smajgl et al., 2016 ). The FEW Nexus is a coupled systems approach to research and global development that accounts for synergies and trade-offs across FEW resource systems (D’Odorico et al., 2018 ; Leck et al., 2015 ; Simpson and Jewitt, 2019 ). For teaching and learning contexts, the FEW Nexus provides a scaffold for incorporating systems thinking and sustainability concepts into courses and across curricula. With global resource consumption outpacing supply, the FEW Nexus is a global priority area for research (Katz et al., 2020 ; Simpson and Jewitt, 2019 ). Understanding the FEW Nexus and the global focus on FEW research and decision-making makes it an ideal concept for exploring complex systems content in introductory IES courses, as FEW resource systems are visible to learners in their daily lives. Students need to develop their systems thinking to fully grasp the importance of the FEW Nexus and how it is impacted and impacts other systems, e.g., climate change, resource scarcity (Brandstädter et al., 2012 ).

Box 1. Systems thinking in the context of sustainability (Redman and Wiek, 2021 )

Ability to collectively apply modeling and complex analytical approaches: (1) to analyze complex systems and sustainability problems across different domains (environmental, social, economic) and across different scales (local to global), including cascading effects, inertia, feedback loops, and other system dynamics; (2) to analyze the impacts of sustainability action plans (strategies) and interventions (how they change systems and problems).

The need for tools to assess interdisciplinary systems thinking

Given the complexity of the relationships within the FEW Nexus and the relatively recent expansion of college-level IESs that incorporate FEW Nexus concepts, assessments that target these more advanced systems-level relationships are lacking. Assessing student conceptual understanding typically requires constructing valid and reliable tests, such as concept inventories (CIs) (Hestenes et al., 1992 ; Libarkin and Anderson, 2005 ; Libarkin and Geraghty Ward, 2011 ; Soltis and McNeal, 2022 ; Stone et al., 2003 ; Tornabee et al., 2016 ). Disciplinary CIs are traditionally used to assess learning using close-response questions (i.e., multiple choice). Existing CIs are inappropriate to assess complex skill development in IESs for two reasons: (1) IESs are interdisciplinary, and existing CIs do not capture the range of concepts typically covered in IES curricula, and (2) Close-ended questions (multiple choice) limit the ability to dissect higher level learning, such as systems thinking. An interdisciplinary, open-ended environmental CI could address these challenges; however, CR or open-ended assessments are labor-intensive to evaluate and can be very subjective for instructors to score. Artificial intelligence (AI) attempts to mimic human intelligent actions, including understanding language via Natural language processing (NLP) and classifying artifacts via ML. In the case of CIs, these approaches (NLP and ML) have been used to classify student written assessments and show promise for use with the first interdisciplinary environmental CI that enables assessment of deeper skill development (i.e., systems thinking, cause and effect, tradeoffs) while alleviating the burden of scoring CR questions. Few studies report on the use of interdisciplinary assessments in STEM (Gao et al., 2020 ), and this dearth of assessment tools also leads to little research about AI-based applications for such assessments (Zhai et al., 2020a , 2020b ). The work presented here is a start towards developing assessments (like CIs) that use CR for more complex concepts, such as systems thinking and connecting concepts across disciplines. Here, we focus on FEW as it is a system that incorporates concept integration that connects environmental processes, management, policy, and socioeconomics of FEW resources. There is a need for education research and collaboration in the FEW Nexus, as evidenced by the recently funded National Collaborative for Research on Food, Energy, and Water Education (NC-FEW), of which author Romulo is a member. FEW concepts are commonly covered in introductory environmental courses (Horne et al., 2023 ), and this project will focus on IES introductory courses for this process of development.

Text classification: using machine learning processes for interdisciplinary assessment

AI has been part of computer science for a number of decades, with the goal of having computers mimic human intelligence in performing complex tasks. AI utilizes approaches from several different computational subfields in computer science depending on the intended use or task performed. NLP is a branch of computer science that is interested in how computers can identify, understand, and support human language. NLP has become foundational for many AI applications, including speech recognition, language translation, and chatbots. NLP has been incorporated into education contexts in a variety of ways, including scoring of student texts, in both summative and formative uses (McNamara and Graesser, 2011 ; Shermis and Burstein, 2013 ), intelligent agents for interactive feedback (Chi et al., 2011 ), and customization of curricula materials and assessments (Mitkov et al., 2006 ). NLP has been applied in science assessment in a variety of ways. For example, NLP coupled with ML techniques has been used to develop predictive scoring models (Nehm et al., 2012 ), as an approach to explore sets of student responses (Zehner et al, 2015 ), and to assist in developing coding rubrics (Sripathi et al., 2023 ). Here, we focus on using NLP as part of text classification approaches to categorize student CR to assessment items (Dogra et al., 2022 ). Specifically, these text based CRs are short in length but rich in disciplinary content and common in STEM assessment practices (Liu et al., 2014 ). Using approaches from AI, these CRs can be automatically categorized according to coding rubrics that are developed with assessment items (Zhai et al., 2021a ).

Machine learning has been described as a “computer program that improves its performance at some task through experience” (Mitchell, 1997 ). “Experience” here refers to some information (e.g., outcomes, labels) available to the program from which it can “learn.” Much of the recent work on automated scoring of student CR has utilized supervised ML approaches, which use text representations from NLP along with assigned human codes as input for text classification models (Zhai et al., 2020b ). Generally, in supervised ML, these data are used to “train” ML algorithms in order to develop a scoring (or classification) model. Once the scoring model is developed, the model can be “tested” by comparing the consistency of human and machine-assigned codes on subsets of the same (or new) data (Jordan and Mitchell, 2015 ; Williamson et al., 2012 ). Various ML scoring approaches have been used to evaluate student CRs in science; these reports cover a range of grade levels and disciplinary topics (Jescovitch et al., 2021 ; Liu et al., 2014 ; Nehm et al., 2012 ; Wilson et al., 2023 ), such as the water cycle in secondary science (Lee et al., 2021 ). These studies and others have identified important considerations when designing assessment items, rubrics, and text classification models for evaluating responses to science CR assessments. Using these ML approaches in automated assessment scoring, important student ideas can be recognized by machines from authentic student work, as opposed to predefined answers. This is important to identify these key ideas as actually expressed by students. Thus a collection of student responses are necessary to train the ML model and to represent the range of possible answers (Shiroda et al., 2022 ; Suresh and Guttag, 2021 ).

Overall, we follow a modified question development cycle (Urban-Lurain et al., 2015 ) (Fig. 1 ) that integrates question, rubric, and text classification model scoring as part of an integrative formative assessment development and validation process. Broadly, this approach uses linguistic feature-based NLP methods (Deane, 2006 ) to extract linguistic features from writing and then uses those extracted features as variables in supervised ML models that predict human raters’ scores of student writing.

Adapted from Urban—Lurain et al. ( 2015 ), each box represents a stage of the process beginning with Question Design with outputs from one stage being used in subsequent stages, as indicated by solid arrows. A predictive model in the top, right corner is the ultimate goal of the cycle, in which a machine learning model can accurately predict classifications of new responses. A dashed arrow represents possible iteration(s) of the cycle depending on the outcomes of previous stages.

In the first stage of the cycle, we begin with Question Design (top) to target student thinking about important interdisciplinary constructs. Data Collection is typically done by administering the questions online to a wide range of students within appropriate courses and levels to collect a diverse range of responses. Exploratory analysis combines automated qualitative and quantitative approaches to the student-supplied text, including NLP, to explore the data corpus. For example, we use text analysis software to extract key terms and disciplinary concepts from the responses and look for patterns and themes among ideas. These terms, concepts and themes are used to assist Rubric Development. We use rubrics, both analytic and holistic, to code for key disciplinary ideas or emergent ideas in responses. These coding rubrics are subsequently used during the Human Coding of student responses in which one or more experts assign codes or scores to student responses. During Confirmatory Analysis, we develop text classification models by extracting text features from student responses using NLP approaches. These text features are subsequently used as independent variables in statistical classification and/or ML algorithms to predict expert human coding of responses, as part of supervised ML. In this stage, the performance of the ML model is measured by comparing the machine-assigned score to the human-assigned score. Once benchmarks for sufficient performance are achieved (Williamson et al., 2012 ), the model is saved and used as a Predictive Model. These Predictive Models can be used to completely automate the scoring of a new set of responses, predicting how experts would categorize or score the data. Often, results from one or more stages of the cycle are used to refine the assessment question (dashed arrow), rubrics, and/or human coding. The overall process is highly iterative, with feedback from each stage informing the refinement of other components. Further, the iterative cycle allows considerations for automated scoring to be addressed throughout the cycle, providing opportunities to collect and examine valid evidence (Rupp, 2018 ).

Concept identification

In previous work, we performed content analysis on IES course materials collected from 30 institutions to identify shared learning objectives across IES courses and programs (Horne et al., 2023 ). We also conducted ~100 semi-structured interviews with undergraduates enrolled in the 10 IES programs used for data collection in this study. From these interviews, we found that students have a broad range of knowledge regarding FEW concepts (Horne et al., 2024 ; Manzanares et al. in review ). We, therefore, sought to create assessment prompts that allowed us to explore a spectrum of student responses about the FEW Nexus. Informed by the previous results of the content analysis (Horne et al. 2023 ) and student interviews (Manzanares et al. in review), we identified two focal areas for assessment item development related to systems thinking (Box 1 ): (1) Identifying sources and Explaining Connections between FEW systems, and (2) Evaluating outcomes and Comparing Trade-offs between FEW systems (e.g., water used for food is water not used to create energy). We note that these assessment item topics align with NGSS standards of Systems & System Models (NGSS Lead States, 2013 ), since students must identify multiple boundaries, components, and connections between components, and they must predict outcomes from alterations in components or connections. We incorporated Bloom’s Taxonomy, a classification system for identifying skills that we intend our students to learn (Krathwohl, 2002 ), to help us scaffold our questions. For example, we recognize that students first must be able to identify sources of FEW and make connections to their environment (Table 1 : Sources of FEW and connections: reservoir) before they can understand the trade-offs of gaining a local energy source while losing land for crops (Table 2 : Trade-offs systems: biomass energy production). As such we have created questions that align with varying levels of student knowledge regarding the FEW Nexus.

Assessment Items

We developed multiple assessment items targeting comprehension of Identifying Sources of FEW and Connections and Trade-offs of FEW Systems using different phenomena (e.g., dams, biomass energy) commonly encountered in IES courses (Table 1 ). Items about important phenomena in IES courses were presented in relevant disciplinary context and broadly focused on one of the three main foci identified previously. For example, the assessment item about reservoirs is designed to have students identify sources of water and energy usage, then explain how these usages may be connected (Identifying sources of FEW, Connections between FEW systems). Items were structured to contain several sub-parts or prompts to better elicit student thinking, each of which was designed to assess a specific construct. For example, in Table 1 , parts A and B of the “Sources of FEW & Connections: Reservoir” item was designed to assess student's ability to identify relevant sources of energy and water resources, while the last sub-question assesses how students understand connections between these sources. Thus, many of these items are multi-dimensional, as they require students to integrate disciplinary knowledge and crosscutting concepts.

Data collection

Higher education institutions were invited to participate in this research from the existing connections of the PIs and via an email to the Association of Environmental Studies and Sciences Listserv. Ten institutions were purposefully selected to represent the three primary categories of 4-year colleges according to the Carnegie Classifications of Institutions of Higher Education (Carnegie Foundation for the Advancement of Teaching, 2011 ) and the three approaches to IES curriculum design outlined by a representative survey of higher education institutions (see Vincent et al., 2013 for further description of the three curricular designs in IES program). The IES program curriculum research conducted by the NCSE found statistical alignment of all undergraduate degree programs in a large, nationally representative sample with one of the three broad approaches to curriculum design (Vincent et al., 2013 ). Our sample includes representation from baccalaureate colleges (4), master’s college and universities (3), and doctoral/research universities (4) and programs/tracks representative of the three approaches to curriculum design—emphasis on natural systems (7), emphasis on societal systems (6), and emphasis on solutions development (4). By selecting programs that represent different types of four-year institutions and the three empirically determined curriculum design approaches, we ensured the inclusion of course materials representative of the diversity of the IES field. We focus on four-year programs for the development of the NGCI due to resource constraints and the lack of equivalent research on community college IES curriculum design that would allow us to select representative programs. Additionally, community college IES degree programs are designed to either articulate with 4-year degree programs or to prepare students for immediate employment (Vincent et al., 2013 ).

Student responses ( n = 698) were collected from introductory IES courses during Fall and Spring semesters from Spring 2022 through Spring 2023 by having students complete the assessment questions pre- and post-course discussion of the FEW Nexus. Demographic information revealed 57.45% identified as female, 4% as non-binary, and the remaining 38.55% as male. Racial and ethnic identities reported were 73.67% white, 5.3% Asian, 4.7% Hispanic/Latino/latinX, 1.78% black or african american, 1.38% american indian or Alaskan native and a majority choosing more than one identity (11.79%). We then added the items in a Qualtrics survey and administered the survey to over 400 IES undergraduates from seven post-secondary institutions across the United States to collect student responses (UNCO IRB#158867-1). Responses were then de-identified for coding to create training and testing data for machine learning.

We surveyed the eight IES instructors who had surveyed their students about the pilot items to collect content validity evidence and feedback on question structure. During this survey instructors were asked to both respond to the question item as if they were a student completing the assignment and then, in a separate survey, instructors were asked questions about the question items in the context of their courses. Instructors indicated that assessment phenomena (e.g., food vs. energy production, and energy flows) were typically covered in their introductory IES courses and the multi-part question structure was accessible to student learners.

Rubric development

Rubric development began by reviewing examples of previously published rubrics that were used in similar assessments and intended for use with automatic scoring (Jescovitch et al., 2021 ; Sripathi et al., 2023 ). We agreed upon a scale that would best represent the students’ varying levels of knowledge (Table 2 ). We created each rubric by first analyzing the range of student answers we had received from the different participating institutions. During this initial review, we used an inductive approach and read student answers to identify common themes that revealed student knowledge regarding food, energy, and water systems and their relationships to each other, to other natural and to human systems. During this process we also re-examined our assessment items and the intended goal(s) of the item and also reviewed instructor responses as examples of “expert responses” and alignment with what types of responses students were providing. This was to ensure that students understood the questions the way we intended and to determine if our questions and, therefore, rubrics would need further alterations. To fully capture students’ knowledge, the majority of the NGCI questions needed to have separate rubrics for each sub-question, i.e., sub-questions A–C would each have their own rubrics. At this stage in rubric development, we relied on the previously acquired instructor responses to define an expert-level answer. Instructor responses were similar, and where there were divergences we identified commonalities across responses. We then compared instructor responses to student responses to create a range of scores reflecting novice to expert knowledge.

We designed dichotomous, analytic rubrics with parallel structures for each node of the FEW Nexus. Each response is categorized based on the ideas it contains, with each response receiving a zero or one score for each code based on the presence or absence of the targeted ideas. We provide an example rubric for parts of the Trade-offs of FEW Systems: Biomass Energy Production question item (hereafter referred to as “Biomass question item”) in Table 2 , and the other rubrics are available in the Supplemental Methods file.

To determine the level of expertise a student displayed in their response, we defined a certain combination of bins to receive a holistic score of one through four (Table 3 ). For example, the following student response to Biomass Part A is considered an expert level response (coded as 4) because it contains the following ideas: water usage will increase (bin A) and there will be changes to the local river (bin D). The student therefore makes two connections between energy and water (water usage increasing, impacts to the river).

While the turn to biomass is a more sustainable option, the use of fresh water is going to increase drastically to be able to sustain such a change to the energy source. More than likely the local river will have drastic impacts from such dependency upon it especially if it is a dry season for rain .

Human coding

After the development of the initial rubrics, we iteratively refined the rubrics over several rounds of human coding. During each iteration, two or three researchers separately assigned scores to a set of 30 randomly selected student responses. Each student response received a 0 or 1 for each bin in the rubric for the absence or presence of the corresponding theme in the response. After scoring the set of 30 responses separately, the researchers compared assigned scores and calculated percent agreement. A percent agreement of at least 85% per bin was considered the acceptable level of agreement between human coders to move forward with coding the rest of the dataset independently (80% agreement is acceptable per Hartmann, 1977 ). The scorers met to discuss agreement for each code; in cases of high percent disagreement, the rubric was revised to improve clarity on those codes. During these discussions, decisions about removing or revising codes with very low agreement or low frequency in the dataset were also made. For example, a reservoir code, “Energy needed for food production or irrigation” originally lacked clarification. It was then further described for coders with the addition of, “Irrigation minimum: POWERING the transport/pumping of water, but not implied movement of water without tying to energy. When to code with machinery: machinery + either harvest, produce, or process food.” Specificities like this helped improve coder agreement.

After revising the rubric, a new sample of 30 student responses was compiled, which were independently scored by two to three researchers against the bins with previously high disagreement. This iteration of separate scoring, calculating percent agreement, and revising the rubric continued until scorers either reached 85% agreement for each code in the rubric or resolved remaining disagreements through discussion until consensus was reached (5 iterations for the Biomass question item and 4 iterations for the Reservoir question item). After reaching a consensus for the rubric, all student responses were divided between two members of the research team and were scored independently. A total of 346 responses were scored for the Reservoir question item and 483 responses for the Biomass question item (Supplemental Tables 5 and 6 , respectively).

Text classification model development

We employed a supervised ML text classification approach to assign student written responses a score. During our ML process, each individual student response was treated as a document and the bins in each scoring rubric were treated as classes (Aggarwal and Zhai, 2012 ). The predicted output of a ML model is a dichotomous outcome of whether a response would be categorized in each rubric bin or not. We decided to combine student responses for both parts of the Sources of FEW & Connections: Reservoir question item (hereafter referred to as “Reservoir question item”) into a single text response for text classification model development for two reasons. First, the final coding rubric for each part of the question was identical, although certain ideas/bins were expected to be more frequent in one part than the other. Second, the human coding team adopted a similar approach when assigning codes: regardless of in which response part the student included the idea, the human coders marked the code as “present” for the response as a whole. For the Biomass question item, student responses to each part of the question were kept separated during model development since different parts have different coding rubrics (see Table 2 ).

Text features (single or strings of words) were extracted as n-grams from each response using NLP methods. We used a default set of extraction settings and processing, including stemming, stop word removal, and number removal, to generate a set of text n-grams. The computerized scoring system then generated predictions on whether each given document was a member of each class (i.e., rubric bin) using the extracted n-grams in a bag-of-words approach as input variables in a series of ML classification algorithms. To generate these predictions, we used an ensemble of eight individual machine-learning algorithms (Jurka et al., 2013 ) to score responses to each question. The predictions of the set of individual algorithms are then combined to produce a single class membership prediction for each response and rubric bin. The text classification, including the ensemble ML model, was generated using a 10-fold cross-validation approach using the Constructed Response Classifier (CRC) tool (Noyes et al., 2020 ). The CRC has been used previously to score short, concept-based CR even in complex disciplinary contexts and is described in more detail elsewhere (Jescovitch et al., 2021 ). For evaluation, we compared the machine-predicted score from the ensemble for each response in each rubric category to the human-assigned score for each response.

For each of the models developed in this study, we optimized model performance based on the training set by starting with a default set of extraction parameters, then adjusted several other common model parameters (e.g., n-gram length, digit removal) and retrained classification models to evaluate model performance. This is what we describe as exploratory, basic feature engineering, and we applied a similar approach to every model for each rubric bin. We used the human-coded data for Reservoir and Biomass questions, and we removed several responses with a missing value for a human-assigned score. We used 345 coded student responses for the Reservoir question item, 480 for the Biomass question item Part A, and 466 for Part B as our initial training and testing sets. During this and further iterative rounds, we used common benchmarks of Cohen’s kappa as our targets (kappa > 0.6 as substantial; kappa > 0.8 as “almost perfect” (Nehm et al., 2012 ). Cohen’s kappa is a measure of agreement between raters (in this case, human and machine) that takes into account chance agreement and is frequently reported in evaluating the overall performance of ML applications to science assessments (Zhai et al., 2021b ). We further considered evaluation metrics of accuracy, sensitivity, specificity, F1 score and Cohen’s kappa to guide iterations of model development and to evaluate the overall performance of models once a benchmark was achieved (Rupp, 2018 ). It is noted that while Cohen’s kappa serves as the primary metric for reporting our model’s overall performance, we also routinely consider other evaluation metrics during model building and evaluation. The assessment of these metrics should not be construed as an all-encompassing validation, as their effectiveness is contingent upon the distribution of scores assigned by humans and the quality of those human scores (Williamson et al., 2012 ). In our specific context, we encountered a challenge with the disproportionate representation of certain score points, particularly in some specific analytic rubric bins where cases scored as 1 were significantly fewer than 0. Such low cases of positive occurrences in the training set led to decreased sensitivity metrics for those rubric bins. In some cases (e.g. Reservoir B3), the overall model still exhibited an acceptable overall performance metric and an acceptable F1 score.

Analysis of model outputs and iterative model development

After performing the initial model development and examining the basic feature turning settings, we examined the outputs of the model for low-performing rubric bins, including model evaluation metrics and groups of responses that showed disagreement between human and machine-assigned scores. We hoped to find possible ways to adjust the model parameters and/or training set of data to improve model performance in subsequent iterations. For example, we collected responses with disagreement in assigned human and machine scores. We examined the false negative and false positive predicted responses (compared to the human coding) in a rubric bin and performed conventional content analysis to try to identify words, phrases, or ideas that were common among these misscored responses (Hsieh and Shannon, 2005 ). We also reexamined the criteria of coding rubrics with low-performing models to ensure the criteria clearly identify important disciplinary ideas and to confirm the original assigned human codes to responses (Sripathi et al., 2023 ). The coding team met to discuss the results of miscode analysis and changes to target during iterative cycles, including possible changes to the rubric, best approaches to tuning model parameters consistent with assessment items and student ideas, and/or adjusting training sets.

The insufficiency of educational data (Crossley et al., 2016 ; Wang and Troia, 2023 ), which often suffers from limited availability of data for training ML models as compared to other sectors, and the observed lack of diversity in undergraduates’ CR (Jescovitch et al., 2021 ) have long posed challenges for educational researchers. These issues present difficulties for ML algorithms in discerning patterns effectively and reliably identifying a broad range of student ideas. To address these challenges, we have adopted a set of extended model tuning strategies, which have been both theoretically and empirically validated (Bonthu et al., 2023 ; Jescovitch et al., 2021 ; Romero et al., 2008 ). We employed these extended strategies beyond our exploratory, basic parameter tuning (described above). The extended strategies we employed are:

Additional feature engineering

In certain instances, we implemented two advanced feature engineering techniques, often arising to address patterns identified during our miscode analysis. These techniques encompassed (1) substituting specific words with synonyms and (2) extending N-gram analysis to more complex levels, including trigrams (three words combined into one feature) and quadgrams (four words combined into one feature).

Data rebalancing

Training sets that heavily represent only certain types of responses can impede model training; therefore, we applied data rebalancing strategies to address situations where the dichotomous coding significantly favored one category (over three times). When our dataset exhibited such imbalances, we implemented data rebalancing techniques by removing responses associated with the most frequently occurring codes to achieve a more equal distribution of the dichotomous codes. In our data set, cases coded as 0 often outnumbered those coded as 1. Since cases coded as 0 sometimes failed to provide meaningful patterns for ML algorithms to learn from, we selectively removed excess cases coded as 0 to equalize or enhance the distribution (i.e., reducing the ratio to equal to or less than two times difference).

Dummy responses

For datasets characterized by a balanced distribution of dichotomous scoring codes, yet still yielding low performance metrics, another extended strategy was devised. In this strategy, we initially ensured dataset balance, saved cases with human rater scores, and ML-predicted scores and outputs of the CRC tool after the initial round of analysis. Subsequently, we filtered out responses that were incorrectly classified, identified by a misalignment between human and ML predicted scores. These misclassified cases underwent further qualitative examination, with notes indicating which phrases and segments included in (or absent from) the response were indicative of the critical concept targeted by the rubric. We then generated new cases (i.e., dummy responses), which only replaced the identified segments of responses with new words or phrases, without altering the sentence’s underlying meaning. This procedure offers advantages, including mitigating overfitting concerns in which the model is only effective on responses very similar to the training set and augmenting the training dataset’s size. The dummy responses were integrated into the overall dataset solely for model training purposes. To derive the final performance metrics of the classifier model, the dummy responses were subsequently removed for model evaluation calculations.

Merging rubric bins

In some instances, despite the explicit indication in the original rubric descriptors that certain ideas are intended to be scored separately as they are designed as mutually exclusive during rubric development, some machine models faced challenges in effectively identifying these subtle textual patterns. Collaborative discussions with expert raters led to a consensus among researchers to combine these rubric bins. This decision was informed by empirical investigation revealing overlapping content, and the re-coding of these bins to a single code / score to enhance the model’s performance, aligning with practical considerations in the procedure.

It is important to note that these strategies can be combined or used consecutively as needed. Nevertheless, the initial round of analysis consistently adhered to the default and basic settings of the CRC tool, utilizing the parameter options provided therein. Further details on the application of these approaches to individual items and rubric bins, along with illustrative examples of dummy response creation, can be found in the supplementary materials.

Here, we report on the use of ML-based text classification models to assess CR questions focused on the FEW Nexus. This section is organized by the research question, beginning by describing the successes and challenges in applying ML to score student CR to questions about sources of FEW resources and trade-offs associated with biomass energy production. We then examine the two questions related to reservoirs and biomass to describe FEW connections in student CR and co-occurrences across responses to understand student system thinking capacities. Co-occurrence suggests evidence of systems thinking as multiple FEW systems are interacting simultaneously in student responses.

Research Question (1): can natural language processing be used to identify instructor-determined important concepts in student responses?

We developed a total of 11 text classification models for the Reservoir item, one each for the 11 “bins” contained in the coding rubric (Table 1 ). These eleven models had a range of overall performance metrics (Table 4 ), ranging from Cohen’s kappa of 0 to 0.957 and accuracies ranging from 0.892 to 0.992. Only one model (D2) failed to detect positive cases, which resulted in an overall Cohen’s kappa = 0.000. This was due to a severe data imbalance in the human-assigned codes in this rubric bin, meaning that there were very few positive codes to responses assigned by humans in this bin. All other ten models met acceptable performance levels as measured by Cohen’s Kappa values (kappa > 0.6 as substantial; kappa > 0.8 as “almost perfect” (Nehm et al., 2012 )), with many models exhibiting “almost perfect” agreement with human assigned codes. We note that most models were tuned to this performance using only basic feature engineering manipulations, as described in the methods. There were also a few bins that met our target threshold of 0.6 only after employing extended strategies (e.g., employing dummy responses for A2), and for one bin (B4), we employed data rebalancing in tuning the model. The model for D2 showed high accuracy but decreased performance on other model metrics due to a severe imbalance of human code occurrence.

One result that emerged from discussions during iterative model development for the Reservoir question item was the similarity of codes A2 (producing hydropower) and B4 (energy transformations). Although we were successful in developing text classification for each code separately, the two models did require slightly different tuning strategies. When examining miscoded responses, the coding team noticed similar patterns in the groups of correctly and miscoded responses for each bin. Human coders reflected that during the coding of the responses, students expressed these ideas similarly, and it was, therefore, sometimes difficult to distinguish when students were explaining hydropower versus describing transformations of energy (e.g., moving water turning turbines). Thus, these two codes (A2, B4), which were initially intended to capture a specific understanding of hydropower and a more general description of energy transformations, ended up being more similar than intended in the context of this item. One potential way forward for text classification is to combine the A2 and B4 codes into a single code and redevelop a text classification model to recognize the single code.

We developed 15 text classification models (eight models for Part A; seven models for Part B) to detect student ideas in response to the Biomass question item (Table 5 ). Overall, for this item, models demonstrated lower performance metrics than models for the Reservoir question item. For the Biomass question item, no model achieved a level of almost perfect agreement (as measured by Cohen’s kappa value of >0.8), although the majority still achieved acceptable agreement with human scores. Due to the reduced maximal performance, these fifteen models had a narrower range of overall performance metrics than models for the Reservoir question item, ranging from Cohen’s kappa of 0–0.674 and accuracies ranging from 0.755 to 0.991. Correspondingly, these Biomass models had a much broader range of sensitivity, specificity, and F1 score metrics too. This reduction in performance metrics is likely due in part to the target of the Biomass item: trade-offs around FEW. Although the item still centers on the FEW Nexus, this item allows students to respond in numerous ways about any number of possible trade-offs between any of the vertices. Thus, this item allows for a much wider possible answer space. As a result, a few models failed to reach the benchmark performance metrics (e.g., B2 in Part A), despite having frequent occurrences of both codes. This also suggests that the text complexity of expressing these ideas or the range of possible ideas in these responses is difficult for these text classification models to reliably identify. Although we attempted extended strategies on models for many of the Biomass models, we report on a few of the bins and attempts as exemplars of this work, or findings that were similar between different bins. We provide more detail on applied strategies for each model in the Supplemental Materials.

The model for B3 code in Biomass question item Part A showed very low-performance metrics despite having a fair number of positive cases. The poor model performance is likely reflective of the range of student ideas covered by this rubric bin: a decrease in water availability for other uses (here, “other uses” means outside the context of bean and corn agriculture given in the question). As such, there is a wide range of possible other uses students could suggest, such as drinking water, home water use, and water for other crops. The broad range of acceptable answers was easy for humans to code, but difficult for the model to detect the underlying similarity. Although we tried some extended strategies for model iterations, these had little effect on overall model performance. During iterative rounds of model development, we decided to merge two codes, B3 and B4, since they both identified similar ideas, about less water available for other things and changes in human behavior due to less water. During our review of miscoded responses by the model, we noticed a number of miscoded responses were somewhat borderline cases of human code assignment between the two bins B3 and B4, with responses often implying or vaguely mentioning effects on community usage of water, without being explicit the change in use or behavior. For example, the response, “Since this is a place of limited rainfall, and the source of water is coming from the river I would expect that water use for the community may need to be diverted more towards the crops, and less towards other measures such as household use.” was coded positive for B3 by human coders but miscoded as missing B3 by the model. After merging these two codes into a single code and model, the performance of the overall model for the merged code was significantly improved for B3 and slightly decreased for B4 (see Table 5 ). After merging these bins into a single model, borderline responses, such as the example, were correctly classified by the model.

Similarly, the initial classification model for C1 in Biomass question item Part B failed to meet performance benchmarks even though student responses were nearly equally distributed between positive and negative cases, and we tried several extended strategies to improve model performance. However, the re-examination of coding rubrics for C1 and C2 presented an opportunity to recombine coding criteria as part of the iterative process of using model outputs to iterate on items and rubrics. The rubric was originally designed to identify student ideas about the production of energy (C1), but not when used in conjunction with trade-offs with other energy sources or energy return on investment (C2). After several rounds of model iteration and discussion with the coding team, we decided to recode the original dichotomous rubric bins C1 and C2 as a single, multi-class code (i.e., a holistic coding rubric, with levels as 0, 1, or 2). This preserved the exclusivity of these two codes (C1 and C2 were intended to be mutually exclusive) while encoding the exclusive classes in the model training set. Making this a single, multi-class prediction increased the overall performance of the model, above the performance for the separate, binary models made for the original rubrics.

Research Question (2): what do our students know about the interconnections between food, energy and water, and how have students assimilated “systems thinking” into their constructed responses about FEW?

Here we apply two different strategies for defining and evaluating student responses as novice to expert. To evaluate student knowledge about interconnections and how they have assimilated “systems thinking” into their constructed responses about FEW, we calculated the co-occurrence of codes. Level of expertise for the Reservoir question item is approximated by co-occurrence of the codes, and level of expertise for the Biomass question item is calculated by the code combination provided in Table 3 .

Sources of FEW and connections: reservoir question item

We examined the predicted codes for each response to the Reservoir item to look for co-occurrence of codes in student responses. This can help identify connections students are making between FEW vertices, since the item prompts students to make these connections. For this analysis, we collapsed individual bins in Table 6 for the Reservoir rubric by grouping letter codes (e.g., A1 and A2 together as A bin), since these groupings indicate similar themes (A codes refer to hydroelectricity, B codes refer to energy production, C codes refer to use of energy; Table 1 in Supplemental Methods).

Responses frequently included ideas from A codes with ideas from C codes, indicating the same response connected generating hydropower to uses of energy for agriculture or infrastructure. The C codes also commonly occurred with the B codes, showing students explained connections between types of energy and uses of energy in agriculture or community resource use. D codes (uses of water) were the least frequently coded; however, when D was coded, these responses were very frequently connected to hydropower (A codes). Co-occurrence within A codes and D codes suggests that students understand that hydropower is powered by water and is needed to create electricity. A–C codes were the most likely to occur together when students were making connections between FEW systems (59 responses). Only 12 students made connections between A–D codes, suggesting that water use beyond hydropower is not as commonly associated with energy use and production in this scenario despite water providing the primary source of energy in the reservoir.

Co-occurrence is how we can approximate the level of understanding of the respondee from novice to expert for the Reservoir question item. The assumption is that the quantity of co-occurrences indicates students have an understanding that there is some sort of connection between Food, Energy, and Water. For example, student responses could be coded in a number of bins regarding the type of energy, and what the energy is used for, e.g., irrigation or powering homes. We assume that students’ answers indicating a greater understanding of the relationships between Food, Energy, and Water will include bin codes for hydroelectricity, irrigation for food, energy for machinery, and energy for housing/farm bins (Table 7 ). Novice responses show they know that the dam is used to create hydropower, but they do not have any further knowledge about how this energy can be used and how it relates to food (Table 7 ).

Trade-offs systems: biomass energy production question item

Overall, students perform at a higher level for explaining changing water usage (Part A) than discussing trade-offs (Part B) (Table 8 ). The large majority of students discuss at least one trade-off in their response for Part B and, therefore, are placed in level 1 or higher (see Table 9 for an example of student responses). Due to the ML model performance for this question item, we have also included the number of responses for each Novice to Expert Level as Supplemental Methods Table 9 .

For the Biomass question item Part A, slightly over half of the responses scored a level 3, with over 20% as Level 4 and about 23% as Level 1, and no level 2 responses (example responses provided in Table 9 ). For Part B, about half of the responses were grouped in level 2 and roughly 20% in level 1; both of these levels) had similar numbers of responses in those levels by ML and human-assigned codes. A small percentage of responses (~11%) were placed in level 3 by the ML model, while human codes had slightly more responses (13.5%) in that level. There were no student responses predicted for level 4 by the ML model and only one response in that level based on human-assigned codes.

The lack of level 2 responses for Part A is due to having only one positive ML predicted for the C component. However, this response was scored to level 3 response because the student response also included one of the other codes. Since this was a poor-performing ML model for the C code (meaning that the model did not recognize any responses for this code), we explored using human scores for this code; even so, only three responses from the data set end up at level 2. Most responses in the dataset which are categorized in code C end up at levels 3 and 4, since these responses tend to incorporate water price increase as an effect of increased water use or water scarcity within their explanation (Table 9 provides an example student response).

For Biomass Part B, we found no level 4 responses in our data set, which was driven by the lack of ML predictions for category A2, which is a requirement for obtaining this level. A2 is an infrequent category in the dataset with only 4 positive cases assigned by human coders. Even when we explored using human scores in place of ML-predicted scores for this specific rubric bin, we observed only a single response in Level 4. About one-third of the student responses score at Level 2, which demonstrates an ability to connect at least two FEW vertices when discussing trade-offs. The largest group of students (~40%) end up at Level 1, which is a trade-off focused on a single vertice of the nexus (food, energy, or water).

Of the 138 responses that do not fit the other patterns in Part A and the 77 responses categorized as Level 0 in Part B, most were a combination of derivations of “I don’t know” or trivial responses such as “it will go up” or “You need all food, energy, and water in this situation.” However, there were also responses that the ML model did not predict any expertise level, but would be considered one of the expertise levels by human coders. For example, this student's response that was not predicted to achieve an expertise level includes concepts that occurred infrequently and, as such, was not provided a code—reduced water availability means that water would need to come from someplace else, and require more labor and cost for transportation:

“A shift from agriculture to biomass production means the community will need to pay for excess water. If there is very minimal rainfall during a year, the community will need to gain a water supply from the surrounding neighborhoods. Buying water, transporting it, and ensuring the corn is watered requires extra labor, which requires extra pay.”

The application of ML for assessing interdisciplinary learning involves both the development of the process as well as using that process to understand student thinking and learning. The ML process here shows promise for use in evaluating complex constructed responses for systems thinking, especially as part of formative assessment practice, and we also report on the evaluation itself. Here we discuss findings in the context of our research questions and results, including limitations pertaining to each topic within each section.

Use of ML to uncover student understanding of FEW Nexus

Considerations for future assessments of student CRs, particularly in the context of science-related items, demand significant attention. Despite the relative success of current applications, there are remaining challenges to using ML approaches to score a broader range of assessment constructs and response types (Zhai et al., 2020a ). These challenges can be characterized by limitations such as insufficient data (or specific types of responses/ideas in CR), subjectivity, imbalances, and the prevalence of noise, and these all present substantial obstacles within the iterative ML training process (Maestrales et al., 2021 ). These challenges, if not effectively addressed, have the potential to compromise the achievement of optimal model accuracy, thereby raising questions about the validity and reliability of ML applications in educational evaluation settings (Suresh and Guttag, 2021 ). Another challenge is the complexity of the assessment target (i.e., what you are trying to measure), and the complexity of expected student responses can pose challenges to such AI-based evaluation (Zhai et al., 2020a ). Others have suggested that features of the assessment item itself, such as the subject domain or scenarios used in the assessment, might impact the accuracy of ML models (Lottridge et al., 2018 ; Zhai et al., 2021b ). To address these challenges, we utilized automated scoring approaches for text classification, which examine complex systems integration. In this study, our technical strategies have introduced a practical solution through data augmentation to help address insufficient data and data imbalance, yielding promising implications. This approach involves generating dummy responses that are subsequently revised with identified synonym sets, thus facilitating the measurement of responses with similar structures and content while preserving the overall meaning and essence. Notably, we found that this approach effectively improved model performance, particularly when dealing with specific descriptors in Reservoir and Biomass question items.

One complexity of assessing complex CRs in postsecondary education is that there are varying disciplinary requirements and usages of student literacy compared to the more consistent expectations of K-12 education. As such, student responses considered holistically consist of a range of literacy abilities, which can impact the “understanding” of natural language processing and text classification models. For this research, student responses were collected from across the United States at different institution types (baccalaureate colleges, master’s colleges, and doctoral universities) to provide a wider range of student responses from which to develop the ML models. The resulting models are thus trained on the many ways that people may write about the question item concepts. High variation in the responses, which can be the result of variation in literacy, language, and understanding, result in more complexity, and are thus more difficult items for model development. Some of this difficulty may be addressed with a larger sample size, but if student responses are too varied or certain types of responses are too infrequent in the sample, then accurate ML models may not be easily achievable. Further, although we refer to the scoring of responses into rubric bins, we posit another important outcome of this work is characterizing students' thinking about FEW concepts. The inclusion of automated text scoring systems into formative assessment evaluation isn’t only for “scoring” but provides a way for instructors to use open-response items and identify complex student ideas, or potential barriers to student learning (Harris et al., 2023 ). This is a critical aspect of formative assessment practice, allowing instructors a richer, more nuanced view of how students’ think about complex systems like the FEW nexus.

Defining criteria for developing text classification models

During the course of our iterative process, models exhibited superior performance in certain rubric categories characterized by well-defined criteria and a robust explanatory framework outlining the expected content under each rubric category. This finding aligns with prior research that underscored the efficacy of ML algorithms in successfully discerning the quality of student responses using fine-grained analytic scoring methodologies ( Ariely et al., 2023 ) . Conversely, challenges become apparent in scenarios where substantial overlap exists between rubric categories, leading to redundancy and a lack of clarity (Liu et al., 2014 ). In such instances, the Kappa value frequently falls short of the desired threshold (Zhai et al., 2021). These insightful observations underscore the imperative need for the refinement of rubric definitions within future assessments. This refinement should be guided by a comprehensive and quantitative delineation of assessment criteria, aimed at mitigating the issues of overlap and ambiguity that our study and prior research have duly highlighted. For example, we revised closely related yet exclusive rubric bins to a single, multi-class prediction after attempting multiple model improvement strategies, yet failing to meet threshold performance metrics. Changing the structure of the rubric maintained the coding criteria of individual bins, now as “levels”, but provided additional information about exclusivity which resulted in better overall model performance. Alternatively, other coding bins with overlapping criteria or developed with too fine-grained of categories than needed to differentiate student ideas, can be merged into a single code. Conversely, other rubric codes that are too broad initially may need to be split or have better-defined coding criteria to better categorize cases (Sripathi et al., 2023 ).

We also note different levels of successful performance metrics for text classification models for the Reservoir versus Biomass question items. Indeed, most models for the Reservoir question item rubric bins achieved very good performance (i.e., “almost perfect” Cohen’s kappa measures), but most models for the Biomass question item rubric bins achieved only “acceptable” performance. This is despite both assessment items being in the Environmental Science domain, being centered on the FEW Nexus as context, and undergoing similar iterations in ML development. We interpret these findings to provide further evidence that the underlying construct of the assessment items and/or the expected complexity in student response can influence ML model performance, as noted by others ( Haudek and Zhai, 2023 ; Lottridge et al., 2018 ) . Thus, a practical implication of this work is that more complex assessment targets (e.g., trade-offs in socio-ecological systems), or assessment items that encompass larger systems will need additional feature engineering or more advanced ML techniques for accurate response evaluation ( Wiley et al., 2017 ; Zhai et al., 2020a ) . Further, this highlights the need for an iterative approach in these research efforts. Although we lay out our approach as a “cycle” (see Fig. 1), in practice, it is highly iterative, with results from all stages informing the work of other stages, often in feedback loops. To improve final model outcomes, all stages of item development, data collection, and rubric alignment should be revisited, not only tuning specific model features. Following principled item design procedures (e.g. Harris et al., 2019 ) and incorporating automated scoring systems into the methodological pipeline (Rupp, 2018 ) are important considerations. Nevertheless, successful item/rubric/model development often takes multiple iterative rounds, which we continue to do, and models should be updated and expanded.

Such challenges to using NLP for short answer scoring are well reported and exist for assessments across science domains (Shermis, 2015 ; Liu et al., 2016 ). This leads to a broad range of scoring model performances (see Zhai et al., 2021b ). These iterative cycles of revision do require an investment of human effort with an outcome of having automated classification models that can predict categories for any number of new responses and for any number of new users. Further, researchers also learn about student thinking about the targeted key concepts (see section “Student understanding of systems thinking in the FEW nexus” below) as they work to design items, rubrics, and models (e.g. Sripathi et al., 2023 ).

Scoring novice to expert levels

Scoring through levels [Level 1–Level 4] allows us to see the real distribution of knowledge for students in introductory courses. Level 4 responses were least frequent, most likely due to this level’s creation being based on an instructor’s expert response. Although, level 4 responses were seldom seen in students, it allows us to set a growth goal and see students who have previous knowledge at the expert level. As seen previously, only one student was able to achieve that level, which suggests that the task at hand is indicative of student ability. The level 4 response level is a baseline for exemplary understanding, it can also be used in the future to see if senior level or graduate students are performing at the expected level or to evaluate different strategies for achieving higher learning outcomes. Another We did have one student be within the level, which suggests that it is possible that students can strive to that level at a beginning level course. In addition, this supports student learning and growth, as we can expect as learning improves the FEW system understanding and could be a good baseline for growth as more students learn better. Additionally, no students fell within the pre-established Level 2 for responses that only included C codes (responses that only addressed a change in water prices), however students who were rated in Level 3 and Level 4 did include that content in their response. This particular content seems to be closely connected with the higher level responses rather than being a piece of information distinct from the other content and future research may delve into the content mapping of student responses.

We report on using the computer predicted scores to place student response in expertise levels. Overall, the computer placement may slightly underpredict student performance on these items as compared to human assignment, especially in the mid-level. This is more notable in the Biomass item, especially for Part A, indicating that the difficulty of this item may affect level assignment. However, although the classifications result from individual models with varying degrees of accuracy, the overall distribution of responses across all levels approximates the distribution from human assigned placements (see Table 9 of the Supplemental Methods for human assigned placements as comparison with the ML predictions of Table 8 ). This supports the use of these automated classification models to evaluate group or large class performance as part of formative assessment practice, even though individual response placement in specific levels may vary. That is, a reasonable approximation of the distribution of a large number of responses collected in an introductory course can be generated in seconds to minutes using the developed classification models, as opposed to the effort of human reading and assigning levels to all collected responses.

Future prospects of generative AI

The recent advancements in generative AI have raised additional considerations about assessment in education, among a host of many different possible applications (Kasneci et al., 2023 ). Although many of these issues are common to uses of classroom assessment in many contexts, some issues are particularly overlapping with the process of assessment development and automated scoring presented here. Recent explorations in using large language models for automated scoring of essays and short responses show great promise (e.g. Cochran et al., 2023 ; Mizumoto and Eguchi, 2023 ; Latif and Zhai, 2024 ). Using such an approach would simplify and expedite the automated scoring process, thus permitting automated scoring for different assessment prompts (Weegar and Idestam-Almquist, 2024 ) and could contribute to generalizability of models (Mayfield and Black, 2020 ). One promising application of generative AI is to do pattern finding,contextualized representation of information, and clustering of collections of student responses to open-ended tasks in support of formative assessment practice (Wang et al., in review ; Wulff et al., 2022 ). This may assist instructors to easily find patterns and capture token-level representations in student responses based on the linguistic context, thus allowing them to attend to student ideas and thinking as exhibited in their classroom, without reading and sorting individual responses.

On the other hand, the use of generative AI in education raises many concerns about academic integrity and students easily finding or asking AI to generate answers to assessments (Chan, 2023 ). Some studies have found that generative AI models still perform less well for producing more complex assessment tasks and tend to do better on quantitative tasks as compared to explanatory (Nguyen Thanh et al., 2023 ). Additionally, regarding the language attributes, current AI-generated responses, when compared to human-authored counterparts, typically manifest a discernible deficiency in cohesive and coherent elements, accompanied by a writing style characterized by uniformity and repetition (Wang et al., 2023 ). It is very likely these shortcomings of these AI models will not last long. Instead, educators should re-evaluate the purposes of assessment (Chan, 2023 ), including how and what content and practices are necessary for students to be “skilled” in a discipline. Therefore, focusing teaching and learning on foundational principles within the discipline, which allows students to see science across contexts and define problem boundaries, like systems and systems models, maybe one such approach. Educators should also consider the purpose learning activities that students engage with, both in the classroom and outside of the classroom. The application of generative AI represents a frontier in the use of technology in support of formative assessment in the classroom (Harris et al., 2023 ).

Student understanding of systems thinking in the FEW Nexus

Systems thinking involves understanding the interdisciplinary connections and relationships between associated components within a system, rather than simply focusing on discrete concepts (Meadows, 2008 ). For teaching and learning contexts, the FEW Nexus provides a scaffold for incorporating systems thinking and sustainability concepts into courses and across curricula. A primary advantage of the NGCI is the potential to capture a student’s understanding of relationships within the FEW Nexus. While the analytic rubrics were developed to score student understanding of FEW isolated discrete parts of the systems in the scenario presented by each item, by examining the constellation of scores a student response achieved across criteria, we quantified student patterns of explanations about these systems.

What our students know about the Food–Energy–Water Nexus

Both the Reservoir and the Biomass question items present students with scenarios about the FEW Nexus relationship centering water with connections to energy production and agriculture. The codes described in the analytic rubric represent the most common concepts students included when presented with these scenarios. The frequency of these concepts may indicate that these ideas are foundational as introductory students construct knowledge about FEW systems. Many of the most common codes could be classified as demonstrating basic knowledge, which is the simplest cognitive task presented in Bloom’s taxonomy model (Bloom and Krathwohl, 1956 ; Krathwohl, 2002 ). For example, in the Reservoir question item rubric, the A codes were the most commonly found in our dataset (Table 6 ), and indicated responses identifying that a dam could be related to hydropower. While this type of statement is reasonable for introductory level courses where students are developing new understanding and aligns with the content presented in introductory IES courses (Horne et al., 2023 ), knowledge statements alone do not achieve the competency goals for IES students (Wiek et al., 2011 ). More complex student responses in our study contained combinations of codes, however exceptionally creative explanations or concepts were not always frequent enough to be included in the analytic rubric or be captured reliably in the ML models.

The Biomass item presents students with an opportunity to consider directionality within trade-offs, and directionality concepts are thus frequent in the associated rubric. In the Biomass sub-questions, students often included at least one statement about directionality of the quantity of food, energy, or water, but responses including predictions across these three ideas were infrequent. Making a statement about change or directionality, such as describing the quantity of food or water decreasing, is a relatively simple task in systems thinking, but is foundational to more complex tasks that consider changes over time (Sweeney and Sterman, 2007 ). Students who described trade-offs in their responses to this question sometimes went beyond discussing the cause-and-effect components of the system and discussed concepts not immediately asked by the question, such as the impact of this scenario on water pricing. However, these types of responses did not always register in the ML models, and some were too infrequent to be included in the rubric. The frequency of simpler codes describing FEW concepts in comparison to codes describing FEW consequences presents a challenge, given that IES curricula prioritizes FEW in relation to socio-environmental topics (Horne et al., 2023 ). The emergence of these concepts in student responses provides insight into what students will need to do with these ideas after the classroom and how students may move from identifying FEW concepts to applying predictions about FEW impacts on people, land, and communities.

How students assimilate systems thinking into their constructed responses

The frequency of co-occurrences in our analysis can serve as a proxy for gauging the level of understanding among respondents, ranging from novice to expert, regarding the relationships between food, energy, and water. The pattern of responses students gave sheds light on the connections between the concepts of food, energy, and water within the context of our study (see Table 6 ). In responses to the Reservoir item, we observed in our data that students frequently combined ideas under A codes (descriptions of hydropower) with C codes (uses of energy). This combination of concepts aligns well with the task presented in the item, and this pattern suggests a moderate association between generating hydropower and its applications in agriculture or infrastructure. Additionally, we noticed a prevalent co-occurrence of connections between C codes and B codes, signifying that students can connect the production of various energy types and their local utilization in agriculture or community resource management. These types of responses represent a robust understanding among students that hydropower is harnessed from water sources and plays a role in electricity generation. When students’ responses were coded into various categories such as the type of energy and its intended purposes (e.g., irrigation or powering homes), we found that responses indicating a more comprehensive understanding tended to include bin codes related to hydroelectricity, energy for irrigation in food production, energy for machinery, and energy for residential or agricultural purposes (see Table 7 ). In contrast, D codes represented facets of student explanations that centered on the use of water , and not necessarily energy, from the reservoir in the prompt. While overall these codes were less frequent than codes describing the use of energy, they were associated with more novice responses that co-occurred with A codes (e.g., stating that hydropower is related to the prompt) but not as frequently with explanations of how energy is produced and used for agriculture. Students commonly linked the concepts of energy generation, energy applications in agriculture, and broader infrastructure. In contrast, novice responses included the basic concept that dams were related to hydropower but lacked further knowledge about how energy is generated, how energy could be employed, or its relevance to food production, and instead offered how water from the reservoir could be used for agricultural purposes (see Table 7 ).

While examining co-occurrences between codes within the Reservoir item explores how students characterize the components of a FEW system, examining co-occurring codes in responses to the Biomass item offers a way to model how students describe trade-offs. The combinations of co-occurring codes reflect the complexity of a students’ response, which serves as the basis for the logic of the Novice to Expert scale (Table 8 ). Without including at least two of the facets of the FEW Nexus, a response to the Biomass item would not describe a trade-off. For example, a Level 1 response to the Biomass item would only include one facet of FEW, while a response including more specific details and more than one FEW element would be more expert-like. Further, moving from describing individual effects to multiple effects may also indicate a student is reasoning about the mechanism behind the system, which is a more expert-like approach to systems thinking (Hmelo-Silver and Pfeffer, 2004 ). However, aligned with previous research in science education indicating the challenge of developing expert-like systems thinking (Hmelo-Silver and Pfeffer, 2004 ; Jacobson and Wilensky, 2006 ; Sweeney and Sterman, 2007 ), expert-like Level 3 and Level 4 responses were infrequent in our dataset compared to responses providing simpler, incomplete explanations of the systems presented in the question.

There is growing support and interest in establishing interdisciplinary environmental education in higher education that integrate concepts and disciplines in addition to providing varied perspectives (Christie et al., 2015 ; Cooke and Vermaire, 2015 ; Wallace and Clark, 2018 ). Most of these IESs do not incorporate systematic evaluation and assessment, and especially non-summative evaluations, with one of the main challenges to developing evaluation being the diversity of content and fields (Vincent et al., 2017 ). There is a need to assess student learning in IESs as well as rigorous evaluation of IES educational practices, especially of complex synthesis concepts. Here, we described initial steps in developing ML text classification models as a tool to assess student systems thinking capabilities using two questions anchored by FEW Nexus phenomena (i.e., water-energy connections, biomass trade-offs). Our two questions are first steps to fulfilling a much-needed gap in educational assessment by providing a means to analyze complex concept integration related to the FEW Nexus using ML. Successes and challenges to ML approaches to scoring student FEW Nexus CR indicate several future research priorities for interdisciplinary, practice-based education research: further development of human scoring methods to specifically prepare training and test data for ML models; developing evaluation systems for student responses on novice to expert scales; developing assessment instruments using multiple CR question items; and examining how students incorporate social competencies and human factors into their explanations of FEW topics. Some of these research priorities address the critical issue of time investment in developing text classification models. Data collection, in the form of hundreds of student responses to the same question, rubric development (an iterative process), human scoring of student responses for training and test data, and model development (also an iterative process), all require a large amount of person-hours. This particular project has included the collaboration of 10 institutions for data collections, as well as two research labs at two additional institutions for scoring and model development with multiple postdoctoral scholars and graduate students. This investment is a severe limitation in the development of such models, and the process information presented here is intended to support other scholars in their model development through in-depth discussion of strategies for model improvement and likely outcomes. However, once a model is developed and has achieved acceptable evaluation metrics, it can be used to very quickly assess large numbers of students’ responses and conduct research on large datasets. This trade-off in investment is also offset by research that makes available resulting models to the scholarly community, as with questions and models presented in this paper (see supplemental information access).

Development of these question items using text classification models and CR assessment items allows evaluation of the relationship between foundational concept understanding and integration of those concepts as well as more nuanced understanding of student comprehension of complex interdisciplinary concepts. This proposed research represents one of the first attempts to assess the links between foundational, discipline-specific concepts and systems thinking and learning. We have been able to engage a range of institutions in all phases of the project thus far. Institutions were chosen as a representative sample of EPs across the US and include baccalaureate colleges (4), master’s colleges (3), and doctoral universities (3). This is critical to ensure that findings and outcomes are applicable to undergraduates across the US. We anticipate that the information gleaned from reviewing environmental curricula across the United States, combined with concept inventory results showing student learning, will better inform those making curricular and staffing decisions regarding college environmental science and studies programs. Thus, students enrolled in IES programs will benefit by having courses and programs evaluated with a valid and reliable instrument. Additionally, combining discipline-specific ideas and phenomena within a new set of CR assessment items focused on complex system thinking will provide faculty with a valid and reliable instrument for evaluating learning. Our instrument development methodology is also applicable to other multidisciplinary assessments. For instance, the Next Generation Science Standards places a strong emphasis on using three-dimensional learning—how science practices, content knowledge, and crosscutting concepts interconnect (Douglas et al., 2020 ). Lastly, environmental and sustainability objectives are becoming commonplace among university mission and vision statements. Providing shared EP objectives with aligned assessments that can inform instruction and student learning helps meet these objectives of undergraduate education.

Data availability

Scored student response data is available through contact with the corresponding author. Source code for the text classification tools used in this study is available at https://github.com/BeyondMultipleChoice/AACRAutoReport . Assessment items are available at https://beyondmultiplechoice.org/ . Text classification models will be saved and published to the public in subsequent papers at https://beyondmultiplechoice.org/ .

Aggarwal CC, Zhai C (eds) (2012) Mining text data. Springer US

Ariely M, Nazaretsky T, Alexandron G (2023) Machine learning and Hebrew NLP for automated assessment of open-ended questions in biology. Int J Artif Intell Educ 33(1):1–34. https://doi.org/10.1007/s40593-021-00283-x

Article Google Scholar

Balint PJ, Stewart RE, Desai A, Walters LC (2011) Wicked environmental problems. Island Press/Center for Resource Economics

Blatti JL, Garcia J, Cave D, Monge F, Cuccinello A, Portillo J, Juarez B, Chan E, Schwebel F (2019) Systems thinking in science education and outreach toward a sustainable future. J Chem Educ 96(12):2852–2862. https://doi.org/10.1021/acs.jchemed.9b00318

Article CAS Google Scholar

Bloom B, Krathwohl D (1956) Taxonomy of educational objectives; the classification of educational goals by a committee of college and university examiners. Handbook I: Cognitive Domain. Longmans, Green, New York, NY

Bonthu S, Rama Sree S, Krishna Prasad MHM (2023) Improving the performance of automatic short answer grading using transfer learning and augmentation. Eng Appl Artif Intell 123:106292. https://doi.org/10.1016/j.engappai.2023.106292

Brandstädter K, Harms U, Großschedl J (2012) Assessing system thinking through different concept-mapping practices. Int J Sci Educ 34(14):2147–2170. https://doi.org/10.1080/09500693.2012.716549

Brew C, Leacock C (2013) Automated short answer scoring: principles and prospects. In: Shermis MD, Burstein J (eds) Handbook of automated essay evaluation. Routledge

Brundiers K, King J, Parnell R, Hiser K (2023) A GCSE proposal statement on key competencies in sustainability: guidance on the accreditation of sustainability and sustainability-related programs in higher education. Global Council for Science and the Environment, p. 40

Bustamante M, Vidueira P, Baker L (2021) Systems thinking and complexity science–informed evaluation frameworks: assessment of the economics of ecosystems and biodiversity for agriculture and food. N Dir Eval 2021(170):81–100

Carnegie Foundation for the Advancement of Teaching (2011) The Carnegie Classification of Institutions of Higher Education, 2010 edition. The Carnegie Classification of Institutions of Higher Education

Chan CKY (2023) A comprehensive AI policy education framework for university teaching and learning. Int J Educ Technol High Educ 20(1):38. https://doi.org/10.1186/s41239-023-00408-3

Chi M, VanLehn K, Litman D, Jordan P (2011) An evaluation of pedagogical tutorial tactics for a natural language tutoring system: a reinforcement learning approach. Int J Artif Intell Educ 21(1–2):83–113. https://doi.org/10.3233/JAI-2011-014

Christie BA, Miller KK, Cooke R, White JG (2015) Environmental sustainability in higher education: what do academics think? Environ Educ Res 21(5):655–686. https://doi.org/10.1080/13504622.2013.879697

Clark SG, Wallace RL (2015) Integration and interdisciplinarity: concepts, frameworks, and education. Policy Sci 48(2):233–255. https://doi.org/10.1007/s11077-015-9210-4

Cochran K, Cohn C, Hastings P, Tomuro N, Hughes S (2023) Using BERT to identify causal structure in students’ scientific explanations. Int J Artif Intell Educ. https://doi.org/10.1007/s40593-023-00373-y

Cooke SJ, Vermaire JC (2015) Environmental studies and environmental science today: inevitable mission creep and integration in action-oriented transdisciplinary areas of inquiry, training and practice. J Environ Stud Sci 5(1):70–78. https://doi.org/10.1007/s13412-014-0220-x

Crossley SA, Allen LK, Snow EL, McNamara DS (2016) Incorporating learning characteristics into automatic essay scoring models: What individual differences and linguistic features tell us about writing quality. J Educ Data Min 8(2):1–19

D’Odorico P, Davis KF, Rosa L, Carr JA, Chiarelli D, Dell’Angelo J, Gephart J, MacDonald GK, Seekell DA, Suweis S, Rulli MC (2018) The global food–energy–water nexus. Rev Geophys 56(3):456–531. https://doi.org/10.1029/2017RG000591

Article ADS Google Scholar

Deane P (2006) Strategies for evidence identification through linguistic assessment of textual responses. In: Williamson DM, Bejar II, Mislevy RJ (eds) Automated scoring of complex tasks in computer-based testing. Lawrence Erlbaum Associates, pp. 313–372

Dogra V, Verma S, Kavita, Chatterjee P, Shafi J, Choi J, Ijaz MF (2022) A complete process of text classification system using state-of-the-art NLP models. Comput Intell Neurosci 2022:1883698. https://doi.org/10.1155/2022/1883698

Article PubMed PubMed Central Google Scholar

Douglas KA, Gane BD, Neumann K, Pellegrino JW. (2020) Contemporary methods of assessing integrated STEM competencies. In: Johnson CC, Mohr-Schroeder MJ, Moore TJ, English LD (eds) Handbook of Research on STEM Education. 1st ed. Routledge: 234–254

Dugan KE, Mosyjowski EA, Daly SR, Lattuca LR (2022) Systems thinking assessments in engineering: a systematic literature review. Syst Res Behav Sci 39(4):840–866

Gao X, Li P, Shen J, Sun H (2020) Reviewing assessment of student learning in interdisciplinary STEM education. Int J STEM Educ 7(1):24. https://doi.org/10.1186/s40594-020-00225-4

Global Council for Science and the Environment (n.d.) Sustainability in Higher Education: Accreditation. https://www.gcseglobal.org/pathways-to-accreditation . Accessed 5 Sept 2023

Gray S, Sterling EJ, Aminpour P, Goralnik L, Singer A, Wei C, Akabas S, Jordan RC, Giabbanelli PJ, Hodbod J, Betley E (2019) Assessing (social-ecological) systems thinking by evaluating cognitive maps. Sustainability 11(20):5753

Grohs JR, Kirk GR, Soledad MM, Knight DB (2018) Assessing systems thinking: a tool to measure complex reasoning through ill-structured problems. Think Skills Creat 28:110–130

Harris CJ, Krajcik JS, Pellegrino JW, DeBarger AH (2019) Designing Knowledge‐In‐Use assessments to promote deeper learning. Educ Meas Issues Pra 38(2):53–67. https://doi.org/10.1111/emip.12253

Harris CJ, Weibe E, Grover S, Pellegrino JW (eds) (2023) Classroom-based STEM assessment: Contemporary issues and perspectives. Community for Advancing Discovery Research in Education (CADRE). Education Development Center, Inc. https://cadrek12.org/resources/classroom-based-stem-assessment-contemporary-issues-and-perspectives

Harrison JS, Josefy MA, Kalm M, Krause R (2023) Using supervised machine learning to scale human-coded data: a method and dataset in the board leadership context. Strateg Manag J 44(7):1780–1802. https://doi.org/10.1002/smj.3480

Hartmann DP (1977) Considerations in the choice of interobserver reliability estimates. J Appl Behav Anal 10(1):103–116. https://doi.org/10.1901/jaba.1977.10-103

Haudek KC, Zhai X (2023) Examining the effect of assessment construct characteristics on machine learning scoring of scientific argumentation. Int J Artif Intell Educ. https://doi.org/10.1007/s40593-023-00385-8

Hestenes D, Wells M, Swackhamer G. (1992) Force concept inventory. Phys Teach 30(3):141–158. https://doi.org/10.1119/1.2343497

Hmelo-Silver CE, Pfeffer MG (2004) Comparing expert and novice understanding of a complex system from the perspective of structures, behaviors, and functions. Cogn Sci 28(1):127–138. https://doi.org/10.1207/s15516709cog2801_7

Hmelo-Silver CE, Marathe S, Liu L (2007) Fish swim, rocks sit, and lungs breathe: expert-novice understanding of complex systems. J Learn Sci 16(3):307–331. https://doi.org/10.1080/10508400701413401

Horne L, Manzanares A, Babin N, Royse EA, Arakawa L, Blavascunas E, Doner L, Druckenbrod D, Fairchild E, Jarchow M, Muchnick BR, Panday P, Perry D, Thomas R, Toomey A, Tucker BH, Washington-Ottombre C, Vincent S, Anderson SW, Romulo C (2023) Alignment among environmental programs in higher education: what Food–Energy–Water Nexus concepts are covered in introductory courses? J Geosci Educ 1–18. https://doi.org/10.1080/10899995.2023.2187680

Horne L, Manzanares AD, Atalan-Helicke N, Vincent S, Anderson SW, Romulo C (2024) An exploratory study of drawings as a tool to evaluate student understanding of the Food–Energy–Water (FEW) Nexus. J Environ Stud Sci. https://doi.org/10.1007/s13412-024-00929-x

Hsieh H-F, Shannon SE (2005) Three approaches to qualitative content analysis. Qual Health Res 15(9):1277–1288. https://doi.org/10.1177/1049732305276687

Article PubMed Google Scholar

Jacobson MJ, Wilensky U (2006) Complex systems in education: scientific and educational importance and implications for the learning sciences. J Learn Sci 15(1):11–34. https://doi.org/10.1207/s15327809jls1501_4

Jescovitch LN, Scott EE, Cerchiara JA, Merrill J, Urban-Lurain M, Doherty JH, Haudek KC (2021) Comparison of machine learning performance using analytic and holistic coding approaches across constructed response assessments aligned to a science learning progression. J Sci Educ Technol 30(2):150–167. https://doi.org/10.1007/s10956-020-09858-0

Jordan MI, Mitchell TM (2015) Machine learning: trends, perspectives, and prospects. Science 349(6245):255–260. https://doi.org/10.1126/science.aaa8415

Article ADS MathSciNet CAS PubMed Google Scholar

Jurka TP, Collingwood L, Boydstun AE, Grossman E, Van Atteveldt W (2013) RTextTools: a supervised learning package for text classification. R J 5(1):6–12

Kasneci E, Sessler K, Küchemann S, Bannert M, Dementieva D, Fischer F, Gasser U, Groh G, Günnemann S, Hüllermeier E, Krusche S, Kutyniok G, Michaeli T, Nerdel C, Pfeffer J, Poquet O, Sailer M, Schmidt A, Seidel T, … Kasneci G (2023) ChatGPT for good? On opportunities and challenges of large language models for education. Learn Individ Differ 103:102274. https://doi.org/10.1016/j.lindif.2023.102274

Katz SL, Padowski JC, Goldsby M, Brady MP, Hampton SE (2020) Defining the nature of the nexus: specialization, connectedness, scarcity, and scale in Food–Energy–Water management. Water 12(4):972. https://doi.org/10.3390/w12040972

Krathwohl DR (2002) A revision of bloom’s taxonomy: an overview. Theory Into Pract 41(4):212–218. https://doi.org/10.1207/s15430421tip4104_2

Latif E, Zhai X (2024) Fine-tuning ChatGPT for automatic scoring. Comput Educ: Artif Intell 6:100210. https://doi.org/10.1016/j.caeai.2024.100210

Laverty JT, Underwood SM, Matz RL, Posey LA, Carmel JH, Caballero MD, Fata-Hartley CL, Ebert-May D, Jardeleza SE, Cooper MM (2016) Characterizing college science assessments: the three-dimensional learning assessment Protocol. PLoS ONE 11(9):e0162333. https://doi.org/10.1371/journal.pone.0162333

Article CAS PubMed PubMed Central Google Scholar

Leal Filho W, Levesque VR, Salvia AL, Paço A, Fritzen B, Frankenberger F, Damke LI, Brandli LL, Ávila LV, Mifsud M, Will M, Pace P, Azeiteiro UM, Lovren VO (2021) University teaching staff and sustainable development: An assessment of competences. Sustain Sci 16(1):101–116. https://doi.org/10.1007/s11625-020-00868-w

Leck H, Conway D, Bradshaw M, Rees J (2015) Tracing the Water–Energy–Food Nexus: description, theory and practice. Geogr Compass 9(8):445–460. https://doi.org/10.1111/gec3.12222

Lee H-S, Gweon G-H, Lord T, Paessel N, Pallant A, Pryputniewicz S (2021) Machine learning-enabled automated feedback: supporting students’ revision of scientific arguments based on data drawn from simulation. J Sci Educ Technol 30(2):168–192. https://doi.org/10.1007/s10956-020-09889-7

Libarkin JC, Anderson SW (2005) Assessment of learning in entry-level geoscience courses: results from the geoscience concept inventory. J Geosci Educ 53(4):394–401

Libarkin JC, Geraghty Ward EM (2011) The qualitative underpinnings of quantitative concept inventory questions. Geological Society of America Special Papers, vol 474. Geological Society of America, pp. 37–48

Liu OL, Rios JA, Heilman M, Gerard L, Linn MC (2016) Validation of automated scoring of science assessments. J Res Sci Teach 53(2):215–233. https://doi.org/10.1002/tea.21299

Liu OL, Brew C, Blackmore J, Gerard L, Madhok J, Linn MC (2014) Automated scoring of constructed-response science items: prospects and obstacles. Educ Meas: Issues Pract 33(2):19–28. https://doi.org/10.1111/emip.12028

Liu SC (2023) Examining undergraduate students’ systems thinking competency through a problem scenario in the context of climate change education. Environ Educ Res 29(12):1780–1795

Article MathSciNet Google Scholar

Lottridge S, Wood S, Shaw D (2018) The effectiveness of machine score-ability ratings in predicting automated scoring performance. Appl Meas Educ 31(3):215–232. https://doi.org/10.1080/08957347.2018.1464452

Maestrales S, Zhai X, Touitou I, Baker Q, Schneider B, Krajcik J (2021) Using machine learning to score multi-dimensional assessments of chemistry and physics. J Sci Educ Technol 30(2):239–254. https://doi.org/10.1007/s10956-020-09895-9

Mambrey S, Timm J, Landskron JJ, Schmiemann P (2020) The impact of system specifics on systems thinking. J Res Sci Teach 57(10):1632–1651. https://doi.org/10.1002/tea.21649

Manzanares AD, Horne L, Royse EA, Azzarello CB, Jarchow M, Druckenbrod D, Babin N, Atalan-Helicke N, Vincent S, Anderson SW, Romulo C (in review). Undergraduate students’ knowledge about the relationships between climate change and the Food–Energy–Water Nexus. Int J Sustain High Educ

Martins FP, Cezarino LO, Liboni LB, Botelho Junior AB, Hunter T (2022) Interdisciplinarity-based sustainability framework for management education. Sustainability 14(19):12289. https://doi.org/10.3390/su141912289

Mayfield E, Black AW (2020) should you fine-tune BERT for automated essay scoring? In: Burstein J, Kochmar E, Leacock C, Madnani N, Pilán I, Yannakoudakis H, Zesch T (eds) proceedings of the fifteenth workshop on innovative use of NLP for building educational applications. Association for Computational Linguistics. 151–162. https://doi.org/10.18653/v1/2020.bea-1.15

McNamara D, Graesser AC (2011) Coh-Metrix: an automated tool for theoretical and applied natural language processing. In Applied Natural Language Processing. IGI Global, pp. 188–205

Meadows DH (2008) Thinking in systems: a primer. Chelsea Green Publishing

Mitchell TM (1997) Machine learning, vol 1. McGraw-hill New York

Mitkov R, Le An H, Karamanis N (2006) A computer-aided environment for generating multiple-choice test items. Nat Lang Eng 12(2):177

Mizumoto A, Eguchi M (2023) Exploring the potential of using an AI language model for automated essay scoring. Res Methods Appl Linguist 2(2):100050. https://doi.org/10.1016/j.rmal.2023.100050

Momsen J, Speth EB, Wyse S, Long T (2022) Using systems and systems thinking to unify biology education. CBE—Life Sci Educ 21(2):es3. https://doi.org/10.1187/cbe.21-05-0118

National Research Council (2012) A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC: The National Academies Press. https://doi.org/10.17226/13165

National Science Foundation (2020) STEM Education for the Future—2020 Visioning Report.pdf. https://www.nsf.gov/ehr/Materials/STEM%20Education%20for%20the%20Future%20-%202020%20Visioning%20Report.pdf

Nehm RH, Ha M, Mayfield E (2012) Transforming biology assessment with machine learning: automated scoring of written evolutionary explanations. J Sci Educ Technol 21(1):183–196. https://doi.org/10.1007/s10956-011-9300-9

NGSS Lead States (2013) Next generation science standards: for states, by states. The National Academies Press

Nguyen Thanh B, Vo DTH, Nguyen Nhat M, Pham TTT, Thai Trung H, Ha Xuan S (2023) Race with the machines: Assessing the capability of generative AI in solving authentic assessments. Australas J Educ Technol 39(5):59–81. https://doi.org/10.14742/ajet.8902

Noyes K, McKay RL, Neumann M, Haudek KC, Cooper, MM (2020) Developing computer resources to automate analysis of students’ explanations of London dispersion forces. J Chem Educ 97(11):3923–3936. https://doi.org/10.1021/acs.jchemed.0c00445

Pellegrino JW, Wilson MR, Koenig JA, Beatty AS (2013) Developing assessments for the next generation science standards. National Academies Press

Randle JM, Stroink ML (2018) The development and initial validation of the paradigm of systems thinking: development and validation of systems thinking. Syst Res Behav Sci 35(5):645–657

Ravi M, Puente-Urbina A, van Bokhoven JA (2021) Identifying opportunities to promote systems thinking in catalysis education. J Chem Educ 98(5):1583–1593. https://doi.org/10.1021/acs.jchemed.1c00005

Redman A, Wiek A, Barth M (2021) Current practice of assessing students’ sustainability competencies: a review of tools. Sustain Sci 16(1):117–135. https://doi.org/10.1007/s11625-020-00855-1

Redman A, Wiek A (2021) Competencies for advancing transformations towards sustainability. Front Educ 6 . https://www.frontiersin.org/articles/10.3389/feduc.2021.785163

Romero C, Ventura S, Espejo PG, Hervás C (2008) Data mining algorithms to classify students. In: Baker RSJd, Barnes T, Beck JE (eds) Educational data mining 2008. The 1st International Conference on Educational Data Mining, Proceedings. Montréal, Québec, Canada

Rupp AA (2018) Designing, evaluating, and deploying automated scoring systems with validity in mind: methodological design decisions. Appl Meas Educ 31(3):191–214. https://doi.org/10.1080/08957347.2018.1464448

Shermis MD (2015) Contrasting state-of-the-art in the machine scoring of short-form constructed responses. Educ Assess 20(1):46–65. https://doi.org/10.1080/10627197.2015.997617

Shermis MD, Burstein J (2013) Handbook of automated essay evaluation: Current applications and new directions. Routledge

Shiroda M, Uhl JD, Urban-Lurain M, Haudek KC (2022) Comparison of computer scoring model performance for short text responses across undergraduate institutional types. J Sci Educ Technol 31(1):117–128. https://doi.org/10.1007/s10956-021-09935-y

Simpson GB, Jewitt GPW (2019) The development of the Water-Energy-Food Nexus as a framework for achieving resource security: a review. Front Environ Sci 7:8. https://doi.org/10.3389/fenvs.2019.00008

Smajgl A, Ward J, Pluschke L (2016) The water–food–energy Nexus – Realising a new paradigm. J Hydrol 533:533–540. https://doi.org/10.1016/j.jhydrol.2015.12.033

Soltis NA, McNeal KS (2022) Development and validation of a concept inventory for earth system thinking skills. J STEM Educ Res 5(1):28–52. https://doi.org/10.1007/s41979-021-00065-z

Sripathi KN, Moscarella RA, Steele M, Yoho R, You H, Prevost LB, Urban-Lurain M, Merrill J, Haudek KC (2024) Machine learning mixed methods text analysis: an illustration from automated scoring models of student writing in biology education. J Mixed Methods Res 18(1):48–70. https://doi.org/10.1177/15586898231153946

Stone A, Allen K, Rhoads TR, Murphy TJ, Shehab RL, Saha C (2003) The statistics concept inventory: a pilot study. 33rd annual frontiers in education, 2003. FIE 2003. T3D_1-T3D_6. https://doi.org/10.1109/FIE.2003.1263336

Suresh H, Guttag J (2021) A framework for understanding sources of harm throughout the machine learning life cycle. In: Proceedings of the 1st ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. Association for Computing Machinery. https://doi.org/10.1145/3465416.3483305

Sweeney LB, Sterman JD (2007) Thinking about systems: Student and teacher conceptions of natural and social systems. Syst Dyn Rev 23(2–3):285–311. https://doi.org/10.1002/sdr.366

Tornabee R, Lavington E, Nehm RH (2016) Testing validity inferences for genetic drift concept inventory scores using Rasch and item order analyses [Conference paper]. National Association for Research in Science Teaching (NARST) Annual International Conference

Underwood SM, Posey LA, Herrington DG, Carmel JH, Cooper MM (2018) Adapting assessment tasks to support three-dimensional learning. J Chem Educ 95(2):207–217. https://doi.org/10.1021/acs.jchemed.7b00645

Urban-Lurain M, Merrill J, Haudek K, Nehm R, Moscarella R, Steele M, Park M (2015) Automated analysis of constructed responses: what are we modeling? [Conference paper]. National Meeting for the Society for the Advancement of Biology Education Research (SABER)

Varela-Losada M, Vega-Marcote P, Pérez-Rodríguez U, Álvarez-Lires M (2016) Going to action? A literature review on educational proposals in formal environmental education. Environ Educ Res 22(3):390–421. https://doi.org/10.1080/13504622.2015.1101751

Vincent S, Bunn S, Sloane S (2013) Interdisciplinary environmental and sustainability education on the nation’s campuses: curriculum design. National Council for Science and the Environment, Washington, DC

Vincent S, Rao S, Fu Q, Gu K, Huang X, Lindaman K, Mittleman E, Nguyen K, Rosenstein R, Suh Y (2017) Scope of interdisciplinary environmental, sustainability, and energy baccalaureate and graduate education in the United States. National Council for Science and the Environment: Washington DC

Wallace RL, Clark SG (2018) Environmental studies and sciences in a time of chaos: problems, contexts, and recommendations. J Environ Stud Sci 8(1):110–113. https://doi.org/10.1007/s13412-018-0469-6

Wang H, Troia GA (2023) Writing quality predictive modeling: integrating register-related factors. Writ Commun 40(4):1070–1112. https://doi.org/10.1177/07410883231185287

Wang H, Li T, Haudek K, Royse EA, Manzanares M, Adams S, Horne L, Romulo C (2023) Is ChatGPT a threat to formative assessment in college-level science? an analysis of linguistic and content-level features to classify response types. In: Schlippe T, Cheng ECK, Wang T (eds) artificial intelligence in education technologies: new development and innovative practices. AIET 2023. Lecture Notes on Data Engineering and Communications Technologies, vol 190. Springer, Singapore. https://doi.org/10.1007/978-981-99-7947-9_13

Wang H, Haudek KC, Manzanares AD, Romulo CL, Royse EA (in review) extending a pretrained language model (BERT) using an ontological perspective to classify students’ scientific expertise level from written responses

Weegar R, Idestam-Almquist P (2024) reducing workload in short answer grading using machine learning. Int J Artif Intell in Educ 34(2):247–273. https://doi.org/10.1007/s40593-022-00322-1

Wiek A, Withycombe L, Redman CL (2011) Key competencies in sustainability: A reference framework for academic program development. Sustain Sci 6(2):203–218. https://doi.org/10.1007/s11625-011-0132-6

Wiek A, Bernstein MJ, Rider WF, Cohen M, Forrest N, Kuzdas C, et al. (2016) operationalising competencies in higher educationfor sustainable Development. In Barth M, Michelsen G, Rieckmann M, Thomas I (eds) handbook of higher education for sustainable development. (London: Routledge), 297–317

Wiley J, Hastings P, Blaum D, Jaeger AJ, Hughes S, Wallace P, Griffin TD, Britt MA (2017) Different approaches to assessing the quality of explanations following a multiple-document inquiry activity in science. Int J Artif Intell Educ 27(4):758–790. https://doi.org/10.1007/s40593-017-0138-z

Williamson DM, Xi X, Breyer FJ (2012) A framework for evaluation and use of automated scoring. Educ Meas: Issues Pract 31(1):2–13. https://doi.org/10.1111/j.1745-3992.2011.00223.x

Wilson CD, Haudek KC, Osborne JF, Buck Bracey ZE, Cheuk T, Donovan BM, Stuhlsatz MAM, Santiago MM, Zhai X (2023) Using automated analysis to assess middle school students’ competence with scientific argumentation. J Res Sci Teach 61:38-69. https://doi.org/10.1002/tea.21864

Wulff P, Buschhüter D, Westphal A, Mientus L, Nowak A, Borowski A (2022) bridging the gap between qualitative and quantitative assessment in science education research with machine learning—a case for pretrained language models-based clustering. J Sci Educ Technol 31(4):490–513. https://doi.org/10.1007/s10956-022-09969-w

Zehner F, Sälzer C, Goldhammer F (2015) Automatic coding of short text responses via clustering in educational assessment. Educ Psychol Meas 76(2):280–303. https://doi.org/10.1177/0013164415590022

Zhai X, Krajcik J, Pellegrino JW (2021a) On the validity of machine learning-based next generation science assessments: a validity inferential network. J Sci Educ Technol 30(2):298–312. https://doi.org/10.1007/s10956-020-09879-9

Zhai X, Shi L, Nehm RH (2021b) A meta-analysis of machine learning-based science assessments: factors impacting machine-human score agreements. J Sci Educ Technol 30(3):361–379. https://doi.org/10.1007/s10956-020-09875-z

Zhai X, Haudek K, Shi L, H. Nehm R, Urban-Lurain M (2020a) From substitution to redefinition: a framework of machine learning-based science assessment. J Res Sci Teach 57(9):1430–1459. https://doi.org/10.1002/tea.21658

Zhai X, Yin Y, Pellegrino JW, Haudek KC, Shi L (2020b) Applying machine learning in science assessment: a systematic review. Stud Sci Educ 56(1):111–151. https://doi.org/10.1080/03057267.2020.1735757

Download references

Acknowledgements

Our team would like to acknowledge and thank the students and faculty who contributed materials, time, and responses to this research. This material is based upon work supported by the National Science Foundation under Grant Nos. 2013373 and 2013359.

Author information

Authors and affiliations.

Aims Community College, Greeley, USA

Emily A. Royse

School of Psychological Sciences at the University of Northern Colorado, Greeley, CO, USA

Amanda D. Manzanares & Caterina Belle Azzarello

Department of Counseling, Educational Psychology and Special Education at Michigan State University, East Lansing, MI, US

Heqiao Wang

Department of Biochemistry and Molecular Biology at Michigan State University, East Lansing, MI, USA

Kevin C. Haudek

Unity Environmental University, Unity, ME, USA

Lydia R. Horne

Department of Earth and Chemical Sciences at Rider University, Lawrenceville, NJ, USA

Daniel L. Druckenbrod

Department of Human Biology at Michigan State University, East Lansing, MI, USA

Megan Shiroda

Metropolitan State University of Denver, Denver, CO, USA

Sol R. Adams

Pacific Northwest National Laboratory, Richland, WA, USA

Ennea Fairchild

Vincent Evaluation Consulting LLC, Tulsa, OK, USA

Shirley Vincent

Department of Earth and Atmospheric Sciences, University of Northern Colorado, Greeley, CO, USA

Steven W. Anderson

Department of Geography, GIS and Sustainability at University of Northern Colorado, Greeley, CO, USA

Chelsie Romulo

You can also search for this author in PubMed Google Scholar

Contributions

CLR, SWA, KCH, and SV conceived the original project; all authors contributed to conceptual design and provided background perspectives; EAR, ADM, LRH, CBA, SRA, EF, and CLR, completed human coding for training and test data and HW, created the machine learning natural language processing algorithms; All authors contributed to the the Introduction, Results and Discussion. All authors discussed results and interpretation, as well as reviewed and edited the manuscript at all stages.

Corresponding author

Correspondence to Chelsie Romulo .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethical approval

This work uses human subject matter data in the form of a survey that collected responses from over 400 IES undergraduates at seven post-secondary institutions across the United States. The protocol was reviewed by the lead institution (University of Northern Colorado) Institutional Review Board (IRB) and approved by all other participating institutions (UNCO IRB#158867-1). Responses were de-identified prior to human coding and for machine learning model development.

Informed consent

Prior to completing the IRB-approved survey, students consented to the research study by agreeing to the consent form provided in supplemental materials.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplemental methods and results file, student survey consent form, constructed response question items, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Royse, E.A., Manzanares, A.D., Wang, H. et al. FEW questions, many answers: using machine learning to assess how students connect food–energy–water (FEW) concepts. Humanit Soc Sci Commun 11 , 1033 (2024). https://doi.org/10.1057/s41599-024-03499-z

Download citation

Received : 22 September 2023

Accepted : 22 July 2024

Published : 13 August 2024

DOI : https://doi.org/10.1057/s41599-024-03499-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

Explore articles by subject
Guide to authors
Editorial policies

IMAGES

Social Work Research: Concept, Scope
Social Work Research Methods
The Basics of Social Work Research
Case Study Research Social Work
Key Concepts in Social Research Methods
Social Work Methods

COMMENTS

Social Work Research: Concept, Scope
Table of Contents. Concept. Purpose of social work research is to produce new knowledge or to increase already available knowledge in the field of social work. Social work research gives new dimensions to social work techniques and methods and provides new ways to deal with problems. Social work research attempts to highlight insights about ...
Social Work Research Methods
Social work researchers will send out a survey, receive responses, aggregate the results, analyze the data, and form conclusions based on trends. Surveys are one of the most common research methods social workers use — and for good reason. They tend to be relatively simple and are usually affordable.
2.2: Concepts, Constructs, and Variables
As shown in Figure 2.1, scientific research proceeds along two planes: a theoretical plane and an empirical plane. Constructs are conceptualized at the theoretical (abstract) plane, while variables are operationalized and measured at the empirical (observational) plane. Thinking like a researcher implies the ability to move back and forth ...
Social Work Research Methods
The aim of social work research is to build the social work knowledge base in order to solve practical problems in social work practice or social policy. Investigating phenomena in accordance with the scientific method requires maximal adherence to empirical principles, such as basing conclusions on observations that have been gathered in a ...
Foundations of Social Work Research
This textbook was created to provide an introduction to research methods for BSW and MSW students, with particular emphasis on research and practice relevant to students at the University of Texas at Arlington. It provides an introduction to social work students to help evaluate research for evidence-based practice and design social work research projects. It can be used with its companion, A ...
Practice Research in Social Work: Themes, Opportunities and Impact
Practice research and social work co-exist within an environment of collaboration and interdisciplinary cooperation, where social workers collaborate with researchers, policymakers, and other professionals to collectively address complex social issues. ... Brix et al. introduce the concept of 'Synthesized Action Research' in the context of ...
Foundations of Social Work Research
Describe the role that theory plays in social work research . The terms paradigm and theory are often used interchangeably in social science, although social scientists do not always agree whether these are identical or distinct concepts. This text makes a clear distinction between the two ideas because thinking about each concept as ...
Sage Research Methods
The book will help readers: critically reflect on their own social work practice; assess, appraise and apply research; effectively advocate on behalf of service-users; confidently engage in debates about the profession; and reflect on legislative and policy developments. is a clear and accessible guide to the subject.
Graduate research methods in social work
We designed our book to help graduate social work students through every step of the research process, from conceptualization to dissemination. Our textbook centers cultural humility, information literacy, pragmatism, and an equal emphasis on quantitative and qualitative methods. It includes extensive content on literature reviews, cultural bias and respectfulness, and qualitative methods, in ...
Scientific Inquiry in Social Work
Chapter 1: Introduction to research. Chapter 2: Beginning a research project. Chapter 3: Reading and evaluating literature. Chapter 4: Conducting a literature review. Chapter 5: Ethics in social work research. Chapter 6: Linking methods with theory. Chapter 7: Design and causality. Chapter 8: Creating and refining a research question.
Social Work Research
Explore a collection of highly cited articles from the NASW journals published in 2020 and 2021. Read now. An official journal of the National Association of Social Workers. Publishes exemplary research to advance the development of knowledge and inform social.
PDF Social Work Research: Meaning, Importance and Scope
Social work research is regarded as the systematic use of research concepts, methods, techniques and strategies to provide information related to the objectives of social work programmes and practices. Thus the unit of analysis of social work research could be individuals, groups, families or programme of the agency.
Journal of Social Work: Sage Journals
The Journal of Social Work is a forum for the publication, dissemination and debate of key ideas and research in social work. The journal aims to advance theoretical understanding, shape policy, and inform practice, and welcomes submissions from all areas of social work.
Key Concepts and Theory in Social Work
Theory in social work according to Hodgson and Watts is an interactive, critical, and interpretive way of understanding social phenomena. The focus on this book is on discovery through the exploration of key concepts, values, and skills. Critical thinking about theory is encouraged by the strategic use of key questions, exercises at the end of ...
15 Important Social Work Theories and Methods
It encourages social workers to address the complex, dynamic interactions between a person and their environment. 7. Empowerment Theory. Empowerment Theory is centered on the process of increasing personal, interpersonal, or political power so individuals and communities can take action to improve their circumstances.
5.2 Conceptualization
Concept- notion or image that we conjure up when we think of some cluster of related observations or ideas. Conceptualization- writing out clear, concise definitions for our key concepts, particularly in quantitative research. Multi-dimensional concepts- concepts that are comprised of multiple elements.
7.3: Social work research paradigms
When social workers speak about social problems impacting societies and individuals, they reference positivist research, including experiments and surveys of the general populations. Positivist research is exceptionally good at producing cause-and-effect explanations that apply across many different situations and groups of people.
Module 2 Chapter 1: The Nature of Social Work Research Questions
Concept mapping to assess community needs of sexual minority youth (Davis, Saltzburg, & Locke, 2010) ... Historically, social work research has focused on studies of the individual, family, group, community, policy and/or organizational level, focusing across the lifespan on prevention, intervention, treatment, aftercare and rehabilitation of ...
11 Important Social Work Theories and Methods
The following 11 social work theories and methods are some of the most important principles in the field today: 1. Psychosocial Theory. Psychosocial theory, which Erik Erikson developed in the 1950s, is the main principle of social work. Also referred to as person-in-environment (PIE) theory, psychosocial theory posits that a person develops a ...
Social Work Research Method
Research methods include, sage research methods, qualitative methods, methods map, research methods in social workers research project, social workers evidence based practice. Especially when researchers have access to large participant groups, surveys are a simple, affordable, and reliable method. The structure is simple: participants answer a ...
Evidence-Based Practice
Evidence-Based Practice. The term evidence-based practice (EBP) was used initially in relation to medicine, but has since been adopted by many fields including education, child welfare, mental heath, and criminal justice. The Institute of Medicine (2001) defines evidence-based medicine as the integration of best researched evidence and clinical ...
9. Writing your research question
Social work research questions must contain a target population. Her study would be very different if she were to conduct it on older adults or immigrants who just arrived in a new country. ... and meanings that people have about the concepts in our research question. These keywords often make an appearance in qualitative research questions ...
Journal of the Society for Social Work and Research
Ranked #500 out of 1,415 "Sociology and Political Science" journals. Founded in 2009, the Journal of the Society for Social Work and Research ( JSSWR) is the flagship publication of the Society for Social Work and Research (SSWR), a freestanding organization founded in 1994 to advance social work research. JSSWR is a peer-reviewed ...
Methods in Social Work and its concept
Social work is a profession that helps individuals, groups, and communities to improve their social and emotional well-being. There are five main methods of social work: social casework, social group work, community organization, social welfare administration, and research. Social work is an art of living, it describes the activities of helping ...
"Why me?": Qualitative research on why patients ask, what they mean
Patients often ask, "why me?" but questions arise regarding what this statement means, how, when and why patients ask, how they answer and why. Interviews were conducted as part of several qualitative research studies exploring how patients view and cope with various conditions, including HIV, cancer, Huntington's disease and infertility. A secondary qualitative analysis was performed ...
One way social work researchers can better understand community needs
Researchers are calling on the social work community to begin incorporating a methodology called "discrete choice experiments" (DCEs) into their research, to better understand the needs and ...
FEW questions, many answers: using machine learning to assess ...
The work presented here is a start towards developing assessments (like CIs) that use CR for more complex concepts, such as systems thinking and connecting concepts across disciplines.
Mind the misalignment: The moderating role of daily social sleep lag in
Circadian processes are important for employees and organizations yet have been relatively underexplored in recovery research. Thus, we embed the concept of circadian misalignment into the recovery literature by investigating the moderating role of employees' daily social sleep lag (i.e., a discrepancy between employees' actual and biologically preferred sleep-wake times) in their recovery ...