framework for research and development

Building an R&D strategy for modern times

The global investment in research and development (R&D) is staggering. In 2019 alone, organizations around the world spent $2.3 trillion on R&D—the equivalent of roughly 2 percent of global GDP—about half of which came from industry and the remainder from governments and academic institutions. What’s more, that annual investment has been growing at approximately 4 percent per year over the past decade. 1 2.3 trillion on purchasing-power-parity basis; 2019 global R&D funding forecast , Supplement, R&D Magazine, March 2019, rdworldonline.com.

While the pharmaceutical sector garners much attention due to its high R&D spending as a percentage of revenues, a comparison based on industry profits shows that several industries, ranging from high tech to automotive to consumer, are putting more than 20 percent of earnings before interest, taxes, depreciation, and amortization (EBITDA) back into innovation research (Exhibit 1).

What do organizations expect to get in return? At the core, they hope their R&D investments yield the critical technology from which they can develop new products, services, and business models. But for R&D to deliver genuine value, its role must be woven centrally into the organization’s mission. R&D should help to both deliver and shape corporate strategy, so that it develops differentiated offerings for the company’s priority markets and reveals strategic options, highlighting promising ways to reposition the business through new platforms and disruptive breakthroughs.

Yet many enterprises lack an R&D strategy that has the necessary clarity, agility, and conviction to realize the organization’s aspirations. Instead of serving as the company’s innovation engine, R&D ends up isolated from corporate priorities, disconnected from market developments, and out of sync with the speed of business. Amid a growing gap in performance  between those that innovate successfully and those that do not, companies wishing to get ahead and stay ahead of competitors need a robust R&D strategy that makes the most of their innovation investments. Building such a strategy takes three steps: understanding the challenges that often work as barriers to R&D success, choosing the right ingredients for your strategy, and then pressure testing it before enacting it.

Overcoming the barriers to successful R&D

The first step to building an R&D strategy is to understand the four main challenges that modern R&D organizations face:

Innovation cycles are accelerating. The growing reliance on software and the availability of simulation and automation technologies have caused the cost of experimentation to plummet while raising R&D throughput. The pace of corporate innovation is further spurred by the increasing emergence of broadly applicable technologies, such as digital and biotech, from outside the walls of leading industry players.

But incumbent corporations are only one part of the equation. The trillion dollars a year that companies spend on R&D is matched by the public sector. Well-funded start-ups, meanwhile, are developing and rapidly scaling innovations that often threaten to upset established business models or steer industry growth into new areas. Add increasing investor scrutiny of research spending, and the result is rising pressure on R&D leaders to quickly show results for their efforts.

R&D lacks connection to the customer. The R&D group tends to be isolated from the rest of the organization. The complexity of its activities and its specialized lexicon make it difficult for others to understand what the R&D function really does. That sense of working inside a “black box” often exists even within the R&D organization. During a meeting of one large company’s R&D leaders, a significant portion of the discussion focused on simply getting everyone up to speed on what the various divisions were doing, let alone connecting those efforts to the company’s broader goals.

Given the challenges R&D faces in collaborating with other functions, going one step further and connecting with customers becomes all the more difficult. While many organizations pay lip service to customer-centric development, their R&D groups rarely get the opportunity to test products directly with end users. This frequently results in market-back product development that relies on a game of telephone via many intermediaries about what the customers want and need.

Projects have few accountability metrics. R&D groups in most sectors lack effective mechanisms to measure and communicate progress; the pharmaceutical industry, with its standard pipeline for new therapeutics that provides well-understood metrics of progress and valuation implications, is the exception, not the rule. When failure is explained away as experimentation and success is described in terms of patents, rather than profits, corporate leaders find it hard to quantify R&D’s contribution.

Yet proven metrics exist  to effectively measure progress and outcomes. A common challenge we observe at R&D organizations, ranging from automotive to chemical companies, is how to value the contribution of a single component that is a building block of multiple products. One specialty-chemicals company faced this challenge in determining the value of an ingredient it used in its complex formulations. It created categorizations to help develop initial business cases and enable long-term tracking. This allowed pragmatic investment decisions at the start of projects and helped determine the value created after their completion.

Even with outcomes clearly measured, the often-lengthy period between initial investment and finished product can obscure the R&D organization’s performance. Yet, this too can be effectively managed by tracking the overall value and development progress of the pipeline so that the organization can react and, potentially, promptly reorient both the portfolio and individual projects within it.

Incremental projects get priority. Our research indicates that incremental projects account for more than half of an average company’s R&D investment, even though bold bets and aggressive reallocation  of the innovation portfolio deliver higher rates of success. Organizations tend to favor “safe” projects with near-term returns—such as those emerging out of customer requests—that in many cases do little more than maintain existing market share. One consumer-goods company, for example, divided the R&D budget among its business units, whose leaders then used the money to meet their short-term targets rather than the company’s longer-term differentiation and growth objectives.

Focusing innovation solely around the core business may enable a company to coast for a while—until the industry suddenly passes it by. A mindset that views risk as something to be avoided rather than managed can be unwittingly reinforced by how the business case is measured. Transformational projects at one company faced a higher internal-rate-of-return hurdle than incremental R&D, even after the probability of success had been factored into their valuation, reducing their chances of securing funding and tilting the pipeline toward initiatives close to the core.

As organizations mature, innovation-driven growth becomes increasingly important, as their traditional means of organic growth, such as geographic expansion and entry into untapped market segments, diminish. To succeed, they need to develop R&D strategies equipped for the modern era that treat R&D not as a cost center but as the growth engine it can become.

Would you like to learn more about our Strategy & Corporate Finance Practice ?

Choosing the ingredients of a winning r&d strategy.

Given R&D’s role as the innovation driver that advances the corporate agenda, its guiding strategy needs to link board-level priorities with the technologies that are the organization’s focus (Exhibit 2). The R&D strategy must provide clarity and commitment to three central elements: what we want to deliver, what we need to deliver it, and how we will deliver it.

What we want to deliver. To understand what a company wants to and can deliver, the R&D, commercial, and corporate-strategy functions need to collaborate closely, with commercial and corporate-strategy teams anchoring the R&D team on the company’s priorities and the R&D team revealing what is possible. The R&D strategy and the corporate strategy must be in sync while answering questions such as the following: At the highest level, what are the company’s goals? Which of these will require R&D in order to be realized? In short, what is the R&D organization’s purpose?

Bringing the two strategies into alignment is not as easy as it may seem. In some companies, what passes for corporate strategy is merely a five-year business plan. In others, the corporate strategy is detailed but covers only three to five years—too short a time horizon to guide R&D, especially in industries such as pharma or semiconductors where the product-development cycle is much longer than that. To get this first step right, corporate-strategy leaders should actively engage with R&D. That means providing clarity where it is lacking and incorporating R&D feedback that may illuminate opportunities, such as new technologies that unlock growth adjacencies for the company or enable completely new business models.

Secondly, the R&D and commercial functions need to align on core battlegrounds and solutions. Chief technology officers want to be close to and shape the market by delivering innovative solutions that define new levels of customer expectations. Aligning R&D strategy provides a powerful forum for identifying those opportunities by forcing conversations about customer needs and possible solutions that, in many companies, occur only rarely. Just as with the corporate strategy alignment, the commercial and R&D teams need to clearly articulate their aspirations by asking questions such as the following: Which markets will make or break us as a company? What does a winning product or service look like for customers?

When defining these essential battlegrounds, companies should not feel bound by conventional market definitions based on product groups, geographies, or customer segments. One agricultural player instead defined its markets by the challenges customers faced that its solutions could address. For example, drought resistance was a key battleground no matter where in the world it occurred. That framing clarified the R&D–commercial strategy link: if an R&D project could improve drought resistance, it was aligned to the strategy.

The dialogue between the R&D, commercial, and strategy functions cannot stop once the R&D strategy is set. Over time, leaders of all three groups should reexamine the strategic direction and continuously refine target product profiles as customer needs and the competitive landscape evolve.

What we need to deliver it. This part of the R&D strategy determines what capabilities and technologies the R&D organization must have in place to bring the desired solutions to market. The distinction between the two is subtle but important. Simply put, R&D capabilities are the technical abilities to discover, develop, or scale marketable solutions. Capabilities are unlocked by a combination of technologies and assets, and focus on the outcomes. Technologies, however, focus on the inputs—for example, CRISPR is a technology that enables the genome-editing capability.

This delineation protects against the common pitfall of the R&D organization fixating on components of a capability instead of the capability itself—potentially missing the fact that the capability itself has evolved. Consider the dawn of the digital age: in many engineering fields, a historical reliance on talent (human number crunchers) was suddenly replaced by the need for assets (computers). Those who focused on hiring the fastest mathematicians were soon overtaken by rivals who recognized the capability provided by emerging technologies.

The simplest way to identify the needed capabilities is to go through the development processes of priority solutions step by step—what will it take to produce a new product or feature? Being exhaustive is not the point; the goal is to identify high-priority capabilities, not to log standard operating procedures.

Prioritizing capabilities is a critical but often contentious aspect of developing an R&D strategy. For some capabilities, being good is sufficient. For others, being best in class is vital because it enables a faster path to market or the development of a better product than those of competitors. Take computer-aided design (CAD), which is used to design and prototype engineering components in numerous industries, such as aerospace or automotive. While companies in those sectors need that capability, it is unlikely that being the best at it will deliver a meaningful advantage. Furthermore, organizations should strive to anticipate which capabilities will be most important in the future, not what has mattered most to the business historically.

Once capabilities are prioritized, the R&D organization needs to define what being “good” and “the best” at them will mean over the course of the strategy. The bar rises rapidly in many fields. Between 2009 and 2019, the cost of sequencing a genome dropped 150-fold, for example. 2 Kris A. Wetterstrand, “DNA sequencing costs: Data,” NHGRI Genome Sequencing Program (GSP), August 25, 2020, genome.gov. Next, the organization needs to determine how to develop, acquire, or access the needed capabilities. The decision of whether to look internally or externally is crucial. An automatic “we can build it better” mindset diminishes the benefits of specialization and dilutes focus. Additionally, the bias to building everything in-house can cut off or delay access to the best the world has to offer—something that may be essential for high-priority capabilities. At Procter & Gamble, it famously took the clearly articulated aspiration of former CEO A. G. Lafley to break the company’s focus on in-house R&D and set targets for sourcing innovation externally. As R&D organizations increasingly source capabilities externally, finding partners and collaborating with them effectively is becoming a critical capability in its own right.

How we will do it. The choices of operating model and organizational design will ultimately determine how well the R&D strategy is executed. During the strategy’s development, however, the focus should be on enablers that represent cross-cutting skills and ways of working. A strategy for attracting, developing, and retaining talent is one common example.

Another is digital enablement, which today touches nearly every aspect of what the R&D function does. Artificial intelligence can be used at the discovery phase to identify emerging market needs or new uses of existing technology. Automation and advanced analytics approaches to experimentation can enable high throughput screening at a small scale and distinguish the signal from the noise. Digital (“in silico”) simulations are particularly valuable when physical experiments are expensive or dangerous. Collaboration tools are addressing the connectivity challenges common among geographically dispersed project teams. They have become indispensable in bringing together existing collaborators, but the next horizon is to generate the serendipity of chance encounters that are the hallmark of so many innovations.

Testing your R&D strategy

Developing a strategy for the R&D organization entails some unique challenges that other functions do not face. For one, scientists and engineers have to weigh considerations beyond their core expertise, such as customer, market, and economic factors. Stakeholders outside R&D labs, meanwhile, need to understand complex technologies and development processes and think along much longer time horizons than those to which they are accustomed.

For an R&D strategy to be robust and comprehensive enough to serve as a blueprint to guide the organization, it needs to involve stakeholders both inside and outside the R&D group, from leading scientists to chief commercial officers. What’s more, its definition of capabilities, technologies, talent, and assets should become progressively more granular as the strategy is brought to life at deeper levels of the R&D organization. So how can an organization tell if its new strategy passes muster? In our experience, McKinsey’s ten timeless tests of strategy  apply just as well to R&D strategy as to corporate and business-unit strategies. The following two tests are the most important in the R&D context:

• Does the organization’s strategy tap the true source of advantage? Too often, R&D organizations conflate technical necessity (what is needed to develop a solution) with strategic importance (distinctive capabilities that allow an organization to develop a meaningfully better solution than those of their competitors). It is also vital for organizations to regularly review their answers to this question, as capabilities that once provided differentiation can become commoditized and no longer serve as sources of advantage.
• Does the organization’s strategy balance commitment-rich choices with flexibility and learning? R&D strategies may have relatively long time horizons but that does not mean they should be insulated from changes in the outside world and never revisited. Companies should establish technical, regulatory, or other milestones that serve as clear decision points for shifting resources to or away from certain research areas. Such milestones can also help mark progress and gauge whether strategy execution is on track.

Additionally, the R&D strategy should be simply and clearly communicated to other functions within the company and to external stakeholders. To boost its clarity, organizations might try this exercise: distill the strategy into a set of fill-in-the-blank components that define, first, how the world will evolve and how the company plans to refocus accordingly (for example, industry trends that may lead the organization to pursue new target markets or segments); next, the choices the R&D function will make in order to support the company’s new focus (which capabilities will be prioritized and which de-emphasized); and finally, how the R&D team will execute the strategy in terms of concrete actions and milestones. If a company cannot fit the exercise on a single page, it has not sufficiently synthesized the strategy—as the famed physicist Richard Feynman observed, the ultimate test of comprehension is the ability to convey something to others in a simple manner.

Cascading the strategy down through the R&D organization will further reinforce its impact. For example, asking managers to communicate the strategy to their subordinates will deepen their own understanding. A useful corollary is that those hearing the strategy for the first time are introduced to it by their immediate supervisors rather than more distant R&D leaders. One R&D group demonstrated the broad benefits of this communication model: involving employees in developing and communicating the R&D strategy helped it double its Organizational Health Index  strategic clarity score, which measures one of the four “power practices”  highly connected to organizational performance.

R&D represents a massive innovation investment, but as companies confront globalized competition, rapidly changing customer needs, and technological shifts coming from an ever-wider range of fields, they are struggling to deliver on R&D’s full potential. A clearly articulated R&D strategy that supports and informs the corporate strategy is necessary to maximize the innovation investment and long-term company value.

Explore a career with us

Related articles.

The innovation commitment

The eight essentials of innovation

The Committed Innovator: An interview with Salesforce’s Simon Mulcahy

An Agile Framework for Research and Development

A quick search of "Agile for research and development" makes two things clear:

• Research and development (R&D) processes need agility beyond the software development space.
• Companies struggle to match agile frameworks to their R&D needs.

As LMI generated an operating model to develop prototypes around innovative and crowdsourced ideas in short timelines (30–45 days), the difficulty became apparent. While LMI has mature teams practicing scrum, Kanban, extreme programming (XP), and the Scaled Agile Framework (SAFe), the compressed timelines and enhanced uncertainty around scope and technology presented unique challenges requiring a new approach. This method (shown in Figure 1) expressly supports the development of rapid prototypes where scope and technology are initially undefined. Why does R&D need something different?

• The open-ended definition of “done” for exploratory efforts clashes with sprint planning (scrum).
• Reaching an endpoint in a timebox is a challenge for Kanban.
• Shortened timelines devalue estimation while augmenting demand for shorter feedback loops.
• Abridged timelines mean that overhead with iteration management needs to be kept tight.

Focusing on the Right Things

To support these efforts LMI developed Ranger™ – a deconstructed agile framework . Framework development must start with goals and principles. Inculcating a team with unified purpose creates alignment and offers team members a beacon when facing an uncertain situation. Standards and principles drive our practices and tools. With each prototype kickoff, we reintroduce our mission and core values.

Our mission is to prove the concept. We aim to establish or demonstrate feasibility, not implement the concept perfectly. We do this through the following core values:

• Creating and preserving momentum
• Practicing ruthless prioritization and ignoring sunk costs
• Emphasizing positioning over planning.

With our mission in mind, the team decided which practices and roles to introduce, alter, and keep from various agile frameworks.

Designing Agile Practices Around Core Values

Ranger™ has similarities to scrum, Kanban, and the dynamic systems development method. However, significant changes and small adjustments make this process unique.

The Significant Changes

Most agile practices use consistent iteration lengths of 2–4 weeks. When working on a 30- or 45-day prototype, this period represents an unacceptable timeline for feedback. Compressed project lengths make capacity measures, like story points, relatively useless (it takes about three iterations to calculate velocity). Ranger™ solves this issue by creating variable-length iterations, called orbits . An orbit lasts 2–5 business days and is goal-focused. The team agrees to the length at each orbit start, based on the goal. Setting a short orbit helps the team right-size goals while variable orbit sizes enable the team to swarm around the true objective without adding filler to round out capacity. When reflecting on our core values, varying length iterations consistently helped our teams create momentum, prioritize ruthlessly, and emphasize positioning.

The other significant adjustment was around roles. Prototype idea generators and subject matter experts often have other commitments preventing them from working as a product owner. We formalized the roles of innovators and sponsors for unique contributions to the prototype vision.

• The innovator has the idea or vision of what a fully developed product can be.
• Market and service line sponsors have insights into proposals and clients that could benefit from the prototype outcome in the near term.
• The product owner defines an achievable prototype scope that marries these insights.

Splitting these responsibilities across technical, business, and functional experts generates a healthy friction. Introducing and sharing multiple vantage points promotes broader education around the problem and how the prototype fits into that narrative.

The Small Adjustments

Although, like with other agile frameworks, teams perform daily stand-ups, iteration reviews, and practice continuous improvement through retrospectives, these ceremonies underwent minor adjustments to account for our context. For example, we have a single formal retrospective at about the 1/3 mark of the prototype and a post-mortem retrospective at its conclusion. This timeline ensures improvement and reflection while preserving momentum and reducing overhead. Figure 2 shows how these ceremonies fit in the framework.

The Results

In one year, 16 prototypes reached completion using the Ranger™ framework. While not every prototype receives further funding for product development, each furnishes enough information for an educated decision regarding further investment. All prototypes have achieved that objective, with a representative proof of concept delivered on time to support decision-making.

Several of the most successful prototypes had significant pivots or concluded with a scope outside of the initial plan, outlining the need for constant positioning based on discoveries and overall agility. Understanding a problem and proving the capacity to solve it in these timeframes has implications for the government’s acquisition and proposal process and can increase confidence in how the government approaches complex problems.

While existing methods might not fit perfectly in every context, adaptations and custom frameworks can bring agility to non-traditional use cases.

Joseph Mariña

Joseph Mariña has experience as a scrum master and agile coach across many agile frameworks. He has more than ten years of experience working on an array of technical solutions.

Related Content

How We Innovate

Operating in an innovation ecosystem, we combine market-leading technical resources, strategic partnerships, and agile development capabilities to accelerate mission success. 

Ranger™ Framework: Exploring with Ranger™

Ranger™ is a deconstructed agile framework designed for the development of quick (~45 business day) prototypes and research projects. It best serves efforts around feasibility rather than perfect implementation. 

Enterprise Agility

Employing an Agile mindset to accelerate organizational transformation, strengthen programs, and meet dynamic business needs. 

Forge™ Technology Studio

Leveraging agile methods and human-centered design, the Forge™ empowers us to deliver technology solutions rapidly. 

• Privacy Policy

Home » Conceptual Framework – Types, Methodology and Examples

Conceptual Framework – Types, Methodology and Examples

Table of Contents

Conceptual Framework

Definition:

A conceptual framework is a structured approach to organizing and understanding complex ideas, theories, or concepts. It provides a systematic and coherent way of thinking about a problem or topic, and helps to guide research or analysis in a particular field.

A conceptual framework typically includes a set of assumptions, concepts, and propositions that form a theoretical framework for understanding a particular phenomenon. It can be used to develop hypotheses, guide empirical research, or provide a framework for evaluating and interpreting data.

Conceptual Framework in Research

In research, a conceptual framework is a theoretical structure that provides a framework for understanding a particular phenomenon or problem. It is a key component of any research project and helps to guide the research process from start to finish.

A conceptual framework provides a clear understanding of the variables, relationships, and assumptions that underpin a research study. It outlines the key concepts that the study is investigating and how they are related to each other. It also defines the scope of the study and sets out the research questions or hypotheses.

Types of Conceptual Framework

Types of Conceptual Framework are as follows:

Theoretical Framework

A theoretical framework is an overarching set of concepts, ideas, and assumptions that help to explain and interpret a phenomenon. It provides a theoretical perspective on the phenomenon being studied and helps researchers to identify the relationships between different concepts. For example, a theoretical framework for a study on the impact of social media on mental health might draw on theories of communication, social influence, and psychological well-being.

Conceptual Model

A conceptual model is a visual or written representation of a complex system or phenomenon. It helps to identify the main components of the system and the relationships between them. For example, a conceptual model for a study on the factors that influence employee turnover might include factors such as job satisfaction, salary, work-life balance, and job security, and the relationships between them.

Empirical Framework

An empirical framework is based on empirical data and helps to explain a particular phenomenon. It involves collecting data, analyzing it, and developing a framework to explain the results. For example, an empirical framework for a study on the impact of a new health intervention might involve collecting data on the intervention’s effectiveness, cost, and acceptability to patients.

Descriptive Framework

A descriptive framework is used to describe a particular phenomenon. It helps to identify the main characteristics of the phenomenon and to develop a vocabulary to describe it. For example, a descriptive framework for a study on different types of musical genres might include descriptions of the instruments used, the rhythms and beats, the vocal styles, and the cultural contexts of each genre.

Analytical Framework

An analytical framework is used to analyze a particular phenomenon. It involves breaking down the phenomenon into its constituent parts and analyzing them separately. This type of framework is often used in social science research. For example, an analytical framework for a study on the impact of race on police brutality might involve analyzing the historical and cultural factors that contribute to racial bias, the organizational factors that influence police behavior, and the psychological factors that influence individual officers’ behavior.

Conceptual Framework for Policy Analysis

A conceptual framework for policy analysis is used to guide the development of policies or programs. It helps policymakers to identify the key issues and to develop strategies to address them. For example, a conceptual framework for a policy analysis on climate change might involve identifying the key stakeholders, assessing their interests and concerns, and developing policy options to mitigate the impacts of climate change.

Logical Frameworks

Logical frameworks are used to plan and evaluate projects and programs. They provide a structured approach to identifying project goals, objectives, and outcomes, and help to ensure that all stakeholders are aligned and working towards the same objectives.

Conceptual Frameworks for Program Evaluation

These frameworks are used to evaluate the effectiveness of programs or interventions. They provide a structure for identifying program goals, objectives, and outcomes, and help to measure the impact of the program on its intended beneficiaries.

Conceptual Frameworks for Organizational Analysis

These frameworks are used to analyze and evaluate organizational structures, processes, and performance. They provide a structured approach to understanding the relationships between different departments, functions, and stakeholders within an organization.

Conceptual Frameworks for Strategic Planning

These frameworks are used to develop and implement strategic plans for organizations or businesses. They help to identify the key factors and stakeholders that will impact the success of the plan, and provide a structure for setting goals, developing strategies, and monitoring progress.

Components of Conceptual Framework

The components of a conceptual framework typically include:

• Research question or problem statement : This component defines the problem or question that the conceptual framework seeks to address. It sets the stage for the development of the framework and guides the selection of the relevant concepts and constructs.
• Concepts : These are the general ideas, principles, or categories that are used to describe and explain the phenomenon or problem under investigation. Concepts provide the building blocks of the framework and help to establish a common language for discussing the issue.
• Constructs : Constructs are the specific variables or concepts that are used to operationalize the general concepts. They are measurable or observable and serve as indicators of the underlying concept.
• Propositions or hypotheses : These are statements that describe the relationships between the concepts or constructs in the framework. They provide a basis for testing the validity of the framework and for generating new insights or theories.
• Assumptions : These are the underlying beliefs or values that shape the framework. They may be explicit or implicit and may influence the selection and interpretation of the concepts and constructs.
• Boundaries : These are the limits or scope of the framework. They define the focus of the investigation and help to clarify what is included and excluded from the analysis.
• Context : This component refers to the broader social, cultural, and historical factors that shape the phenomenon or problem under investigation. It helps to situate the framework within a larger theoretical or empirical context and to identify the relevant variables and factors that may affect the phenomenon.
• Relationships and connections: These are the connections and interrelationships between the different components of the conceptual framework. They describe how the concepts and constructs are linked and how they contribute to the overall understanding of the phenomenon or problem.
• Variables : These are the factors that are being measured or observed in the study. They are often operationalized as constructs and are used to test the propositions or hypotheses.
• Methodology : This component describes the research methods and techniques that will be used to collect and analyze data. It includes the sampling strategy, data collection methods, data analysis techniques, and ethical considerations.
• Literature review : This component provides an overview of the existing research and theories related to the phenomenon or problem under investigation. It helps to identify the gaps in the literature and to situate the framework within the broader theoretical and empirical context.
• Outcomes and implications: These are the expected outcomes or implications of the study. They describe the potential contributions of the study to the theoretical and empirical knowledge in the field and the practical implications for policy and practice.

Conceptual Framework Methodology

Conceptual Framework Methodology is a research method that is commonly used in academic and scientific research to develop a theoretical framework for a study. It is a systematic approach that helps researchers to organize their thoughts and ideas, identify the variables that are relevant to their study, and establish the relationships between these variables.

Here are the steps involved in the conceptual framework methodology:

Identify the Research Problem

The first step is to identify the research problem or question that the study aims to answer. This involves identifying the gaps in the existing literature and determining what specific issue the study aims to address.

Conduct a Literature Review

The second step involves conducting a thorough literature review to identify the existing theories, models, and frameworks that are relevant to the research question. This will help the researcher to identify the key concepts and variables that need to be considered in the study.

Define key Concepts and Variables

The next step is to define the key concepts and variables that are relevant to the study. This involves clearly defining the terms used in the study, and identifying the factors that will be measured or observed in the study.

Develop a Theoretical Framework

Once the key concepts and variables have been identified, the researcher can develop a theoretical framework. This involves establishing the relationships between the key concepts and variables, and creating a visual representation of these relationships.

Test the Framework

The final step is to test the theoretical framework using empirical data. This involves collecting and analyzing data to determine whether the relationships between the key concepts and variables that were identified in the framework are accurate and valid.

Examples of Conceptual Framework

Some realtime Examples of Conceptual Framework are as follows:

• In economics , the concept of supply and demand is a well-known conceptual framework. It provides a structure for understanding how prices are set in a market, based on the interplay of the quantity of goods supplied by producers and the quantity of goods demanded by consumers.
• In psychology , the cognitive-behavioral framework is a widely used conceptual framework for understanding mental health and illness. It emphasizes the role of thoughts and behaviors in shaping emotions and the importance of cognitive restructuring and behavior change in treatment.
• In sociology , the social determinants of health framework provides a way of understanding how social and economic factors such as income, education, and race influence health outcomes. This framework is widely used in public health research and policy.
• In environmental science , the ecosystem services framework is a way of understanding the benefits that humans derive from natural ecosystems, such as clean air and water, pollination, and carbon storage. This framework is used to guide conservation and land-use decisions.
• In education, the constructivist framework is a way of understanding how learners construct knowledge through active engagement with their environment. This framework is used to guide instructional design and teaching strategies.

Applications of Conceptual Framework

Some of the applications of Conceptual Frameworks are as follows:

• Research : Conceptual frameworks are used in research to guide the design, implementation, and interpretation of studies. Researchers use conceptual frameworks to develop hypotheses, identify research questions, and select appropriate methods for collecting and analyzing data.
• Policy: Conceptual frameworks are used in policy-making to guide the development of policies and programs. Policymakers use conceptual frameworks to identify key factors that influence a particular problem or issue, and to develop strategies for addressing them.
• Education : Conceptual frameworks are used in education to guide the design and implementation of instructional strategies and curriculum. Educators use conceptual frameworks to identify learning objectives, select appropriate teaching methods, and assess student learning.
• Management : Conceptual frameworks are used in management to guide decision-making and strategy development. Managers use conceptual frameworks to understand the internal and external factors that influence their organizations, and to develop strategies for achieving their goals.
• Evaluation : Conceptual frameworks are used in evaluation to guide the development of evaluation plans and to interpret evaluation results. Evaluators use conceptual frameworks to identify key outcomes, indicators, and measures, and to develop a logic model for their evaluation.

Purpose of Conceptual Framework

The purpose of a conceptual framework is to provide a theoretical foundation for understanding and analyzing complex phenomena. Conceptual frameworks help to:

• Guide research : Conceptual frameworks provide a framework for researchers to develop hypotheses, identify research questions, and select appropriate methods for collecting and analyzing data. By providing a theoretical foundation for research, conceptual frameworks help to ensure that research is rigorous, systematic, and valid.
• Provide clarity: Conceptual frameworks help to provide clarity and structure to complex phenomena by identifying key concepts, relationships, and processes. By providing a clear and systematic understanding of a phenomenon, conceptual frameworks help to ensure that researchers, policymakers, and practitioners are all on the same page when it comes to understanding the issue at hand.
• Inform decision-making : Conceptual frameworks can be used to inform decision-making and strategy development by identifying key factors that influence a particular problem or issue. By understanding the complex interplay of factors that contribute to a particular issue, decision-makers can develop more effective strategies for addressing the problem.
• Facilitate communication : Conceptual frameworks provide a common language and conceptual framework for researchers, policymakers, and practitioners to communicate and collaborate on complex issues. By providing a shared understanding of a phenomenon, conceptual frameworks help to ensure that everyone is working towards the same goal.

When to use Conceptual Framework

There are several situations when it is appropriate to use a conceptual framework:

• To guide the research : A conceptual framework can be used to guide the research process by providing a clear roadmap for the research project. It can help researchers identify key variables and relationships, and develop hypotheses or research questions.
• To clarify concepts : A conceptual framework can be used to clarify and define key concepts and terms used in a research project. It can help ensure that all researchers are using the same language and have a shared understanding of the concepts being studied.
• To provide a theoretical basis: A conceptual framework can provide a theoretical basis for a research project by linking it to existing theories or conceptual models. This can help researchers build on previous research and contribute to the development of a field.
• To identify gaps in knowledge : A conceptual framework can help identify gaps in existing knowledge by highlighting areas that require further research or investigation.
• To communicate findings : A conceptual framework can be used to communicate research findings by providing a clear and concise summary of the key variables, relationships, and assumptions that underpin the research project.

Characteristics of Conceptual Framework

key characteristics of a conceptual framework are:

• Clear definition of key concepts : A conceptual framework should clearly define the key concepts and terms being used in a research project. This ensures that all researchers have a shared understanding of the concepts being studied.
• Identification of key variables: A conceptual framework should identify the key variables that are being studied and how they are related to each other. This helps to organize the research project and provides a clear focus for the study.
• Logical structure: A conceptual framework should have a logical structure that connects the key concepts and variables being studied. This helps to ensure that the research project is coherent and consistent.
• Based on existing theory : A conceptual framework should be based on existing theory or conceptual models. This helps to ensure that the research project is grounded in existing knowledge and builds on previous research.
• Testable hypotheses or research questions: A conceptual framework should include testable hypotheses or research questions that can be answered through empirical research. This helps to ensure that the research project is rigorous and scientifically valid.
• Flexibility : A conceptual framework should be flexible enough to allow for modifications as new information is gathered during the research process. This helps to ensure that the research project is responsive to new findings and is able to adapt to changing circumstances.

Advantages of Conceptual Framework

Advantages of the Conceptual Framework are as follows:

• Clarity : A conceptual framework provides clarity to researchers by outlining the key concepts and variables that are relevant to the research project. This clarity helps researchers to focus on the most important aspects of the research problem and develop a clear plan for investigating it.
• Direction : A conceptual framework provides direction to researchers by helping them to develop hypotheses or research questions that are grounded in existing theory or conceptual models. This direction ensures that the research project is relevant and contributes to the development of the field.
• Efficiency : A conceptual framework can increase efficiency in the research process by providing a structure for organizing ideas and data. This structure can help researchers to avoid redundancies and inconsistencies in their work, saving time and effort.
• Rigor : A conceptual framework can help to ensure the rigor of a research project by providing a theoretical basis for the investigation. This rigor is essential for ensuring that the research project is scientifically valid and produces meaningful results.
• Communication : A conceptual framework can facilitate communication between researchers by providing a shared language and understanding of the key concepts and variables being studied. This communication is essential for collaboration and the advancement of knowledge in the field.
• Generalization : A conceptual framework can help to generalize research findings beyond the specific study by providing a theoretical basis for the investigation. This generalization is essential for the development of knowledge in the field and for informing future research.

Limitations of Conceptual Framework

Limitations of Conceptual Framework are as follows:

• Limited applicability: Conceptual frameworks are often based on existing theory or conceptual models, which may not be applicable to all research problems or contexts. This can limit the usefulness of a conceptual framework in certain situations.
• Lack of empirical support : While a conceptual framework can provide a theoretical basis for a research project, it may not be supported by empirical evidence. This can limit the usefulness of a conceptual framework in guiding empirical research.
• Narrow focus: A conceptual framework can provide a clear focus for a research project, but it may also limit the scope of the investigation. This can make it difficult to address broader research questions or to consider alternative perspectives.
• Over-simplification: A conceptual framework can help to organize and structure research ideas, but it may also over-simplify complex phenomena. This can limit the depth of the investigation and the richness of the data collected.
• Inflexibility : A conceptual framework can provide a structure for organizing research ideas, but it may also be inflexible in the face of new data or unexpected findings. This can limit the ability of researchers to adapt their research project to new information or changing circumstances.
• Difficulty in development : Developing a conceptual framework can be a challenging and time-consuming process. It requires a thorough understanding of existing theory or conceptual models, and may require collaboration with other researchers.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

APA Table of Contents – Format and Example

Figures in Research Paper – Examples and Guide

Research Process – Steps, Examples and Tips

Purpose of Research – Objectives and Applications

Critical Analysis – Types, Examples and Writing...

Evaluating Research – Process, Examples and...

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Methodology
• What Is a Conceptual Framework? | Tips & Examples

What Is a Conceptual Framework? | Tips & Examples

Published on August 2, 2022 by Bas Swaen and Tegan George. Revised on March 18, 2024.

A conceptual framework illustrates the expected relationship between your variables. It defines the relevant objectives for your research process and maps out how they come together to draw coherent conclusions.

Keep reading for a step-by-step guide to help you construct your own conceptual framework.

Table of contents

Developing a conceptual framework in research, step 1: choose your research question, step 2: select your independent and dependent variables, step 3: visualize your cause-and-effect relationship, step 4: identify other influencing variables, frequently asked questions about conceptual models.

A conceptual framework is a representation of the relationship you expect to see between your variables, or the characteristics or properties that you want to study.

Conceptual frameworks can be written or visual and are generally developed based on a literature review of existing studies about your topic.

Your research question guides your work by determining exactly what you want to find out, giving your research process a clear focus.

However, before you start collecting your data, consider constructing a conceptual framework. This will help you map out which variables you will measure and how you expect them to relate to one another.

In order to move forward with your research question and test a cause-and-effect relationship, you must first identify at least two key variables: your independent and dependent variables .

• The expected cause, “hours of study,” is the independent variable (the predictor, or explanatory variable)
• The expected effect, “exam score,” is the dependent variable (the response, or outcome variable).

Note that causal relationships often involve several independent variables that affect the dependent variable. For the purpose of this example, we’ll work with just one independent variable (“hours of study”).

Now that you’ve figured out your research question and variables, the first step in designing your conceptual framework is visualizing your expected cause-and-effect relationship.

We demonstrate this using basic design components of boxes and arrows. Here, each variable appears in a box. To indicate a causal relationship, each arrow should start from the independent variable (the cause) and point to the dependent variable (the effect).

It’s crucial to identify other variables that can influence the relationship between your independent and dependent variables early in your research process.

Some common variables to include are moderating, mediating, and control variables.

Moderating variables

Moderating variable (or moderators) alter the effect that an independent variable has on a dependent variable. In other words, moderators change the “effect” component of the cause-and-effect relationship.

Let’s add the moderator “IQ.” Here, a student’s IQ level can change the effect that the variable “hours of study” has on the exam score. The higher the IQ, the fewer hours of study are needed to do well on the exam.

Let’s take a look at how this might work. The graph below shows how the number of hours spent studying affects exam score. As expected, the more hours you study, the better your results. Here, a student who studies for 20 hours will get a perfect score.

But the graph looks different when we add our “IQ” moderator of 120. A student with this IQ will achieve a perfect score after just 15 hours of study.

Below, the value of the “IQ” moderator has been increased to 150. A student with this IQ will only need to invest five hours of study in order to get a perfect score.

Here, we see that a moderating variable does indeed change the cause-and-effect relationship between two variables.

Mediating variables

Now we’ll expand the framework by adding a mediating variable . Mediating variables link the independent and dependent variables, allowing the relationship between them to be better explained.

Here’s how the conceptual framework might look if a mediator variable were involved:

In this case, the mediator helps explain why studying more hours leads to a higher exam score. The more hours a student studies, the more practice problems they will complete; the more practice problems completed, the higher the student’s exam score will be.

Moderator vs. mediator

It’s important not to confuse moderating and mediating variables. To remember the difference, you can think of them in relation to the independent variable:

• A moderating variable is not affected by the independent variable, even though it affects the dependent variable. For example, no matter how many hours you study (the independent variable), your IQ will not get higher.
• A mediating variable is affected by the independent variable. In turn, it also affects the dependent variable. Therefore, it links the two variables and helps explain the relationship between them.

Control variables

Lastly,  control variables must also be taken into account. These are variables that are held constant so that they don’t interfere with the results. Even though you aren’t interested in measuring them for your study, it’s crucial to be aware of as many of them as you can be.

A mediator variable explains the process through which two variables are related, while a moderator variable affects the strength and direction of that relationship.

A confounding variable is closely related to both the independent and dependent variables in a study. An independent variable represents the supposed cause , while the dependent variable is the supposed effect . A confounding variable is a third variable that influences both the independent and dependent variables.

Failing to account for confounding variables can cause you to wrongly estimate the relationship between your independent and dependent variables.

Yes, but including more than one of either type requires multiple research questions .

For example, if you are interested in the effect of a diet on health, you can use multiple measures of health: blood sugar, blood pressure, weight, pulse, and many more. Each of these is its own dependent variable with its own research question.

You could also choose to look at the effect of exercise levels as well as diet, or even the additional effect of the two combined. Each of these is a separate independent variable .

To ensure the internal validity of an experiment , you should only change one independent variable at a time.

A control variable is any variable that’s held constant in a research study. It’s not a variable of interest in the study, but it’s controlled because it could influence the outcomes.

A confounding variable , also called a confounder or confounding factor, is a third variable in a study examining a potential cause-and-effect relationship.

A confounding variable is related to both the supposed cause and the supposed effect of the study. It can be difficult to separate the true effect of the independent variable from the effect of the confounding variable.

In your research design , it’s important to identify potential confounding variables and plan how you will reduce their impact.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Swaen, B. & George, T. (2024, March 18). What Is a Conceptual Framework? | Tips & Examples. Scribbr. Retrieved September 3, 2024, from https://www.scribbr.com/methodology/conceptual-framework/

Is this article helpful?

Other students also liked

Independent vs. dependent variables | definition & examples, mediator vs. moderator variables | differences & examples, control variables | what are they & why do they matter, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Theories, Models, & Frameworks

One of the cornerstones of implementation science is the use of theory.

Unfortunately, the vast number of theories, models, and frameworks available in the implementation science toolkit can make it difficult to determine which is the most appropriate to address or frame a research question. There are dozens of theories, models, and frameworks used in implementation science that have been developed across a wide range of disciplines, and more are published each year.

Two reviews provide schemas to organize implementation science theories, models, and frameworks and narrow the range of choices:

Bridging research and practice: models for dissemination and implementation research (Tabak, Khoong, Chambers, & Brownson, 2013) Tabak et al’s schema organizes 61 dissemination and implementation models based on three variables: 1) construct flexibility, 2) focus on dissemination and/or implementation activities, and 3) socio-ecological framework level.

Doing Research

Frame your question, ⇥ pick a theory, model, or framework, identify implementation strategies, select research method, select study design, choose measures, get funding, report results.

The authors argue that classification of a model based on these three variables will assist in selecting a model to inform D&I science study design and execution. For more information, check out this archived NCI webinar with presenters Dr. Rachel Tabak and Dr. Ted Skolarus: 💻 Applying Models and Frameworks to D&I Research: An Overview & Analysis .

✪ Making sense of implementation theories, models, and frameworks (Nilsen, 2015) Per Nilsen's schema sorts implementation science theories, models, and frameworks into five categories: 1) process models, 2) determinants frameworks, 3) classic theories, 4) implementation theories, and 5) evaluation frameworks.

Adapted from: Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci . 2015;10(1):1-13.

Below, we borrow from Nilsen’s schema to organize overviews of a selection of implementation science theories, models, and frameworks. In each overview, you will find links to additional resources.

Open Access articles will be marked with ✪ Please note some journals will require subscriptions to access a linked article.

What are you using implementation science to accomplish.

• To describe or guide the process of translating research into practice
• To understand and/or explain what influences implementation outcomes
• To evaluate implementation

Process Models

Examples of use.

• ✪ Results-based aid with lasting effects: Sustainability in the Salud Mesoamérica Initiative ( Globalization and Health , 2018)
• ✪ Study Protocol: A Clinical Trial for Improving Mental Health Screening for Aboriginal and Torres Strait Islander Pregnant Women and Mothers of Young Children Using the Kimberley Mum's Mood Scale ( BMC Public Health , 2019)
• ✪ Sustainability of Public Health Interventions: Where Are the Gaps? ( Health Research Policy and Systems , 2019)

In 2018 the authors refined the EPIS model into the cyclical EPIS Wheel, allowing for closer alignment with rapid-cycle testing. A model for rigorously applying the Exploration, Preparation, Implementation, Sustainment (EPIS) framework in the design and measurement of a large scale collaborative multi-site study is available Open Access (✪) from Health & Justice .

• ✪ Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework ( Implementation Science , 2019)
• A Review of Studies on the System-Wide Implementation of Evidence-Based Psychotherapies for Posttraumatic Stress Disorder in the Veterans Health Administration ( Administration and Policy in Mental Health and Mental Health Services Research , 2016)
• Advancing Implementation Research and Practice in Behavioral Health Systems ( Administration and Policy in Mental Health and Mental Health Services Research , 2016)
• ✪ A model for rigorously applying the Exploration, Preparation, Implementation, Sustainment (EPIS) framework in the design and measurement of a large scale collaborative multi-site study ( Health and Justice , 2018)
• Characterizing Shared and Unique Implementation Influences in Two Community Services Systems for Autism: Applying the EPIS Framework to Two Large-Scale Autism Intervention Community Effectiveness Trials ( Administration and Policy in Mental Health and Mental Health Services Research , 2019)
• 💻 WEBINAR: Use of theory in implementation research: The EPIS framework: A phased and multilevel approach to implementation
• ✪ A two-way street: bridging implementation science and cultural adaptations of mental health treatments ( Implementation Science , 2013)
• ✪ “Scaling-out” evidence-based interventions to new populations or new health care delivery systems ( Implementation Science , 2017)
• ✪ Implementing measurement based care in community mental health: a description of tailored and standardized methods ( Implementation Science , 2018)
• ✪ "I Had to Somehow Still Be Flexible": Exploring Adaptations During Implementation of Brief Cognitive Behavioral Therapy in Primary Care ( Implementation Science , 2018)
• An Implementation Science Approach to Antibiotic Stewardship in Emergency Departments and Urgent Care Centers ( Academic Emergency Medicine , 2020)
• Using the Practical, Robust Implementation and Sustainability Model (PRISM) to Qualitatively Assess Multilevel Contextual Factors to Help Plan, Implement, Evaluate, and Disseminate Health Services Programs ( Translational Behavioral Medicine , 2019)
• Stakeholder Perspectives on Implementing a Universal Lynch Syndrome Screening Program: A Qualitative Study of Early Barriers and Facilitators ( Genetics Medicine , 2016)
• Evaluating the Implementation of Project Re-Engineered Discharge (RED) in Five Veterans Health Administration (VHA) Hospitals ( The Joint Commission Journal on Quality and Patient Safety , 2018)

In 2012 Meyers, Durlak, and Wandersman synthesized information from 25 implementation frameworks with a focus on identifying specific actions that improve the quality of implementation efforts. The result of this synthesis was the Quality Implementation Framework (QIF) , published in the American Journal of Community Psychology . This framework is comprised of fourteen critical steps across four phases of implementation, and has been used widely in child and family services, behavioral health, and hospital settings.

• ✪ Practical Implementation Science: Developing and Piloting the Quality Implementation Tool ( American Journal of Community Psychology , 2012)
• Survivorship Care Planning in a Comprehensive Cancer Center Using an Implementation Framework ( The Journal of Community and Supportive Oncology , 2016)
• ✪ The Application of an Implementation Science Framework to Comprehensive School Physical Activity Programs: Be a Champion! ( Frontiers in Public Health , 2017)
• ✪ Developing and Evaluating a Lay Health Worker Delivered Implementation Intervention to Decrease Engagement Disparities in Behavioural Parent Training: A Mixed Methods Study Protocol ( BMJ Open , 2019)
• Implementation Process and Quality of a Primary Health Care System Improvement Initiative in a Decentralized Context: A Retrospective Appraisal Using the Quality Implementation Framework ( The International Journal of Health Planning and Management , 2019)

Determinant Frameworks

Learn more:.

• Statewide Implementation of Evidence-Based Programs ( Exceptional Children , 2013)
• Active Implementation Frameworks for Successful Service Delivery: Catawba County Child Wellbeing Project ( Research on Social Work Practice , 2014)
• The Active Implementation Frameworks: A roadmap for advancing implementation of Comprehensive Medication Management in primary care ( Research in Social and Administrative Pharmacy , 2017)

For additional resources, please visit the CFIR Technical Assistance Website . The website has tools and templates for studying implementation of innovations using the CFIR framework, and these tools can help you learn more about issues pertaining to inner and outer contexts. You can read the original framework development article in the Open Access (✪) journal Implementation Science .

• ✪ Evaluating and Optimizing the Consolidated Framework for Implementation Research (CFIR) for use in Low- and Middle-Income Countries: A Systematic Review ( Implementation Science , 2020)
• ✪ A systematic review of the use of the Consolidated Framework for Implementation Research ( Implementation Science , 2017)
• Using the Consolidated Framework for Implementation Research (CFIR) to produce actionable findings: A rapid-cycle evaluation approach to improving implementation ( Implementation Science , 2017)
• ✪ The Consolidated Framework for Implementation Research: Advancing implementation science through real-world applications, adaptations, and measurement ( Implementation Science , 2015)
• 💻 WEBINAR: Use of theory in implementation research: Pragmatic application and scientific advancement of the Consolidated Framework for Implementation Research (CFIR) (Dr. Laura Damschroder, National Cancer Institute of NIH Fireside Chat Series )

In 2005, Dr. Susan Michie and colleagues published the Theoretical Domains Framework in BMJ Quality & Safety , the result of a consensus process to develop a theoretical framework for implementation research. The primary goals of the development team were to determine key theoretical constructs for studying evidence based practice implementation and for developing strategies for effective implementation, and for these constructs to be accessible and meaningful across disciplines.

• ✪ Validation of the theoretical domains framework for use in behaviour change and implementation research ( Implementation Science , 2012)
• ✪ Theoretical domains framework to assess barriers to change for planning health care quality interventions: a systematic literature review ( Journal of Multidisciplinary Healthcare , 2016)
• ✪ Combined use of the Consolidated Framework for Implementation Research (CFIR) and the Theoretical Domains Framework (TDF): a systematic review ( Implementation Science , 2017)
• ✪ Applying the Theoretical Domains Framework to identify barriers and targeted interventions to enhance nurses’ use of electronic medication management systems in two Australian hospitals ( Implementation Science , 2017)
• ✪ A guide to using the Theoretical Domains Framework of behaviour change to investigate implementation problems ( Implementation Science , 2017)
• ✪ Hospitals Implementing Changes in Law to Protect Children of Ill Parents: A Cross-Sectional Study ( BMC Health Services Research , 2018)
• Addressing the Third Delay: Implementing a Novel Obstetric Triage System in Ghana ( BMJ Global Health , 2018)

The original framework development article, Enabling the implementation of evidence based practice: a conceptual framework is available Open Access (✪) from BMJ Quality & Safety .

• Ingredients for change: revisiting a conceptual framework ( BMJ Quality & Safety , 2002)
• Evaluating the successful implementation of evidence into practice using the PARIHS framework: theoretical and practical challenges ( Implementation Science , 2008)
• ✪ A critical synthesis of literature on the promoting action on research implementation in health services (PARIHS) framework ( Implementation Science , 2010)
• ✪ A Guide for applying a revised version of the PARIHS framework for implementation ( Implementation Science , 2011)
• 💻 WEBINAR: Use of theory in implementation research; Pragmatic application and scientific advancement of the Promoting Action on Research Implementation in Health Services (PARiHS) framework

Classic Theories

In 2017 Dr. Sarah Birken and colleagues published their application of four organizational theories to published accounts of evidence-based program implementation. The objective was to determine whether these theories could help explain implementation success by shedding light on the impact of the external environment on the implementing organizations.

Their paper, ✪ Organizational theory for dissemination and implementation research , published in the journal Implementation Science utilized transaction cost economics theory , institutional theory , contingency theories , and resource dependency theory for this work.

In 2019, Dr. Jennifer Leeman and colleagues applied these same three organizational theories to case studies of the implementation of colorectal cancer screening interventions in Federally Qualified Health Centers, in ✪ Advancing the use of organization theory in implementation science ( Preventive Medicine , 2019).

In 2005 the NIH published ✪ Theory at a Glance: A Guide For Health Promotion Practice 2.0, an overview of behavior change theories. Below are selected theories from the intrapersonal and interpersonal ecological levels most relevant to implementation science.

There are two intrapersonal behavioral theories most often used to interpret individual behavior variation:

The Health Belief Model : An initial theory of health behavior, the HBM arose from work in the 1950s by a group of social psychologists in the U.S. wishing to understand why health improvement services were not being used. The HBM posited that in the health behavior context, readiness to act arises from six factors: perceived susceptibility , perceived severity . perceived benefits , perceived barriers , a cue to action , and self-efficacy . To learn more about the Health Belief Model, please read "Historical Origins of the Health Belief Model" ( Health Education Monographs ).

The Theory of Planned Behavior : This theory, developed by Ajzen in the late 1980s and formalized in 1991 , sees the primary driver of behavior as being behavioral intention . Through the lens of the TPB, behavioral intention is believed to be influenced by an individual's attitude , their perception of peers' subjective norms , and the individual's perceived behavioral control .

At the interpersonal behavior level , where individual behavior is influenced by a social environment, Social Cognitive Theory is the most widely used theory in health behavior research.

Social Cognitive Theory : Published by Bandera in the 1978 article, Self-efficacy: Toward a unifying theory of behavioral change , SCT consists of six main constructs: reciprocal determinism , behavioral capability , expectations , observational learning , reinforcements , and self-efficacy (which is seen as the most important personal factor in changing behavior).

Examples of use in implementation science:

The Health Belief Model

• ✪ Using technology for improving population health: comparing classroom vs. online training for peer community health advisors in African American churches ( Implementation Science , 2015)

The Theory of Planned Behavior

• ✪ Assessing mental health clinicians’ intentions to adopt evidence-based treatments: reliability and validity testing of the evidence-based treatment intentions scale ( Implementation Science , 2016)

Social Cognitive Theory

• ✪ Systematic development of a theory-informed multifaceted behavioural intervention to increase physical activity of adults with type 2 diabetes in routine primary care: Movement as Medicine for Type 2 Diabetes ( Implementation Science , 2016)
• Diffusion of preventive innovations ( Addictive Behaviors , 2002)
• ✪ Diffusion of Innovation Theory ( Canadian Journal of Nursing Informatics , 2011)

Implementation Theories

• ✪ Implementing community-based provider participation in research: an empirical study ( Implementation Science , 2012)
• ✪ Context matters: measuring implementation climate among individuals and groups ( Implementation Science , 2014)
• ✪ Determining the predictors of innovation implementation in healthcare: a quantitative analysis of implementation effectiveness ( BMC Health Services Research , 2015)
• Review: Conceptualization and Measurement of Organizational Readiness for Change ( Medical Care Research and Review , 2008)
• ✪ Organizational factors associated with readiness to implement and translate a primary care based telemedicine behavioral program to improve blood pressure control: the HTN-IMPROVE study ( Implementation Science , 2013)
• ✪ Towards evidence-based palliative care in nursing homes in Sweden: a qualitative study informed by the organizational readiness to change theory ( Implementation Science , 2018)
• ✪ Assessing the reliability and validity of the Danish version of Organizational Readiness for Implementing Change (ORIC) ( Implementation Science , 2018)
• ✪ Development of a theory of implementation and integration: Normalization Process Theory ( Implementation Science , 2009)
• ✪ Implementation, context and complexity ( Implementation Science , 2016)
• ✪ Exploring the implementation of an electronic record into a maternity unit: a qualitative study using Normalisation Process Theory ( BMC Medical Informatics and Decision Making , 2017)
• ✪ Implementation of cardiovascular disease prevention in primary health care: enhancing understanding using normalisation process theory ( BMC Family Practice , 2017)
• ✪ Using Normalization Process Theory in feasibility studies and process evaluations of complex healthcare interventions: a systematic review ( Implementation Science , 2018)

Evaluation Frameworks

The framework development article, ✪ Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda , is available through Administration and Policy in Mental Health and Mental Health Services Research .

In 2023, Dr. Proctor and several colleagues published Ten years of implementation outcomes research: a scoping review in the journal Implementation Science , a scoping review of 'the field’s progress in implementation outcomes research.'

• Toward Evidence-Based Measures of Implementation: Examining the Relationship Between Implementation Outcomes and Client Outcomes ( Journal of Substance Abuse Treatment , 2016)
• ✪ Toward criteria for pragmatic measurement in implementation research and practice: a stakeholder-driven approach using concept mapping ( Implementation Science , 2017)
• ✪ German language questionnaires for assessing implementation constructs and outcomes of psychosocial and health-related interventions: a systematic review ( Implementation Science , 2018)
• The Elusive Search for Success: Defining and Measuring Implementation Outcomes in a Real-World Hospital Trial ( Frontiers In Public Health , 2019)

In 1999, authors Glasgow, Vogt, and Boles developed this framework because they felt tightly controlled efficacy studies weren’t very helpful in informing program scale-up or in understanding actual public health impact of an intervention. The RE-AIM framework has been refined over time to guide the design and evaluation of complex interventions in order to maximize real-life public health impact.

This framework helps researchers collect information needed to translate research to effective practice, and may also be used to guide implementation and potential scale-up activities. You can read the original framework development article in The American Journal of Public Health . Additional resources, support, and publications on the RE-AIM framework can be found at RE-AIM.org . The 2021 special issue of Frontiers in Public Health titled Use of the RE-AIM Framework: Translating Research to Practice with Novel Applications and Emerging Directions includes more than 20 articles on RE-AIM.

• What Does It Mean to “Employ” the RE-AIM Model? ( Evaluation & the Health Professions , 2012)
• The RE-AIM Framework: A Systematic Review of Use Over Time (The American Journal of Public Health , 2013)
• ✪ Fidelity to and comparative results across behavioral interventions evaluated through the RE-AIM framework: a systematic review ( Systematic Reviews , 2015)
• ✪ Qualitative approaches to use of the RE-AIM framework: rationale and methods ( BMC Health Services Research , 2018)
• ✪ RE-AIM in Clinical, Community, and Corporate Settings: Perspectives, Strategies, and Recommendations to Enhance Public Health Impact ( Frontiers in Public Health , 2018)
• ✪ RE-AIM Planning and Evaluation Framework: Adapting to New Science and Practice With a 20-Year Review ( Frontiers in Public Health , 2019)
• ✪ RE-AIM in the Real World: Use of the RE-AIM Framework for Program Planning and Evaluation in Clinical and Community Settings ( Frontiers in Public Health , 2019)

Be boundless

Connect with us:.

© 2024 University of Washington | Seattle, WA

Educational resources and simple solutions for your research journey

What is a Theoretical Framework? How to Write It (with Examples) 

Theoretical framework 1,2 is the structure that supports and describes a theory. A theory is a set of interrelated concepts and definitions that present a systematic view of phenomena by describing the relationship among the variables for explaining these phenomena. A theory is developed after a long research process and explains the existence of a research problem in a study. A theoretical framework guides the research process like a roadmap for the research study and helps researchers clearly interpret their findings by providing a structure for organizing data and developing conclusions.   

A theoretical framework in research is an important part of a manuscript and should be presented in the first section. It shows an understanding of the theories and concepts relevant to the research and helps limit the scope of the research.  

Table of Contents

What is a theoretical framework ?  

A theoretical framework in research can be defined as a set of concepts, theories, ideas, and assumptions that help you understand a specific phenomenon or problem. It can be considered a blueprint that is borrowed by researchers to develop their own research inquiry. A theoretical framework in research helps researchers design and conduct their research and analyze and interpret their findings. It explains the relationship between variables, identifies gaps in existing knowledge, and guides the development of research questions, hypotheses, and methodologies to address that gap.  

Now that you know the answer to ‘ What is a theoretical framework? ’, check the following table that lists the different types of theoretical frameworks in research: 3

Developing a theoretical framework in research can help in the following situations: 4

• When conducting research on complex phenomena because a theoretical framework helps organize the research questions, hypotheses, and findings  
• When the research problem requires a deeper understanding of the underlying concepts  
• When conducting research that seeks to address a specific gap in knowledge  
• When conducting research that involves the analysis of existing theories  

Summarizing existing literature for theoretical frameworks is easy. Get our Research Ideation pack  

Importance of a theoretical framework  

The purpose of theoretical framework s is to support you in the following ways during the research process: 2  

• Provide a structure for the complete research process  
• Assist researchers in incorporating formal theories into their study as a guide  
• Provide a broad guideline to maintain the research focus  
• Guide the selection of research methods, data collection, and data analysis  
• Help understand the relationships between different concepts and develop hypotheses and research questions  
• Address gaps in existing literature  
• Analyze the data collected and draw meaningful conclusions and make the findings more generalizable  

Theoretical vs. Conceptual framework  

While a theoretical framework covers the theoretical aspect of your study, that is, the various theories that can guide your research, a conceptual framework defines the variables for your study and presents how they relate to each other. The conceptual framework is developed before collecting the data. However, both frameworks help in understanding the research problem and guide the development, collection, and analysis of the research.  

The following table lists some differences between conceptual and theoretical frameworks . 5

How to write a theoretical framework  

The following general steps can help those wondering how to write a theoretical framework: 2

• Identify and define the key concepts clearly and organize them into a suitable structure.  
• Use appropriate terminology and define all key terms to ensure consistency.  
• Identify the relationships between concepts and provide a logical and coherent structure.  
• Develop hypotheses that can be tested through data collection and analysis.  
• Keep it concise and focused with clear and specific aims.  

Write a theoretical framework 2x faster. Get our Manuscript Writing pack  

Examples of a theoretical framework  

Here are two examples of a theoretical framework. 6,7

Example 1 .   

An insurance company is facing a challenge cross-selling its products. The sales department indicates that most customers have just one policy, although the company offers over 10 unique policies. The company would want its customers to purchase more than one policy since most customers are purchasing policies from other companies.  

Objective : To sell more insurance products to existing customers.  

Problem : Many customers are purchasing additional policies from other companies.  

Research question : How can customer product awareness be improved to increase cross-selling of insurance products?  

Sub-questions: What is the relationship between product awareness and sales? Which factors determine product awareness?  

Since “product awareness” is the main focus in this study, the theoretical framework should analyze this concept and study previous literature on this subject and propose theories that discuss the relationship between product awareness and its improvement in sales of other products.  

Example 2 .

A company is facing a continued decline in its sales and profitability. The main reason for the decline in the profitability is poor services, which have resulted in a high level of dissatisfaction among customers and consequently a decline in customer loyalty. The management is planning to concentrate on clients’ satisfaction and customer loyalty.  

Objective: To provide better service to customers and increase customer loyalty and satisfaction.  

Problem: Continued decrease in sales and profitability.  

Research question: How can customer satisfaction help in increasing sales and profitability?  

Sub-questions: What is the relationship between customer loyalty and sales? Which factors influence the level of satisfaction gained by customers?  

Since customer satisfaction, loyalty, profitability, and sales are the important topics in this example, the theoretical framework should focus on these concepts.  

Benefits of a theoretical framework  

There are several benefits of a theoretical framework in research: 2  

• Provides a structured approach allowing researchers to organize their thoughts in a coherent way.  
• Helps to identify gaps in knowledge highlighting areas where further research is needed.  
• Increases research efficiency by providing a clear direction for research and focusing efforts on relevant data.  
• Improves the quality of research by providing a rigorous and systematic approach to research, which can increase the likelihood of producing valid and reliable results.  
• Provides a basis for comparison by providing a common language and conceptual framework for researchers to compare their findings with other research in the field, facilitating the exchange of ideas and the development of new knowledge.  

Frequently Asked Questions 

Q1. How do I develop a theoretical framework ? 7

A1. The following steps can be used for developing a theoretical framework :  

• Identify the research problem and research questions by clearly defining the problem that the research aims to address and identifying the specific questions that the research aims to answer.
• Review the existing literature to identify the key concepts that have been studied previously. These concepts should be clearly defined and organized into a structure.
• Develop propositions that describe the relationships between the concepts. These propositions should be based on the existing literature and should be testable.
• Develop hypotheses that can be tested through data collection and analysis.
• Test the theoretical framework through data collection and analysis to determine whether the framework is valid and reliable.

Q2. How do I know if I have developed a good theoretical framework or not? 8

A2. The following checklist could help you answer this question:  

• Is my theoretical framework clearly seen as emerging from my literature review?  
• Is it the result of my analysis of the main theories previously studied in my same research field?  
• Does it represent or is it relevant to the most current state of theoretical knowledge on my topic?  
• Does the theoretical framework in research present a logical, coherent, and analytical structure that will support my data analysis?  
• Do the different parts of the theory help analyze the relationships among the variables in my research?  
• Does the theoretical framework target how I will answer my research questions or test the hypotheses?  
• Have I documented every source I have used in developing this theoretical framework ?  
• Is my theoretical framework a model, a table, a figure, or a description?  
• Have I explained why this is the appropriate theoretical framework for my data analysis?  

Q3. Can I use multiple theoretical frameworks in a single study?  

A3. Using multiple theoretical frameworks in a single study is acceptable as long as each theory is clearly defined and related to the study. Each theory should also be discussed individually. This approach may, however, be tedious and effort intensive. Therefore, multiple theoretical frameworks should be used only if absolutely necessary for the study.  

Q4. Is it necessary to include a theoretical framework in every research study?  

A4. The theoretical framework connects researchers to existing knowledge. So, including a theoretical framework would help researchers get a clear idea about the research process and help structure their study effectively by clearly defining an objective, a research problem, and a research question.  

Q5. Can a theoretical framework be developed for qualitative research?  

A5. Yes, a theoretical framework can be developed for qualitative research. However, qualitative research methods may or may not involve a theory developed beforehand. In these studies, a theoretical framework can guide the study and help develop a theory during the data analysis phase. This resulting framework uses inductive reasoning. The outcome of this inductive approach can be referred to as an emergent theoretical framework . This method helps researchers develop a theory inductively, which explains a phenomenon without a guiding framework at the outset.  

Q6. What is the main difference between a literature review and a theoretical framework ?  

A6. A literature review explores already existing studies about a specific topic in order to highlight a gap, which becomes the focus of the current research study. A theoretical framework can be considered the next step in the process, in which the researcher plans a specific conceptual and analytical approach to address the identified gap in the research.  

Theoretical frameworks are thus important components of the research process and researchers should therefore devote ample amount of time to develop a solid theoretical framework so that it can effectively guide their research in a suitable direction. We hope this article has provided a good insight into the concept of theoretical frameworks in research and their benefits.  

References  

• Organizing academic research papers: Theoretical framework. Sacred Heart University library. Accessed August 4, 2023. https://library.sacredheart.edu/c.php?g=29803&p=185919#:~:text=The%20theoretical%20framework%20is%20the,research%20problem%20under%20study%20exists .  
• Salomao A. Understanding what is theoretical framework. Mind the Graph website. Accessed August 5, 2023. https://mindthegraph.com/blog/what-is-theoretical-framework/  
• Theoretical framework—Types, examples, and writing guide. Research Method website. Accessed August 6, 2023. https://researchmethod.net/theoretical-framework/  
• Grant C., Osanloo A. Understanding, selecting, and integrating a theoretical framework in dissertation research: Creating the blueprint for your “house.” Administrative Issues Journal : Connecting Education, Practice, and Research; 4(2):12-26. 2014. Accessed August 7, 2023. https://files.eric.ed.gov/fulltext/EJ1058505.pdf  
• Difference between conceptual framework and theoretical framework. MIM Learnovate website. Accessed August 7, 2023. https://mimlearnovate.com/difference-between-conceptual-framework-and-theoretical-framework/  
• Example of a theoretical framework—Thesis & dissertation. BacherlorPrint website. Accessed August 6, 2023. https://www.bachelorprint.com/dissertation/example-of-a-theoretical-framework/  
• Sample theoretical framework in dissertation and thesis—Overview and example. Students assignment help website. Accessed August 6, 2023. https://www.studentsassignmenthelp.co.uk/blogs/sample-dissertation-theoretical-framework/#Example_of_the_theoretical_framework  
• Kivunja C. Distinguishing between theory, theoretical framework, and conceptual framework: A systematic review of lessons from the field. Accessed August 8, 2023. https://files.eric.ed.gov/fulltext/EJ1198682.pdf  

Editage All Access is a subscription-based platform that unifies the best AI tools and services designed to speed up, simplify, and streamline every step of a researcher’s journey. The Editage All Access Pack is a one-of-a-kind subscription that unlocks full access to an AI writing assistant, literature recommender, journal finder, scientific illustration tool, and exclusive discounts on professional publication services from Editage.  

Based on 22+ years of experience in academia, Editage All Access empowers researchers to put their best research forward and move closer to success. Explore our top AI Tools pack, AI Tools + Publication Services pack, or Build Your Own Plan. Find everything a researcher needs to succeed, all in one place –  Get All Access now starting at just $14 a month !    

Related Posts

Back to School – Lock-in All Access Pack for a Year at the Best Price

Journal Turnaround Time: Researcher.Life and Scholarly Intelligence Join Hands to Empower Researchers with Publication Time Insights 

• APOS Theory

A Framework for Research and Curriculum Development in Mathematics Education

• © 2014
• Ilana Arnon 0 ,
• Jim Cottrill 1 ,
• Ed Dubinsky 2 ,
• Asuman Oktaç 3 ,
• Solange Roa Fuentes 4 ,
• Maria Trigueros 5 ,
• Kirk Weller 6

College of Education, Givat Washington Academic, Tel Aviv, Israel

You can also search for this author in PubMed   Google Scholar

Department of Mathematics, Ohio Dominican University, Columbus, USA

School of education, university of miami, miami, usa, departamento de matemática educativa, cinvestav-ipn, mexico city, mexico, escuela de matematicas edificio camilo torres, oficina 201, universidad industrial de santander, bucaramanga, colombia, departamento de matematicas, instituto tecnologico autonomo de mexico, col. tizapan, san angel, mexico, department of mathematics, ferris state university, big rapids, usa.

• The first book on APOS Theory in Mathematics Education, written by the people who developed APOS Theory
• Provides examples of curriculum development utilizing APOS Theory
• Links research and teaching APOS Theory in a coherent manner
• Describes relation of APOS Theory to Functions, Mathematical Induction, Discrete Mathematics, Linear and Abstract Algebra? and more

26k Accesses

111 Citations

4 Altmetric

This is a preview of subscription content, log in via an institution to check access.

Access this book

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as EPUB and PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Other ways to access

Licence this eBook for your library

Institutional subscriptions

About this book

Similar content being viewed by others.

Combining and Contrasting Formal Concept Analysis and APOS Theory

Using conceptual analyses to resolve the tension between advanced and secondary mathematics: the cases of equivalence and inverse

“Much Palaver About Greater Than Zero and Such Stuff” – First Year Engineering Students’ Recognition of University Mathematics

• ACE cycle and mathematics education
• APOS theory and abstract algebra
• APOS theory and curriculum development
• APOS theory and discrete mathematics
• APOS theory and linear algebra
• APOS theory and mathematical induction
• APOS theory and mathematics functions
• APOS theory and technology
• Cognitive phenomena in mathematics education
• Construction of mathematical knowledge
• Genetic decomposiition and APOS Theory
• ISETL in mathematics education
• Mental constructions and mechanisms in math ed
• Reflective abstraction in mathematics education
• schemas in mathematics education
• learning and instruction

Table of contents (12 chapters)

Front matter, introduction.

• Ilana Arnon, Jim Cottrill, Ed Dubinsky, Asuman Oktaç, Solange Roa Fuentes, María Trigueros et al.

From Piaget’s Theory to APOS Theory: Reflective Abstraction in Learning Mathematics and the Historical Development of APOS Theory

Mental structures and mechanisms: apos theory and the construction of mathematical knowledge, genetic decomposition, the teaching of mathematics using apos theory, the apos paradigm for research and curriculum development, schemas, their development and interaction, totality as a possible new stage and levels in apos theory, use of apos theory to teach mathematics at elementary school, frequently asked questions, conclusions, annotated bibliography, back matter.

“This book is clearly intended to persuade researchers to consider adopting an APOS-based approach. … the authors describe APOS theory as a Kuhnian paradigm, and set out to explain the paradigmatic questions, assumptions, and methods that within-paradigm researchers adopt. … In sum, for a reader interested in understanding APOS theory, this is an excellent book. It lays out APOS’s theoretical assumptions and standard research methods with clarity and precision, and it gives helpful examples of research conducted within the programme.” (Matthew Inglis, International Journal of Research in Undergraduate Mathematics Education, Vol. 1, 2015)

Authors and Affiliations

Ilana Arnon

Jim Cottrill

Ed Dubinsky

Asuman Oktaç

Solange Roa Fuentes

Maria Trigueros

Kirk Weller

About the authors

Ed Dubinsky is a Visiting Adjunct Professor at the University of Miami, FL, USA. Dr. Dubinsky is considered the father of APOS Theory. Ilana Arnon is a lecturer of Mathematics Education for prospective middle school mathematics teachers at Givat Washington Academic College of Education, Israel. Jim Cottrill is an assistant professor of Mathematics at Ohio Dominican University, OH, USA. Asuman Oktaç  is a professor in the Department of Mathematics Education at CINVESTAV-IPN, Mexico. Dora Solange Roa is an associate professor in the School of Mathematics at the Universidad Industrial de Santander, Colombia. Maria Trigueros is a professor in the Department of Mathematics at Instituto Tecnológico Autónomo de México, Mexico. Kirk Weller is a professor and head of the Mathematics Department at Ferris State University, MI, USA.

Bibliographic Information

Book Title : APOS Theory

Book Subtitle : A Framework for Research and Curriculum Development in Mathematics Education

Authors : Ilana Arnon, Jim Cottrill, Ed Dubinsky, Asuman Oktaç, Solange Roa Fuentes, Maria Trigueros, Kirk Weller

DOI : https://doi.org/10.1007/978-1-4614-7966-6

Publisher : Springer New York, NY

eBook Packages : Humanities, Social Sciences and Law , Education (R0)

Copyright Information : Springer Science+Business Media New York 2014

Hardcover ISBN : 978-1-4614-7965-9 Published: 05 August 2013

Softcover ISBN : 978-1-4899-9825-5 Published: 27 August 2015

eBook ISBN : 978-1-4614-7966-6 Published: 04 August 2013

Edition Number : 1

Number of Pages : XI, 254

Topics : Mathematics Education , Learning & Instruction

• Publish with us

Policies and ethics

• Find a journal
• Track your research

School of Social and Political Science

Researcher development framework (rdf).

The Vitae Researcher Development Framework (RDF), is an internationally recognised framework for planning, promoting and supporting the personal, professional and career development of researchers in higher education.

It is an approach to researcher development, which aims to provide researchers with a universal language for communicating their capabilities.

RDF Structure

The RDF was created using empirical data, collected through interviewing researchers to identify the characteristics of excellent researchers (expressed in the RDF as 'descriptors').

The descriptors are structured into four domains and twelve sub-domains, encompassing the knowledge, intellectual abilities, techniques and professional standards to do research, as well as the personal qualities, knowledge and skills to work with others and ensure the wider impact of research.

About the Vitae Researcher Development Framework

Using the RDF to plan your own development

All SDO skills training activities are mapped against the RDF. This helps you to keep track of your development during your PhD. Keep a note of the training that you attend and use the skills in the framework as a 'check-list'.

Tracking attendance against the RDF will help to identify where your strengths lie and which areas you should focus on in your future development.

Download the full RDF (PDF) 

Each sub-domain lists a range of skills focussed around a core objective. Within each skill, the RDF identifies between three to five phases, representing distinct stages of development or levels of performance within that descriptor.

Vitae Framework RDF Planner

A1: Knowledge Base

This sub-domain describes the knowledge, research methods and information management needed to do excellent research.

A2: Cognitive Abilities

This sub-domain describes the intellectual abilities needed to do excellent research.

A3: Creativity

This sub-domain describes the creative abilities and attributes needed to do excellent research.

B1: Personal Qualities

This sub-domain describes the personal attributes needed to be an excellent researcher.

B2: Self-Management

This sub-domain describes the behaviours and attributes needed to be a committed and effective researcher.

B3: Professional and Career Development

This sub-domain describes the knowledge and behaviours needed to take control of own professional development and reputation.

C1: Professional Conduct

This sub-domain describes the knowledge, behaviours and attributes needed for appropriate professional conduct as a researcher.

C2: Research Management

This sub-domain describes the knowledge and behaviours needed for the effective management of research projects.

C3: Finance, Funding and Resources

This sub-domain describes the knowledge needed of the processes for the effective management of finance, funding and resources.

D1: Working with Others

This sub-domain describes the attitudes and behaviours to work effectively with other people through different relationships.

D2: Communication and Dissemination

This sub-domain describes the knowledge and behaviours relating to effective communication and publication through a variety of methods and media.

D3: Engagement and Impact

This sub-domain describes the knowledge, behavoiur and attributes needed to engage with, impact and influence teaching, society, culture and the economy.

Student Development Office

The Student Development Office provide professional development and employability support to all students within the School of Social and Political Science.

Contact:  [email protected]

Researcher Hub

The researcher development framework (rdf).

The RDF is a professional development tool that was developed by Vitae . The purpose of the framework is to support researchers in realising and planning their career development. The RDF articulates knowledge, behaviours and attributes of successful researchers at all stages of the research career. It can be used as an aspiration or as a hands on planning tool.

Information about the RDF can be accessed via the Vitae website:

RDF frontpage

Full content of the framework

Framework lenses

The framework consists of four domains: Domain A: Knowledge and Intellectual Abilities Domain B: Personal Effectiveness Domain C: Research Governance and Organisation Domain D: Engagement, Influence and Impact

Download the RDF (PDF)

How do researchers use the Vitae Researcher Development Framework?

Vitae asked researchers in different disciplines and at different stages of their careers how they have benefitted from using the RDF. Read how they collected evidence of strengths and areas for development and reflected on how it was useful to them in the researcher profiles .

RDF briefing papers

For research staff For Principal Investigators

Choose the experimental features you want to try

Do you want to help improving EUR-Lex ? This is a list of experimental features that you can enable. These features are still under development; they are not fully tested, and might reduce EUR-Lex stability. Don't forget to give your feedback!

• Replacement of CELEX identifiers by short titles - experimental feature. It replaces clickable CELEX identifiers of treaties and case-law by short titles.
• Visualisation of document relationships. It displays a dynamic graph with relations between the act and related documents. It is currently only available for legal acts.
• Deep linking. It enables links to other legal acts referred to within the documents. It is currently only available for documents smaller than 900 KB.

This document is an excerpt from the EUR-Lex website

• EUR-Lex home
• EUR-Lex - 52022XC1028(03) - EN

Document 52022XC1028(03)

Communication from the Commission Framework for State aid for research and development and innovation 2022/C 414/01

C/2022/7388

OJ C 414, 28.10.2022, p. 1–38 (BG, ES, CS, DA, DE, ET, EL, EN, FR, GA, HR, IT, LV, LT, HU, MT, NL, PL, PT, RO, SK, SL, FI, SV)

  Languages, formats and link to OJ

  Multilingual display

Communication from the Commission

Framework for State aid for research and development and innovation

(2022/C 414/01)

Introduction

1.    Scope of application and definitions

1.1.    Scope of application

1.2.    Aid measures covered by the framework

13.

The Commission has identified a series of R&D&I measures for which State aid may, under specific conditions, be compatible with the internal market: (a) aid for R&D projects where the aided part of the research project falls within the categories of fundamental research and applied research, of which the latter can be divided into industrial research and experimental development  ). Such aid is mainly targeted at the market failure related to positive externalities (knowledge spill-overs), but may also address a market failure caused by imperfect and asymmetric information or (mainly in collaboration projects) a coordination failure; (b) aid for feasibility studies related to R&D projects, which helps overcoming a market failure primarily related to imperfect and asymmetric information; (c) aid for the construction and upgrade of research infrastructures , which mainly addresses the market failure stemming from coordination difficulties but also from imperfect and asymmetric information. High-quality research infrastructures are increasingly necessary for ground-breaking research, as they attract global talent and are essential for example for information and communication technologies and key enabling technologies  ). High up-front investment costs for acquiring state of the art scientific facilities and equipment used for early stage research activities, predominantly by the scientific community, make it often impossible to find the necessary financing on the market; (d) aid for the construction and upgrade of testing and experimentation infrastructures, mainly addresses the market failure stemming from imperfect and asymmetric information or a coordination failure. Constructing or upgrading a state of the art testing and experimentation infrastructure involves high up-front investment costs, which together with an uncertain client base can render access to financing difficult. Access to publicly funded testing and experimentation infrastructures must be granted on a transparent and non-discriminatory basis and on market terms to several users. To facilitate users’ access to testing and experimentation infrastructures, their user fees can be reduced in compliance with other provisions of this Framework or of Regulation (EU) No 651/2014  ) or the de minimis Regulation  ); (e) aid for innovation activities , which is mainly targeted at market failures related to positive externalities (knowledge spill-overs), coordination difficulties and, to a lesser extent, asymmetric information. With respect to small and medium-sized enterprises (‘SMEs’), such innovation aid may be awarded for obtaining, validating and defending patents and other intangible assets, for the secondment of highly qualified personnel, and for acquiring innovation advisory and support services, for example those provided by research and knowledge dissemination organisations, research infrastructures, testing and experimentation infrastructures or innovation clusters; (f) aid for process and organisational innovation , which is mainly targeted at market failures related to positive externalities (knowledge spill-overs), coordination difficulties and, to a lesser extent, asymmetric information. These aid measures can be awarded predominantly to SMEs. Aid to large undertakings shall only be compatible if they effectively collaborate with at least one SME in the aided activity; (g) aid for innovation clusters , which aims at tackling market failures linked with coordination problems hampering the development of clusters, limiting the interactions and knowledge flows within and between clusters. State aid could contribute to resolving this problem, first by supporting the investment in open and shared infrastructures for innovation clusters, and second by supporting the operation of clusters with a view to enhancing collaboration, networking and learning. Operating aid for clusters would need to be duly justified by the Member State especially when it exceeds ten years. The fees charged for using the innovation cluster’s facilities and for participating in the innovation cluster’s activities shall correspond to the market price or reflect their costs (including a reasonable margin). To facilitate access to the innovation cluster’s facilities or participation in the innovation cluster’s activities, access can be offered at reduced prices in compliance with other provisions of this Framework or of Regulation (EU) No 651/2014  ) or the de minimis Regulation  ) rules as applicable to the users of the services provided by the innovation cluster. 14. Member States must notify R&D&I aid pursuant to Article 108(3) of the Treaty, with the exception of measures that fulfil the conditions laid down in a block exemption Regulation adopted by the Commission pursuant to Article 1 of Council Regulation (EU) No 2015/1588  ). 15. This framework sets out the compatibility criteria for R&D&I aid schemes and individual aid which are subject to the notification requirement and must be assessed on the basis of Article 107(3)(c) of the Treaty  ). 1.3.    Definitions 16. For the purposes of this framework, the following definitions apply: (a) ‘ ad hoc aid’ means aid not awarded on the basis of an aid scheme; (b) ‘aid’ means any measure fulfilling the criteria laid down in Article 107(1) of the Treaty; (c) ‘aid intensity’ means the gross aid amount expressed as a percentage of the eligible costs, before any deduction of tax or other charge. Where aid is awarded in a form other than a grant, the aid amount is the grant equivalent of the aid. Aid payable in several instalments is discounted to its value at the date of award. The interest rate to be used for this purpose is the discount rate  ) applicable at the date of award. The aid intensity is calculated per beneficiary; (d) ‘aid scheme’ means any act on the basis of which, without further implementing measures being required, individual aid may be awarded to undertakings defined therein in a general and abstract manner and any act on the basis of which aid which is not linked to a specific project may be awarded to one or several undertakings; (e) ‘applied research’ means industrial research, experimental development, or any combination of both; (f) ‘’arm’s length’ means that the conditions of the transaction between the contracting parties do not differ from those which would be stipulated between independent enterprises and contain no element of collusion. Any transaction that results from an open, transparent and non-discriminatory procedure is considered as meeting the arm’s length principle; (g) ‘date of award of the aid’ means the date on which the legal right to receive the aid is conferred on the beneficiary under the applicable national legal regime; (h) ‘effective collaboration’ means collaboration between at least two independent parties to exchange knowledge or technology, or to achieve a common objective based on the division of labour where the parties jointly define the scope of the collaborative project, contribute to its implementation and share its risks, as well as its results. One or several parties may bear the full costs of the project and thus relieve other parties of its financial risks. Contract research and provision of research services are not considered forms of collaboration; (i) ‘evaluation plan’ means a document covering one or more aid schemes and containing at least the following minimum aspects: the objectives to be evaluated, the evaluation questions, the result indicators, the envisaged method to conduct the evaluation, the data collection requirements, the proposed timing of the evaluation including the date of submission of the interim and the final evaluation reports, the description of the independent body that will carry out the evaluation or the criteria that will be used for its selection and the modalities for making the evaluation publicly available; (j) ‘exclusive development’ means the public procurement of research and development services of which all benefits accrue exclusively to the contracting authority or contracting entity, and which it may use in the conduct of its own affairs on condition that it fully remunerates them; (k) ‘experimental development’ means acquiring, combining, shaping and using existing scientific, technological, business and other relevant knowledge and skills with the aim of developing new or improved products, processes or services, including digital products, processes or services, in any area, technology, industry or sector (including, but not limited to, digital industries and technologies, such as for example super-computing, quantum technologies, block chain technologies, artificial intelligence, cyber security, big data and cloud or edge technologies). This may also encompass, for example, activities aiming at the conceptual definition, planning and documentation of new products, processes or services. Experimental development may comprise prototyping, demonstrating, piloting, testing and validation of new or improved products, processes or services in environments representative of real life operating conditions where the primary objective is to make further technical improvements on products, processes or services that are not substantially set. This may include the development of a commercially usable prototype or pilot which is necessarily the final commercial product and which is too expensive to produce for it to be used only for demonstration and validation purposes. Experimental development does not include routine or periodic changes made to existing products, production lines, manufacturing processes, services and other operations in progress, even if those changes may represent improvements; (l) ‘feasibility study’ means the evaluation and analysis of the potential of a project, which aims at supporting the process of decision making by objectively and rationally uncovering its strengths and weaknesses, opportunities and threats, as well as identifying the resources required to carry it through and ultimately its prospects for success; (m) ‘full allocation’ means that the research organisation, research infrastructure or public purchaser enjoys the full economic benefit of intellectual property rights by retaining the right to make unrestricted use of them, particularly the right of ownership and the right to license. This may also be the case where the research organisation or research infrastructure (respectively, public purchaser) decides to conclude further contracts concerning those rights, including licensing them to a collaboration partner (respectively, undertakings); (n) ‘fundamental research’ means experimental or theoretical work undertaken primarily to acquire new knowledge of the underlying foundations of phenomena and observable facts, without any direct commercial application or use in view; (o) ‘gross grant equivalent’ means the amount of the aid if it had been awarded in the form of a grant, before any deduction of tax or other charge; (p) ‘highly qualified personnel’ means staff having a tertiary education degree and at least five years of relevant professional experience which may also include doctoral training; (q) ‘individual aid’ means aid that is not awarded on the basis of an aid scheme and notifiable awards of aid on the basis of an aid scheme; (r) ‘industrial research’ means the planned research or critical investigation aimed at the acquisition of new knowledge and skills for developing new products, processes or services or aimed at bringing about a significant improvement in existing products, processes or services, including digital products, processes or services, in any area, technology, industry or sector (including, but not limited to, digital industries and technologies, such as super-computing, quantum technologies, block chain technologies, artificial intelligence, cyber security, big data and cloud technologies). Industrial research comprises the creation of components parts of complex systems, and may include the construction of prototypes in a laboratory environment or in an environment with simulated interfaces to existing systems as well as of pilot lines, when necessary for the industrial research and notably for generic technology validation; (s) ‘innovation advisory services’ means consultancy, assistance, or training in the fields of knowledge transfer, acquisition, protection or exploitation of intangible assets or the use of standards and regulations embedding them, as well as consultancy, assistance or training on the introduction or use of innovative technologies and solutions (including digital technologies and solutions); (t) ‘innovation clusters’ means structures or organised groups of independent parties (such as innovative start-ups, small, medium and large enterprises, as well as research and knowledge dissemination organisations, research infrastructures, testing and experimentation infrastructures, Digital Innovation Hubs, non-for-profit organisations and other related economic actors), designed to stimulate innovative activity and new ways of collaboration, such as by digital means, by sharing and/or promoting the sharing of facilities and exchange of knowledge and expertise, and by contributing effectively to knowledge transfer, networking, information dissemination and collaboration among the undertakings and other organisations in the cluster  ); (u) ‘innovation support services’ means the provision of office space, data banks, cloud and data storage services, libraries, market research, laboratories, quality labelling, testing, experimentation, and certification or other related services, including those services provided by research and knowledge dissemination organisations, research infrastructures, testing and experimentation infrastructures or innovation clusters, for the purpose of developing more effective or technologically advanced products, processes or services, including the implementation of innovative technologies and solutions (including digital technologies and solutions); (v) ‘intangible assets’ means assets that do not have a physical or financial embodiment such as patents, licences, know-how or other intellectual property; (w) ‘knowledge transfer’ means any process which has the aim of acquiring, collecting and sharing explicit and tacit knowledge, including skills and competence in both economic and non-economic activities such as research collaborations, consultancy, licensing, spin-off creation, publication and mobility of researchers and other personnel involved in those activities. Besides scientific and technological knowledge, it includes other kinds of knowledge such as knowledge on the use of standards and regulations embedding them and on conditions of real life operating environments and methods for organisational innovation, as well as management of knowledge related to identifying, acquiring, protecting, defending and exploiting intangible assets; (x) ‘large enterprises’ means undertakings which do not fall within the definition of small and medium-sized enterprises; (y) ‘net extra costs’ means the difference between the expected net present values of the aided project or activity and a viable counterfactual investment that the beneficiary would have carried out in the absence of aid; (z) ‘organisational innovation  )’ means the implementation of a new organisational method at the level of the undertaking (at group level in the given industry sector in the EEA), workplace organisation or external relations, including for instance by making use of novel or innovative digital technologies. Excluded from this definition are changes that are based on organisational methods already in use in the undertaking, changes in management strategy, mergers and acquisitions, ceasing to use a process, simple capital replacement or extension, changes resulting purely from changes in factor prices, customisation, localisation, regular, seasonal and other cyclical changes and trading of new or significantly improved products; (aa) ‘personnel costs’ means the cost of researchers, technicians and other supporting staff to the extent employed on the relevant project or activity; (bb) ‘pre-commercial procurement’ means the public procurement of research and development services where the contracting authority or contracting entity does not reserve all the results and benefits of the contract exclusively for itself for use in the conduct of its own affairs, but shares them with the providers under market conditions. The contract, the object of which falls within one or several categories of research and development defined in this framework, must be of limited duration and may include the development of prototypes or limited volumes of first products or services in the form of a test series. The purchase of commercial volumes of products or services must not be an object of the same contract; (cc) ‘process innovation  )’ means the implementation of a new or significantly improved production or delivery method (including significant changes in techniques, equipment or software) at the level of the undertaking (at group level in the given industry sector in the EEA), including for instance by making use of novel or innovative digital technologies or solutions. Excluded from this definition are minor changes or improvements, increases in production or service capabilities through the addition of manufacturing or logistical systems which are very similar to those already in use, ceasing to use a process, simple capital replacement or extension, changes resulting purely from changes in factor prices, customisation, localisation, regular, seasonal and other cyclical changes and trading of new or significantly improved products; (dd) ‘R&D project’ means an operation that includes activities spanning over one or several categories of research and development defined in this framework, and that is intended to accomplish an indivisible task of a precise economic, scientific or technical nature with clearly pre-defined goals. A R&D project may consist of several work packages, activities or services, and includes clear objectives, activities to be carried out to achieve those objectives (including their expected costs), and concrete deliverables to identify the outcomes of those activities and compare them with the relevant objectives. When two or more R&D projects are not clearly separable from each other and in particular when they do not have independent probabilities of technological success, they are considered as a single project; (ee) ‘repayable advance’ means a loan for a project which is paid in one or more instalments and the conditions for the reimbursement of which depend on the outcome of the project; (ff) ‘research and knowledge dissemination organisation’ or ‘research organisation’ means an entity (such as universities or research institutes, technology transfer agencies, innovation intermediaries, research-oriented physical or virtual collaborative entities), irrespective of its legal status (organised under public or private law) or way of financing, whose primary goal is to independently conduct fundamental research, industrial research or experimental development or to widely disseminate the results of such activities by way of teaching, publication or knowledge transfer. Where such entity also pursues economic activities, the financing, the costs and the revenues of those economic activities must be accounted for separately. Undertakings that can exert a decisive influence upon such an entity, for example in the quality of shareholders or members, may not enjoy a preferential access to the results generated by it; (gg) ‘research infrastructure’ means facilities, resources and related services that are used by the scientific community to conduct research in their respective fields and covers scientific equipment or set of instruments, knowledge-based resources such as collections, archives or structured scientific information, enabling information and communication technology-based infrastructures such as grid, computing, software and communication, or any other entity of a unique nature essential to conduct research. Such infrastructures may be ‘single-sited’ or ‘distributed’ (an organised network of resources)  ); (hh) ‘secondment’ means temporary employment of staff by a beneficiary with the right for the staff to return to the previous employer; (ii) ‘small and medium-sized enterprises’ or ‘SMEs’, ‘small enterprises’ and ‘medium-sized enterprises’ means undertakings fulfilling the criteria laid down in the Commission recommendation on the definition of micro, small and medium-sized enterprises  ); (jj) ‘start of works’ or ‘start of the project’ means either the start of R&D&I activities, or the first agreement between the beneficiary and the contractors to conduct the project, whichever comes first. Preparatory works such as obtaining permits and conducting feasibility studies are not considered as start of works; (kk) ‘tangible assets’ means assets consisting of land, buildings and plants, machinery and equipment; (ll) ‘testing and experimentation infrastructure  )’ means facilities, equipment, capabilities and resources, such as test beds, pilot lines, demonstrators, testing facilities or living labs, and related support services that are used predominantly by undertakings, especially SMEs, which seek support for testing and experimentation, in order to develop new or improved products, processes and services, and to test and upscale technologies, to advance through industrial research and experimental development. Access to publicly funded testing and experimentation infrastructures is open to several users and must be granted on a transparent and non-discriminatory basis and on market terms. 2.    State aid within the meaning of Article 107(1) of the Treaty 17. Generally, any measure meeting the criteria set out in Article 107(1) of the Treaty constitutes State aid. Whilst a separate Commission Notice  ) on the notion of State aid clarifies how the Commission understands the notion of State aid in general, this Section considers situations typically arising in the field of R&D&I activities without prejudice to the interpretation of the Court of Justice of the European Union. 2.1.    Research and knowledge dissemination organisations and research infrastructures as recipients of State aid 18. Research and knowledge dissemination organisations (‘research organisations’) and research infrastructures are recipients of State aid if their public funding fulfils all conditions of Article 107(1) of the Treaty. Pursuant to the Commission Notice on the notion of State aid, and in accordance with the case-law of the Court of Justice, the beneficiary must qualify as an undertaking, but that qualification does not depend upon its legal status, that is to say whether it is organised under public or private law, or its economic nature, that is to say whether it seeks to make profits or not. Rather, what is decisive for that qualification as an undertaking is whether it carries out an economic activity consisting of offering products or services on a given market  ). 2.1.1.    Public funding of non-economic activities 19. Where the same entity carries out activities of both economic and non-economic nature, the public funding of the non-economic activities will not fall under Article 107(1) of the Treaty if the two kinds of activities and their costs, funding and revenues can be clearly separated so that cross-subsidisation of the economic activity is effectively avoided. Evidence of due allocation of costs, funding and revenues can consist of annual financial statements of the relevant entity. 20. The Commission considers that the following activities are generally of a non-economic character: (a) primary activities of research organisations and research infrastructures, in particular: (i) education for more and better skilled human resources. In line with case-law  ) and decisional practice of the Commission  ), and as explained in the Notice on the notion of State aid and the SGEI Communication  ), public education organised within the national educational system, predominantly or entirely funded by the State and supervised by the State is considered as a non-economic activity  ); (ii) independent R&D for more knowledge and better understanding, including collaborative R&D where the research organisation or research infrastructure engages in effective collaboration  ); (iii) wide dissemination of research results on a non-exclusive and non-discriminatory basis, for example through teaching, open-access databases, open publications or open software; (b) knowledge transfer activities, where they are conducted either by the research organisation or research infrastructure (including their departments or subsidiaries) or jointly with, or on behalf of other such entities, and where all profits from those activities are reinvested in the primary activities of the research organisation or research infrastructure. The non-economic nature of those activities is not prejudiced by contracting the provision of corresponding services to third parties by way of open tenders. 21. Where a research organisation or research infrastructure is used for both economic and non-economic activities, public funding falls under State aid rules only insofar as it covers costs linked to the economic activities  ). Where the research organisation or research infrastructure is used almost exclusively for a non-economic activity, its funding may fall outside State aid rules in its entirety  ), provided that the economic use remains purely ancillary, that is to say corresponds to an activity which is directly related to and necessary for the operation of the research organisation or research infrastructure or intrinsically linked to its main non-economic use, and which is limited in scope. For the purposes of this framework, the Commission will consider this to be the case where the economic activities consume exactly the same inputs (such as material, equipment, labour and fixed capital) as the non-economic activities and the capacity allocated each year to such economic activities does not exceed 20 % of the relevant entity’s overall annual capacity. 2.1.2.    Public funding of economic activities of research organisations and research infrastructures 22. Without prejudice to point 21, where research organisations or research infrastructures are used to perform economic activities, such as renting out equipment or laboratories to undertakings, supplying services to undertakings or performing contract research, public funding of those economic activities will generally be considered State aid. 23. However, the Commission will not consider the research organisation or research infrastructure to be a beneficiary of State aid if it acts as a mere intermediary for passing on to the final recipients the totality of the public funding and any advantage acquired through such funding. This is generally the case where: (a) both the public funding and any advantage acquired through such funding are quantifiable and demonstrable, and there is an appropriate mechanism which ensures that they are fully passed on to the final recipients, for example through reduced prices; and (b) no further advantage is awarded to the intermediary because it is either selected through an open tender procedure or the public funding is available to all entities which satisfy the necessary objective conditions, so that customers as final recipients are entitled to acquire equivalent services from any relevant intermediary. 24. Where the conditions in point 23 are fulfilled, State aid rules apply at the level of the final recipients. 2.2.    Indirect State aid to undertakings through public funded research and knowledge dissemination organisations and research infrastructures 25. The question of whether and under which conditions undertakings obtain an advantage within the meaning of Article 107(1) of the Treaty in cases of contract research or research services provided by a research organisation or research infrastructure, as well as in cases of collaboration with a research organisation or research infrastructure must be answered in accordance with general State aid principles. To this purpose, as explained in the Notice on the notion of State aid, it may in particular be necessary to assess whether the behaviour of the research organisation or research infrastructure can be imputed to the State  ). 2.2.1.    Research on behalf of undertakings (contract research or research services) 26. Where a research organisation or research infrastructure is used to perform contract research or provide a research service to an undertaking, which typically specifies the terms and conditions of the contract, owns the results of the research activities and carries the risk of failure, no State aid will usually be passed to the undertaking if the research organisation or research infrastructure receive payment of an adequate remuneration for its services, particularly where one of the following conditions is fulfilled: (a) the research organisation or research infrastructure provides its research service or contract research at market price  ); or (b) where there is no market price, the research organisation or research infrastructure provides its research service or contract research at a price which: — reflects the full costs of the service and generally includes a margin established by reference to those commonly applied by undertakings active in the sector of the service concerned, or — is the result of arm’s length negotiations where the research organisation or research infrastructure, in its capacity as service provider, negotiates in order to obtain the maximum economic benefit at the moment when the contract is concluded and covers at least its marginal costs. 27. Where the ownership of, or access rights to intellectual property rights (‘IPR’) remain with the research organisation or research infrastructure, their market value may be deducted from the price payable for the services concerned. 2.2.2.    Collaboration with undertakings 28. A project is considered to be carried out through effective collaboration where at least two independent parties pursue a common objective based on the division of labour and jointly define its scope, participate in its design, contribute to its implementation and share its financial, technological, scientific and other risks, as well as its results. One or several parties may bear the full costs of the project and thus relieve other parties of its financial risks. The terms and conditions of a collaboration project, in particular as regards contributions to its costs, the sharing of risks and results, the dissemination of results, access to and rules for allocation of IPR, must be concluded prior to the start of the project  ). Contract research and provision of research services are not considered to be forms of collaboration. 29. Where collaboration projects are carried out jointly by undertakings and research organisations or research infrastructures, the Commission considers that no indirect State aid is awarded to the participating undertakings through those entities due to favourable conditions of the collaboration  ) if one of the following conditions is fulfilled: (a) the participating undertakings bear the full cost of the project; or (b) the results of the collaboration which do not give rise to IPR may be widely disseminated and any IPR resulting from the activities of research organisations or research infrastructures are fully allocated to those entities; or (c) any IPR resulting from the project, as well as related access rights are allocated to the different collaboration partners in a manner which adequately reflects their work packages, contributions and respective interests; or (d) the research organisations or research infrastructures receive compensation equivalent to the market price for the IPR which result from their activities and are assigned to the participating undertakings, or to which participating undertakings are allocated access rights. The absolute amount of the value of any contribution, both financial and non-financial, of the participating undertakings to the costs of the research organisations or research infrastructures’ activities that resulted in the IPR concerned, may be deducted from that compensation. 30. For the purpose of point 29(d), the Commission will consider that the compensation received is equivalent to the market price if it enables the research organisations or research infrastructures concerned to enjoy the full economic benefit of those rights, where one of the following conditions is fulfilled: (a) the amount of the compensation has been established by means of an open, transparent and non-discriminatory competitive sale procedure; or (b) an independent expert valuation confirms that the amount of the compensation is at least equal to the market price; or (c) the research organisation or research infrastructure, as seller, can demonstrate that it effectively negotiated the compensation, at arm’s length conditions, in order to obtain the maximum economic benefit at the moment when the contract is concluded, while considering its statutory objectives; or (d) in cases where the collaboration agreement provides the collaborating undertaking with a right of first refusal as regards IPR generated by the collaborating research organisations or research infrastructures, where those entities exercise a reciprocal right to solicit more economically advantageous offers from third parties so that the collaborating undertaking has to match its offer accordingly. 31. If none of the conditions in point 29 are fulfilled, the full value of the contribution of the research organisations or research infrastructures to the project will be considered as an advantage for the collaborating undertakings, to which State aid rules apply. 2.3.    Public procurement of research and development services 32. Public purchasers may procure research and development services from undertakings, through both exclusive development and pre-commercial procurement procedures  ). 33. As long as an open tender procedure for the public procurement is carried out in accordance with the applicable directives  ), the Commission will generally consider that no State aid within the meaning of Article 107(1) of the Treaty is awarded to the undertakings delivering the relevant services  ). 34. In all other cases, including pre-commercial procurement, the Commission will consider that no State aid is awarded to undertakings where the price paid for the relevant services fully reflects the market value of the benefits received by the public purchaser and the risks taken by the participating providers, in particular where all of the following conditions are fulfilled: (a) the selection procedure is open, transparent and non-discriminatory, and is based on objective selection and award criteria specified in advance of the bidding procedure; (b) the envisaged contractual arrangements describing all rights and obligations of the parties, including with regard to IPR, are made available to all interested bidders in advance of the bidding procedure; (c) the procurement does not give any of the participant providers any preferential treatment in the supply of commercial volumes of the final products or services to a public purchaser in the Member State concerned  ); and (d) one of the following conditions is fulfilled: — all results which do not give rise to IPR may be widely disseminated, for example through publication, teaching or contribution to standardisation bodies in a way that allows other undertakings to reproduce them, and any IPR are fully allocated to the public purchaser, or — any service provider to which results giving rise to IPR are allocated is required to grant the public purchaser unlimited access to those results free of charge, and to grant access to third parties, for example by way of non-exclusive licenses, under market conditions. 35. Where the conditions set out in point 34 are not fulfilled, Member States may rely on an individual assessment of the terms of the contract between the public purchaser and the undertaking, without prejudice to the general obligation to notify R&D&I aid pursuant to Article 108(3) of the Treaty. 3.    Compatibility assessment of R&D&I aid 36. On the basis of Article 107(3)(c) of the Treaty, the Commission may consider compatible with the internal market State aid to facilitate the development of certain economic activities within the European Union, where such aid does not adversely affect trading conditions to an extent contrary to the common interest. 37. In this section, the Commission clarifies how it will apply the compatibility assessment principles and, where applicable, lays down specific conditions for aid schemes and additional conditions for individual aid which is subject to the obligation of notification  ). 38. In order to assess whether State aid for R&D&I can be considered compatible with the internal market, the Commission will determine whether the aid measure facilitates the development of a certain economic activity and whether it adversely affects trading conditions to an extent contrary to the common interest. 39. In order to make the assessment referred to in point 39, the Commission will consider the following aspects: (a) First condition: R&D&I aid facilitates the development of an economic activity (i) identify the economic activity (section 3.1.1); (ii) incentive effect: evaluating whether the aid changes the behaviour of the undertaking or undertakings concerned in such a way that such undertaking or undertakings engage in additional activity, which would not be carried out without the aid or which would be carried out in a restricted or different manner or location (section 3.1.2); (iii) the aid does not contravene relevant provisions and principles of EU law (section 3.1.3). (b) Second condition: R&D&I aid does not unduly affect trading conditions to an extent contrary to the common interest (i) need for State intervention: the aid measure must bring about a material improvement that the market cannot deliver by itself, for example by remedying a market failure or addressing an equity or cohesion concern where applicable (section 3.2.1); (ii) appropriateness of the aid measure: the proposed aid measure must be an appropriate policy instrument to facilitate the development of the economic activity (section 3.2.2); (iii) proportionality of the aid (aid to the minimum): the amount and intensity of the aid must be limited to the minimum needed to induce the additional investment or activity by the undertaking(s) concerned (section 3.2.3); (iv) transparency of aid: Member States, the Commission, economic operators, and the public, must have easy access to all relevant acts and to pertinent information about the aid awarded thereunder (section 3.2.4); (v) negative effects that R&D&I aid can have on competition and trade between Member States must be minimized or avoided: (section 3.2.5); (vi) weighing up the positive and negative effects of the aid (section 3.2.6). 3.1.    First condition: R&D&I aid facilitates the development of an economic activity 3.1.1.    Identifying the supported economic activity 40. The Commission will assess, based on the information provided by the Member State, which economic activity will be supported by the notified measure. 3.1.2.    Incentive effect 3.1.2.1.    General conditions 41. R&D&I aid can be found compatible with the internal market if it has an incentive effect. The Commission considers that aid without incentive effect does not facilitate the development of an economic activity. 42. An incentive effect occurs where the aid changes the behaviour of an undertaking in such a way that it engages in additional activities, which it would not carry out or it would carry out in a restricted or different manner without the aid. The aid must however not subsidise the costs of an activity that an undertaking would anyhow incur and must not compensate for the normal business risk of an economic activity  ). 43. The Commission considers that aid does not present an incentive for the beneficiary wherever work on the relevant R&D&I activity  ) has already started prior to the aid application by the beneficiary to the national authorities  ). Where start of works takes place before the aid application is submitted by the beneficiary to the national authorities, the project will not be eligible for aid. 44. The aid application must include at least the applicant’s name and size, a description of the project, including its location and start and end dates, the amount of public support needed to carry it out, and a list of eligible costs. 45. To the extent they constitute State aid, the Commission may consider that fiscal measures have an incentive effect, by stimulating higher R&D&I spending by undertakings, on the basis of evaluation studies  ) provided by Member States. 3.1.2.2.    Additional conditions for individual aid 46. For notifiable individual aid, Member States must demonstrate to the Commission that the aid has an incentive effect and therefore need to provide clear evidence that the aid has a positive impact on the decision of the undertaking to pursue R&D&I activities which would otherwise not have been pursued. In order to enable the Commission to carry out a comprehensive assessment of the aid measure in question, the Member State concerned must provide not only information concerning the aided project but also, to the extent possible, a comprehensive description of what would have happened or could reasonably have been expected to happen without aid, that is to say the counterfactual scenario. The counterfactual scenario may consist in the absence of an alternative project, when supported by evidence, or in a clearly defined and sufficiently predictable alternative project considered by the beneficiary in its internal decision making, and may relate to an alternative project that is wholly or partly carried out outside the Union. 47. In its analysis, the Commission will take into consideration the following elements: (a) specification of intended change: the change in behaviour which is expected to result from State aid, that is to say whether a new project is triggered, or the size, scope or speed of a project is enhanced, has to be well specified; (b) counterfactual analysis: the change of behaviour has to be identified by comparing what the expected outcome and level of intended activity would be with and without aid. The difference between the two scenarios shows the impact of the aid measure and its incentive effect; (c) level of profitability: where a project or investment would not, in itself, be profitable to carry out for an undertaking, but would generate important benefits for society, it is more likely that the aid has an incentive effect; (d) amount of investment and timeframe of cash flows: a high start-up investment, a low level of appropriable cash flows and a significant fraction of the cash flow arising in the very far future or in a very uncertain manner, will be considered positive elements in assessing the incentive effect; (e) level of risk involved: the assessment of risk will in particular take into account the irreversibility of the investment, the probability of commercial failure, the risk that the project will be less productive than expected, the risk that the project undermines other activities of the aid beneficiary and the risk that the project costs undermine its financial viability. 48. Member States are in particular invited to rely on board documents, risk assessments, financial reports, internal business plans, expert opinions and other studies related to the project under assessment. Documents containing information on demand forecasts, cost forecasts, financial forecasts, documents that are submitted to an investment committee and that describe in detail various investment scenarios, or documents provided to financial institutions could help Member States demonstrate the incentive effect. 49. In order to ensure that the incentive effect is established on an objective basis, the Commission may in its assessment compare company-specific data with data concerning the industry in which the aid beneficiary is active. In particular, Member States should where possible provide industry-specific data demonstrating that the beneficiary’s counterfactual scenario, its required level of profitability and its expected cash-flows are reasonable. 50. In that context, the level of profitability can be evaluated by reference to methodologies which are demonstrably used by the beneficiary undertaking or are standard practice in the particular industry concerned, and which may include methods for evaluating the net present value of the project (NPV)  ), the internal rate of return (IRR)  ) or the average return on capital employed (ROCE). 51. Furthermore, in the case of investment support for cross-border R&D activities, research infrastructures, testing and experimentation infrastructures as well as innovation clusters, the Commission will consider investments which facilitate cross-border cooperation or are financed by more than one Member State, to be an element which may strengthen the incentive effect of the aid. In such cases there can be a strong presumption that the aid incentivises R&D&I activities bigger in size or scope, or it facilitates their speedier implementation, or the total project costs become higher (see point 142 below) because of the increased activities, as compared to a project aimed at meeting only national needs. 52. Therefore, aid will not be considered compatible with the internal market in cases where it appears that the same activities could and would be pursued even without the aid. 3.1.3.    No breach of relevant Union law 53. If a State aid measure, the conditions attached to it, including its financing method when the financing method forms an integral part of the State aid measure, or the activity it finances entails a violation of relevant Union law, the aid cannot be declared compatible with the internal market  ). 54. In assessing the compatibility of any individual aid with the internal market, the Commission will notably take into account any infringement proceedings relative to Articles 101 or 102 of the Treaty which may concern the beneficiary of the aid and which may be relevant for its assessment under Article 107(3) of the Treaty  ). 3.2.    Second condition: R&D&I aid does not unduly affect trading conditions to an extent contrary to the common interest 55. Pursuant to Article 107(3)(c) of the Treaty aid to facilitate the development of certain economic activities or of certain economic areas can be declared compatible, but only ‘where such aid does not adversely affect trading conditions to an extent contrary to the common interest.’ 56. The assessment of the negative effects on the internal market involves complex economic and social assessments. This Section sets out the method of exercise of the Commission’s discretion in carrying out the assessment under the second condition of the compatibility assessment referred to in point 39 (b). 57. By its very nature, any aid measure generates distortions of competition and has an effect on trade between Member States. However, in order to establish if the distortive effects of the aid are limited to the minimum, the Commission will verify whether the aid is necessary, appropriate, proportionate and transparent. 58. The Commission will then assess the distortive effect of the R&D&I aid in question on competition and trading conditions. More specifically, aid in the field of R&D&I can cause specific product market distortions and location effects. The Commission will then balance the positive effects of the aid with its negative effects on competition and trade. Where the positive effects outweigh the negative effects, the Commission will declare the aid compatible. 3.2.1.    Need for State intervention 59. A State aid measure must be targeted towards a situation where aid can bring about a material development that the market cannot deliver by itself, for example by remedying a market failure affecting the R&D&I activity or investment in question. 3.2.1.1.    General conditions 60. State aid may be necessary to increase R&D&I in a situation where the market, on its own, fails to deliver an efficient outcome. In order to assess whether State aid is effective in reaching the intended outcome, it is first necessary to identify the problem to be addressed. State aid should be targeted towards situations where it can bring about a material development that the market cannot deliver on its own. Member States should explain how the aid measure can effectively mitigate the identified market failures, hindering the implementation of the R&D&I activity or investment in question by the market on its own. 61. R&D&I takes place through a series of activities, which are usually upstream to a number of product markets and exploit available capabilities to develop new or improved products, services and processes in those product markets or completely new ones, thereby fostering growth in the economy, contributing to territorial and social cohesion or furthering the general consumer interest. However, given the available R&D&I capabilities, market failures may be an obstacle to reaching the optimal output and may lead to an inefficient outcome for the following reasons: (a) positive externalities or knowledge spill-overs: R&D&I often generate benefits for society in the form of positive spill-over effects, for example knowledge spill-overs or enhanced opportunities for other economic actors to develop complementary products and services. However, if left to the market, a number of projects might have an unattractive rate of return from a private perspective, although they would be beneficial for society, because profit seeking undertakings cannot sufficiently appropriate the benefits of their actions when deciding about the amount of R&D&I they should carry out. State aid may therefore contribute to the implementation of projects which result in an overall societal or economic benefit and which would otherwise not be pursued. However, neither are all benefits of R&D&I activities externalities, nor does the presence of externalities alone automatically mean that State aid is compatible with the internal market. In general, consumers are willing to pay for the direct benefit of new products and services while firms can appropriate the benefits from their investment through other existing instruments, such as IPR. In some cases, however, those means are imperfect and leave a residual market failure that may be corrected by State aid. For instance, as is often argued for fundamental research, it may be difficult to exclude others from gaining access to the results of some activities, which might therefore have a public good character. On the other hand, more specific knowledge related to production can often be well protected, for example through patents, allowing the inventor to reap a higher return on the invention; (b) imperfect and asymmetric information: R&D&I activities are characterised by a high degree of uncertainty. Under certain circumstances, due to imperfect and asymmetric information, private investors may be reluctant to finance valuable projects and highly-qualified personnel may be unaware of recruitment possibilities in innovative undertakings. As a result, the allocation of human and financial resources may not be adequate and projects which may be valuable for society or the economy may not be carried out. In certain cases, imperfect and asymmetric information may also hamper access to finance. However, imperfect information and the presence of risk do not automatically justify the need for State aid. Projects with lower private returns on investments not being financed can very well be a sign of market efficiency. Moreover, risk is part of every business activity and is not a market failure in itself. However, in a context of asymmetric information, risk may exacerbate financing problems; (c) coordination and network failures: the ability of undertakings to coordinate with each other or to interact in order to deliver R&D&I may be impaired for various reasons, including difficulties in coordinating among a large number of collaboration partners where some of them have diverging interests, problems in designing contracts, and difficulties in coordinating collaboration due for example to sensitive information being shared. 3.2.1.2.    Additional conditions for individual aid 62. Whilst certain market failures may hamper R&D&I generally, not all undertakings and sectors in the economy are affected by them to the same extent. Consequently, for notifiable individual aid, Member States should provide adequate information about whether the aid addresses a general market failure regarding R&D&I, or a specific market failure to R&D&I affecting, for example, a particular sector or line of business. 63. The Commission will take into consideration the following elements: (a) knowledge spill-overs: level of knowledge dissemination envisaged; specificity of the knowledge created; availability of IPR protection; degree of complementarity with other products and services; (b) imperfect and asymmetric information: level of risk and complexity of R&D&I activities; need for external finance; characteristics of the aid beneficiary regarding access to external finance; (c) coordination failures: number of collaborating undertakings; intensity of collaboration; diverging interests among collaborating partners; problems in designing contracts; problems to coordinate collaboration. 64. In its analysis of an alleged market failure hindering the R&D&I activities to be triggered by the aid measure, the Commission will in particular take into account any available sectoral comparisons and other studies, which should be provided by the Member State concerned. 65. When notifying investment or operating aid for clusters, Member States must provide information on the planned or expected specialisation of the innovation cluster, existing regional potential and presence of clusters in the Union with similar purposes. Where relevant, Member States should also explain how the cluster can have a positive effect on the technological advancement and digital transformation of the Union economy. Where the supported innovation cluster is a Digital Innovation Hub, the Commission may presume such positive effect. In its analysis, the Commission will analyse whether the collaborations which would be stimulated or incentivised by the innovation cluster’s activities might aim, amongst others, to shorten the time needed from the creation of new knowledge to transposing it into innovative applications. These may include new products, services or processes or solutions, also based on digital technologies, or to help transforming the Union economy in accordance with the Green Deal and the Digital Europe Communication, among others. 66. When notifying investment aid for a testing and experimentation infrastructure, Member States must provide detailed and precise information on its planned or expected specialisation, its state of the art character and the role the testing and experimentation infrastructure could play in facilitating at regional, national or Union level the digital and green transition of the Union economy. Member States must also provide information on whether there are similar testing and experimentation infrastructures, whether publicly funded or not, in the Union. In addition, Member States should provide information on the profile of the users, such as size, sector and other relevant information. The Commission will consider in its assessment the extent to which the capacity of the infrastructure would be allocated to services provided to SMEs and hence provide opportunities for SMEs to improve the efficiency of own production processes and their ability to innovate products and business models, in particular facilitated by access to digital technologies. 67. With respect to State aid which is awarded for projects or activities that are also financed by the Union, either directly or indirectly (that is to say by the Commission, by its executive agencies, by joint undertakings established on the basis of Articles 185 and 187 of the Treaty, or by any other implementing bodies where the Union funding is not directly or indirectly under the control of Member States), the Commission will consider that the need for State intervention has been established. 68. On the other hand, where State aid is awarded for projects or activities which, with respect to their technological content, level of risk and size, are similar to those already delivered within the Union at market conditions, the Commission will in principle presume that no market failure is present and will require further evidence of and justification for the need for State intervention. In particular, in the case of testing and experimentation infrastructures and innovation clusters, Member States must demonstrate that the public support will not lead to duplication in services already offered by existing structures operating within the Union, which could lead to idle capacities and put into question the economic viability of the supported investment. 3.2.2.    Appropriateness of the aid measure 69. The proposed aid measure must be an appropriate policy instrument to achieve the intended objective of the aid, that is to say there must not be a better placed and less distortive policy and aid instrument capable of achieving the same results. 3.2.2.1.    Appropriateness among alternative policy instruments 70. State aid is not the only policy instrument available to Member States to promote R&D&I activities. It is important to keep in mind that there may be other, better placed instruments such as demand-side measures involving regulation, public procurement or standardisation, as well as an increase in funding of public research and education and general fiscal measures. The appropriateness of a policy instrument in a given situation is normally linked to the nature of the problem that is being addressed. For instance, reducing market barriers may be more appropriate than State aid to deal with a new entrant’s difficulty to appropriate R&D&I results. Increased investment in education may be more appropriate to deal with a lack of qualified personnel than awarding State aid. 71. Aid for R&D&I can be authorised as an exception to the general prohibition of State aid, where it is necessary to enable the R&D&I in question. An important element in that respect is therefore whether and to what extent aid for R&D&I can be considered an appropriate instrument to increase R&D&I activities, given that other less distortive instruments may achieve the same results. 72. In its compatibility analysis, the Commission takes particularly into account the impact assessment of the proposed measure carried out by the Member State concerned. Measures, for which Member States have considered other policy options and for which the advantages of using a selective instrument such as State aid are established and submitted to the Commission, are considered to constitute an appropriate instrument. 73. With respect to State aid which is awarded for projects or activities that are also financed by the Union, either directly or indirectly, that is to say by the Commission, by its executive agencies, by joint undertakings established on the basis of Articles 185 and 187 of the Treaty, or by any other implementing bodies where the Union funding is not directly or indirectly under the control of Member States, the Commission will consider that the appropriateness of the aid measure has been established. Member States should demonstrate that the State aid for the assessed project or activity would create synergies with any funding or co-financing from Union programmes. 3.2.2.2.    Appropriateness among different aid instruments 74. State aid for R&D&I can be awarded in various forms. Member States should therefore ensure that the aid is awarded in the form that is likely to generate the least distortions of competition and trade. In this respect, where the aid is awarded in forms that provide a direct pecuniary advantage, such as direct grants, exemptions or reductions in taxes or other compulsory charges, or the supply of land, products or services at favourable prices, the Member State concerned must include an analysis of other options and explain why or how other potentially less distortive forms of aid such as repayable advances or forms of aid that are based on debt or equity instruments, such as State guarantees, the purchase of a share-holding or an alternative provision of debt or capital on favourable terms, are less appropriate. 75. The choice of the aid instrument should be made on the basis of the market failure it seeks to address. For instance, where the underlying market failure is a problem of access to external debt finance due to asymmetric information, Member States should normally resort to aid in the form of liquidity support, such as a loan or guarantee, rather than a grant. Where it is also necessary to provide the firm with a certain degree of risk sharing, a repayable advance should normally be the aid instrument of choice. In particular, where aid is awarded in a form other than liquidity support or a repayable advance for activities that are close to the market, Member States must justify the appropriateness of the chosen instrument for tackling the specific market failure in question. For aid schemes implementing the objectives and priorities of Operational Programmes, the financing instrument chosen in those pro7grammes is in principle presumed to be an appropriate instrument. 3.2.3.    Proportionality of the aid 3.2.3.1.    General conditions 76. For any R&D&I aid to be considered proportional, its amount must be limited to the minimum needed for carrying out the aided activity. 3.2.3.1.1.   Maximum aid intensities 77. In order to ensure that the level of aid is proportionate to the market failures which it is intended to address hindering the implementation of the R&D&I activities to be triggered by the aid measure in question, the aid must be determined in relation to a predefined set of eligible costs and limited to a certain proportion of those eligible costs (‘aid intensity’). The aid intensity must be established for each beneficiary of aid, including in a collaboration project. 78. To ensure predictability and a level playing field, the Commission applies maximum aid intensities for R&D&I aid, which are established on the basis of three criteria: (i) the closeness of the aid to the market, as a proxy for its expected negative effects and the need for it, taking into account the potential higher revenues that can be expected from the aided activities; (ii) the size of the beneficiary as a proxy for the more acute difficulties generally faced by smaller undertakings to finance a risky project; and (iii) the acuteness of any market failure, such as the expected externalities in terms of dissemination of knowledge. Therefore, aid intensities should generally be lower for activities linked to development and innovation than for research activities. These considerations apply similarly to the aid intensity for aid for testing and experimentation infrastructures, which, by definition, would provide predominantly services to undertakings for R&D activities closer to the market. 79. The eligible costs for each aid measure covered by this framework are set out in Annex I. When an R&D project encompasses different tasks, each eligible task must fall under the categories of fundamental research, industrial research or experimental development  ). When classifying different activities according to the relevant category  ), the Commission will refer to its own practice as well as to the specific examples and explanations provided in the OECD Frascati Manual  ). 80. The R&D&I eligible costs shall be supported by the most recently available documentary evidence which shall be clear and specific. Additional overheads and other operating expenses, including costs of materials, supplies and similar products, incurred directly as a result of the project, may alternatively be calculated on the basis of a simplified cost approach in the form of a flat-rate of up to 20 %, applied to total eligible direct R&D project costs defined in Annex I, point (a) to (d) and point (g) for health relevant/related R&D projects. In this case, the R&D project costs used for the calculation of indirect costs shall be established on the basis of normal accounting practices and shall comprise only eligible R&D project costs listed in Annex I, points (a) to (d) and point (g) for health relevant/related R&D projects. For projects co-funded under Horizon Europe programme Member States may use the Horizon Europe simplified cost methodology to calculate indirect R&D project costs. 81. The maximum aid intensities generally applicable to all eligible R&D&I measures are set out in Annex II  ). Unless otherwise specified in the Framework, all aid intensities applicable to R&D&I measures under the General Block Exemption Regulation will guide the Commission’s assessment of the categories of notifiable measures. 82. In the case of State aid for a project being carried out in collaboration between research organisations and undertakings, the combination of direct public support and, where they constitute aid, contributions from research organisations to the same project must not exceed the applicable aid intensities for each beneficiary undertaking. 3.2.3.1.2.   Repayable advances 83. If a Member State awards a repayable advance which qualifies as State aid within the meaning of Article 107(1) of the Treaty, the rules laid down in this section apply. 84. Where a Member State can demonstrate, on the basis of a valid methodology based on sufficient verifiable data, that it is possible to calculate the gross grant equivalent of a repayable advance, it may notify an aid scheme and the associated methodology to the Commission. If the Commission accepts the methodology and deems the scheme compatible, the aid may be awarded on the basis of the gross grant equivalent of the repayable advance, up to the aid intensities laid down in Annex II. 85. In all other cases, the repayable advance is expressed as a percentage of the eligible costs and may exceed the applicable maximum aid intensities by 10 percentage points, provided that the following conditions are fulfilled: (a) in case of a successful outcome, the measure must provide that the advance is to be repaid with an interest rate not less than the discount rate resulting from the application of the Communication from the Commission on the revision of the method for setting the reference and discount rates  ); (b) in case of a success exceeding the outcome defined as successful, the Member State concerned should request payments beyond repayment of the advance amount including interest according to the applicable discount rate; (c) in case the project fails, the advance does not have to be fully repaid. In case of partial success, the repayment should be proportional to the degree of success achieved. 86. For the Commission to assess the measure, it must include detailed provisions on the repayment in case of success, which clearly define what will be considered as a successful outcome, on the basis of reasonable and prudent hypothesis. 3.2.3.1.3.   Fiscal measures 87. To the extent it constitutes State aid, the aid intensity of a fiscal measure can be calculated either on the basis of individual projects or, at the level of an undertaking, as the ratio between the overall tax relief and the sum of all eligible R&D&I costs incurred in a period not exceeding three consecutive fiscal years. In the latter case, the fiscal measure may apply without distinction to all eligible activities, but must not exceed the applicable aid intensity for experimental development  ). 3.2.3.1.4.   Cumulation of aid 88. Aid may be awarded concurrently under several aid schemes or cumulated with ad hoc aid, provided that the total amount of State aid for an activity or project does not exceed the aid ceilings laid down in this framework. As noted in point 10, Union funding centrally managed by the institutions, agencies, joint undertakings or other bodies of the Union that is not directly or indirectly under the control of Member States does not constitute State aid and should not be taken into account. Where such Union funding is combined with State aid, the total amount of public funding awarded in relation to the same eligible costs must however not exceed the most favourable funding rate laid down in the applicable rules of Union law. 89. Testing and experimentation infrastructures co-funded by Union funding, agencies, joint undertakings or other bodies of the Union, could benefit from public support up to 100 % of eligible investment costs, provided that the necessary amount of total public funding (i.e. State aid and other sources of public funding) for the project is demonstrated on the basis of a credible funding gap assessment to ensure that the total amount of public funding does not lead to overcompensation  ). 90. Where the expenditure eligible for R&D&I aid is also potentialy eligible in whole or in part for aid for other purposes, the overlapping portion will be subject to the most favourable ceiling under any of the relevant rules. 91. Aid for R&D&I may not be cumulated with de minimis support in respect of the same eligible costs if that would result in an aid intensity exceeding those laid down in this framework. 3.2.3.2.    Additional conditions for individual aid 92. For notifiable individual aid, mere compliance with a set of predefined maximum aid intensities is not sufficient to ensure proportionality. 93. As a general rule, and in order to establish whether the aid is proportional, the Commission will verify that its amount does not exceed the minimum necessary for the aided project to be sufficiently profitable, for example by making possible to achieve an IRR corresponding to the sector or firm specific benchmark or hurdle rate. Normal rates of return required by the beneficiary in other R&D&I projects, its cost of capital as a whole or returns commonly observed in the industry concerned may also be used for this purpose. All relevant expected costs and benefits must be considered over the lifetime of the project, including the costs and revenues stemming from the results of R&D&I activities. 94. Where it is shown, for example by means of internal company documents, that the aid beneficiary faces a clear choice between carrying out either an aided project or an alternative one without aid, the aid will be considered to be limited to the minimum necessary only if its amount does not exceed the net extra costs of implementing the activities concerned, compared to the counterfactual project that would be carried out in the absence of aid. In order to establish the net extra costs, the Commission will compare the expected net present values of the investment in the aided project and the counterfactual project, account being taken of the probabilities of different business scenarios occurring  ). 95. Where aid is awarded for R&D projects or for the construction or upgrade of research infrastructures or for the construction or upgrade of testing and experimentation infrastructures and the Commission can establish, on the basis of the methodology laid down in points 93 or 94, that the aid is strictly limited to the minimum necessary, higher maximum aid intensities than those laid down in Annex II may be allowed, up to the levels set out in the following table:   Small enterprise Medium-sized enterprise Large enterprise Aid for R&D projects       Fundamental research 100 % 100 % 100 % Applied research 80 % 70 % 60 % — subject to effective collaboration between undertakings (for large enterprises cross-border or with at least one SME) or between an undertaking and a research organisation, or — subject to wide dissemination of results, or — subject to the R&D project being carried out in assisted regions fulfilling the conditions of Article 107(3)(a) of the Treaty, or — subject to the R&D project being carried out in assisted regions fulfilling the conditions of Article 107(3)(c) of the Treaty Aid for the construction and upgrade of research infrastructures — subject to at least two Member States providing public funding, or — for research infrastructures evaluated and selected at EU level Aid for the construction and upgrade of testing and experimentation infrastructures — subject to at least two Member States providing the public funding, or — for TEIs evaluated and selected at EU level (covering co-funding or ‘seal-of-excellence’ type of scenario), and/or — subject to the testing and experimentation infrastructure providing services predominantly to SMEs (at least allocating 80 % of its capacity for that purpose) 70 % (65 +5 ) 60 % (55 +5 ) 50 % (45 +5 ) 40 % (35 +5 ) 96. In order to demonstrate that aid is limited to the minimum necessary, Member States must explain how the aid amount has been established. Documentation and calculations used for the analysis of the incentive effect can also be used to assess whether the aid is proportionate. Insofar as the identified need for aid relates mainly to difficulties in attracting debt finance from the market, rather than to a lack of profitability, a particularly apt way to ensure that the aid is kept to the minimum may be to provide it in the form of a loan, guarantee or repayable advance instead of a non-repayable form, such as a grant. 97. Where there are multiple potential candidates for carrying out the aided activity, the proportionality requirement is more likely to be met if the aid is awarded on the basis of transparent, objective and non-discriminatory criteria. 98. In order to address actual or potential direct or indirect distortions of international trade, higher intensities than generally permissible under this framework may be authorised if, directly or indirectly, competitors located outside the Union have received in the last three years or are going to receive aid of an equivalent intensity for similar projects. However, where distortions of international trade are likely to occur after more than three years, given the particular nature of the sector in question, the reference period may be extended accordingly. Where possible, the Member State concerned must provide the Commission with sufficient information to enable it to assess the situation, in particular the need to take account of the competitive advantage enjoyed by a third country competitor. Where the Commission does not have evidence concerning the awarded or proposed aid, it may also base its decision on circumstantial evidence. 99. When gathering evidence, the Commission may use its investigative powers  ). 3.2.4.    Transparency 100. Member States must publish in the European Commission’s transparency award module  ) or on a comprehensive State aid website, at national or regional level: (a) the full text of the individual aid granting decision or the approved aid scheme and its implementing provisions, or a link to it; (b) the following information on each individual aid award granted ad hoc or under an aid scheme approved on the basis of this framework and exceeding EUR 100 000: — Identity of the individual beneficiary  ) — Name — Beneficiary’s identifier — Type of beneficiary undertaking at the time of granting: — SME — Large undertaking — Region in which the beneficiary is located, at NUTS level II or below — The principal economic sector in which the beneficiary has its activities, at NACE group level  ) — Aid element, and, where different, the nominal amount of aid, expressed as full amount in national currency  ) — Aid instrument  ): — Grant/Interest rate subsidy/Debt write-off — Loan/Repayable advances/Reimbursable grant — Guarantee — Tax advantage or tax exemption — Risk finance — Other (please specify) — Date of award and the date of publication — Objective of the aid — Identity of the granting authority or authorities — Where applicable, name of the entrusted entity, and the names of the selected financial intermediaries — Reference of the aid measure  ) 101. Member States must organise their comprehensive State aid websites, as referred to in paragraph 100, in such a way as to allow easy access to the information. Information must be published in a non-proprietary spreadsheet data format, which allows data to be effectively searched, extracted, downloaded and easily published on the internet, for instance in CSV or XML format. The general public must be allowed to access the website without any restrictions, including prior user registration. 102. For schemes in the form of tax advantages, the conditions set out in paragraph 100 (b) will be considered to be fulfilled if Member States publish the required information on individual aid amounts in the following ranges (in EUR million): [0,1-0,5]; [0,5-1]; [1-2]; [2-5]; [5-10]; [10-30]; [30-60]; [60-100]; [100-250]; and [250 and over]. 103. The information referred to in paragraph 100 (b) must be published within six months from the date of award of the aid, or, for aid in the form of tax advantages, within one year from the date the tax declaration is due  ). For aid that is unlawful but subsequently found to be compatible, Member States must publish this information within six months from the date of the Commission’s decision declaring the aid compatible. To enable the enforcement of State aid rules under the Treaty, the information must be available for at least 10 years from the date on which the aid was granted. 104. The Commission will publish on its website the link to the State aid website referred to in paragraph 101. 3.2.5.    Verifying that specific negative effects of R&D&I aid on competition and trading conditions are minimized or avoided 3.2.5.1.    General considerations 105. The Commission will identify the markets that are affected by the aid, taking into account the information provided by the Member State on the product markets concerned, that is to say the markets affected by the change in behaviour of the aid beneficiary. To the extent that a specific innovative R&D&I activity will be associated with multiple future product markets, the impact of State aid will be considered on the set of markets concerned. The Commission will also identify the affected geographic market, which corresponds to the area in which the undertakings of affected product markets operate, and for which competition conditions are sufficiently homogeneous and clearly can be distinguished from those of neighbouring areas. 106. The Commission will further assess the distortions of competition based on the foreseeable impact of the R&D&I aid on competition between undertakings in the product and geographic markets concerned  ), that are likely to be negatively affected by the aid, taking also into account the information provided by the Member States on the competitors and customers or consumers affected. In doing so, where appropriate, the Commission may also identify the competitive interactions (substitutes, complements, including also upstream or downstream markets) where the distortions caused by an aid measure are more likely to occur. 107. The aid allows the aid beneficiary, typically, to gain competitive advantage through, among others, (i) reduced production costs; or (ii) increased production capacity; or (iii) new product development. The Commission considers that the negative effects of the aid will first operate on competitors. For that reason, the Commission should focus at first place on identifying the aid beneficiary’s actual or potential competitors that are likely to be negatively affected by the aid. 108. The Commission identifies two main potential distortions of competition and trade between Member States caused by R&D&I aid, namely product market distortions and location effects. Both types may lead to allocative inefficiencies, undermining the economic performance of the internal market, and distributional concerns, in that the aid affects the distribution of economic activity across regions. 109. As far as distortions on the product markets are concerned, State aid for R&D&I may have an impact on competition in innovation processes and in the product markets where the results of the R&D&I activities are exploited. 3.2.5.1.1.   Effects on product markets 110. State aid for R&D&I can hamper competition in innovation processes and product markets in three ways, namely by distorting the competitive entry and exit process, by distorting dynamic investment incentives and by creating or maintaining market power. (a)    Distorting the competitive entry and exit processes 111. R&D&I aid may prevent the market mechanism from rewarding the most efficient producers and putting pressure on the least efficient to improve, restructure or exit the market. That might lead to a situation where, due to the aid awarded, competitors that would otherwise be able to stay on are forced out of the market, or never enter in the first place. Similarly, State aid can prevent inefficient firms from leaving the market or even induce them to enter and gain market shares from otherwise more efficient competitors. If not correctly targeted, R&D&I aid may therefore support inefficient undertakings and lead to market structures in which many players operate significantly below efficient scale. In the long run, interfering with the competitive entry and exit processes may stifle innovation and slow down industry-wide productivity improvements. (b)    Distorting dynamic incentives 112. R&D&I aid may distort the dynamic incentives to invest of competitors of the aid beneficiary. When an undertaking receives aid, the likelihood of successful R&D&I activities on its part generally increases, leading to an increased presence on the relevant product market(s) in the future. That increased presence may lead competitors to reduce the scope of their original investment plans (crowding out effect). 113. Furthermore, the presence of aid may make potential beneficiaries complacent or more risk seeking. The long term effect on the overall performance of the sector is in that case likely to be negative. R&D&I aid may therefore, if not correctly targeted, support inefficient undertakings and lead to market structures where many market players operate significantly below efficient scale. (c)    Creating or maintaining market power 114. Aid for R&D&I may also have distortive effects in terms of increasing or maintaining the degree of market power in product markets. Market power is the power to influence market prices, output, the variety or quality of products and services, or other parameters of competition for a significant period of time, to the detriment of consumers. Even where aid does not strengthen market power directly, it may do so indirectly, by discouraging the expansion of existing competitors or inducing their exit or discouraging the entry of new competitors. 3.2.5.1.2.   Effects on trade and location choice 115. State aid for R&D&I may also give rise to distortions of competition when it influences the choice of a location. Those distortions can arise across Member States, either when firms compete across borders or consider different locations. Aid aimed at relocating an activity in another region within the internal market may not lead directly to a distortion in the product market, but it displaces activities or investments from one region into another. 3.2.5.1.3.   Manifest negative effects 116. In principle, an aid measure and the context in which it is applied need to be analysed to identify the extent to which it can be deemed distortive. However, certain situations can be identified where the negative effects manifestly outweigh any positive effects, meaning that aid cannot be found to be compatible with the internal market. 117. In particular, according to the general principles of the Treaty, State aid cannot be considered compatible with the internal market if the aid measure is discriminatory to an extent not justified by its State aid character. As explained in section 3.1.3, the Commission will thus not allow any measure where such measure or the conditions attached to it entail a violation of relevant Union law  ). This is particularly the case for aid measures where the award of aid is subject to the obligation for the beneficiary to have its central seat in the relevant Member State (or to be predominantly established in that Member State) or to use national products or services, as well as for aid measures restricting the possibility for the beneficiary to exploit the R&D&I results in other Member States  ). 3.2.5.2.    Aid schemes 118. In order to be compatible with the internal market, notifiable aid schemes must not lead to significant distortions of competition and trade. In particular, even where distortions may be considered limited at individual level (provided the aid is necessary and proportional to achieve the common objective), on a cumulative basis aid schemes might still lead to high levels of distortions. Such distortions may for instance result from aid that negatively affects dynamic incentives to innovate on the part of competitors. In the case of a scheme focusing on certain sectors, the risk of that kind of distortions is even more pronounced. 119. Without prejudice to point 145, Member States therefore must demonstrate that any negative effects will be limited to the minimum taking into account, for example, the size of the projects concerned, the individual and cumulative aid amounts, the number of expected beneficiaries as well as the characteristics of the targeted sectors. In order to enable the Commission to assess the likely negative effects of notifiable aid schemes, Member States may submit any impact assessment as well as ex post evaluations carried out for similar predecessor schemes. 3.2.5.3.    Additional conditions for individual aid 3.2.5.3.1.   Distortions in product markets 120. For notifiable individual aid, in order to enable the Commission to identify and assess potential distortions of competition and trade, Member States should provide information on (i) the product markets concerned, that is to say the markets affected by the change in behaviour of the aid beneficiary; and (ii) the competitors and customers or consumers affected. 121. In assessing the negative effects of the aid measure, the Commission will focus its analysis of the distortions of competition on the foreseeable impact of the R&D&I aid on competition between undertakings in the product markets concerned. The Commission will give more weight to risks for competition and trade that arise in the near future and with particular likelihood. 122. To the extent that a specific innovative activity will be associated with multiple future product markets, the impact of State aid will be considered on the set of markets concerned. In certain cases the results of R&D&I activities, for example in the form of IPR, are themselves traded in technology markets, for instance through patent licensing or trading. In those cases, the Commission may also consider the effect of the aid on competition in technology markets. 123. The Commission will use various criteria to assess the potential distortions of competition, namely distorting dynamic incentives, creating or maintaining market power, and maintaining inefficient market structures. (a)    Distorting dynamic incentives 124. In its analysis of the potential distortion of dynamic incentives, the Commission will consider the following elements: (i) Market growth: the more the market is expected to grow in the future, the less likely that the competitors’ incentives will be negatively affected by the aid, given that there remain ample opportunities to develop a profitable business; (ii) Aid amount: aid measures which involve significant amounts of aid are more likely to lead to significant crowding out effects. The significance of the aid amount will be measured mainly with reference to the amount spent by the main market players on projects of a similar kind; (iii) Closeness to the market / category of the aid: the more the aid measure is aimed at activities close to the market, the more it is liable to develop significant crowding out effects; (iv) Open selection process: where the aid is awarded on the basis of transparent, objective and non-discriminatory criteria, the Commission will take a more positive stance; (v) Exit barriers: competitors are more likely to maintain, or even to increase their investment plans when exit barriers to the innovation process are high. That may be the case when many of the competitors’ past investments are locked in to a particular R&D&I trajectory; (vi) Incentives to compete for a future market: R&D&I aid may lead to a situation where competitors of the aid beneficiary renounce competing for a future ‘winner takes all’ market, because the advantage provided by the aid, in terms of degree of technological advance, economies of scale, network effects or timing, reduces their possibility to potentially successfully enter that future market; (vii) Product differentiation and intensity of competition: where product innovation is rather about developing differentiated products, related for example to distinct brands, standards, technologies or consumer groups, competitors are less likely to be affected. The same situation arises where there are many effective competitors in the market. (b)    Creating or maintaining market power 125. The Commission is concerned mainly about those R&D&I measures which enable the aid beneficiary to strengthen market power held on existing product markets or to transfer it to future product markets. The Commission is therefore unlikely to identify competition concerns related to market power in cases where the aid beneficiary has a market share below 25 % and in markets with a market concentration below 2 000 on the Herfindahl-Hirschman Index (HHI). 126. In its analysis of market power, the Commission will consider the following elements: (i) Market power of the aid beneficiary and market structure: where the aid recipient is already dominant on a product market, the aid measure may reinforce that dominance by further weakening the competitive constraint that competitors can exert on the recipient undertaking. Similarly, State aid measures may have a significant impact in oligopolistic markets where only a few players are active; (ii) Level of entry barriers: in the field of R&D&I, there may be significant barriers to entry for new entrants. Those barriers include legal entry barriers (in particular in respect of IPR), economies of scale and scope, access barriers to networks and infrastructure, and other strategic barriers to entry or expansion; (iii) Buyer power: the market power of an undertaking may also be limited by the market position of the buyers. The presence of strong buyers can serve to counter a finding of a strong market position if it is likely that the buyers will seek to preserve sufficient competition in the market; (iv) Selection process: aid measures which allow undertakings with a strong market position to influence the selection process, for example by having the right to recommend undertakings in the selection process or influencing the research path in a way which disfavours alternative paths on unjustified grounds, are liable to raise concern by the Commission. (c)    Maintaining inefficient market structures 127. In its analysis of market structures, the Commission will consider whether the aid is awarded in markets featuring overcapacity or in declining industries. However, in situations where the market is growing or where State aid for R&D&I is likely to change the overall growth dynamics or in particular the GHG emissions’ footprint of the sector (in accordance with the European Green Deal and European Digital Strategy Communications), notably as a result of introducing new technologies, for example to achieve decarbonisation or the digitalisation of the production, or both, without an increase in capacities, such aid is not likely to raise concerns. 3.2.5.3.2.   Location effects 128. In particular where R&D&I aid is close to the market, it may result in some territories benefiting from more favourable conditions in respect of subsequent production, particularly because of comparatively lower production costs as a result of the aid or due to higher levels of R&D&I activities pursued through the aid. That may lead undertakings to re-locate to those territories. 129. Location effects may also be relevant to research infrastructures and testing and experimentation infrastructures. If aid is mainly used to attract an infrastructure to a particular region at the expense of another, it will not contribute to promoting further R&D&I activities in the Union. 130. In its analysis of notifiable individual aid, the Commission will accordingly take into account any evidence that the aid beneficiary has considered alternative locations. 131. Likewise, aid that merely leads to a change in location of R&D&I activities within the internal market without changing the nature, size or scope of the project will not be considered compatible. 3.2.6.    Weighing up the positive and the negative effects of the aid 132. The Commission assesses whether the identified negative effects on competition and trading conditions of the aid measure outweigh the positive effects of the planned aid. 3.2.6.1.    Identifying the positive effects to be taken into account 133. There is a correlation between economic growth and R&D&I investment. R&D&I activity increases productivity and boosts economic development. Hence, R&D&I is an important factor for the Union undertakings to ensure economic development by means of developing new products, technologies, services or production processes or both. 134. Investments in R&D&I are of high importance for the development of all sectors of the economy, since they are strongly linked to productivity. 135. As a first step of the balancing test, the Commission will assess the positive effects of the aid on the aided economic activity, duly taking into account the R&D&I activity which is triggered by the aid measure in question, or the size, scope or speed of the R&D&I project which is to be enhanced by the aid measure. 136. In addition, the Commission may also assess whether the aid brings about wider R&D&I-related positive effects. Where such broader positive effects reflect those embodied in Union policies, such as the new ERA for Research and Innovation, the European Green Deal, the European Digital Strategy and the New Industrial Strategy for Europe Communications, then the R&D&I aid aligned with such Union policies can be presumed to have such wider positive effects. 137. The Commission acknowledges that both private and public investments are required to support and speed up R&D&I activities into critical technologies, which when deployed on the market would facilitate the digital transformation of the Union industry and Union’s transition to a zero/low-carbon economy, as well as to a circular and zero-pollution one, where natural capital is protected. The Commission takes a favourable view when the R&D&I activities supported by Member States are in accordance with the Regulation (EU) 2020/852 of the European Parliament and of the Council  ), the latter constituting one of the possible methodologies to identify R&D&I activities for technologies, products or other solutions for environmentally sustainable economic activities. 138. Member States considering awarding State aid for R&D&I, must precisely define the objective pursued, and in particular explain how the measure intends to promote R&D&I. For measures co-financed by the European Structural and Investments Funds, Member States may rely on the reasoning in the relevant Operational Programmes. 139. The Commission takes a favourable view of aid measures, which are an integral part of a comprehensive programme or action plan to stimulate R&D&I activities or smart specialisation strategies, and are supported by rigorous evaluations of similar past aid measures demonstrating their effectiveness. 140. With respect to State aid, which is awarded for projects or activities that are also financed by the Union, either directly or indirectly, that is to say by the Commission, by its executive agencies, by joint undertakings established in accordance with Articles 185 and 187 of the Treaty, or by any other implementing bodies where the Union funding is not directly or indirectly under the control of Member States, the Commission will consider that the related positive effects have been established. 3.2.6.1.1.   Additional considerations for individual aid 141. In order to demonstrate that individual aid subject to the notification obligation (‘notifiable individual aid’) contributes to an increased level of R&D&I activities, Member States may use the following indicators, together with other relevant quantitative or qualitative elements: (a) increase in project size: increase in the total project costs (without a decrease in spending by the aid beneficiary when compared to the situation without aid); increase in the number of people assigned to R&D&I activities; (b) increase in scope: increase in the number of the expected deliverables of the project; increase in the level of ambition of the project evidenced by a higher number of partners involved; increase in cross-border R&D&I activities; a higher probability of a scientific or technological break-through or a higher risk of failure (notably linked to the long-term nature of the project and uncertainty about its results); (c) increase in speed: the completion of the project requires less time when compared to the completion time necessary for the same project carried out without aid; (d) increase in total amount spent: increase in total R&D&I spending by the aid beneficiary, in absolute terms or as a proportion of turnover; changes in the committed budget for the project (without a corresponding decrease in the budget allocated to other projects). 142. In order to conclude that the aid contributes to increasing the level of R&D&I in the Union, the Commission will consider not only the net increase of R&D&I carried out by the undertaking, but also the contribution of the aid to the overall increase of R&D&I spending in the sector concerned, the increase in cross-border R&D&I activities within the Union, as well as to the improvement of the Union situation with regard to R&D&I in the international context. A favourable view will be taken regarding aid measures, for which a publicly available ex post evaluation of their positive effects is envisaged. 3.2.6.2.    Balancing of the positive effects against the negative effects of the aid 143. Finally, the Commission will balance the identified negative effects of the aid measure in terms of distortions of competition and impact on trade between Member States (see section 3.2.1 to 3.2.5) against the positive effects of the planned aid (see section 3.2.6.1) for the development of the economic activities and the Union’s economy or society, or both, and conclude on the compatibility of the aid measure with the internal market only where the positive effects outweigh the negative ones. 144. In cases where the proposed aid measure does not address a well-identified market failure in an appropriate and proportionate way, the negative distortive effects on competition will tend to outweigh the positive effects of the measure hence the Commission is likely to conclude that the proposed aid measure is incompatible. 145. The overall balance of certain categories of aid schemes may further be made subject to a requirement of ex post evaluation referred to in Section 4. In such cases, the Commission may limit the duration of those schemes to four years or less with a possibility of re-notifying their prolongation afterwards. 4.    Evaluation 146. To further ensure that distortion of competition and trade is limited, the Commission may require that aid schemes as referred to in paragraph 147 are subject to an ex post evaluation. Evaluations will be carried out for schemes where the potential distortion of competition and trade is particularly high, i.e. that may risk significantly restricting or distorting competition if implementation is not reviewed in due time. 147. Ex post evaluation may be required for schemes with large aid budgets, or containing novel characteristics, or when significant market, technology or regulatory changes are foreseen. In any case, evaluation will be required for schemes with a State aid budget or accounted expenditure over EUR 150 million in any given year or EUR 750 million over their total duration, i.e. the combined duration of the scheme and any predecessor scheme covering a similar objective and geographical area, starting from 1 January 2022. Given the objectives of the evaluation, and to avoid putting a disproportionate burden on Member States, ex post evaluations are only required for aid schemes the total duration of which exceeds three years, starting from 1 January 2022. 148. The ex post evaluation requirement may be waived for aid schemes that are an immediate successor of a scheme covering a similar objective and geographical area that has been subject to an evaluation, delivered a final evaluation report in compliance with the evaluation plan approved by the Commission and has not generated any negative findings. Where the final evaluation report of a scheme is not in compliance with the approved evaluation plan, that scheme must be suspended with immediate effect. 149. The aim of the evaluation should be to verify whether the assumptions and conditions underlying the compatibility of the scheme have been achieved, in particular the necessity and the effectiveness of the aid measure in the light of its general and specific objectives. It should also assess the impact of the scheme on competition and trade. 150. For aid schemes subject to the evaluation requirement according to paragraph 147, Member States must notify a draft evaluation plan, which will form an integral part of the Commission’s assessment of the scheme, as follows: (a) together with the aid scheme, if the State aid budget of the scheme exceeds EUR 150 million in any given year or EUR 750 million over its total duration; (b) within 30 working days following a significant change that increases the budget of the scheme to over EUR 150 million in any given year or EUR 750 million over the total duration of the scheme; (c) within 30 working days following the recording in official accounts of expenditure under the scheme in excess of EUR 150 million in any year. 151. The draft evaluation plan must be in line with the common methodological principles provided by the Commission  ). Member States must publish the evaluation plan approved by the Commission. 152. The ex post evaluation must be carried out by an expert independent from the aid granting authority on the basis of the evaluation plan. Each evaluation must include at least one interim and one final evaluation report. Member States must publish both reports. 153. The final evaluation report must be submitted to the Commission in due time to assess any prolongation of the aid scheme and at the latest nine months before its expiry. That period may be reduced for schemes triggering the evaluation requirement in their last two years of implementation. The precise scope and arrangements for each evaluation will be set out in the decision approving the aid scheme. The notification of any subsequent aid measure with a similar objective must describe how the results of the evaluation have been taken into account. 5.    Reporting and Monitoring 154. In accordance with Council Regulation (EC) No 2015/1589  ) and Commission Regulation (EC) No 794/2004  ) Member States must submit annual reports to the Commission. 155. Member States must maintain detailed records regarding all aid measures. Such records must contain all information necessary to establish that the conditions regarding eligible costs and maximum aid intensities have been fulfilled. Those records must be maintained for ten years from the date of award of the aid and must be provided to the Commission upon request. 6.    Applicability 156. The Commission will follow the principles and guidelines set out in this Communication for the compatibility assessment of all notified R&D&I aid in respect of which it is called upon to take a decision after 19 October 2022. Unlawful R&D&I aid will be assessed in accordance with the rules applicable on the date on which the aid was awarded. 157. Pursuant to Article 108(1) of the Treaty, the Commission proposes that Member States amend, where necessary, their existing R&D&I aid schemes to ensure compliance with this framework no later than 6 months as of the entry into force of this Framework. 158. Member States are invited to give their explicit unconditional agreement to the appropriate measures referred to in point 157 within two months from the date of publication of this framework in the Official Journal of the European Union. In the absence of a reply from any of the Member States, the Commission will consider that the Member State in question does not agree with the proposed measures. 7.    Revision 159. The commission may decide to review or amend this framework at any time should it be necessary for reasons associated with competition policy or in order to take account of other Union policies and international commitments or for any other justified reason. ( 1 )   The term ‘market failure’ refers to situations in which markets, left to their own devices, are unlikely to produce efficient outcomes. ( 2 )   Amongst others, the rules aim to support R&D&I for digitalisation activities which are understood for the purpose of this Framework as the adoption of innovative technologies carried out by electronic devices and/or systems which make it possible to increase product functionality, develop online services, modernise processes, or migrate to business models based on the disintermediation of goods production and service delivery, eventually producing a transformative impact. R&D&I for digitalisation activities under this Framework are eligible for State aid unless they constitute purely replacement investments in the case of which the necessity and incentive effect of aid is questionable. ( 3 )   Communication from the Commission to the European Parliament, the European Council, the Council, the European Economic and Social Committee and the Committee of the Regions – ‘The European Green Deal’, COM(2019) 640 final, 11 December 2019. ( 4 )   Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions – ‘Shaping Europe’s digital future’, COM(2020) 67 final, 19 February 2020. ( 5 )   Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions – ‘2030 Digital Compass: the European way for the Digital Decade’, COM(2021), 118 final, 9 March 2021. ( 6 )   Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions – ‘A European strategy for data’, COM(2020) 66 final, 19 February 2020. ( 7 )   Communication from the Commission to the European Parliament, the European Council, the Council, the European Economic and Social Committee and the Committee of the Regions – ‘A New Industrial Strategy for Europe’, COM(2020) 102 final, 10 March 2020. ( 8 )   Communication from the Commission to the European Parliament, the European Council, the Council, the European Economic and Social Committee and the Committee of the Regions – ‘Updating the 2020 New Industrial Strategy: Building a stronger Single Market for Europe’s recovery’, COM(2021) 350 final, 5 May 2021. ( 9 )   Communication from the Commission to the European Parliament, the European Council, the Council, the European Economic and Social Committee and the Committee of the Regions – ‘Europe’s moment: Repair and Prepare for the Next Generation’, COM(2020) 456 final, 27 May 2020. ( 10 )   Communication from the Commission to the European Parliament, the European Council, the Council, the European Economic and Social Committee and the Committee of the Regions – ‘Building a European Health Union – preparedness and resilience’, COM(2020)724 final, 11 November 2020. ( 11 )   Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions – ‘A new ERA for Research and Innovation’, COM(2020) 628 final, 30 September 2020. ( 12 )   Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions – ‘A new Circular Economy Action Plan For a cleaner and more competitive Europe’, COM(2020) 98 final, 11 March 2020. ( 13 )   This target was endorsed in the Council (Competitiveness) Conclusions of 1 December 2020. ( 14 )   This framework does not apply to patent box regimes. ( 15 )   Including funding provided under Horizon Europe or the Digital Europe Programme. ( 16 )   Communication from the Commission, ‘Community Guidelines on State aid for rescuing and restructuring firms in difficulty’ ( OJ C 244, 1.10.2004, p. 2 ). ( 17 )   See Judgment of the Court of First Instance of 13 September 1995, TWD Textilwerke Deggendorf GmbH v. Commission , Joined Cases T-244/93 and T-486/93, ECLI:EU:T:1995:160. ( 18 )   The Commission considers that it is useful to maintain different categories of R&D activities regardless of the fact that those activities may follow an interactive model rather than a linear model. ( 19 )   Key enabling technologies are defined and identified in the Communication from the Commission, ‘A European strategy for Key Enabling Technologies – A bridge to growth and jobs’, COM(2012) 341 final, 26 June 2012. ( 20 )   Commission Regulation (EU) No 651/2014 of 17 June 2014 declaring certain categories of aid compatible with the internal market in application of Articles 107 and 108 of the Treaty ( OJ L 187, 26.6.2014, p. 1 ). ( 21 )   Commission Regulation (EU) No 1407/2013 of 18 December 2013 on the application of Articles 107 and 108 of the Treaty on the Functioning of the European Union to de minimis aid ( OJ L 352, 24.12.2013, p. 1 ). ( 22 )   Commission Regulation (EU) No 651/2014 of 17 June 2014 declaring certain categories of aid compatible with the internal market in application of Articles 107 and 108 of the Treaty ( OJ L 187, 26.6.2014, p. 1 ). ( 23 )   Commission Regulation (EU) No 1407/2013 of 18 December 2013 on the application of Articles 107 and 108 of the Treaty on the Functioning of the European Union to de minimis aid ( OJ L 352, 24.12.2013, p. 1 ). ( 24 )   Council Regulation (EU) No 2015/1588 of 13 July 2015 on the application of Articles 107 and 108 of the Treaty on the Functioning of the European Union to certain categories of horizontal State aid ( OJ L 248, 24.9.2015, p. 1 ), as amended by Council Regulation (EU) 2018/1911 of 26 November 2018 ( OJ L 311, 7.12.2018, p. 8 ). ( 25 )   The criteria for the analysis of the compatibility with the internal market of State aid to promote the execution of important projects of common European interest, including R&D&I aid assessed on the basis of Article 107(3)(b) of the Treaty, are laid down in a separate Communication from the Commission. ( 26 )   See the Communication from the Commission on the revision of the method for setting the reference and discount rates ( OJ C 14, 19.1.2008, p. 6 ). ( 27 )   Digital Innovation Hubs (including also European Digital Innovation Hubs supported under the centrally managed DEP programme), whose aim is to stimulate the broad uptake of digital technologies like, but not limited to artificial intelligence, High Performance Computing and cybersecurity by industry (in particular SMEs) and public sector organisations, may qualify as an innovation cluster by themselves in the meaning of this framework, depending on the specific objectives pursued or activities/ functionalities offered by the Digital Innovation Hub. ( 28 )   Organisational innovation may also include social innovation providing that social innovation activities fall within the scope of the definition. ( 29 )   Process innovation may also include social innovation providing that the social innovation activities fall within the scope of the definition. ( 30 )   See Article 2(a) Council Regulation (EC) No 723/2009 of 25 June 2009 on the Community legal framework for a European Research Infrastructure Consortium (ERIC) ( OJ L 206, 8.8.2009, p. 1 ). ( 31 )   Commission Recommendation of 6 May 2003 concerning the definition of micro, small and medium-sized enterprises ( OJ L 124, 20.5.2003, p. 36 ). ( 32 )   Testing and experimentation infrastructures may also be known by the term technology infrastructures, see Commission Staff Working Document, “Technology Infrastructures”, SWD(2019) 158 final, 8 April 2019. ( 33 )   European Commission, ‘Commission Notice on the notion of State aid as referred to in Article 107(1) of the Treaty on the Functioning of the European Union’ ( OJ C 262, 19.7.2016, p. 1 ). ( 34 )   See Judgment of the Court of Justice of 16 June 1987, Commission v. Italy , C-118/85, ECLI:EU:C:1987:283, paragraph 7; Judgment of the Court of Justice of 18 June 1998, Commission v. Italy , C-35/96, ECLI:EU:C:1998:303, paragraph 36; Judgment of the Court of Justice of 19 February 2002, Wouters , C-309/99, ECLI:EU:C:2002:98, paragraph 46. ( 35 )   See Judgment of the Court of Justice of 27 September 1988, Humble and Edel , C-263/86, ECLI:EU:C:1988:451, paragraphs 9-10, 15-18; Judgment of the Court of Justice of 7 December 1993, Wirth , C-109/92, ECLI:EU:C:1993:916, paragraph 15. ( 36 )   See for instance cases NN54/2006, Přerov logistics College , and N 343/2008, Individual aid to the College of Nyíregyháza for the development of the Partium Knowledge Centre . ( 37 )   See Communication from the Commission on the application of the European Union State aid rules to compensation granted for the provision of services of general economic interest ( OJ C 8, 11.1.2012, p. 4 ), paragraphs 26-29. ( 38 )   Workforce training, in the sense of State aid rules for training aid, does not qualify as a non-economic primary activity of research organisations. ( 39 )   Provision of R&D services and R&D carried out on behalf of undertakings are not considered as independent R&D. ( 40 )   Where a research organisation or research infrastructure is both publicly and privately funded, the Commission will consider this to be the case where the public funding allocated to the relevant entity for a specific accounting period exceeds the costs of non-economic activities incurred in that period. ( 41 )   Since the research community, when conducting ancillary economic activities, derives improved and enhanced expertise and knowledge that can be used to perform the primary non-economic activities of the research organisation or the research infrastructure to the benefit of the society at large. ( 42 )   See Judgment of the Court of Justice of 16 May 2002, France v . Commission, C-482/99, ECLI:EU:C:2002:294, paragraph 24. ( 43 )   Where the research organisation or research infrastructure provides a specific research service or carries out contract research for the first time on behalf of a given undertaking, on a trial basis and during a clearly limited period of time, the Commission will normally consider the price charged as a market price where that research service or contract research is unique and it can be shown that there is no market for it. ( 44 )   This does not include definite agreements on the market value of resulting IPR and the value of contributions to the project. ( 45 )   Including in the form of material transfer agreements, where a research organisation or a research infrastructure transfers materials to an undertaking for the recipient’s own R&D activities. ( 46 )   See the Communication and associated staff working document – Communication from the Commission, ‘Pre-commercial Procurement: Driving innovation to ensure sustainable high quality public services in Europe’, COM(2007) 799 final, 14 December 2007. ( 47 )   See Article 27 of Directive 2014/24/EU of the European Parliament and of the Council of 26 February 2014 on public procurement and repealing Directive 2004/18/EC ( OJ L 94, 28.3.2014, p. 65 ) and Article 45 of Directive 2014/25/EU of the European Parliament and of the Council of 26 February 2014 on procurement by entities operating in the water, energy, transport and postal services sectors and repealing Directive 2004/17/EC ( OJ L 94, 28.3.2014, p. 243 ). Likewise, in the case of a restricted procedure within the meaning of respectively Articles 28 of Directive 2014/24/EU and Article 46 of Directive 2014/25/EU, the Commission will also consider that no State aid is awarded to undertakings, unless interested providers are prevented from tendering without valid reasons. ( 48 )   This will also be the case where public purchasers procure innovative solutions resulting from a preceding R&D procurement, or non-R&D products and services that are to be delivered to a performance level requiring a product, process or organisational innovation. ( 49 )   Without prejudice to procedures that cover both the development and the subsequent purchase of unique or specialised products or services. ( 50 )   The compatibility conditions laid down in a block exemption Regulation remain fully applicable to all other cases of individual aid, including where such aid is awarded on the basis of an aid scheme which is subject to the notification obligation. ( 51 )   See Judgment of the Court of Justice, HGA and Others v. Commission , Joined Cases C-630/11 P to C-633/11 P, ECLI:EU:C:2013:387. ( 52 )   If the aid application is for an R&D project, this does not exclude that the potential beneficiary would have already carried out feasibility studies, which are not covered by the request for aid. ( 53 )   In the case of aid for projects or activities that are carried out in successive phases which may be subject to separate aid awarding procedures, this means that start of works must not take place before the first aid application. In the case of aid awarded under an automatic fiscal aid scheme, such scheme must have been adopted and entered into force before any work on the aided project or activity starts. ( 54 )   Even though this may not be possible ex ante for a newly introduced measure, Member States will be expected to provide evaluation studies on the incentive effect of their own fiscal aid schemes (so that planned or intended methodologies for ex post evaluations should normally be part of the design of such measures). In the absence of any evaluation studies, the incentive effect of fiscal aid schemes may be presumed only for incremental measures. ( 55 )   The net present value of a project is the difference between the positive and negative cash flows over the lifetime of the investment, discounted to their current value (using the weighted average cost of capital). ( 56 )   The IRR is not based on accounting earnings in a given year, but takes into account the stream of future cash flows that the investor expects to receive over the entire lifetime of the investment. It is defined as the discount rate for which the NPV of a stream of cash flows equals zero. ( 57 )   See for instance Judgment of the Court of Justice of 19 September 2000, Germany v. Commission, C-156/98, ECLI:EU:C:2000:467, paragraph 78; Judgment of the Court of Justice of 22 December 2008, Société Régie Networks v. Rhône-Alpes Bourgogne , C-333/07, ECLI:EU:C:2008:764, paragraphs 94-116; Judgment of the Court of Justice of 22 September 2020, Austria v. Commission, C-594/18 P, ECLI:EU:C:2020:742, paragraph 44; Judgment of the Court of Justice of 14 October 2010, Nuova Agricast v . Commission , C-67/09 P, ECLI:EU:C:2010:607, paragraph 51. ( 58 )   See Judgment of the Court of Justice of 15 June 1993, Matra v. Commission , C-225/91, ECLI:EU:C:1993:239, paragraph 42. ( 59 )   This qualification does not necessarily need to follow a chronological approach, moving sequentially over time from fundamental research to activities closer to the market. Accordingly, nothing will prevent the Commission from classifying a task, which is carried out at a later stage of a project as industrial research, while finding that an activity carried out at an earlier stage constitutes experimental development or is not research at all. ( 60 )   For practical purposes and unless it is shown that a different scale should be used in individual cases, the different R&D categories can also be considered to correspond to Technology Readiness Levels 1 (fundamental research), 2-4 (industrial research) and 5-8 (experimental development) – see Communication from the Commission, ‘A European strategy for Key Enabling Technologies – A bridge to growth and jobs’, COM(2012) 341 final, 26 June 2012. ( 61 )   OECD Frascati Manual 2015: Guidelines for Collecting and Reporting Data on Research and Experimental Development, as amended or replaced. ( 62 )   Without prejudice to specific provisions applying to aid for research and development in the agricultural and fisheries sectors, as laid down in a block exemption Regulation. ( 63 )   Communication from the Commission on the revision of the method for setting the reference and discount rates ( OJ C 14, 19.1.2008, p. 6 ). ( 64 )   Conversely, where a fiscal aid measure distinguishes between different R&D categories, the relevant aid intensities must not be exceeded. ( 65 )   A claw back mechanism may be implemented as a safeguard. ( 66 )   In the particular case where aid merely allows for an increase in the speed of completion of the project, the comparison should mostly reflect the different timelines in terms of cash flows and delayed entry in the market. ( 67 )   See Article 25 of Council Regulation (EU) 2015/1589 of 13 July 2015 laying down detailed rules for the application of Article 108 of the Treaty on the Functioning of the European Union ( OJ L 248, 24.9.2015, p. 9 ). ( 68 )   ‘State Aid Transparency Public Search’, available at the following website: https://webgate.ec.europa.eu/competition/transparency/public?lang=en. ( 69 )   With the exception of business secrets and other confidential information in duly justified cases and subject to the Commission’s agreement (Commission communication of 1 December 2003 on professional secrecy in State aid decisions, C(2003) 4582 ( OJ C 297, 9.12.2003, p. 6 )). ( 70 )   Regulation (EC) No 1893/2006 of the European Parliament and of the Council of 20 December 2006 establishing the statistical classification of economic activities NACE Revision 2 and amending Council Regulation (EEC) No 3037/90 as well as certain EC Regulations on specific statistical domains ( OJ L 393, 30.12.2006, p. 1 ). ( 71 )   Gross grant equivalent, or where applicable, the amount of the investment. For operating aid, the annual amount of aid per beneficiary can be provided. For fiscal schemes this amount can be provided by the ranges set out in paragraph 143. The amount to be published is the maximum allowed tax benefit and not the amount deducted each year (e.g. in the context of a tax credit, the maximum allowed tax credit shall be published rather than the actual amount which might depend on the taxable revenues and vary each year). ( 72 )   If the aid is granted through multiple aid instruments, the aid amount shall be provided by instrument. ( 73 )   As provided by the Commission under the electronic procedure referred to in paragraph 21. ( 74 )   If there is no formal requirement for an annual declaration, 31 December of the year for which the aid was granted will be considered as the granting date for encoding purposes. ( 75 )   This analysis may address, where relevant, both input and output markets. ( 76 )   Judgment of the Court of Justice of 22 March 1977, Iannelli & Volpi SpA v. Ditta Paolo Meroni (C-74/76, ECLI:EU:C:1977:51). ( 77 )   Judgment of the Court of Justice of 10 March 2005, Laboratoires Fournier SA v. Direction des vérifications nationales et internationales (C-39/04, ECLI:EU:C:2005:161). ( 78 )   Regulation (EU) 2020/852 of 18 June 2020 on the establishment of a framework to facilitate sustainable investment, and amending Regulation (EU) 2019/2088 ( OJ L 198, 22.6.2020, p. 13 ). ( 79 )   Commission staff working document, Common methodology for State aid evaluation, Brussels, 28 May 2014, SWD(2014) 179 final, or any of its successors. ( 80 )   Council Regulation (EU) 2015/1589 of 13 July 2015 laying down detailed rules for the application of Article 108 of the Treaty on the Functioning of the European Union ( OJ L 248, 24.9.2015, p. 9 ). ( 81 )   Commission Regulation (EC) No 794/2004 of 21 April 2004 implementing Council Regulation (EC) No 659/1999 laying down detailed rules for the application of Article 93 of the EC Treaty ( OJ L 140, 30.4.2004, p. 1 ). Eligible costs Aid for R&D projects (a) Personnel costs: researchers, technicians and other supporting staff to the extent employed on the project. (b) Costs of instruments and equipment to the extent and for the period used for the project. If such instruments and equipment are not used for their full life for the project, only the depreciation costs corresponding to the life of the project, as calculated on the basis of good accounting practice, are considered as eligible. (c) Costs of buildings and land, to the extent and for the period used for the project. With regard to buildings, only the depreciation costs corresponding to the life of the project, as calculated on the basis of good accounting practice are considered as eligible. For land, costs of commercial transfer or actually incurred capital costs are eligible. (d) Cost of contractual research, knowledge and patents bought or licensed from outside sources at arm’s length conditions, as well as costs of consultancy and equivalent services used exclusively for the project. (e) Additional overheads incurred directly as a result of the project. (f) Other operating expenses, including costs of materials, supplies and similar products incurred directly as a result of the project. (g) Specifically for health relevant/related R&D projects ): all costs necessary for the R&D project during its duration, amongst others, personnel costs, costs for digital and computing equipment, for diagnostic tools, for data collection and processing tools, for R&D services, for pre-clinical and clinical trials (trial phases I-IV); phase-IV trials are eligible as long as they allow further scientific or technological advance. Aid for feasibility studies Costs of study. Investment costs in intangible and tangible assets. Aid for the construction and upgrade of testing and experimentation infrastructure Innovation aid for SMEs (a) Costs for obtaining, validating and defending patents and other intangible assets. (b) Costs for secondment of highly qualified personnel from a research and knowledge dissemination organisation or a large enterprise, working on R&D&I activities in a newly created function within the beneficiary and not replacing other personnel. (c) Costs for innovation advisory and support services. Aid for process and organisational innovation (a) Personnel costs to the extent employed on the project. (d) Costs of contractual research, knowledge and patents bought or licensed from outside sources at arm’s length conditions, as well as costs of consultancy and equivalent services used exclusively for the project. (f) Other operating costs, including costs of materials, supplies and similar products, incurred directly as a result of the project. Aid for innovation clusters Investment aid Investment costs in tangible and intangible assets. Operating aid Personnel and administrative costs (including overhead costs) relating to: (a) animation of the cluster to facilitate collaboration, information sharing and the provision or channelling of specialised and customised business support services; (b) marketing of the cluster to increase participation of new undertakings or organisations and to increase visibility; (c) management of the cluster’s facilities; and (d) organisation of training programmes, workshops and conferences to support knowledge sharing and networking and transnational cooperation. ( 1 )   Health relevant/related research includes research into vaccines, medicinal products and treatments, medical devices and hospital and medical equipment, disinfectants, and protective clothing and equipment, and into relevant process innovations for an efficient production of the required products. Maximum aid intensities   Small enterprise Medium-sized enterprise Large enterprise Aid for R&D projects       Fundamental research 100 % 100 % 100 % Industrial research 70 % 60 % 50 % — subject to effective collaboration between undertakings (for large enterprises, cross-border or with at least one SME) or between an undertaking and a research organisation; or — subject to the R&D project being carried out in assisted regions fulfilling the conditions of Article 107(3)(c) of the Treaty or — subject to the R&D project being carried out in assisted regions fulfilling the conditions of Article 107(3)(a) of the Treaty Experimental development — subject to effective collaboration between undertakings (for large enterprises, cross-border or with at least one SME) or between an undertaking and a research organisation; or — subject to the R&D project being carried out in assisted regions fulfilling the conditions of Article 107(3)(c) of the Treaty, or — in assisted regions fulfilling the conditions of Article 107(3)(c) of the Treaty, or — in assisted regions fulfilling the conditions of Article 107(3)(a) of the Treaty — subject to at least two Member States providing the public funding, or the testing and experimentation infrastructures has been evaluated and selected at EU level, and/or 60 % (55 + 5 ) 50 % (45 + 5 ) 30 % (25 +5 ) — aid for large undertakings is subject to effective collaboration with at least one SME Site Contents Where We Work News & Media Politics and Governance Economic Growth and Poverty Reduction Human Development Peacebuilding and Humanitarian Support Global Environment Development Cooperation Strategies Joint Research JICA Development Studies Program Research Papers Discussion Papers Working Papers Policy Notes Books and Reports Literature Reviews Knowledge Reports Project History Field Reports Policy Briefs Other Publications and Papers News & Topics Interview & Comment Video Contents Email Alert Mail Magazine Updates from JICA Ogata Research Institute on the latest events, publications and more. How to Find Us Blog: “Exploring an Indicator Framework for the Post-2030 International Development Goals that will Follow the United Nations Sustainable Development Goals (SDGs)” X (Twitter) At the JICA Ogata Sadako Research Institute for Peace and Development (JICA Ogata Research Institute), researchers with a wide range of experience and backgrounds forge partnerships with diverse stakeholders to undertake work on important projects. We will share their knowledge and perspectives gained from their research activities in this blog post series. Sato Ichiro , executive senior research fellow, who is part of the research project “ Study on the Indicator Framework for Post-2030 International Development Goals ” by the JICA Ogata Research Institute, writes this blog post to share his insights. Author: Sato Ichiro, executive senior research fellow, JICA Ogata Research Institute The 2030 Agenda for Sustainable Development (the 2030 Agenda), which sets out the 17 Sustainable Development Goals (SDGs), was adopted at the United Nations (UN) in 2015. 2025 will mark its tenth anniversary. We have a little over five years until 2030, the year the SDGs are to be met, but will we be able to meet these 17 goals in time? According to The Sustainable Development Goals Report 2024 , released by the UN in June 2024, the progress of the SDGs has faced many issues. Not only are a significant number of targets under the SDGs showing insufficient progress toward achievement, some are even showing a reversal of progress. How to accelerate efforts by each country and international cooperation to achieve the SDGs is a pressing issue. In fact, this issue is expected to be discussed at the Summit of the Future , to be held by the UN this September. The report points out that in addition to serious delays in progress, even though it has been nearly a decade since the adoption of the 2030 Agenda, data to measure the degree of achievement are still insufficient and are unable to be obtained in a timely manner. Why focus on the indicators? There are 17 SDGs with 169 targets and to measure the progress of each of these targets, the SDG indicator framework sets out 231 indicators. Some of these are tied to multiple targets, so when such duplications are counted as separate indicators they add up to 248 in total. These indicators draw less attention compared to the goals and targets but are actually very important. How far on the journey toward achieving the set goals and targets have member states and our world reached and how much effort is still required to complete that journey? Are we moving toward or away from the goals and targets? Answers to such questions can be found based on the indicator data. Furthermore, based on our understanding of the progress so far and the current standings derived from indicator data, measures to be taken from here need be considered. What is more, the indicators sometimes practically define the contents of the targets. For the SDGs, the goals and targets were first negotiated and agreed, and then discussions on the indicators to measure each target followed. Therefore, the indicators were set only to measure targets that were already defined and were not expected to affect their contents. However, when the definition of a target is broad or vague, its content may essentially only be defined by its indicators (Kim 2023). One example is, “Target 5.b: enhance the use of enabling technologies, in particular ICT, to promote women’s empowerment,” set under “Goal 5: Gender equality.” The only indicator set under this target is Indicator 5.b.1, which looks at the percentage of mobile phone ownership disaggregated by gender. Although it is obvious that it is not realistic to measure how well technology is being used to promote women’s empowerment just by level of mobile phone ownership, when this becomes the only indicator, all attention goes there. Policies to increase women’s mobile phone ownership may be implemented, while attention toward other important aspects may be lost, possibly causing the growth in the percentage of women’s mobile phone ownership to essentially be perceived as the only relevant target. Thus, the indicators, which may seem mundane and relevant only to a limited number of experts, are in fact crucial. JICA Ogata Research Institute is working on the research project “ Study on the Indicator Framework for Post-2030 International Development Goals .” In this project, the challenges now seen in the existing indicator framework of the SDGs are being analyzed. Based on the results, we will attempt to propose what a new indicator framework could look like if intended for the new international development goals that should be set after 2030, the target year of the SDGs. The outline of this project will be shared in this blog post, but first an overview of the challenges seen in the existing indicator framework of the SDGs is presented in the next section. The challenges of the SDG indicator framework: too many or too few? Many challenges relating to the SDG indicators have been pointed out to date. The first is that there are too many indicators. As already explained, there are 231 indicators excluding duplicates. However, some indicators have multiple sub-indicators set within them, and/or data needs to be disaggregated based on categories like gender, age group and area classification (e.g., urban or rural), upon submission. When these are added up, more than 1,000 types of indicator data are required. On top of this, data submission is not a one-off event. Data for the same indicators must be submitted periodically although submission frequency depends on each indicator. Of the roughly 200 UN member states, not a single one has been able to submit all the requested data to date. Data sufficiency is extremely low for many indicators (Dang and Serajuddin 2020). The burden of data collection is heavy especially for developing countries where government resources are limited. Some researchers estimate that if low- and middle-income countries try to get data for all SDG indicators in place, the total cost by 2030 could amount to 45 billion USD (Global Partnership for Sustainable Development Data 2016). Moreover, issues in accuracy, reliability and comparability have been noted with data that have already been submitted (Biggeri et al. 2019). Meanwhile, some researchers are making a seemingly contradictory argument, saying that there are not enough indicators in the current framework. For some of the SDGs, multiple different elements or broad topics are covered by a single target. Yet, as explained earlier, there are also a considerable number of targets with indicators that look at only limited aspects of each one. The story behind this is that when the SDG indicators were being discussed it seems like there was political pressure to restrict the number of indicators (MacFeely 2020). Some academics have therefore proposed increases in the number of SDG indicators and an expansion of their framework so that all important aspects of the targets can be captured (Kim 2023). In light of such seemingly contradicting views—to increase or decrease the number of indicators—how should the indicator framework for the post-2030 international development goals be improved? Exploring an indicator framework for post-2030 international development goals In the ongoing study by the JICA Ogata Research Institute we are working on a few recommendations. These are being made based on the existing SDG indicator framework but are intended for the post-2030 international development goals. The first recommendation is to categorize the global indicators into two groups: core indicators, for which all member states should submit data on a regular basis; and optional indicators, which are all other global indicators. For the core indicators, it is recommended that submitted data are checked by the international organizations in charge of the indicators in question. Moreover, international support to countries with limited statistical capacity should be reinforced and concentrated on data collection and management of the core indicators. This is to ensure that the data submitted are developed and managed following the same definitions and methods to make them comparable between different member states. Considering the burden of data collection for member states and the scale of available international support to increase statistical capacity, a realistic size of the core indicators would be around 50 or less. We further suggest that collection of other non-core global indicator data be optional and be selectively applied based on the development strategies and monitoring needs of each member state. As they are optional, their number does not have to be limited but should be sufficient to include enough indicators to cover the various aspects of the goals and targets. In the study, a trial set of core indicators will be considered based on the current SDG indicators. First, indicators with globally established definitions and data management methods for which data have already been submitted by many member states will be selected from the current SDG indicators, and then these will be further narrowed down by analyzing correlations between them. Previous studies (such as Shuai et al. 2021) have pointed out that the number of indicators can be reduced by taking advantage of the correlation between indicators and our study follows this approach. As the prototype of the core indicators identified with the above method may not be well balanced or lack indicators to capture some important aspects, the complementary addition and replacement of indicators will also be considered. For instance, some recent issues such as the use of artificial intelligence (AI) and measures against pandemics were not seen as significant when the 2030 Agenda was adopted in 2015 but have become important since then. Indicators relevant to such emerging issues may be considered for addition. The second recommendation is to further facilitate setting local targets and indicators by countries and subnational regions. This is not something new and is in fact contemplated in the current SDG framework. Paragraph 75 of Transforming our world: the 2030 Agenda for Sustainable Development , which was adopted at the UN in 2015 and included the SDGs, says that “(t)he Goals and targets will be followed up and reviewed using a set of global indicators. These will be complemented by indicators at the regional and national levels that will be developed by Member States, in addition to the outcomes of work undertaken for the development of the baselines for those targets where national and global baseline data does not yet exist” (note though, that the term “regional” here may refer to supranational regions rather than subnational regions). However, in reality, there are not many countries or subnational regions that set their own local indicators that are distinct from the global indicators of the SDGs, although some global indicators may not be meaningful in the unique local context of countries and regions. We believe in the merit of setting local indicators aligned with development priorities and local contexts. We consider demonstrating the significance of setting local indicators as well as how to develop them, referring to actual cases. As Japan has relatively abundant cases of local SDG indicators set by local authorities, we will start from reviewing such cases in Japan, and then extend to other countries for extracting useful experience and lessons, and presenting synthesized knowledge. As an example of local indicators set by a Japanese municipality let us look at the case of Kanazawa in Ishikawa Prefecture. The city of Kanazawa selected 45 local indicators and a corresponding data list to measure the progress of the Kanazawa Future Visions (Kanazawa SDGs Action Plan). Comments from the public were taken into account during this process. One example is the indicator, “Preserve and pass on the culture and scenery”, which is unique to Kanazawa, a historical city with traditional streetscapes. Relevant data listed under this indicator include the “percentage of residents who feel that opportunities to experience the history and culture are near at hand” and the “number of registered Kanazawa machiya (traditional townhouses with architectural significance that are unique to Kanazawa).” In a few years’ time discussions on post-2030 international development goals will ramp up. We plan to release interim research outcomes in FY 2024 and FY 2025 to contribute to international discussions on this subject. Disclaimer: All opinions expressed in this blog post are the author’s and do not reflect opinions of JICA or the JICA Ogata Research Institute. Biggeri, Mario, David A. Clark, Andrea Ferrannini, and Vincenzo Mauro. 2019. “Tracking the SDGs in an ‘Integrated’ Manner: A Proposal for a New Index to Capture Synergies and Trade-Offs between and within Goals.” World Development 122 (October): 628–47. https://doi.org/10.1016/j.worlddev.2019.05.022. Dang, Hai-Anh H., and Umar Serajuddin. 2020. “Tracking the Sustainable Development Goals: Emerging Measurement Challenges and Further Reflections.” World Development 127 (March):104570. https://doi.org/10.1016/j.worlddev.2019.05.024. Global Partnership for Sustainable Development Data. 2016. “The State of Development Data Funding 2016.” https://opendatawatch.com/wp-content/uploads/2016/09/development-data-funding-2016.pdf. Kim, Rakhyun E. 2023. “Augment the SDG Indicator Framework.” Environmental Science & Policy 142 (April): 62–67. https://doi.org/10.1016/j.envsci.2023.02.004. MacFeely, Steve. 2020. “Measuring the Sustainable Development Goal Indicators: An Unprecedented Statistical Challenge.” Journal of Official Statistics 36 (2): 361–78. https://doi.org/10.2478/jos-2020-0019. Shuai, Chenyang, Long Yu, Xi Chen, Bu Zhao, Shen Qu, Ji Zhu, Jianguo Liu, Shelie A. Miller, and Ming Xu. 2021. “Principal Indicators to Monitor Sustainable Development Goals.” Environmental Research Letters 16(12): 124015. https://doi.org/10.1088/1748-9326/ac3697. About the Author Sato Ichiro Sato is an executive senior research fellow at the JICA Ogata Research Institute since 2022. He joined Japan International Cooperation Agency in 1997 and has worked at its Mexico Office, Brazil Office, Disaster Risk Reduction Group, and the Office for Climate Change. He was seconded to the World Resources Institute from 2018 to 2020. Related research projects Study on the Indicator Framework for Post-2030 International Development Goals Related Experts SATO Ichiro Related news & column 2023.07.12 Blog: Realizing Development That Can Withstand Uncertain Changes in Climate and Society Reccommend Content of the same tag as this article 技協（地球規模課題対応国際科学技術協力） 事業事前評価表 2010 ... www2.jica.go.jp › evaluation › pdf 事業事前評価表（地球規模課題対応国際科学技術協力(SATREPS)）.国際協力機構 地球環境部 . 防災第一チーム. 1．案件名.国 名： フィリピン共和国. Small Language Model Computer Vision Federated Learning Reinforcement Learning Natural Language Processing New Releases Open Source AI AI Webinars 🔥 Promotion/Partnership The Critic-CoT framework operates by engaging LLMs in a step-wise critique process. The model first generates a solution to a given problem and then critiques its output, identifying errors or areas of improvement. Following this, the model refines the solution based on the critique, and this process is repeated iteratively until the solution is either corrected or validated. For example, during experiments on the GSM8K and MATH datasets, the Critic-CoT model could detect and correct errors in its solutions with high accuracy. The iterative nature of this process allows the model to continuously improve its reasoning capabilities, making it more adept at handling complex tasks. The effectiveness of the Critic-CoT framework was demonstrated through extensive experiments. On the GSM8K dataset, which consists of grade-school-level math word problems, the accuracy of the LLM improved from 89.6% to 93.3% after iterative refinement, with a critic filter further increasing accuracy to 95.4%. Similarly, on the more challenging MATH dataset, which includes high school math competition problems, the model’s accuracy increased from 51.0% to 57.8% after employing the Critic-CoT framework, with additional gains observed when applying the critic filter. These results highlight the significant improvements in task-solving performance that can be achieved through the Critic-CoT framework, particularly when the model is tasked with complex reasoning scenarios. In conclusion, the Critic-CoT framework represents a substantial advancement in developing self-critique capabilities for LLMs. This research addresses the critical challenge of enabling AI models to evaluate and improve their reasoning by introducing a structured and iterative refinement process. The impressive gains in accuracy observed in both the GSM8K and MATH datasets demonstrate the potential of Critic-CoT to enhance the performance of AI systems across various complex tasks. This framework improves the accuracy and reliability of AI reasoning and reduces the need for human intervention, making it a scalable and efficient solution for future AI development. Check out the Paper . All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on  Twitter and LinkedIn . Join our  Telegram Channel . If you like our work, you will love our  newsletter.. Don’t Forget to join our  50k+ ML SubReddit Sana Hassan Sana Hassan, a consulting intern at Marktechpost and dual-degree student at IIT Madras, is passionate about applying technology and AI to address real-world challenges. With a keen interest in solving practical problems, he brings a fresh perspective to the intersection of AI and real-life solutions. CircuitNet: A Brain-Inspired Neural Network Architecture for Enhanced Task Performance Across Diverse Domains Harvard researchers introduce a machine learning approach based on gaussian processes that fits single-particle energy levels. CSGO: A Breakthrough in Image Style Transfer Using the IMAGStyle Dataset for Enhanced Content Preservation and Precise Style Application Across Diverse Scenarios Enhancing Machine Learning ML Education Through No-Code AI: Integrating Lightweight AI Tools in Non-Technical Higher Education Programs RELATED ARTICLES MORE FROM AUTHOR Llama-3.1-storm-8b: a groundbreaking ai model that outperforms meta ai’s llama-3.1-8b-instruct and hermes-3-llama-3.1-8b models on diverse benchmarks, minig released by causallm: a groundbreaking scalable ai-language model trained on a synthesis dataset of 120 million entries, why gpu utilization falls short: understanding streaming multiprocessor (sm) efficiency for better llm performance, what if game engines could run on neural networks this ai paper from google unveils gamengen and explores how diffusion models are revolutionizing real-time..., llama-3.1-storm-8b: a groundbreaking ai model that outperforms meta ai’s llama-3.1-8b-instruct and hermes-3-llama-3.1-8b models on..., minig released by causallm: a groundbreaking scalable ai-language model trained on a synthesis dataset..., harvard researchers introduce a machine learning approach based on gaussian processes that fits single-particle..., what if game engines could run on neural networks this ai paper from google..., wavtokenizer: a breakthrough acoustic codec model redefining audio compression, llavaolmobitnet1b: the first ternary multimodal llm capable of accepting image(s) and text inputs to..., the art of ai persuasion: a study on large language model interactions, re-laion 5b dataset released: improving safety and transparency in web-scale datasets for foundation model.... AI Magazine Privacy & TC Cookie Policy 🐝 Partnership and Promotion Join the Fastest Growing AI Research/Dev Newsletter 🔥 Thank You 🙌 Privacy Overview For the Public FINRA Data provides non-commercial use of data, specifically the ability to save data views and create and manage a Bond Watchlist. For Industry Professionals Registered representatives can fulfill Continuing Education requirements, view their industry CRD record and perform other compliance tasks. For Member Firms FINRA GATEWAY Firm compliance professionals can access filings and requests, run reports and submit support tickets. For Case Participants Arbitration and mediation case participants and FINRA neutrals can view case information and submit documents through this Dispute Resolution Portal. Need Help? | Check System Status Log In to other FINRA systems FINRA’s R&D Program Explores Emerging and Innovative Technologies FINRA’s Research and Development (R&D) program is an innovation engine for FINRA - a sandbox for staff to take risks, explore new and disruptive technologies, assess potential usefulness to FINRA use cases, as well as investigate and test solutions to known problems. The program invites interested participants to engage in a framework for collaboration and experimentation, aimed at accelerating emerging technology adoption across the enterprise. This is done through the open submission of ideas, which are then reviewed, prioritized and refined. The ideas with the most innovative and potential significance are then funded as three-to-six-month long R&D projects. “FINRA’s culture is built on our mission and our values,” says Ivy Ho, Vice President of Enterprise Shared Applications, FINRA Research and Development and Nationwide Mortgage Licensing System technology. “The Research and Development program is deliberate in fostering and promoting an environment for collaboration and innovation, which are two core values at FINRA. Innovation, especially that which can be disruptive in nature, is hard. Disruptive, new ideas go against the grain of how an organization is doing things currently.  It might never come naturally. It’s important to create the right environment to experiment.” Research topics have included broad business and technology use cases for advanced analytics, such as data science, machine learning and artificial intelligence. Outcomes have included enabling data and engineering advancements, and operationalization and governance of advanced analytics. “We explore more than just new technologies,” says Greg Wolff, Senior Principal Architect in Technology Development and Services. “We also explore new approaches to our business activities. Our objective is to empower our staff to think creatively and innovatively, to provide a sandbox for staff to take risks, explore ideas and to experiment.  This is how we disrupt incremental thinking and encourage continuous innovation. We want to stay ahead of the technology and business innovation cycles and leverage leading technology to become an even more effective regulator.” The R&D program’s first project originated from Createathon, FINRA’s premiere innovation event, where FINRA staff collaborate to solve essential business problems over the course of several days. “The Three Bounties Project was three Createathon ideas that we combined into one R&D project,” Ivy says. “The project focused on deep learning models for identifying potentially manipulative conduct in the US capital markets. The outcome of the project became the basis for our road map with developing deep learning-based surveillance patterns. FINRA has been granted a US patent for this work and two individuals on the R&D project are named inventors in the patent, it was a proud moment for us.” That first project was just the beginning. “The program is in its fifth year and, thus far, we’ve launched over 60 R&D projects which have the potential to affect all aspects of FINRA,” Ivy says. “Through the program, we’re able to evaluate the applicability of latest research breakthroughs and to test technology feasibility before making technology investments. R&D projects are experiments and with any experimentation, the outcome is uncertain. Having said that, the success of the program has far exceeded our expectations.” Outcomes from the R&D program fall into three categories: proof-of-concept prototypes, recommendations, and best practices and operational guidance and knowledge. “Proof-of-concept prototypes demonstrate the technology and feasibility of its use within specific business use cases,” Greg explained. “This also provides essential information to launch the implementation of new software products or features within an existing product. The second category of outcomes can result in introducing new tools and platforms as enterprise capabilities and support for an enterprise function. The third category of outcome informs our longer-term technology and business strategy and roadmap.” These days, the R&D program has been focusing on generative AI (GenAI) and large language models (LLM). “Recognizing the transformative and disruptive nature of GenAI and LLM technology, the Research & Development program initiated over a dozen projects to gain working knowledge and to determine how best to leverage this technology for FINRA,” Ivy says. “In addition, LLMs present new and unique risks that must be identified, reviewed, and addressed prior to implementation and use at FINRA or other financial firms.” FINRA Technology’s Security Hackathon Energizes Participants and Organizers Technology Staffers Receive FINRA’s Award of Distinction Honors Createathon’s Executive Sponsor Shares Excitement for the 2024 Event To Commemorate Stonewall Rebellion Day, a Q&A with the FINRA Pride Alliance Co-Chair NAEYC Login Member Profile Hello Community Accreditation Portal Online Learning Online Store Popular Searches:   DAP ;  Coping with COVID-19 ;  E-books ;  Anti-Bias Education ;  Online Store Code of Ethics You are here *NEW* Thousands of you have since shared your ideas, needs, and feedback through surveys and focus groups. With that guidance, and led by extraordinary workgroups made up of Governing Board members, educators, faculty, researchers, partners, and advisors, we are honored to be able to launch a public comment period with draft versions of Code of Ethics for your review through November 15, 2024.   English   Español You are invited to engage in the process of collective revision with us. Here are three ways you can provide feedback during this time:    1. Take a survey. NAEYC has prepared surveys for both statements, available in English and Spanish . In addition to offering general feedback opportunities, these surveys will help guide you towards some areas where we are seeking specific feedback on open or unresolved questions.    English Survey   Encuesta en español 2. Email your reflections. NAEYC is committed to reading, and translating if needed, all comments that come our way, so feel free to send your thoughts, in your preferred language, directly to [email protected] .      3. Participate in conferences and focus groups. NAEYC and many Affiliates and Interest Forums will be holding conferences, meetings, and focus groups exploring one or both of these position statement drafts this fall, providing you with opportunities to share feedback in person and/or virtually.    Thank you for helping us shape these collective, shared resources that support early childhood educators, partnering with families, in creating joyful, equitable learning environments for all.  Thank you to the workgroup members who have done tremendous heavy lifting in bringing us to this point.  Leah Austin, President and CEO, The National Black Child Development Institute  Raquel Diaz, Implementation Consultant for Triple P  Cynthia DiCarlo, Professor of Early Childhood Education, Louisiana State University  Christyn Dundorf, Co-director, Teaching Preschool Partners  Zeynep Isik-Ercan, President, National Association of Early Childhood Teacher Educators and Department Chair of Early Childhood, Rowan University  Benita Flores-Muñoz, Member of the NAEYC Commission on Early Childhood Higher Education Accreditation and Retired ECE faculty , Del Mar College  Robin Fox, Interim Provost, University of Wisconsin Whitewater  *Stacey French-Lee, NAEYC Governing Board Member, and Clinical Assistant Professor, Executive Director of the Campus Child Development Program, Early Childhood and Elementary Education, Georgia State University  Heidi Friedel, NAEYC Faith Based Interest Forum Facilitator, Early Childhood Consultant, and Staff Support Specialist for ECE Subhub  Eugene Geist, Associate Professor, Louisiana State University  Georgia Goldburn, Executive Director, Hope For New Haven and Co-founder,CERCLE  *Brian Johnson, NAEYC Governing Board Member, and Assistant Dean, James Madison College at Michigan State University  Sim Loh, Public Policy Specialist, First Up: Champions for Early Education  Andrea Maldonado, Director of Quality Assessment and Recognition, National Association for Family Child Care  Meir Muller, Associate Professor of Early Childhood Education,University of South Carolina  Ernesto Muñoz, Senior Project Manager of Curriculum Literacy, University of Texas  Richelle Patterson, Senior Policy Analyst,  National Education Association Anu Sachdev, President, ACCESS and Adjunct ECE Faculty, East Stroudsburg University  **Ian Schiefelbein, ECE Faculty, Central New Mexico Community College  Ashley Simpson, BIPOC Educator Recruitment and Retention Strategies Program Manager, Aurora Public School District  *Toni Sturdivant, NAEYC Governing Board Member, and Director of Early Learning, Mid-America Regional Council  Tracy Weston, GAEYC District 1 Representative and Co-Founder, Noah's Ark Preschool Academy of Terrell, Inc.  **Reginald Williams, Full Professor of Early Childhood Education, South Carolina State University  *Current NAEYC Governing Board Members  **Former NAEYC Governing Board Members  NAEYC is grateful to our funders and supporters who make this work possible, including those who have donated through the Marilyn M. Smith Applied Research Fund *NUEVO* Miles de ustedes compartieron sus opiniones, necesidades y comentarios a través de encuestas y grupos de discusión. Con esa guía, y liderados por grupos de trabajo extraordinarios compuestos por miembros del Directorio, docentes, socios y asesores, nos honra poder lanzar un período abierto a comentarios del público con versiones borrador de El Código de Conducta Ética y Declaración de Compromiso revisada para su lectura.    Inglés   Español Están invitado a participar en el proceso de revisión colectiva con nosotros. Estas son tres maneras en las que puede enviar sus comentarios durante este período:    1. Responda una encuesta: La NAEYC preparó encuestas para ambas declaraciones, disponibles en inglés y en español . Además de ofrecer oportunidades generales para hacer comentarios, estas encuestas sirven de ayuda para guiarlo hacia algunas áreas en las que buscamos recibir comentarios específicos o preguntas abiertas o sin respuesta.    Encuesta en inglés   Encuesta en español 2. Envíe sus reflexiones por correo electrónico. La NAEYC asume el compromiso de leer, y traducir si es necesario, todos los comentarios que recibamos, de manera que puede enviar libremente sus ideas, en su idioma de preferencia, directamente a [email protected] .     3. Participe en conferencias y grupos de discusión. La NAEYC y muchas Afiliadas y Foros de interés organizarán conferencias, reuniones y grupos de discusión y estudiarán uno o ambos borradores de esta declaración de posición durante este otoño y le ofrecerán oportunidades para compartir sus comentarios de manera presencial y/o virtual. Gracias a ustedes por ayudarnos a dar forma a estos recursos colectivos y compartidos que apoyan a los docentes de educación inicial, en colaboración con las familias, para crear ambientes educativos, disfrutables e igualitarios para todos.   Gracias a los miembros del grupo de trabajo que han hecho un tremendo trabajo para llegar a este punto.  Richelle Patterson, Senior Policy Analyst,  National Education AssociationAnu Sachdev, President,  ACCESS and Adjunct ECE Faculty, East Stroudsburg University  Tracy Weston, GAEYC District 1 Representative and Co-Founder,Noah's Ark Preschool Academy of Terrell, Inc.  NAEYC agradece a nuestros financiadores y patrocinadores que hacen posible este trabajo, incluidos aquellos que han donado a través del Marilyn M. Smith Applied Research Fund. Position Statements (Reaffirmation and Updated, 2011)   NAEYC Code of Ethical Conduct Brochure Supplements More ethics resources. Teaching the NAEYC Code of Ethical Conduct: A Resource Guide, Revised Edition Ethics and the Early Childhood Educator: Using the NAEYC Code, Second Edition Sample Activities from Teaching the NAEYC Code of Ethical Conduct: A Resource Guide, Revised Edition Why naeyc has updated the ethics position statements. In May 2011, the NAEYC Governing Board reaffirmed the 2005 Code and updated this position statement to reflect consistency with the “Supplement for Early Childhood Program Administrators,” which was initially approved in July 2006. Specifically, Section III-C of the Code (Ethical Responsibilities to Colleagues / Responsibilities to Employees) was deleted, as these Ideals and Principles are addressed in the Supplement. Other minor modifications were also made to ensure clarity and consistency. In addition, changes were made to Ideals and Principles that regard responsibilities to families to ensure alignment with current family engagement best practices in the field. The “Supplement for Early Childhood Program Administrators” was also reaffirmed by the NAEYC Governing Board in May 2011, and changes were made to Ideals and Principles that regard responsibilities to families to ensure alignment with current family engagement best practices in the field. In addition, references to the Code of Ethical Conduct, Section III, Part C: Responsibilities to Employees were deleted, as Section III, Part C was deleted in the May 2011 update of the Code. Announcements About AOSIS SA Journal of Industrial Psychology Editorial Team Submission Procedures Submission Guidelines Submit and Track Manuscript Publication fees Make a payment Journal Information Journal Policies Frequently Asked Questions Reviewer Guidelines Article RSS Support Enquiry (Login required) Aosis Newsletter Department of Industrial Psychology and People Management. Open Journal Systems Original research, a relational bureaucracy framework for meaningful internal stakeholder engagement post-covid 19, about the author(s). Orientation:  The coronavirus disease 2019 (COVID-19) pandemic forced organisations to rapidly redesign workplace structures to adapt to a changed and disrupted business world and improve stakeholder relationships. The relational bureaucracy theory (RBT) provides a valuable foundation for increasing stakeholder participation. Research purpose:  We investigate how a relational bureaucracy’s organisational structure promotes internal stakeholders’ involvement in a post-Covid workplace. Motivation for the study:  Limited frameworks illustrate how a newly emergent relational bureaucratic structure can enhance stakeholder involvement and engagement in the new world of work. Research approach/design and method:  The researchers followed a literature review to derive shared meanings in constructing an RBT framework for promoting stakeholder involvement. Main findings:  According to our preliminary research, the organisational type known as the engaged ambassador could be named the relational bureaucratic stakeholder prototype. Seven zones crucial to the business’s overall operation are identified in the stakeholder landscape. Additionally, we illustrate the relational bureaucracy design ideas that promote stakeholder participation. Practical/managerial implications:  We propose that organisations could benefit from stakeholder engagement through interpersonal coordination mechanisms that create, maintain and improve stakeholder relationships through strategic human resource management (HRM) frameworks and RBT. We further argue that a relational bureaucracy’s structure raises stakeholder participation for organisational leadership, coordination and coproduction. Contribution/value-add:  This article integrates some main effects of relational bureaucratic theory to provide a landscape for the needs of internal stakeholders in a disrupted workplace. Sustainable Development Goal Crossref citations. Subscribe to our newsletter Get specific, domain-collection newsletters detailing the latest CPD courses, scholarly research and call-for-papers in your field. SA Journal of Industrial Psychology    |    ISSN: 0258-5200 (PRINT)    |    ISSN: 2071-0763 (ONLINE) ISSN: 2071-0763 Information Author Services Initiatives You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader. All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess . Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications. Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers. Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal. Original Submission Date Received: . Active Journals Find a Journal Proceedings Series For Authors For Reviewers For Editors For Librarians For Publishers For Societies For Conference Organizers Open Access Policy Institutional Open Access Program Special Issues Guidelines Editorial Process Research and Publication Ethics Article Processing Charges Testimonials Preprints.org SciProfiles Encyclopedia Article Menu Subscribe SciFeed Recommended Articles Google Scholar on Google Scholar Table of Contents Find support for a specific problem in the support section of our website. Please let us know what you think of our products and services. Visit our dedicated information section to learn more about MDPI. JSmol Viewer Sustainable development of digital cultural heritage: a hybrid analysis of crowdsourcing projects using fsqca and system dynamics. 1. Introduction 2. problem description and research framework, 2.1. defining digital humanities cultural heritage crowdsourcing projects and their sustainability implications, 2.2. “resource synergy–subject interaction–value co-creation” analytical framework, 2.3. integrated research paradigm based on fsqca-sd, 3. research process and results, 3.1. configuration analysis of digital humanities cultural heritage crowdsourcing projects’ sustainable development based on fsqca, 3.1.1. case selection and data collection, 3.1.2. measurement of condition variables and outcome variable, 3.1.3. data analysis and configuration analysis, 3.2. development of system dynamics simulation model, 3.2.1. model boundary determination and key variable definition, 3.2.2. causal loop diagrams of subsystems and their system dynamics modeling simulation. Configuration elements are not static combinations in project operation but engage in dynamic interactions. The impact of various configuration elements on project development involves a combination of immediate and cumulative effects. The effects of element combinations exhibit path dependence and positive feedback self-reinforcing effects. 3.2.3. Analysis of Simulation Results 3.2.4. theoretical correspondence between simulation results and fsqca findings, 4. discussion, 4.1. research summary, 4.2. theoretical contributions, 5. conclusions, 5.1. summary of research findings, 5.2. practical implications, author contributions, data availability statement, conflicts of interest. Beccherle, P.; Lazzeretti, L. The role of digital technologies for culture-driven local development in Europe: A policy review. Capital Cult. 2023 , 28 , 25–58. [ Google Scholar ] [ CrossRef ] Estermann, B. Diffusion of Open Data and Crowdsourcing among Heritage Institutions: Results of a Pilot Survey in Switzerland. J. Theor. Appl. Electron. Commer. Res. 2014 , 9 , 15–31. [ Google Scholar ] [ CrossRef ] Jones, S.A.N.; Jeffrey, S.; Maxwell, M.; Hale, A.; Jones, C. 3D heritage visualisation and the negotiation of authenticity: The ACCORD project. Int. J. Herit. Stud. 2018 , 24 , 333–353. [ Google Scholar ] [ CrossRef ] Owens, T. Digital cultural heritage and the crowd. Curator 2013 , 56 , 121–130. [ Google Scholar ] [ CrossRef ] Lascarides, M.; Vershbow, B. What’s on the Menu?: Crowdsourcing at the New York Public Library. In Crowdsourcing Our Cultural Heritage ; Routledge: London, UK, 2016; pp. 113–138. [ Google Scholar ] Bhuyan, B.P.; Tomar, R. Crowdsourcing Mechanisms for Reviving Cultural Heritage. In Social Media and Crowdsourcing ; Auerbach Publications: London, UK, 2023; pp. 194–216. [ Google Scholar ] Alam, S.L.; Campbell, J. Temporal motivations of volunteers to participate in cultural crowdsourcing work. Inf. Syst. Res. 2017 , 28 , 744–759. [ Google Scholar ] [ CrossRef ] Zhang, X.; Zhang, W.; Zhao, Y.C.; Zhu, Q. Imbalanced volunteer engagement in cultural heritage crowdsourcing: A task-related exploration based on causal inference. Inf. Process. Manag. 2022 , 59 , 103027. [ Google Scholar ] [ CrossRef ] Ch Ng, E.; Cai, S.; Zhang, T.E.; Leow, F. Crowdsourcing 3D cultural heritage: Best practice for mass photogrammetry. J. Cult. Herit. Manag. Sustain. Dev. 2019 , 9 , 24–42. [ Google Scholar ] [ CrossRef ] Bonacchi, C.; Bevan, A.; Keinan-Schoonbaert, A.; Pett, D.; Wexler, J. Participation in heritage crowdsourcing. Mus. Manag. Curatorship 2019 , 34 , 166–182. [ Google Scholar ] [ CrossRef ] Annad, O.; Bendaoud, A.; Goria, S.E.P. Web information monitoring and crowdsourcing for promoting and enhancing the Algerian geoheritage. Arab. J. Geosci. 2017 , 10 , 276. [ Google Scholar ] [ CrossRef ] Vrbík, D.; Lábus, V. Crowdsourcing of Popular Toponyms: How to Collect and Preserve Toponyms in Spoken Use. ISPRS Int. J. Geo-Inf. 2021 , 10 , 303. [ Google Scholar ] [ CrossRef ] Saker, M.; Frith, J. Coextensive space: Virtual reality and the developing relationship between the body, the digital and physical space. Media Cult. Soc. 2020 , 42 , 1427–1442. [ Google Scholar ] [ CrossRef ] Wang, Y.; Kaplan, N.; Newman, G.; Scarpino, R. CitSci. org: A new model for managing, documenting, and sharing citizen science data. PLoS. Biol. 2015 , 13 , e1002280. [ Google Scholar ] [ CrossRef ] Edmond, J.; Morselli, F. Sustainability of digital humanities projects as a publication and documentation challenge. J. Doc. 2020 , 76 , 1019–1031. [ Google Scholar ] [ CrossRef ] Berry, D.M. Introduction: Understanding the digital humanities. In Understanding Digital Humanities ; Springer: London, UK, 2012; pp. 1–20. [ Google Scholar ] Peng, Q. Digital humanities approach to comparative literature: Opportunities and challenges. Comp. Lit. Stud. 2020 , 57 , 595–610. [ Google Scholar ] [ CrossRef ] Dunn, S.; Hedges, M. How the crowd can surprise us: Humanities crowdsourcing and the creation of knowledge. In Crowdsourcing Our Cultural Heritage ; Routledge: London, UK, 2016; pp. 231–246. [ Google Scholar ] Muenster, S. Digital 3D technologies for humanities research and education: An overview. Appl. Sci. 2022 , 12 , 2426. [ Google Scholar ] [ CrossRef ] Zhao, Y.; Zhu, Q. Evaluation on crowdsourcing research: Current status and future direction. Inform. Syst. Front. 2014 , 16 , 417–434. [ Google Scholar ] [ CrossRef ] O’ Sullivan, J.; Pidd, M. The born-digital in future digital scholarly editing and publishing. Hum. Soc. Sci. Commun. 2023 , 10 , 930. [ Google Scholar ] [ CrossRef ] Brundtland, G.H. Our common future—Call for action. Environ. Conserv. 1987 , 14 , 291–294. [ Google Scholar ] [ CrossRef ] Smithies, J.; Westling, C.; Sichani, A.; Mellen, P.; Ciula, A. Managing 100 Digital Humanities Projects: Digital scholarship and archiving in King’s Digital Lab. Digit. Humanit. Q. 2019 , 13 , 1–45. Available online: http://www.digitalhumanities.org/dhq/vol/13/1/000411/000411.html (accessed on 31 June 2024). Poole, A.H.; Garwood, D.A. Interdisciplinary scholarly collaboration in data-intensive, public-funded, international digital humanities project work. Libr. Infor. Sci. Res. 2018 , 40 , 184–193. [ Google Scholar ] [ CrossRef ] Wald, D.M.; Longo, J.; Dobell, A.R. Design principles for engaging and retaining virtual citizen scientists. Conserv. Biol. 2016 , 30 , 562–570. [ Google Scholar ] [ CrossRef ] [ PubMed ] Heras, V.C.; Moscoso Cordero, M.I.A.S.; Wijffels, A.; Tenze, A.; Jaramillo Paredes, D.E. Heritage values: Towards a holistic and participatory management approach. J. Cult. Herit. Manag. Sustain. Dev. 2019 , 9 , 199–211. [ Google Scholar ] [ CrossRef ] Liu, A. Toward a diversity stack: Digital humanities and diversity as technical problem. PMLA Publ. Mod. Lang. Assoc. Am. 2020 , 135 , 130–151. [ Google Scholar ] [ CrossRef ] Toscano, M.; Cobo, M.J.; Herrera-Viedma, E. Software solutions for web information systems in digital humanities: Review, analysis and comparative study. Prof. Inf. 2022 , 31 , e310211. [ Google Scholar ] [ CrossRef ] Geels, F.W. Ontologies, socio-technical transitions (to sustainability), and the multi-level perspective. Res. Policy 2010 , 39 , 495–510. [ Google Scholar ] [ CrossRef ] Barney, J. Firm resources and sustained competitive advantage. J. Manag. 1991 , 17 , 99–120. [ Google Scholar ] [ CrossRef ] Afuah, A.; Tucci, C.L. Crowdsourcing as a solution to distant search. Acad. Manag. Rev. 2012 , 37 , 355–375. [ Google Scholar ] [ CrossRef ] Michelucci, P.; Dickinson, J.L. The power of crowds. Science 2016 , 351 , 32–33. [ Google Scholar ] [ CrossRef ] Jones, T.M. Instrumental stakeholder theory: A synthesis of ethics and economics. Acad. Manag. Rev. 1995 , 20 , 404–437. [ Google Scholar ] [ CrossRef ] Wu, W.; Gong, X. Motivation and sustained participation in the online crowdsourcing community: The moderating role of community commitment. Internet Res. 2021 , 31 , 287–314. [ Google Scholar ] [ CrossRef ] Cricelli, L.; Grimaldi, M.; Vermicelli, S. Crowdsourcing and open innovation: A systematic literature review, an integrated framework and a research agenda. Rev. Manag. Sci. 2022 , 16 , 1269–1310. [ Google Scholar ] [ CrossRef ] Vargo, S.L.; Lusch, R.F. Institutions and axioms: An extension and update of service-dominant logic. J. Acad. Market. Sci. 2016 , 44 , 5–23. [ Google Scholar ] [ CrossRef ] Bonacchi, C.; Krzyzanska, M. Digital heritage research re-theorised: Ontologies and epistemologies in a world of big data. Int. J. Herit. Stud. 2019 , 25 , 1235–1247. [ Google Scholar ] [ CrossRef ] Joo, S.; Hootman, J.; Katsurai, M. Exploring the digital humanities research agenda: A text mining approach. J Doc 2022 , 78 , 853–870. [ Google Scholar ] [ CrossRef ] Ragin, C.C. Fuzzy-Set Social Science ; University of Chicago Press: Chicago, IL, USA, 2000; pp. 34–36. [ Google Scholar ] Parente, T.C.; Federo, R. Qualitative comparative analysis: Justifying a neo-configurational approach in management research. RAUSP Manag. J. 2019 , 54 , 399–412. [ Google Scholar ] [ CrossRef ] Fiss, P.C. Building better causal theories: A fuzzy set approach to typologies in organization research. Acad. Manag. J. 2011 , 54 , 393–420. [ Google Scholar ] [ CrossRef ] Forrester, J.W. Industrial dynamics. J. Oper. Res. Soc. 1997 , 48 , 1037–1041. [ Google Scholar ] [ CrossRef ] Forrester, J.W. System dynamics—A personal view of the first fifty years. Syst. Dyn. Rev. 2007 , 23 , 345–358. [ Google Scholar ] [ CrossRef ] Armenia, S.; Barnab, F.; Franco, E.; Iandolo, F.; Pompei, A.; Tsaples, G. Identifying policy options and responses to water management issues through System Dynamics and fsQCA. Technol. Forecast. Soc. Chang. 2023 , 194 , 122737. [ Google Scholar ] [ CrossRef ] Roberts, R.E. Qualitative Interview Questions: Guidance for Novice Researchers. Qual. Rep. 2020 , 25 , 3185–3203. [ Google Scholar ] [ CrossRef ] Eisenhardt, K.M. Building theories from case study research. Acad. Manag. Rev. 1989 , 14 , 532–550. [ Google Scholar ] [ CrossRef ] Mariani, M.M.; Machado, I.; Magrelli, V.; Dwivedi, Y.K. Artificial intelligence in innovation research: A systematic review, conceptual framework, and future research directions. Technovation 2023 , 122 , 102623. [ Google Scholar ] [ CrossRef ] Arora, S.K.; Li, Y.; Youtie, J.; Shapira, P. Using the wayback machine to mine websites in the social sciences: A methodological resource. J. Assoc. Inf. Sci. Tech. 2016 , 67 , 1904–1915. [ Google Scholar ] [ CrossRef ] Bengtsson, M. How to plan and perform a qualitative study using content analysis. Nurs. Open 2016 , 2 , 8–14. [ Google Scholar ] [ CrossRef ] Zhao, Y.C.; Lian, J.; Zhang, Y.; Song, S.; Yao, X. Value co-creation in cultural heritage information practices: Literature review and future agenda: An Annual Review of Information Science and Technology (ARIST) paper. J. Assoc. Inf. Sci. Tech. 2024 , 75 , 298–323. [ Google Scholar ] [ CrossRef ] Basurto, X.; Speer, J. Structuring the calibration of qualitative data as sets for qualitative comparative analysis (QCA). Field Methods 2012 , 24 , 155–174. [ Google Scholar ] [ CrossRef ] Rihoux, B.I.T. Qualitative comparative analysis (QCA) and related systematic comparative methods: Recent advances and remaining challenges for social science research. Int. Sociol. 2006 , 21 , 679–706. [ Google Scholar ] [ CrossRef ] Ragin, C.C. Redesigning Social Inquiry: Fuzzy Sets and Beyond ; University of Chicago Press: Chicago, IL, USA, 2009; pp. 106–107. [ Google Scholar ] Sony, M.; Naik, S. Industry 4.0 integration with socio-technical systems theory: A systematic review and proposed theoretical model. Technol. Soc. 2020 , 61 , 101248. [ Google Scholar ] [ CrossRef ] Damschroder, L.J.; Reardon, C.M.; Widerquist, M.A.O.; Lowery, J. The updated Consolidated Framework for Implementation Research based on user feedback. Implement. Sci. 2022 , 17 , 75. [ Google Scholar ] [ CrossRef ] Cao, X.; Ali, A.; Pitafi, A.H.; Khan, A.N.; Waqas, M. A socio-technical system approach to knowledge creation and team performance: Evidence from China. Inf. Technol. People 2021 , 34 , 1976–1996. [ Google Scholar ] [ CrossRef ] Cohen, W.M.; Levinthal, D.A. Absorptive capacity: A new perspective on learning and innovation. Admin. Sci. Quart. 1990 , 35 , 128–152. [ Google Scholar ] [ CrossRef ] Parmar, B.L.; Freeman, R.E.; Harrison, J.S.; Wicks, A.C.; Purnell, L.; De Colle, S. Stakeholder theory: The state of the art. Acad. Manag. Ann. 2010 , 4 , 403–445. [ Google Scholar ] [ CrossRef ] Luna-Reyes, L.F.; Andersen, D.L. Collecting and analyzing qualitative data for system dynamics: Methods and models. Syst. Dyn. Rev. 2003 , 19 , 271–296. [ Google Scholar ] [ CrossRef ] Ghezzi, A.; Gabelloni, D.; Martini, A.; Natalicchio, A. Crowdsourcing: A review and suggestions for future research. Int. J. Manag. Rev. 2018 , 20 , 343–363. [ Google Scholar ] [ CrossRef ] Zhang, X.; Xia, E.; Shen, C.; Su, J. Factors influencing solvers’ behaviors in knowledge-intensive crowdsourcing: A systematic literature review. J. Theor. Appl. Electron. Commer. Res. 2022 , 17 , 1297–1319. [ Google Scholar ] [ CrossRef ] Kantaros, A.; Soulis, E.; Alysandratou, E. Digitization of ancient artefacts and fabrication of sustainable 3D-printed replicas for intended use by visitors with disabilities: The case of Piraeus archaeological museum. Sustainability 2023 , 15 , 12689. [ Google Scholar ] [ CrossRef ] Buche, J.; Siewert, M.B. Qualitative Comparative Analysis (QCA) and Sociology-Perspectives, Potential, and Areas of Application. Z. Soziol. 2015 , 44 , 386–406. [ Google Scholar ] [ CrossRef ] Cappa, F.; Oriani, R.; Peruffo, E.; McCarthy, I. Big data for creating and capturing value in the digitalized environment: Unpacking the effects of volume, variety, and veracity on firm performance. J. Prod. Innov. Manag. 2021 , 38 , 49–67. [ Google Scholar ] [ CrossRef ] Linnenluecke, M.K. Resilience in business and management research: A review of influential publications and a research agenda. Int. J. Manag. Rev. 2017 , 19 , 4–30. [ Google Scholar ] [ CrossRef ] Sterman, J.D. System dynamics modeling: Tools for learning in a complex world. Calif. Manag. Rev. 2001 , 43 , 8–25. [ Google Scholar ] [ CrossRef ] Furnari, S.; Crilly, D.; Misangyi, V.F.; Greckhamer, T.; Fiss, P.C.; Aguilera, R.V. Capturing causal complexity: Heuristics for configurational theorizing. Acad. Manag. Rev. 2021 , 46 , 778–799. [ Google Scholar ] [ CrossRef ] Click here to enlarge figure No. Case Name Initiating Organization Academic Field Task Type 1 Ancient Lives University of Oxford History Transcription and Translation of Papyri 2 By the People Library of Congress History Transcription and Tagging of Historical Documents 3 Smithsonian Digital Volunteers Smithsonian Institution Multidisciplinary Enhancing Accessibility of Digital Collections 4 MicroPasts UK Cultural Heritage Institutions Archaeology and History Crowdsourcing Tasks for Archaeology and Historical Documents 5 Zooniverse International Crowdsourcing Platform Multidisciplinary Various Fields Including Humanities and Natural Sciences 6 Old Weather Zooniverse Project Meteorology Transcription of Ship’s Logs 7 Europeana 1914–1918 Europeana History Collection and Digitization of WWI-Related Items 8 Prokudin-Gorskii Crowdsourcing Project Photography Restoration of Color Photos 9 Transcribe Bentham University College London Philosophy Transcription of Philosopher’s Manuscripts 10 What’s on the Menu? New York Public Library Food Culture Transcription of Historical Menus 11 Wikidata Sister Project of Wikipedia Multidisciplinary Construction of a Knowledge Graph 12 Papers of the War Department US War Department Archives Project History Transcription and Annotation of War Department Documents 13 Cultural Heritage Imaging Non-profit Organization Cultural Heritage Digitization and Crowdsourcing Projects 14 Yad Vashem Yad Vashem Memorial History Entry and Annotation of Holocaust Victim Information 15 Library of Congress Flickr Commons Library of Congress Photo Annotation Tagging and Commenting on Historical Photos 16 The Great War Archive University of Oxford History Collection and Digitization of WWI-Related Items and Letters 17 Field Expedition: Mongolia National Geographic and Mongolian Academy of Sciences Archaeology Marking Potential Archaeological Sites on Satellite Images 18 Measuring the ANZACs New Zealand National Archives and University of Waikato History Transcription and Annotation of Soldiers’ Records Condition SUS_High SUS_ Low Cons_High Cov_High Cons_Low Cov_Low PLA 0.891892 0.871795 0.727273 0.173913 ~PLA 0.310811 0.469565 0.454545 0.168067 DAT 0.891892 0.871795 0.727273 0.173913 ~DAT 0.310811 0.469565 0.454545 0.168067 KNO 0.905405 0.870370 0.727273 0.170732 ~KNO 0.297297 0.458333 0.454545 0.171429 SOC 0.878378 0.872727 0.772727 0.188406 ~SOC 0.324324 0.480000 0.409091 0.148148 MOT 0.891892 0.868421 0.727273 0.173913 ~MOT 0.310811 0.469565 0.454545 0.168067 INT 0.864865 0.888889 0.772727 0.194444 ~INT 0.337838 0.480769 0.409091 0.142857 DIG 0.905405 0.859649 0.681818 0.158537 ~DIG 0.297297 0.458333 0.500000 0.188679 CRO 0.878378 0.872727 0.727273 0.177215 ~CRO 0.324324 0.480000 0.454545 0.164179 SOI 0.878378 0.875000 0.727273 0.177215 ~SOI 0.324324 0.480000 0.454545 0.164179 Condition SUS_High SUS_Low High_1 High_2 High_3 Low_1 Low_2 PLA ● ● ● ⊗ ⊗ DAT ● ● ● ⊗ ⊗ KNO ● ● ● ⊗ SOC ○ ⊗ MOT ○ ○ ⊗ INT ○ ⊗ DIG ● ● ● ⊗ CRO ○ ⊗ SOI ○ ⊗ Consistency 0.963 0.958 0.955 0.912 0.895 Raw Coverage 0.718 0.701 0.729 0.632 0.587 Unique Coverage 0.031 0.014 0.042 0.165 0.120 Solution Consistency 0.951 0.903 Solution Coverage 0.785 0.752 The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. Share and Cite Zhang, Y.; Dong, C. Sustainable Development of Digital Cultural Heritage: A Hybrid Analysis of Crowdsourcing Projects Using fsQCA and System Dynamics. Sustainability 2024 , 16 , 7577. https://doi.org/10.3390/su16177577 Zhang Y, Dong C. Sustainable Development of Digital Cultural Heritage: A Hybrid Analysis of Crowdsourcing Projects Using fsQCA and System Dynamics. Sustainability . 2024; 16(17):7577. https://doi.org/10.3390/su16177577 Zhang, Yang, and Changqi Dong. 2024. "Sustainable Development of Digital Cultural Heritage: A Hybrid Analysis of Crowdsourcing Projects Using fsQCA and System Dynamics" Sustainability 16, no. 17: 7577. https://doi.org/10.3390/su16177577 Article Metrics Article access statistics, further information, mdpi initiatives, follow mdpi. Subscribe to receive issue release notifications and newsletters from MDPI journals 123 IMAGES Key Phases Of Research Development Framework Researcher Development Skills Framework The Researcher Development Framework describes areas of knowledge Research Framework FREE 10+ Research and Development Plan Samples & Templates in MS Word Research Methodology for Framework Development VIDEO THEORETICAL FRAMEWORK CHECKLISTS l PART 2 Implementation of Simplified Review Framework for Research Project Grants What is a Theoretical Framework really? simple explanation Conceptual Framework Theoretical Framework Design Driven External Analysis: a Framework for Adaptation & Innovation in Digitally Enterprises COMMENTS What is a framework? Understanding their purpose, value, development Frameworks are important research tools across nearly all fields of science. They are critically important for structuring empirical inquiry and theoretical development in the environmental social sciences, governance research and practice, the sustainability sciences and fields of social-ecological systems research in tangent with the associated disciplines of those fields (Binder et al. 2013 ... Building an R&D strategy for modern times The global investment in research and development (R&D) is staggering. In 2019 alone, organizations around the world spent $2.3 trillion on R&D—the equivalent of roughly 2 percent of global GDP—about half of which came from industry and the remainder from governments and academic institutions. What is a Conceptual Framework and How to Make It (with Examples) A conceptual framework in research is used to understand a research problem and guide the development and analysis of the research. It serves as a roadmap to conceptualize and structure the work by providing an outline that connects different ideas, concepts, and theories within the field of study. A conceptual framework pictorially or verbally ... An Agile Framework for Research and Development An Agile Framework for Research and Development. A quick search of "Agile for research and development" makes two things clear: Research and development (R&D) processes need agility beyond the software development space. Companies struggle to match agile frameworks to their R&D needs. As LMI generated an operating model to develop prototypes ... Conceptual Framework A conceptual framework is a structured approach to organizing and understanding complex ideas, theories, or concepts. It provides a systematic and coherent way of thinking about a problem or topic, and helps to guide research or analysis in a particular field. A conceptual framework typically includes a set of assumptions, concepts, and ... PDF CHAPTER CONCEPTUAL FRAMEWORKS IN RESEARCH distribute conceptual framework guides every facet of research. In this chapter, we build on that text and the work it builds on and seek to conceptualize the term and highlight the roles and uses of the conceptual framework, as well as the process of developing one, since a conceptual framework is a generative source of thinking, planning, conscious ac. What is a framework? Understanding their purpose, value, development A meta-framework outlining the central role frameworks play in scientific advancement through their development and use. In the center, frameworks provide two core bridging values: knowledge ... What Is a Conceptual Framework? Developing a conceptual framework in research. Step 1: Choose your research question. Step 2: Select your independent and dependent variables. Step 3: Visualize your cause-and-effect relationship. Step 4: Identify other influencing variables. Frequently asked questions about conceptual models. A research and development framework for integrated project delivery The framework aims to help both researchers and practitioners navigate the different components of IPD and guide research efforts to further their development. The R&D framework is built upon established frameworks and a systematic literature review of 175 papers from 2017 to 2022 and was validated using data from three case studies. PDF What is a framework? Understanding their purpose, value, development Positioning frameworks within a theory of science can aid in knowing the purpose and value of framework use. This article provides a meta-framework for visualizing and engaging the four mediating processes for framework development and application: (1) empirical generalization, (2) theoretical fitting, (3) application, and (4) hypothesizing. Theories, Models, & Frameworks Tabak et al's schema organizes 61 dissemination and implementation models based on three variables: 1) construct flexibility, 2) focus on dissemination and/or implementation activities, and 3) socio-ecological framework level. Frame Your Question. Select Research Method. Select Study Design. PDF A GUIDE TO RESEARCH DEVELOPMENT As a research institution, this investment is most often in man-hours spent. developing the initial research proposal. In 2018, the United States government spent $142.9 billion funding research and. development activities.1 This funding makes up only a portion of the overall research. What is a research framework and why do we need one? As above, a framework helps us to determine, based on what we're trying to learn, the right approach and methods to apply in a given situation. It also helps to structure and plan our research activities, according to the breadth and scope of what we're trying to learn. For example, we might reasonably anticipate more foundational ... What is a Theoretical Framework? How to Write It (with Examples) A theoretical framework guides the research process like a roadmap for the study, so you need to get this right. Theoretical framework 1,2 is the structure that supports and describes a theory. A theory is a set of interrelated concepts and definitions that present a systematic view of phenomena by describing the relationship among the variables for explaining these phenomena. Agile manufacturing: A framework for research and development A framework for the design of agile manufacturing systemsIn this section, a framework for the design of AMSs is developed. This development is based on the literature survey and its analysis. It can be seen that most of the literature on AM and related issues either deal with strategies or techniques, but not an integrated view of developing an ... APOS Theory: A Framework for Research and Curriculum Development in A Framework for Research and Curriculum Development in Mathematics Education. Book ... The APOS Paradigm for Research and Curriculum Development. Ilana Arnon, Jim Cottrill, Ed Dubinsky, Asuman Oktaç, Solange Roa Fuentes, María Trigueros et al. Pages 93-108. Download chapter PDF A framework for R&D A framework for R&D. Impactful innovation is what Research & Development (R&D) teams around the world are working hard towards. Yet, there don't seem to be blueprints that are reused and trusted ... Researcher Development Framework (RDF) The Vitae Researcher Development Framework (RDF), is an internationally recognised framework for planning, promoting and supporting the personal, professional and career development of researchers in higher education. It is an approach to researcher development, which aims to provide researchers with a universal language for communicating their capabilities. WfCommons: A framework for enabling scientific workflow research and In this paper, we present WfCommons [17], an open-source framework that aims at supporting and at bridging theoretical and practical aspects of workflow systems research and development.The broad objective of WfCommons is to enable simulation-driven engineering of workflow systems (Fig. 1) [16].As such, it must address three main technical challenges: (i) the gathering and archiving of real ... Agile manufacturing: A framework for research and development A framework for the design of agile manufacturing systems. In this section, a framework for the design of AMSs is developed. This development is based on the literature survey and its analysis. It can be seen that most of the literature on AM and related issues either deal with strategies or techniques, but not an integrated view of developing ... The Researcher Development Framework (RDF) The RDF is a professional development tool that was developed by Vitae. The purpose of the framework is to support researchers in realising and planning their career development. The RDF articulates knowledge, behaviours and attributes of successful researchers at all stages of the research career. It can be used as an aspiration or as a hands ... State aid framework for research and development and ... The RDI framework applies to all technologies, industries and sectors. This includes sectors subject to specific EU State aid rules, unless those rules provide otherwise. The framework distinguishes between non-economic and economic activities of research and knowledge dissemination organisations and research infrastructures, with only the ... EUR-Lex Communication from the Commission Framework for State aid for research and development and innovation 2022/C 414/01 ... ' R&D project ' means an operation that includes activities spanning over one or several categories of research and development defined in this framework, and that is intended to accomplish an indivisible task of a precise ... Blog: "Exploring an Indicator Framework for the Post-2030 International At the JICA Ogata Sadako Research Institute for Peace and Development (JICA Ogata Research Institute), researchers with a wide range of experience and backgrounds forge partnerships with diverse stakeholders to undertake work on important projects. ... Based on the results, we will attempt to propose what a new indicator framework could look ... Critic-CoT: A Novel Framework Enhancing Self-Critique and Reasoning In conclusion, the Critic-CoT framework represents a substantial advancement in developing self-critique capabilities for LLMs. This research addresses the critical challenge of enabling AI models to evaluate and improve their reasoning by introducing a structured and iterative refinement process. FINRA's R&D Program Explores Emerging and Innovative Technologies FINRA's Research and Development (R&D) program is an innovation engine for FINRA - a sandbox for staff to take risks, explore new and disruptive technologies, ... The program invites interested participants to engage in a framework for collaboration and experimentation, aimed at accelerating emerging technology adoption across the enterprise. ... Code of Ethics *NEW* Thousands of you have since shared your ideas, needs, and feedback through surveys and focus groups. With that guidance, and led by extraordinary workgroups made up of Governing Board members, educators, faculty, researchers, partners, and advisors, we are honored to be able to launch a public comment period with draft versions of Code of Ethics for your review through November 15, 2024. A relational bureaucracy framework for meaningful internal stakeholder Research approach/design and method: The researchers followed a literature review to derive shared meanings in constructing an RBT framework for promoting stakeholder involvement. Main findings: According to our preliminary research, the organisational type known as the engaged ambassador could be named the relational bureaucratic stakeholder ... How meeting the ten pedometrics challenges can deliver healthy‐soil Not only cutting-edge research can contribute to defining indicators and thresholds. A hundred years of research has produced many valuable insights and methodologies that can be applied as well. The: 'better' can be the enemy of the: 'good'. The sustainable development challenge is highly urgent: there is no time to lose. Sustainability Cultural heritage crowdsourcing has emerged as a promising approach to address the challenges of digitizing and preserving cultural heritage, contributing to the sustainable development goals of cultural preservation and digital inclusivity. However, the long-term sustainability of these projects faces numerous obstacles. This study explores the key configurational determinants and dynamic ... essay homework paper phd project resume review study thesis