importance of research in healthcare

• Fact sheets
• Facts in pictures

Publications

• Questions and answers
• Tools and toolkits
• Endometriosis
• Excessive heat
• Mental disorders
• Polycystic ovary syndrome
• All countries
• Eastern Mediterranean
• South-East Asia
• Western Pacific
• Data by country
• Country presence 
• Country strengthening 
• Country cooperation strategies 
• News releases
• Feature stories
• Press conferences
• Commentaries
• Photo library
• Afghanistan
• Cholera 
• Coronavirus disease (COVID-19)
• Greater Horn of Africa
• Israel and occupied Palestinian territory
• Disease Outbreak News
• Situation reports
• Weekly Epidemiological Record
• Surveillance
• Health emergency appeal
• International Health Regulations
• Independent Oversight and Advisory Committee
• Classifications
• Data collections
• Global Health Observatory
• Global Health Estimates
• Mortality Database
• Sustainable Development Goals
• Health Inequality Monitor
• Global Progress
• World Health Statistics
• Partnerships
• Committees and advisory groups
• Collaborating centres
• Technical teams
• Organizational structure
• Initiatives
• General Programme of Work
• WHO Academy
• Investment in WHO
• WHO Foundation
• External audit
• Financial statements
• Internal audit and investigations 
• Programme Budget
• Results reports
• Governing bodies
• World Health Assembly
• Executive Board
• Member States Portal
• Health topics /

Research is indispensable for resolving public health challenges – whether it be tackling diseases of poverty, responding to rise of chronic diseases,  or ensuring that mothers have access to safe delivery practices.

Likewise, shared vulnerability to global threats, such as severe acute respiratory syndrome, Ebola virus disease, Zika virus and avian influenza has mobilized global research efforts in support of enhancing capacity for preparedness and response. Research is strengthening surveillance, rapid diagnostics and development of vaccines and medicines.

Public-private partnerships and other innovative mechanisms for research are concentrating on neglected diseases in order to stimulate the development of vaccines, drugs and diagnostics where market forces alone are insufficient.

Research for health spans 5 generic areas of activity:

• measuring the magnitude and distribution of the health problem;
• understanding the diverse causes or the determinants of the problem, whether they are due to biological, behavioural, social or environmental factors;
• developing solutions or interventions that will help to prevent or mitigate the problem;
• implementing or delivering solutions through policies and programmes; and
• evaluating the impact of these solutions on the level and distribution of the problem.

High-quality research is essential to fulfilling WHO’s mandate for the attainment by all peoples of the highest possible level of health. One of the Organization’s core functions is to set international norms, standards and guidelines, including setting international standards for research.

Under the “WHO strategy on research for health”, the Organization works to identify research priorities, and promote and conduct research with the following 4 goals:

• Capacity - build capacity to strengthen health research systems within Member States.
• Priorities - support the setting of research priorities that meet health needs particularly in low- and middle-income countries.
• Standards - develop an enabling environment for research through the creation of norms and standards for good research practice.
• Translation - ensure quality evidence is turned into affordable health technologies and evidence-informed policy.
• Prequalification of medicines by WHO
• Global Observatory on Health R&D
• Global Observatory on Health Research and Development
• Implementation research toolkit
• Ethics in implementation research: participant's guide
• Ethics in implementation research: facilitator's guide
• Ethics in epidemics, emergencies and disasters: Research, surveillance and patient care: WHO training manual
• WHA58.34 Ministerial Summit on Health Research
• WHA60.15 WHO's role and responsibilities in health research
• WHA63.21 WHO's role and responsibilities in health research
• EB115/30 Ministerial Summit on Health Research: report by the Secretariat
• Science division

WHO consults on action plan for sustainable clinical research infrastructure

WHO advisory group convenes its first meeting on responsible use of the life sciences in Geneva

Challenging harmful masculinities and engaging men and boys in sexual and reproductive health

Stakeholders convene in Uganda on responsible use of the life sciences

Target product profile for a diagnostic test to confirm visceral leishmaniasis

In 2007, the Sixtieth World Health Assembly adopted resolution WHA60.13 on control of leishmaniasis, urging Member States, among other actions: to strengthen...

Ebola and Marburg disease outbreaks: infection prevention and control research priorities in health care...

Ebola virus (EBOV) and Marburg virus (MARV) are associated with severe, potentially fatal, systemic diseases. During the development of the Infection Prevention...

WHO Global Clinical Trials Forum, Geneva, 20-21 November 2023: summary of proceedings

This report offers a summary of the discussions during the first Global Clinical Trials Forum (GCTF) where experts from 43 countries deliberated on how...

WHO global research priorities for sexually transmitted infections

Sexually transmitted infections (STIs) are widespread globally and negatively affect sexual and reproductive health. Gaps in evidence and in available...

Coordinating R&D on antimicrobial resistance

Ensuring responsible use of life sciences research

Prioritizing diseases for research and development in emergency contexts

Promoting research on Buruli ulcer

Research in maternal, perinatal, and adolescent health

Undertaking health law research

Feature story

One year on, Global Observatory on Health R&D identifies striking gaps and inequalities

Video: Open access to health: WHO joins cOAlition S

Video: Multisectional research on sleeping sickness in Tanzania in the context of climate change

Related health topics

Clinical trials

Global health ethics

Health Laws

Intellectual property and trade

Related links

Research and Development Blueprint

WHO Collaborating Centres

R&D Blueprint for Action to Prevent Epidemics

International Clinical Trials Registry Platform

The Benefits of Research in Healthcare

• February 1, 2023

Table of Contents

The healthcare industry is always changing and evolving. With new technologies and treatments constantly being developed, it can be hard to keep up. That’s where research comes in.

Research plays a vital role in healthcare, providing the evidence that we need to decide which treatments are effective and how best to use limited resources. It can also help us understand the causes of diseases and identify potential new therapies or preventative measures. In this article, we will show you ten benefits of research in healthcare.

What Is Research in Healthcare?

Research in healthcare is an investigation of human health issues in order to gain more knowledge and understanding. It is usually funded by the government, private organizations, and pharmaceutical companies.

It aims to gain new information that may benefit patients, the community, and other researchers. Healthcare research can take different forms, such as clinical trials, which involve examining and observing people with various conditions, research on blood samples or other tissues, or tests such as scans or X-rays.

Public health research addresses larger health issues that affect society as a whole, such as the impact of giving up smoking, and social care research looks at improving the lives of people who receive care and support from social care services. Health research helps us learn why we become ill and what we can do to stay well. It is an integral part of any healthcare environment and is essential for scientific and academic innovation.

? Learn more: Why is Research Important in Healthcare?

Let’s look at some of the benefits of research in healthcare.

1. Increased knowledge and understanding of diseases and treatments

Research increases knowledge and understanding of diseases and treatments by providing doctors and scientists with a better understanding of the causes and symptoms. This allows them to identify more effective diagnosis, detection, treatment, and prevention methods.

Research leads to the development of more effective medicines and treatments and improved medical procedures, such as MRIs, CT scans, and X-rays. It also provides valuable insights into how certain cohorts of people react to treatments and medications, which can lead to more targeted and effective treatments.

2. Better care and treatment for patients

Healthcare research can improve care and treatment for patients in several ways.

Firstly, research-active hospitals are often better set up to provide more comprehensive treatment options and opportunities to be included in clinical trials, leading to better health outcomes.

Secondly, participating in research can give patients a more in-depth understanding of their condition and more time with clinical team members to identify any potential problems earlier.

Furthermore, research can allow clinicians to develop their skills and knowledge in their field while also creating a culture of learning and development. Research funded by health charities, pharmaceutical companies , and universities can provide financial support for healthcare institutions, which can help improve patient care quality.

3. Advances in medical technology

Research in healthcare has advanced medical technology in many ways. Through the development of new drugs, treatments, and cures, health research has enabled us to live longer and healthier lives. Medical research studies provide new knowledge that leads to the development of new drugs, vaccines, medical devices, and surgeries, which help to improve current treatment methods.

4. Increased safety and efficacy of drugs and treatments

Research in healthcare has increased the safety and effectiveness of drugs and medicines by providing important information about disease trends and risk factors, treatment outcomes or public health interventions, functional abilities, patterns of care, and healthcare costs and use.

Through clinical trials, researchers can control the variables that could impact the study results and provide important information about medical interventions’ efficacy and adverse effects. After approval, tracking clinical experience with the drugs is important for identifying relatively rare adverse effects and determining effectiveness in different populations or in various circumstances.

Lastly, and according to F. Barigozzi, I. Jelovac, public integration of basic research can result in lowest prices and highest efficiency .

5. Improved quality of care

Research can improve the quality of care in healthcare by providing relevant, high-quality, and safe real-world data. Quality of care can be enhanced by utilizing research to develop and implement evidence-based solutions tailored to individual patients’ needs, which can also be related to the use of AI and the digitalization of the industry.

6. Greater awareness of public health issues

Healthcare research can help increase general health awareness by providing valuable data and evidence on the efficacy of different treatment options and interventions. For example, studies such as those conducted by the Centers for Disease Control and Prevention (CDC) have provided valuable information on the safety of various drugs, treatments, and procedures, as well as the associated risks.

Research can also help to identify areas of public health that need to be addressed, such as disparities in access to care and unequal outcomes for minority populations, allowing healthcare providers to better target resources and interventions that can improve public health.

7. Greater understanding of patient perspectives

Research helps to better understand patient perspectives by providing firsthand documentation of their experiences, offering an overview of suggestions for improvement, detecting common concerns and desires associated with the quality of care, and identifying obstacles to growth and change.

Qualitative research can provide a detailed and personal account of individual experiences, allowing for a comprehensive description of the research method and participant demographics. Ethicists have long suggested greater community involvement in health research studies, including more communication about research results, to build public trust in the research community and ensure the protection process is open and accessible to all interested parties.

? Learn more : Professional ethics: An overview from health research ethics point of view by Aceme Nyika

8. Creation of new knowledge through research partnerships

Research partnerships, such as those between universities, medical schools, and funding bodies, have the potential to create new knowledge in healthcare by allowing for innovative ideas, evidence-based care, and practice to be tested and developed.

Through such collaborations, researchers can explore the causes of diseases, evaluate the effectiveness of prevention techniques, diagnose medical conditions, and create new treatments and screening programs. By harnessing the power of new genomic research and the vast amount of health-related data and stored tissue specimens, research partnerships can lead to groundbreaking discoveries that can improve the quality of life and develop safer, more effective medical care tailored to individual needs.

9. Increased funding for research

Research into causes of disease, methods of prevention, techniques for diagnosis, and new treatment approaches has led to an increase in life expectancy, reduction of infant mortality, and improved outcomes for patients with chronic diseases. This has increased funding for healthcare as more resources are put towards researching and developing new treatments and methods of care.

10. Improvements in scientific rigor and quality of research

Healthcare research can potentially improve the scientific rigor and quality of research significantly. Through increased participation in research and data sharing, health research can generate new knowledge and accelerate the pace of discovery, ultimately improving the scientific rigor and quality of research in healthcare.

Healthcare Research Benefits

Research is critical to the advancement of healthcare. By understanding the benefits of research, we can see why it is so important to invest in research initiatives and understand the industry’s digitalization . By doing so, we can improve the quality of care for all patients, advance medical technology, and create new knowledge that can be used to improve healthcare systems around the world.

Health Analytics bridges the gap between sound scientific data and market access. Phone: (410) 997-3314 Email: [email protected]

• Health Care
• Evidence Generation Planning
• Evidence Generation
• Translating to Market Access

Cookies on the NHS England website

We’ve put some small files called cookies on your device to make our site work.

We’d also like to use analytics cookies. These send information about how our site is used to a service called Google Analytics. We use this information to improve our site.

Let us know if this is OK. We’ll use a cookie to save your choice. You can  read more about our cookies before you choose.

Change my preferences I'm OK with analytics cookies

Maximising the benefits of research: Guidance for integrated care systems

England has a vibrant research and development ecosystem, with well-developed research infrastructure and research expertise within our health and care workforce. The value of research in transforming health and care is significant; additionally, staff satisfaction, recruitment and retention is higher among staff who are involved in research. The inception of integrated care systems (ICSs) provides the opportunity for systems to embed research within health and care for the benefit of our population. Supporting this opportunity, a clear research thread runs through ICS strategies and plans, from joint strategic needs assessments and joint health and wellbeing strategies , integrated care strategies , joint forwards plans , integrated care board (ICB) annual reports and the assessment by NHS England of the discharge of duties by ICBs.

The Health and Care Act 2022 (the 2022 Act) sets new legal duties on ICBs around the facilitation and promotion of research in matters relevant to the health service, and the use in the health service of evidence obtained from research. NHS England will assess ICBs for their discharge of these duties. The ICS design framework sets the expectation that in arranging provision of health services, ICBs will facilitate their partners in the health and care system to work together, combining expertise and resources to foster and deploy research and innovations. This guidance supports ICBs in fulfilling their research duties.

ICSs are encouraged to develop a research strategy that aligns to or could be included in their integrated care strategy. This strategy will enable the unification of research across ICS partners, and be consistently embedded to:

• identify and address local research priorities and needs, and work collaboratively to address national research priorities
• improve the quality of health and care and outcomes for all through the evidence generated by research
• increase the quality, quantity and breadth of research undertaken locally
• extend and expand research in settings such as primary care, community care, mental health services, public health and social care
• drive the use of research evidence for quality improvement and evidence-based practice
• influence the national research agenda to better meet local priorities and needs
• improve co-ordination and standardisation within and between localities for the set up and delivery of research
• harness the patient and economic benefits of commercial contract research
• co-ordinate and develop the research workforce across all settings.

1. Introduction

This guidance sets out what good research practice looks like. It supports integrated care systems (ICSs) to maximise the value of their duties around research for the benefit of their population’s health and care and, through co-ordination across ICSs, for national and international impact. It supports integrated care boards (ICBs), integrated care partnerships (ICPs) and their partners to develop a research strategy that aligns to or can be incorporated into their integrated care strategy, and helps them and their workforce to build on existing research initiatives and activities across health and social care to improve sector-wide performance and best practice

• explains the ICB legal duties and other requirements around research and the use of evidence from research, and that research is included in forward planning and reporting
• encourages system leaders to develop a footprint-wide research strategy that aligns to local and national research priorities, develops and supports their workforce, takes the opportunities offered by commercial research and includes plans to embed research in their system’s governance and leadership
• identifies best practice examples and other resources that ICBs may find useful as they develop their research strategies.

This guidance provides comprehensive information for use by:

• those with senior responsibility, including at board level, for research strategy development and/or operationalising research
• managers responsible for developing joint strategic needs assessments, integrated care strategies, joint health and wellbeing strategies, joint forward plans, other linked strategies, or reporting on ICB activities
• research managers
• research and development/innovation leads
• heads of services
• knowledge and library specialists.

It may also be useful to individuals involved in research, education, and partner organisations such as local authorities, social care services, the voluntary, community and social enterprise sector (VCSE) and other providers of healthcare services.

NHS England provides guidance on embedding research in the NHS and secure data environments, and the Office for Life Sciences (OLS ) champions research, innovation and the use of technology to transform health and care service. Other sources of guidance, support and information are signposted in this guidance to support ICSs in aligning to national visions, strategies and plans around research.

1.1 Definition of research

NHS England uses the UK Policy Framework for Health and Social Care Research definition of research:

“… the attempt to derive generalisable or transferable new knowledge to answer or refine relevant questions with scientifically sound methods. This excludes audits of practice and service evaluation. It includes activities that are carried out in preparation for or as a consequence of the interventional part of the research, such as screening potential participants for eligibility, obtaining participants’ consent and publishing results. It also includes non-interventional health and social care research (that is, projects that do not involve any change in standard treatment, care, or other services), projects that aim to generate hypotheses, methodological research and descriptive research”.

This broad definition encompasses the range of types of research:

• clinical trials and other clinical investigations into the safety and effectiveness of medicines, devices and health technologies
• public health research
• observational studies
• discovery science and experimental medicine
• translational research in which results from basic research are developed into results that directly benefit people
• applied research
• research to support policy-making and commissioning
• social care research and research in social care settings
• research into NHS services and care pathways.

1.2 Why research is important

The UK is a world leader for research and invention in healthcare, with around 25% of the world’s top 100 prescription medicines being discovered and developed in the UK ( The impact of collaboration: The value of UK medical research to EU science and health ). Research in the health and care system is important because it underpins all advances in health and care and is the basis for evidence-based practice. Engaging clinicians and healthcare organisations in research is associated with improvements in delivery of healthcare ( Does the engagement of clinicians and organisations in research improve healthcare performance: a three-stage review) . To benefit service users and the public, the NHS and local government, and achieve return on investment, it is vital that research is disseminated, shared and translated into practice.

The National Institute for Health and Care Research (NIHR) is funded by the Department of Health and Social Care (DHSC) to transform research in the health and social care system, including through support for NHS research. Research led to the first proven treatments for Covid, for example the use of dexamethasone, estimated to have saved over a million lives worldwide . This success was in part due to how research is undertaken in the unique environment of the NHS, innovative trial designs, the support provided by the NIHR, frontline staff enabling research, and the awareness and readiness of the public to support research. We need to learn from these and other successes, and translate this across all health and care settings. ICSs will play a vital role in enabling research to be embedded in evolving patient pathways across their footprints.

Example: PRINCIPLE trial – finding treatments for Covid recovery at home

The Platform Randomised Trial of Treatment in the Community for Epidemic and Pandemic Illnesses (PRINCIPLE) was a UK-wide, clinical study to find Covid treatments for recovery at home without the need to attend hospital. The study was open to all with ongoing Covid symptoms, registration was easy, and the trial was run entirely remotely by delivering ‘participant packs’ to people’s homes. It was one of the first trials in the world to show that azithromycin and doxycycline did not benefit patients with Covid and to identify the effectiveness of a commonly used drug – inhaled budesonide –in reducing time to recovery.

The PRINCIPLE study team demonstrated the integral role that primary, secondary and ambulatory care staff can play in the delivery of studies. Local collaborators were trained in good clinical practice to allow them to assess and confirm the eligibility of potential participants, and were commended specifically for their use of patient data to contact people soon after they received a positive test result. It is this network of local staff contributing to research within their healthcare setting that has enabled over 10,000 people to be recruited onto this study so far – one of the largest at home Covid treatment studies worldwide.

This is an example of a study design that incorporates the vital contributions of healthcare providers across the system.

Policy-makers and commissioners need evidence to support their decision-making around the delivery and system-wide transformation of health and care services, including how health inequalities will be reduced.

There is also evidence that:

• staff involved in research have greater job satisfaction and staff turnover is lower in research active trusts ( Academic factors in medical recruitment: evidence to support improvements in medical recruitment and retention by improving the academic content in medical posts)
• research active hospitals have lower mortality rates, and not just among research participants ( Research activity and the association with mortality )
• 83% of people believe that health research is very important ( Survey of the general public: attitudes towards health research)
• healthcare performance improvements have been seen from the creation of academic research placements ( Experiences of hospital allied health professionals in collaborative student research projects: a qualitative study )
• clinical academic research, and in particular the practice changes resulting from it, is associated with improved patient and carer experiences ( A qualitative systematic review and thematic synthesis exploring the impacts of clinical academic activity by healthcare professionals outside medicine ).

Key to having research embedded in health and care is having staff who can understand, undertake, use and generate new research, and share actionable research finding as part of a pro-research culture. Education and training are therefore critical for research to be sustainably embedded within health and care, and for people to develop careers in research and support it in their clinical or care roles.

DHSC, NHS England, the devolved administrations, NIHR and other partners expect to publish a clinical research workforce strategy in 2023/24 to help the UK realise the national clinical research vision outlined in Saving and Improving Lives: The Future of UK Clinical Research Delivery and deliver the Life Sciences Vision to see research embedded in the NHS as part of health and care pathways.

Research will support ICSs to deliver on their four key aims:

Improving outcomes

The NHS 2023/34 priorities and operational planning guidance emphasises the importance of research in improving patient care, outcomes and experience.

Research evidence will inform commissioning decisions to improve experience and outcomes. Research activities should align with the local health priorities identified through local joint strategic needs assessments, and may be best designed and delivered by collaborating with partners. Research priorities may be best addressed by collaborating with partners nationally to design and deliver research.

Tackling inequalities

Research can give a better understanding of local populations and the wider determinants of health, and with this the steps to maintain health and narrow health inequalities.

Enhancing productivity

The development of ICSs creates the opportunity to consider research delivery within the ICS and across ICS boundaries, increasing flexibility of workforce or recruitment while reducing bureaucracy and improving research productivity and value for money.

Supporting social and economic development

An active research ecosystem working in a co-ordinated way and to national standards brings revenue and jobs to regions. The NIHR Clinical Research Network (CRN) supports service users, the public and health and care organisations across England to participate in high-quality research. The 2019 impact and value report detailed the significant income and cost savings that commercial research generates for NHS trusts. Between 2016/17 and 2018/19 the NHS received on average £9,000 per patient recruited to a commercial clinical trial and saved over £5,800 in drug costs for each of these patients. This equates to income of £355 million and cost savings of £26.8 million in 2018/19.

In 2021 150 members of the Association of Medical Research Charities funded £1.55 billion of medical research, including the salaries of 20,000 researchers. Every £1 million spent by charities on medical research in the UK contributes £1.83 million to the economy.

Example: Research that cut problematic prescribing and generated cost savings in general practice – a local health priority

Analysis of routine patient data identified the need for strategies targeting clinicians and patients to curb rising opioid prescribing. From this, the Campaign to Reduce Opioid Prescription (CROP) was launched in 2016, urging GPs across West Yorkshire to ‘think-twice’ before prescribing opioids. This promoted the NICE guidance on chronic pain , which recommends reducing the use of opioids because there is little or no evidence that they make any difference to people’s quality of life, pain or psychological distress, but they can cause harm, including possible addiction.

Over a year 15,000 fewer people were prescribed opioids (a 5.63% relative reduction), a net saving to the NHS of £700,000. The biggest reduction was in people aged over 75, who are at higher risk of opioid-related falls and death, and there was no compensatory rise in the prescribing of other painkillers or referrals to musculoskeletal services.

The CROP campaign, led by researchers at the University of Leeds, has subsequently been rolled out across all ICBs in Yorkshire and the Humber, and the North East and North Cumbria ICB, and the 1,045 practices to which it has been delivered are reporting results similar to the above.

Foy R, Leaman B, McCrorie C, Petty D, House A, Bennett M, et al (2016) Prescribed opioids in primary care: cross-sectional and longitudinal analyses of influence of patient and practice characteristics | BMJ Open 69(5).

Alderson SL, Faragher TM, Willis TA, Carder P, Johnson S, Foy R (2021) The effects of an evidence- and theory-informed feedback intervention on opioid prescribing for non-cancer pain in primary care: A controlled interrupted time series analysis. PLOS Med .

2. ICS, ICP and ICB responsibilities and requirements

ICBs have legal duties and other requirements that relate to research. These are additional to the duties and responsibilities of individual providers within ICS footprints. This section sets out what these duties mean in practical terms and gives examples of how to meet them.

2.1 Legal duties relating to research in the Health and Care Act 2022

Part 1 of the 2022 Act includes specific legal duties for ICBs and NHS England in respect of research. In the Explanatory Notes to the 2022 Act, government sets out how ICBs could discharge their research duty.

Duty to facilitate or otherwise promote research

The ICB duty builds on the previous clinical commissioning group (CCG) duty to promote research, by requiring each ICB, in the exercise of its functions, to facilitate or otherwise promote research on matters relevant to the health service. This duty is intended to include a range of activities to enable research. Section 3 of this guidance outlines ways in which ICBs can do this.

The NHS Constitution also makes clear that patients should be enabled to take part in research: “the NHS pledges … to inform you of research studies in which you may be eligible to participate”.

The Provider Selection Regime (PSR) will be a new set of rules for arranging healthcare services in England, introduced by regulations made under the 2022 Act. The research component should be referred to once the PSR is published.

Duty to facilitate or otherwise promote the use in the health service of evidence obtained from research

This duty similarly builds on the CCG requirement to promote the use of evidence. ICBs must, in the exercise of their functions, facilitate or otherwise promote the use in the health service of evidence obtained from research. For example, ICBs should facilitate or otherwise promote the use of evidence in care, clinical and commissioning decisions.

Duty for ICSs to include research in their joint forward plans and annual reports

Joint forward plans are five-year plans developed by ICBs and their partner NHS trusts and foundation trusts. Systems are encouraged to use the joint forward plan as a shared delivery plan for the integrated care strategy and joint health and wellbeing strategy, aligned to the NHS’s universal commitments. The plan must explain how the ICB will discharge its duties around research, and the ICB must report on the discharge of its research duties in its annual report. These inclusions will raise the profile of research at board level and help embed research as a business-as-usual activity.

The joint forward plan and NHS Oversight Framework guidance set the minimum requirements for what needs to be included in plans and reports.

NHS England duty to include how each ICB is carrying out its duties relating to research in its annual performance assessment of each ICB

NHS England has a new legal duty to annually assess the performance of each ICB and publish a summary of its findings. For 2022/23 NHS England will complete a narrative assessment, identifying areas of good and/or outstanding performance, areas for improvement and any areas that are particularly challenged, drawing on national expertise as required and having regard to relevant guidance. This assessment will include a section considering how effectively the ICB has discharged its duties to facilitate or otherwise promote research and the use of evidence obtained from research.

This, alongside the implementation of the NHS Long Term Plan commitment to develop research metrics for NHS providers, will increase transparency across the system and enable more targeted support for research. Research metrics from NHS England, the Care Quality Commission (CQC) and NIHR will enable the monitoring of progress over time, and are under development with sector colleagues, including providers.

2.2 Legal requirement to work with people and communities

Working with people and communities is a requirement of ICBs, and statutory guidance is available to support them and their partner providers meet this legal duty. A co-ordinated approach across healthcare delivery and research will make it more likely that research reflects what matters to people and communities.

This will also help ICBs to fulfil their legal duty in the 2022 Act to reduce health inequalities in access to health services and the outcomes achieved. Section 3.9 includes links to resources to help guide engagement with underserved communities around research.

The Public Sector Equality Duty also applies and requires equality of opportunities between persons who share a relevant protected characteristic and persons who do not.

2.3 Research governance

While research can address local priorities, it typically operates across ICS boundaries and at national and international levels. Health and social care research is governed by a range of laws, policies, and international, national and professional standards.

The Health Research Authority (HRA ) is responsible for ensuring such regulation is co-ordinated and standardised across the UK to make it easier to do research that people can trust. The HRA is an executive non-departmental public body created by the Care Act 2014 to protect and promote the interests of patients and the public in health and social care research, including by co-ordinating and standardising the practice of research regulation. Local authorities and the NHS are obliged to have regard to its guidance on the management and conduct of research.

Before a research project can start in the NHS in England it must receive approval from the HRA. This includes research taking place in NHS trusts, NHS foundation trusts, ICBs or primary care providers of NHS commissioned services in England, and all research under an NHS duty of care, including that undertaken by NHS staff working in social care or other non-NHS environments.

The HRA schemes indemnify NHS organisations accepting these assurances against any claim covered by the NHS Litigation Authority arising as a result of incorrect assurances. If an NHS organisation duplicates the HRA assessments, it will be liable for any consequences of the decisions it bases on its own checks.

ICBs and partner organisations should have processes for the set up and delivery of research that comply with national laws and systems, and does not duplicate them. Such national systems include confirmation of capacity, National Contract Value Review (NCVR), management of Excess Treatment Costs (ETCs) and contracting arrangements (see section 2.4).

The UK Policy Framework for Health and Social Care sets out the roles and responsibilities of individuals and organisations involved in research.

2.4 Contractual requirements around research

NHS England mandates commissioner use of the NHS Standard Contract for all contracts for healthcare services other than primary care. The contract is updated annually. References to research in the current NHS Standard Contract and service conditions fall into three main areas.

Recruitment of service users and staff into approved research studies

The NHS Standard Contract obliges every provider of NHS-funded services to assist the recruitment of suitable subjects (whether patients or staff) into approved research studies. This requirement aligns to those in the 2022 Act that require ICBs to facilitate or otherwise promote research (see section 2.1). Section 3 considers how this requirement can best be met. Research involving people or their data requires ethical and potentially other approvals (see section 2.3).

National Directive on Commercial Contract Research Studies

Adherence to the National Directive is mandated as part of the NHS Standard Contract. The directive states that providers must:

• Use the unmodified model agreements for sponsor-to-site contracting; HRA and Health and Care Research Wales (HCRW) approval of studies will be dependent on use of these templates.
• Use the standard costing methodology to set prices for commercial contract research undertaken by NHS providers; this is currently in the NIHR interactive costing tool (NIHR iCT).
• Introduce the National Contract Value Review (NCVR) process in line with national rollout. NCVR is a standardised national approach to costing commercial contract research within the NHS. It currently covers acute, specialist and mental health trusts, but the intention is to roll it out to all NHS providers. The creation of ICSs is the ideal opportunity to explore how commercial study set up can be supported across these footprints, reducing the resource needed and time taken.

Comply with HRA/NIHR research reporting guidance

The provider must comply with HRA/NIHR research reporting guidance, as applicable.

2.5 Excess treatment costs

Patients in a research study may receive healthcare that differs from what is standard in the NHS, requires more clinician time or is delivered in a different location. The associated NHS treatment costs may exceed or be less than those of standard treatment. If greater, the difference is referred to as the NHS Excess Treatment Costs (ETCs).

In the case of commercial contract research, the commercial funder will pay the full cost of the study. In the case of non-commercial research, the commissioner of the service in which the study operates is responsible for funding the ETCs.

ICBs as commissioners of services are responsible for ETCs in services that they commission. Guidance for the management of ETCs is available.

DHSC and NIHR are piloting interim arrangements to support non-NHS ETCs for research in public health and social care (non-NHS intervention costs). Please refer to the further detail on the NIHR website .

2.6 Care Quality Commission

The CQC is currently developing its approach for ICS-level assessments, and its new assessment framework will be introduced towards the end of 2023 .

CQC inspection of NHS providers continue, with research assessed as part of the review of the trust-level Well-led framework. Providers are asked:

• Are divisional staff aware of research undertaken in and through the trust, how it contributes to improvement and the service level needed across departments to support it?
• How do senior leaders support internal investigators initiating and managing clinical studies?
• Does the vision and strategy incorporate plans for supporting clinical research activity as a key contributor to best patient care?
• Does the trust have clear internal reporting systems for its research range, volume, activity, safety and performance?
• How are service users and carers given the opportunity to participate in or become actively involved in clinical research studies in the trust?

3. Developing a research strategy

3.1 why develop a research strategy.

Like the health and care system, the research environment is complex. Developing a research strategy will help bring together the legal and other duties around research in a coherent way, and help the ICS understand its local research capability, workforce, activity and needs, set ambitions around research and maximise the benefits associated with commercial research. It will help demonstrate the benefit of research locally, nationally and internationally, and guide the production of clear plans.

Example: Value of research partnerships and integration with ICSs

Bristol Health Partners (BHP) Academic Health Science Centre (AHSC) has a fully integrated relationship as the new Research and Innovation Steering Group for the Bristol, North Somerset and South Gloucestershire (BNSSG) ICS, and reports directly to ICB chief executives.

The group provides the strategic direction and oversight for all research undertaken and delivered across the system. Membership includes directors of research, clinical strategy, public health, social care, senior innovation and education leaders from its core funding partners. It also includes public contributors and senior representatives from primary care, NIHR Applied Research Collaboration West, NIHR CRN West of England, West of England Academic Health Science Network (WEAHSN), Healthier Together ICS, university research institutes and People in Health West of England.

The group has reviewed ICS programmes, identified current and potential research and innovation connections, and begun to establish new connections. It has also supported work with the ICS Ageing Well programme and secured funding for innovative pilots to improve dementia care and increase physical activity for older adults.

Since 2016 BHP has directly contributed an estimated additional £1.1 million to support ICS priorities through Health Integration Team projects and other activities, and has attracted more than £33 million of external research, service redesign and infrastructure into the region.

3.2 General considerations

In developing its research strategy, the ICS may find it helpful to consider these overarching questions alongside the suggested focused content covered in the sections below:

• What do you hope to achieve within a given timeframe?
• Are all the right organisations involved in developing the research strategy?
• How will the health and care workforce be enabled to deliver the research strategy?
• How can research be embedded in existing health and care delivery and pathways?
• What mechanisms are in place to translate actionable research findings into practice and decision-making?
• What inequalities exist in different areas, communities or groups? How will you ensure planning and delivery of research aligns to CORE20plus5 priorities?
• Are you considering equality, diversity and inclusivity and the Public Sector Equality Duty in facilitating and promoting research opportunities for service users and for health and care staff?
• Is the ICS considering the opportunities of developing their commercial research portfolio?
• Is research informing or being informed by population health management?
• How will you plan and deliver research in a sustainable manner, aligning it to the Greener NHS agenda and the ICB’s duties in relation to climate change ?

Buy-in from NHS staff, patients and the public will be vital if ICBs are to discharge their research duties and deliver on their research plans. An important consideration is how to develop sustainable, routine and accessible information flows to ensure the ICB, partners, staff, patients and public can access up-to-date and appropriate information around local research activity, regional, national and international research opportunities and findings, and contact information.

3.3 Leadership and governance across the ICS

Executive leadership.

The Explanatory Notes to the 2022 Act suggest that ICBs have board-level discussions on research activity, the use of the evidence from research, the research workforce and research culture within the ICS. ICSs should refer to the NHS Leadership Competency Framework for board-level leaders at organisation and ICS level for the competencies relating to the research duties of ICSs, once published.

All ICBs are encouraged to have an executive lead responsible for fulfilling the research duties conferred by the 2022 Act. They should help give the ICB a clear understanding of research across the area, regularly reporting on progress towards agreed aims. An executive lead can take responsibility for ensuring clear research ambitions and a research strategy are developed; oversight of organisational research portfolios, diversity in research, alignment to national priorities; promotion of research skills and the need for research skills training; and succession planning.

Senior leaders could engage, consult and be supported by representatives of each registered health and social care professional group when developing strategic plans, and for oversight of training, succession planning, and equality and inclusivity. They could use the capacity and capability of the research and development leads within provider organisations, although established lead roles across social care settings are rare so extra effort may be needed to garner social care research insight.

Research steering group, board or forum

Some CCGs had research steering groups and some of these have expanded with the widening remit of ICBs. ICSs that do not have a such a group should consider adopting a model similar to one in other ICSs where research is effectively embedded in ICS governance structures.

A dedicated steering research group, board and/or forum can:

• provide dedicated time to plan, oversee and report on research
• bring a range of representatives from research infrastructure organisations, patients and the public together with representation from across the ICS, to develop a common aim and objective
• ensure board-level sight of research
• take a cross-ICS approach to research, increasing participation and diversity in research, and reducing bureaucracy.

Example: A dedicated research and innovation subgroup

East and North Hertfordshire Health Care Partnership established a formal research and innovation subgroup to support its objectives to transform services, reduce health inequalities and improve patient health and wellbeing. This subgroup is dedicated to determining and supporting local research priorities and developing an innovation agenda. With effective patient and public involvement, it is working to ensure the local population has access to more research opportunities.

Bringing together the NIHR, academia, industry and local health and care services, the subgroup develops collaborative work plans that support the design, implementation and evaluation of local transformation needs, sharing resources, staff, expertise and facilities. Its work exemplifies a sustainable approach to partnership working and supports Hertfordshire and West Essex ICS’s developing strategy.

HWE ICS Partnership Board 14 September 2021

3.4 Understanding your research activity and working with local and national research infrastructure

Research in NHS and non-NHS settings across an ICS footprint will be supported by different organisations. In some areas networks or collaboratives already exist to bring these organisations together, but in others the links are not as well formed. ICBs would benefit from having a clear map of the research infrastructure and pre-existing local or national investment into research in their area.

It may be valuable to consider:

• Who are the research leaders in your local health and care system, NIHR, higher education institutions, VCSE sector and businesses?
• Are there any pre-existing local or regional research, researcher or research engagement networks?
• What are the opportunities to inform, participate in, collaborate with or lead national and international research efforts in addition to local opportunities?

A list of organisations involved in research including NIHR-funded infrastructure and programmes is included in Annex 1 .

Much of the research undertaken in NHS and other health and care settings is funded though national calls and grants provided by funders such as NIHR, research charities , UK Research and Innovation (UKRI) , including the Medical Research Council (MRC ) and Economic and Social Research Council (ESRC) , and is aligned to national priorities. Other research may include national or international commercial or non-commercial clinical trials funders.

Partners within ICS systems can use NIHR research portfolio data to monitor and plan research activity; however, not all research is included within the NIHR’s portfolio, so this will not give a full picture of the research within the footprint. Mechanisms to map and monitor research more widely could be incorporated in ICB research strategies.

Some local needs may best be addressed through public health or social care research rather than research in primary, secondary or tertiary healthcare settings. Public health and social care research are described in Annex 2 .

Example: Mapping health and care research activity, expertise, interests and infrastructure

The Nottingham and Nottinghamshire Integrated Care System Research Partners Group meets bi-monthly and is chaired by the ICB Head of Research and Evidence. It brings together senior managers from the NHS providers, ICB, two local authorities, two universities and the NIHR CRN East Midlands, providing a forum for ICS-wide research discussions and the development of a system-wide collaborative approach to health and care research across the ICS. Among its aims, the group seeks to increase participation in research at both the organisational and population level, enable equity of access to research opportunities and generate impact on health and care pathways.

The group have mapped health and care research activity, expertise, interests and infrastructure in the constituent organisations. With this the ICS can see the research capabilities, strengths, expertise, and areas of synergy and opportunities for future collaboration that align to its needs and priorities, and also gaps for future development, recognising that organisations are at different stages of research development.

3.5 Understanding local needs

Universal NHS priorities will be reflected in local research needs, and each ICS footprint is likely to have its own specific local research needs. Joint strategic needs assessments (JSNAs) are undertaken jointly by local authorities and ICBs through health and wellbeing boards (HWBs) to identify current health and social care needs of local communities, where more information is needed to do so or to understand how best to address the need. People and communities should be directly involved in identifying local need, including by working with local charities, specific communities or groups who face inequalities in access to, experience of or outcomes from healthcare, eg to target health research at those areas and populations with greatest need.

ICPs are required to develop an integrated care strategy informed by JSNAs and the joint health and wellbeing strategy (JHWS). The integrated care strategy sets out how the assessed needs can be met through the exercise of the functions of the ICB, partner local authorities or NHS England, and is informed by research and practice-based evidence, as stated in the health and wellbeing guidance. In considering where such evidence is lacking, HWBs should identify in JSNAs those research needs that ICBs, local authorities and NHS England could meet through the exercise of their research functions.

Systems are encouraged to use their joint forward plan to develop a shared delivery plan for the Integrated Care Strategy and the JHWS that is supported by the whole system, including local authorities and VCSE partners. ICBs and trusts must also use their Joint Forward Plan to describe how the ICB will discharge its duty in respect of research.

The Explanatory Notes to the 2022 Act suggest how ICBs can discharge their duties around research. These include the articulating local research needs when assessing local needs and how they will be addressed when preparing strategies and plans, and encouraging partner organisations to play an active and collaborative role in pursuing these.

3.6 Supporting delivery of research

Once an ICS has a clear picture of its local research infrastructure it can consider how best to target and support research and the research workforce across its footprint and how research findings will be used. For this, the ICB should ensure that its approaches reflect national approaches to costing, contracting, approvals and information governance, and that they are also informed by learning from effective practices across equivalent ICBs.

As healthcare shifts into communities, ICSs should support the parallel shift in research by embedding research in health and care. Increasing access to research opportunities will give service users earlier access to new treatments, and faster research set up and delivery may provide the evidence needed to support improvements to local care sooner. Inclusive recruitment practices will be needed to ensure that all groups in society have the opportunity to help shape and take part in research, and benefit from research findings.

In developing its research strategy, an ICS has opportunities to reduce bureaucracy, and make research more efficient and effective across its own and with other ICS footprints, and across NHS and non-NHS boundaries, while meeting national regulatory guidance. ICBs will be expected to work with the HRA to co-develop, build on and implement strategies for further co-ordination and standardisation of study set-up and delivery processes. Any regional systems and processes that ICBs do establish must support consistent national practice in relation to the management and regulation of research, and should not duplicate them. The HRA will work with ICBs to address barriers to efficient and rapid study set-up, including model agreements, information governance and R&D office functions.

Other potential areas for streamlining and cross-organisational working include:

• cross-ICS research proposals to identify research needs
• research delivery – identifying how ICS-wide approaches could accelerate patient recruitment and deployment of research delivery staff
• shared data architecture, including the NHS Secure Data Environment for Research Network and its subnational secure data environments (SDEs). Subnational SDEs cover multiple ICSs to achieve access to multimodal data at a scale of approximately 5 million citizens, and over time will achieve technical and governance interoperability
• a greater focus on translation and implementation of research findings into health and care practice, supporting faster improvements
• sharing access to and funding for knowledge and library services
• shared processes and repositories for research assets.

The Explanatory Notes to the 2022 Act suggest that one way an ICB could discharge its research duty would be to have a dedicated research office or team supporting research.

3.7 Enabling cross-provider research

Health and care priorities can often only be addressed with complex, multiorganisational approaches and as such the research to inform these needs to span organisational boundaries. Organisational policies should promote cross-organisational research and dissemination of research findings, including through participation in collaborative research to address national priorities, joint staff posts, honorary contracts, and administratively easier movement of researchers between health and care organisations and other sector partners, including higher education, industry, charities and local authorities.

The HRA and ICS partners are developing national guidance to support cross-provider research.

The NIHR CRN can offer ICSs opportunities to participate in national and international research studies, including those the NIHR, industry and others commission.

3.8 Commercial research

Commercial contract research is research funded solely by industry, where NHS providers are contracted to carry out the research. Most of these research studies in the NHS are interventional clinical trials, such as the NHS-Galleri trial and Astra Zeneca’s COVID-19 vaccine development . Commercial research can give patients access to a wider range of research opportunities, earlier access to novel therapies and treatments, provide drugs free of charge to patients in trials, accelerate the development of new treatments and devices, generate income for providers, and fund NHS staff. It is vitally important for the benefit of patients, the NHS and the UK economy that we create an environment in the NHS that makes it easy and efficient for the NHS to undertake commercial research. This is particularly important when it comes to international commercial research, where companies can place their studies in a number of different countries and consideration of anticipated set up and recruitment times informs where they place trials.

Data gathered during some commercial research is specific to the study and is the property of the company, as is any Intellectual Property (IP) generated. In other cases, where the NHS contributes to the foreground IP – such as through the use of NHS data for research or where NHS expertise provides important contributions to a commercial product – it is important that the NHS shares in the value of IP generated as a consequence of its contributions.

The establishment of ICSs is an ideal opportunity for their creation of ambitions to enable, grow and benefit from commercial research. ICSs should explore how efficient commercial study set up and delivery could be streamlined across sites within their footprint, and should set ambitions around commercial research.

3.9 Involving patients, service users, carers and the public in research

In developing a research strategy ICSs should set out their approach to diverse public and patient involvement (PPI) in relation to research.

Areas where working with people and communities could add value in the context of research include:

• identification of local research needs, including through JSNAs and JHWSs
• designing research proposals in partnership with local or national experts
• raising awareness of research opportunities and recruitment of participants
• developing research outcome reports and identification of how and when participants will be able to access these
• consideration of how members of the public can access the outputs from publicly-funded research
• how volunteers should be involved and what they should be paid.

The UK Standards for Public Involvement sets out the core components of good public involvement. A guide outlining good practice in engaging underserved communities around research is available from NHS England. Resources about good practice around PPI in designing and delivering research, including around incentivisation , are also available from the HRA and NIHR .

It will be useful to link into established community involvement approaches. NIHR infrastructure organisations may have established networks of expert PPI representatives, and ICSs have extensive VCSE Alliances. A co-ordinated community engagement approach across health and care delivery and research will reduce the risk of overburdening communities with organisations wanting to work with them, and will support the identification of under-served communities.

3.10 Ensuring anyone can participate in research

Making research more visible within communities and increasing the public’s understanding of research can ensure greater diversity in research participation. Research findings will then be more generalisable to a broader range of groups or communities, or can be targeted and specific to relevant communities.

ICSs should seek mechanisms to ensure that opportunities to take part in research are available to all. They should consider encouraging patients and members of the public to register on NIHR Be Part of Research (a national registry where people can express their interest in being contacted about research that is relevant to them), widely disseminate research opportunities and make provision for inclusive access for communities to take part in research. Decentralised or virtual trials are remote access trials recruited to and delivered using electronic tools, making it easier for people to participate in some studies without needing to visit a recruiting hospital or attend appointments in person. ICBs should consider ways in which research delivery can increase access to research opportunities for people within their area. ICBs should also advise the public how they can access research outputs.

NIHR and UK Research and Innovation provide resources that help organisations address issues of equality, diversity and inclusion in research settings.

Example: RELIEVE-IBS decentralised trial

In 2020, Newcastle researchers launched RELIEVE-IBS, one of the first interventional decentralised clinical studies in the UK to trial Enterosgel, a new treatment for irritable bowel syndrome with diarrhoea (IBS-D). Decentralised trials are remote access trials that use electronic tools for trial recruitment and delivery, without the patient needing to visit a recruiting hospital site, which could be miles from their homes – a convenient option for patients with IBS-D. By running the trial remotely, researchers could reach beyond the small proportion of those with this condition who attend specialist clinics, as well as save resource for the sponsor.

Not only did this trial embrace technological developments to deliver research, but it empowered more patients to become involved regardless of where they lived. With in-depth patient input, the research team were able to shape the recruitment approach to be highly accessible to participants and were offered feedback on how to refine the trial design by the sponsors. The resulting patient-centric design ensured a good recruitment response when the trial opened.

NIHR (2020) Virtual trial recruits 67% faster led by NIHR Patient Recruitment Centre in Newcastle in collaboration with Enteromed

NIHR (2021) Pushing virtual boundaries to improve patient engagement and accessibility

NIHR (2022) RELIEVE IBS-D trial case study

3.11 Health data in research

Health data generated through care of service users in the NHS can fuel a revolution in the research and development of new diagnostics and treatments, maximising the potential to improve service user outcomes and experiences, support diversity in research, and minimise health inequalities through research. To do this, researchers need access to high quality and timely data to generate insights. The public expect data to be used legally and efficiently to conduct and support research.

National commitments around data for research can be found in Data saves lives: reshaping health and social care with data . This strategy shows how data will be used to bring benefits to all parts of health and social care. To achieve this vision, the NHS will be making a strategic move away from a system of data dissemination to one of data access when making NHS health and social care data available for research and analysis. This will be facilitated by the implementation of secure data environments (SDEs).

SDEs are data storage and access platforms with features that enable organisations to have greater control and oversight over their data. SDEs allow approved users to view and analyse data without it having to leave the environment. The SDE policy guidelines provide a clear signal to the sector that SDEs will become the default way of accessing NHS data for research.

This change is supported by major investments in digital infrastructure through the Data for Research & Development Programme, which is funding the development of national and subnational SDEs. The subnational SDEs will cover the entirety of England and individual platforms will cover several ICS.

ICBs should seek ways to promote and enable the use of these rich data sources for research and include them in their research strategy.

3.12 Using evidence for planning, commissioning and improving health and care

Evidence-based commissioning has advantages for the commissioner, workforce and service users, as it can:

• lead to innovation in service design and delivery
• enhance the quality of health and care provision
• reduce clinical variation between locations and providers
• improve equity of access to services
• improve patient and population outcomes.

As part of the commissioning process, commissioners are expected to use evidence-based clinical policies, as per the Roadmap for integrating specialised services within integrated care systems . Knowledge and library services can help source and interpret evidence.

The Provider Selection Regime will reflect the research duties of the 2022 Act and should be referred to when commissioning provider services, once it has been published.

NHS knowledge and library services provide access to evidence and support for knowledge management; they train people in searching for, handling and publishing information. The Knowledge for Healthcare strategy encourages and equips NHS knowledge and library services to support NHS organisations with the translation of knowledge for the spread and adoption of research and innovation. To fulfil their obligations under the 2022 Act, ICBs could commit to active knowledge translation.

Evidence for commissioning information is available from a number of sources:

• NHS Library and Knowledge Hub
• Health Libraries and Information Services Directory
• NICE guidance
• NIHR evidence
• NHS evidence works toolkit
• Academy of Medical Royal Colleges: Evidence-based Intervention
• A million decisions

The infographic for the role of research and evidence in commissioning also provides sources for evidence-based commissioning.

Example: Evidence mobilisation, knowledge sharing and improving outcomes

The STEMClub (Sustaining Transformation by Evidence Mobilisation) is a network in the North East and North Cumbria that brings together local policy and decision-makers with NHS knowledge and library specialists to facilitate evidence-based decision-making. The input of knowledge specialists ensures timely access to published research and provides knowledge management expertise to shape how soft intelligence is translated into knowledge assets.

As members within the STEMClub network, knowledge and library specialists are providing ongoing detailed evidence reviews and information management expertise to facilitate system-wide working , eg:

• North East North Cumbria Frailty Framework
• North East and North Cumbria Maternity Clinical Network
• a review of optimal patient transfer times in the North East and North Cumbria
• regular evidence summaries for the ICS Mental Health Evidence and Evaluation subgroup.

3.13 The health and care workforce and research

Staff involved in research have greater job satisfaction and research active trusts have lower staff turnover [3] . Clinical academic roles [7] , having research colleagues within services [8] and taking students on research placements [6] are felt to foster an increase in knowledge and skills across the wider staff workforce. The General Medical Council (GMC) and the Royal College of Physicians (RCP) and NIHR have issued position statements and recommendations around research, with additional signatories including UKRI, UKRD, the Academy of Medical Royal Colleges and the Royal College of Surgeons of England. Learning resources, including programmes for ongoing professional development of the research delivery workforce, are available through NIHR Learn.

In developing a research strategy ICSs could ensure that, as part of their people function and approach to workforce planning :

• Staff roles in leading, delivering or facilitating research and in supervising those developing research skills are recognised, supported and enabled across all staff groups and health and care settings as part of a positive research culture.
• The value of evidence is recognised, and education and training around research are facilitated. Opportunities to develop research careers or in overseeing the development of other researchers are enabled; this may include having protected time, inclusion in job plans and joint appointments across health and care providers and academic institutions.
• Ensuring that there is capacity and systems that support research through services like imaging, pathology and pharmacy, as well as finance and human resources.
• Individual organisations do not always have the necessary skills or services to support effective research and its impact, such as IP management, methodological expertise, regulatory compliance, statistical analysis, knowledge mobilisation expertise, genomics expertise, health informatics and data analytics. Mechanisms are needed to ensure that these can readily and rapidly be accessed across other health and care organisations, including from local authorities and other non-NHS care providers.

A UK Clinical Research Workforce Strategy is under development. ICSs should update their approaches to their research workforce once DHSC publishes this in 2023/24.

Example: Investing in the research workforce – developing capacity for chief investigators

Across the West Midlands NIHR CRN, an investment of approximately £750,000 to develop capacity for chief investigators returned additional research grant income of over £18 million in three years. This was achieved primarily by increasing the programme activity for consultants in areas where chief investigators were underrepresented.

The funding was provided through a competitive process and co-supported by the local NIHR CRN, with several local trusts jointly funded these scholars.

Kirk J, Willcocks J, Boyle P, Brocklehurst P, Morris K, Kearney R, et al (2022) Developing chief investigators within the NHS: the West Midlands clinical trials scholars programme. Clin Med 22(2): 149–52.

Kirk J, Reynolds F, Adey E, Boazman M, Brookes M, Brocklehurst P (2022) Developing paediatric chief investigators within the NHS: the Clinical Trials Scholars programme . Arch Dis Child Educ Pract Published online first: 22 February 2022. doi: 10.1136/archdischild-2021-322186

4. References

• Varnai P, Rentel M, Dave A, De Scalzi M, Timmerman W, Rosemberg-Mantes C, Simmonds P, Technopolis Group (2017) The impact of collaboration: The value of UK medical research to EU science and health .
• Boaz A, Hanney S, Jones T, Soper B (2015) Does the engagement of clinicians and organisations in research improve healthcare performance: a three-stage review. BMJ Open 5: e009415. doi:10.1136/ bmjopen-2015-009415 .
• Rees MR, Bracewell M (2019) Academic factors in medical recruitment: evidence to support improvements in medical recruitment and retention by improving the academic content in medical posts. Postgrad Med J 95(1124): 323-327. doi:10.1136/postgradmedj-2019-136501 .
• Ozdemir BA, Karthikesalingham A, Singha S, Poloniecki JD, Hinchliffe RJ, Thompson MM, et al (2015) Research activity and the association with mortality. PLoS ONE 10(2): doi.org/10.1371/journal.pone.0118253 .
• Hunn A (2017) Survey of the general public: attitudes towards health research . Health Research Authority.
• Angus RL, Hattingh HL, Weir KA (2022) Experiences of hospital allied health professionals in collaborative student research projects: a qualitative study. BMC Health Services Research 22(1). Available at: https://doi.org/10.1186/s12913-022-08119-7 .
• Newington L, Wells M, Adonis A, Bolten L, Bolton Saghdaoui L, Coffey M, et al (2021) A qualitative systematic review and thematic synthesis exploring the impacts of clinical academic activity by healthcare professionals outside medicine. BMC Health Serv Res 21(1). Available at: https://doi.org/10.1186/s12913-021-06354-y .
• Wenke RJ, Hickman I, Hulcombe J, Phillips R, Mickan S (2017) Allied health research positions: A qualitative evaluation of their impact. Health Res Policy Syst 15(6). Available at: https://doi.org/10.1186/s12961-016-0166-4

Annex 1 – Organisations that may be involved in regional or local research

• Clinical Research Networks (CRNs) , which will be retendered and renamed regional research delivery networks (RRDNs) from April 2024
• Applied Research Collaborations (ARCs)
• Biomedical Research Centres (BRCs)
• Experimental Cancer Medicine Centres (ECMCs) , jointly funded with Cancer Research UK
• Research Design Services (RDSs) and Clinical Trials Units (CTUs) which will be replaced by the NIHR Research Support Service from 1 October 2023
• Patient Recruitment Centres (PRCs)
• MedTech and In vitro diagnostic Co-operatives (MICs) , which will be replaced with HealthTech research centres from April 2024
• School of Public Health Research, School of Primary Care Research and School of Social Care Research
• Health Determinants Research Collaborations (HDRCs)
• Clinical Research Facilities (CRFs)
• Patient Safety Research Collaborations (PSRCs)
• Translational Research Collaborations (TRCs)
• Academic Health Science Centres (AHSCs)
• university teaching hospitals and all trusts that deliver research activity
• primary care organisations, including GP practices, that deliver research activity
• higher education institutions (HEIs)
• local authorities
• social care partners
• Local Government Association
• local and national charities that fund, collaborate in or support participation in research
• research and development offices in providers or CSUs, including primary care providers and ambulance, community and mental health trusts, and those in the VCSE sector
• UKRD members
• NHS subnational secure data environments for research
• NHS R&D Forum
• NHS Genomic Medicines Service Research Collaborative
• NHS Knowledge and Library Services
• Academic Health Science Networks (AHSNs) are often well linked with research organisations and infrastructure as part of their roles in development, adoption and spread of innovation.

Annex 2 – Public health and social care research

Public health research investigates issues that impact at a population rather than an individual level. This can be done within the NHS with system-level studies, such as secondary prevention of cardiovascular disease and examining the impact on health inequalities of changes to the NHS resource allocation formula, and outside the NHS for the wider determinants of health such as air quality, transport systems and housing. There is a substantial body of public health evidence for the clinical and cost effectiveness of prevention, health protection, health service redesign and addressing health inequalities.

Social care research aims to improve the lives of children and adults who need to draw on personal or practical care and support, and family members or other unpaid carers. It can include research around the introduction, use and impact of technologies, and changing social care interventions, policies and practice. Social care research also examines issues pertaining to the safeguarding of adults and children and workforce, commissioning of services, and questions about organisational and professional practice, including decision-making, training and the quality of care.

Publication reference: PR1662

Loading metrics

Open Access

Policy Forum

Policy Forum articles provide a platform for health policy makers from around the world to discuss the challenges and opportunities in improving health care to their constituencies.

See all article types »

Defining Research to Improve Health Systems

Affiliation UNICEF/UNDP/World Bank/World Health Organization Special Programme for Research and Training in Tropical Diseases, Geneva, Switzerland

Affiliation The Alliance for Health Policy and Systems Research, Geneva, Switzerland

Affiliation Council on Health Research for Development, Mexico City, Mexico

Affiliation World Health Organization, Department of Reproductive Health and Research, Geneva, Switzerland

Affiliation Council on Health Research for Development, Geneva, Switzerland

Affiliation Global Forum for Health Research, Geneva, Switzerland

Affiliation World Health Organization, Initiative for Vaccine Research, Geneva, Switzerland

Affiliation World Health Organization, Department of Public Health, Innovation and Intellectual Property, Geneva, Switzerland

Affiliation World Health Organization, Department of Research Policy and Cooperation, Geneva, Switzerland

•  [ ... ],

* E-mail: [email protected]

• [ view all ]
• [ view less ]
• Jan H. F. Remme, 
• Taghreed Adam, 
• Francisco Becerra-Posada, 
• Catherine D'Arcangues, 
• Michael Devlin, 
• Charles Gardner, 
• Abdul Ghaffar, 
• Joachim Hombach, 
• Jane F. K. Kengeya, 

Published: November 16, 2010

• https://doi.org/10.1371/journal.pmed.1001000
• Reader Comments

Citation: Remme JHF, Adam T, Becerra-Posada F, D'Arcangues C, Devlin M, Gardner C, et al. (2010) Defining Research to Improve Health Systems. PLoS Med 7(11): e1001000. https://doi.org/10.1371/journal.pmed.1001000

Copyright: © 2010 World Health Organization; licensee Public Library of Science (PLoS). This is an Open Access article in the spirit of the Public Library of Science (PLoS) principles for Open Access http://www.plos.org/oa/, without any waiver of WHO's privileges and immunities under international law, convention, or agreement. This article should not be reproduced for use in association with the promotion of commercial products, services or any legal entity. There should be no suggestion that WHO endorses any specific organisation or products. The use of the WHO logo is not permitted. This notice should be preserved along with the article's original URL.

Funding: No direct funding was received for this study. JHFR was employed under contract from TDR (Special Programme for Research and Training in Tropical Diseases) to write the initial draft. The salary for RFT was provided under a grant from the Bill & Melinda Gates Foundation (number 49275.01). Contributions from all other authors were salaried by their institutions during the period of writing.

Competing interests: The authors have declared that no competing interests exist. The views expressed here are those of the authors and not of the organization they represent.

Abbreviations: MDG, Millennium Development Goal

Provenance: Not commissioned; externally peer reviewed.

Summary Points

• Research has an important role to play in strengthening health systems to improve system performance and public health impact.
• The multiple definitions of operational research, implementation research, and health systems research creates confusion and negatively affects the credibility and progress of the research.
• The aim of this paper is to present working definitions of operational research, implementation research, and health systems research to provide greater clarity for non-specialists, scientists, policymakers, and donors working to strengthen health systems.

Introduction

A major obstacle to achieving the health-related Millennium Development Goals (MDGs) is the weakness of the health systems in many low- and middle-income countries, and their struggle to effectively provide health care to populations in need [1] , [2] . Several global health initiatives have been created over the last decade to support the delivery of available interventions for priority health problems, and in recent years there have been some major new initiatives to support health system strengthening [1] , [3] , [4] . These developments have been accompanied by a growing recognition of the role of research in improving health systems and health care delivery. The ministerial summit on health research that was held in Mexico in 2004 concluded that research has a crucial part to play in strengthening health systems and in improving the equitable distribution of quality health services for populations in need, and the summit called for greater support for such research [5] . Since then, the number of research initiatives on health systems in low- and middle-income countries has increased substantially [6] , [7] . This is a positive development that we would like to see expanded and accelerated to build up evidence-based knowledge to improve the effectiveness of health systems. Unfortunately, these initiatives have also led to growing confusion about what type of research is involved and at whom that research is targeted. The fact that the various research initiatives originate from different research backgrounds (biomedicine, social sciences, organization of services, health economics, etc.) has led to an inconsistent use of terminology to describe the research. Multiple definitions of operational research, implementation research, and health systems research have been proposed in recent years [2] , [6] – [18] , and many of these define the scope of their research very broadly, resulting in considerable overlap between definitions. Operational research and implementation research are sometimes used interchangeably in the literature, or are classified as health systems research [10] – [12] , [17] .

So does this matter?

We believe it does, because the resulting confusion may create duplications and inefficiencies both in the funding for different research efforts and among those seeking to understand and use the evidence. It indicates a lack of shared conceptual clarity among scientists and decision-makers about the scope, nature, methodologies, and issues to be addressed by the research involved [6] . This makes efforts to retrieve relevant evidence on particular topics even more complex than it already is, negatively affecting the credibility of the research itself.

The aim of this paper is to present working definitions of operational research, implementation research, and health systems research in the context of research to strengthen health systems, with the intention of providing greater clarity and consistency for non-specialists, scientists, policymakers, and donors.

As a starting point, the three research areas described here refer to research domains that differ at their core in the type of research questions they address, in how they are organized, and in how they interface with the health system. Below, we briefly describe the proposed three domains, indicate where they differ and overlap, how they complement each other, and how they could more effectively interact for greater impact of the overall research effort. We hope that, in this year with an unprecedented focus on research to strengthen health systems with the First Global Symposium on Health Systems Research this paper will contribute to greater clarity and more efficient approaches to fulfil the overall objective of strengthening health systems to improve population health [19] .

Defining the Research Domains

Building on the numerous definitions that exist in the literature under the overall umbrella of research to improve health systems, three domains of research can be defined using their primary characteristics: the focus of the research, the users of the research outputs, and the utility of the research outputs (see Figure 1 and Table 1 ).

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pmed.1001000.g001

https://doi.org/10.1371/journal.pmed.1001000.t001

The focus of the research within a health system [20] is explained more fully in the following sections.

The users of the research outputs (published results, findings, methodologies, etc.) fall broadly into three groups with operational research being predominantly, but not exclusively, of use to health care providers; implementation research predominantly of use to managers of programmes scaling up an intervention; and research on the health system as a whole (or one of its building blocks) of most use to those who manage or need to make policy for the health system.

In another characteristic, the importance of how amenable the research is to adaptation and use in other contexts or locations—also varies across the three domains. We define this characteristic as the utility of the research, where utility describes the fact, character, or quality of something being useful or serviceable [21] . While it is well established that all research to support health systems is context-specific, careful consideration of study design and reporting of context-specific factors generally improve the application of this research to other settings. This can be achieved in varying degrees of success across the three domains. For example, operational research tends to address a local problem, taking into account the particular context in which it occurs. Therefore, the research utility of the outputs would not be readily applicable to other settings without careful and considerable adaptation.

As it moves through implementation research and onto health systems research, the utility of the research tends to be broader and have increasingly common points of comparison with other contexts. Therefore, it is not uncommon to be able to generate lessons from this type of research that are applicable to other settings within the country or even for other countries. So, using research utility as defined here, operational research generates research outputs that are generally local in their utility, and health systems research has a primary characteristic of generally being more amenable to adaptation and application in other contexts, i.e., a broad utility. Again the degree of utility is wholly dependent on the use of an appropriate study design or protocol that takes into consideration of contextual factors.

In subsequent sections we will elaborate on these definitions using case studies and examples to illustrate them.

1. Research Domain: Operational

Operational research aims to develop solutions to current operational problems of specific health programmes or specific service delivery components of the health system, e.g., a health district or a hospital. Table 2 gives some selected examples of research questions that illustrate the local nature of the type of problems that this research addresses. These are problems that confront a local disease control programme, health district, or health clinic during the execution of its routine operations, and for which practically useful answers or solutions are urgently needed to allow operations to proceed more effectively. This research is characterized by a strong problem-solving focus and an urgency to find solutions. Its demand-driven nature and close association with health care delivery and routine health care operations ensure operational relevance of the research activities and rapid uptake and local utilization of research findings. The operational problems are often identified through routine monitoring and evaluation activities for which this research, where it exists, can be an important complement that allows a health programme to explore new ideas and experiment with potentially more effective approaches to its operations.

https://doi.org/10.1371/journal.pmed.1001000.t002

A wide range of study designs and research methods are used, ranging from descriptive and analytical studies to operational experiments and the use of mathematical modelling. The research often starts with exploratory studies to better define the problem and its determinants, and to identify potential solutions that can subsequently be tested under operational conditions. The research requires the involvement of, and is usually executed by, health staff who have the necessary research experience and/or by scientists from collaborating local research institutions. In their definition of operational research, Zachariah et al. [8] stress the local, programme-based focus of research that “can enhance the quality, effectiveness or coverage of programs in which the research is being done.” Similarly, WHO refers to research “for programme decision making to achieve a specific outcome” [22] ; and the Global Fund refers to research that “provides decision-makers with information to enable them to improve the performance of their programs” [9] . Hence, the results of this research tend to have a local utility and, because of its design, are not generally amenable for adaptation and use in other settings. This in no way devalues the research, and useful lessons—such as methodological approaches—could be applied elsewhere if reported. However, operational research is still not commonly undertaken, and many of the lessons that could be learnt remain unpublished.

Many health programmes and health system managers do not see operational research as a priority, and it is sometimes perceived as a waste of time and resources, distracting from the need for operational action on the basis of “common sense” [23] . Such attitudes tend to soften with exposure to properly executed operational research that delivers practical results, but quality operational research does not come easily, given the general lack of research capacity and research funding at the operational level. Several global health initiatives offer additional funding for operational research but most of these funds are not taken up at country and programme level because of the lack of appreciation for this type of research and insufficient local research capacity [10] .

2. Research Domain: Implementation

Implementation research aims to develop strategies for available or new health interventions in order to improve access to, and the use of, these interventions by the populations in need. Table 2 provides some examples of this type of research for which the starting point is the availability of an intervention or intervention package that has been proven efficacious in previous research, but for which major questions remain as to how to scale up the intervention and ensure effective integration within the health system. This research is characterized by a focus on the need for innovative approaches and/or ensuring the effectiveness of implemented interventions. Examples may include mass treatment with ivermectin for onchocerciasis or the introduction of new evidence-based birth practices for isolated communities where there are no formal health services or maternity clinics. This research often addresses implementation of newly developed products, such as a pharmaceutical, medical device, or vaccine, where this research represents the last phase of the product development pipeline. However, in this definition implementation research also covers such interventions as service delivery approaches, behavioural interventions, or understanding the impact of a payment mechanism.

Many promising health interventions have had only limited impact on the burden of disease in low- and middle-income countries because of implementation problems that were not identified, researched, and addressed. For example, research on the impact of insecticide-treated nets to reduce malaria was stopped too soon: phase IV effectiveness trials were not followed up by implementation research, and 15 years later the utilization of nets is still low in Africa. Hence, it is critical to include research on implementation as an extension of the development phase or R&D pipeline when testing a new intervention.

The examples of implementation research above tend to be developed as focused studies with clear research questions. Multicentre and multicountry studies are often used, as these help clarify which findings are location-specific and which are more generalisable. Large-scale implementation studies often have two phases. The first phase consists of descriptive, formative research to better understand the major implementation challenges and to design potential implementation strategies. This is often followed by a second phase, in which the most promising implementation strategies are tested and compared in large-scale experimental studies in settings at the appropriate level of the health system [24] , [25] . Social science research methods are extensively used (including qualitative research methods for stakeholder analysis and process evaluation) as well as methods for determining the cost of implementation strategies at different levels of the formal and informal health system.

Implementation research is usually undertaken by multidisciplinary research groups drawn from many countries, including those where the study is located. There is often a network of such groups, supported by international research institutions and expertise as required. The necessary local research capacity in behavioural sciences, health economics, and epidemiology is still inadequate in many low-income countries, and building such research capacity remains a top priority [7] , [14] . Where adequate research capacity does exist, it is often isolated, and special efforts may be needed to identify and involve such groups in research initiatives.

With regard to existing definitions, Sanders et al. refer to “research to promote the uptake and successful implementation of evidence-based interventions and policies” [12] , and Allottey et al. to “evidence that informs effective, sustained and embedded adoption of interventions by health systems and communities” [13] . The Special Programme for Research and Training in Tropical Diseases (TDR) defines implementation research as “research to significantly improve access to efficacious interventions by developing practical solutions to common implementation problems” [14] . As these definitions indicate, implementation research is intervention-specific, but in contrast to operational research, it is often designed with the intention of creating outputs that can be applicable beyond the local environment in which the research is done.

The relevance of implementation research is increasingly being recognized, and several convincing examples in recent years have demonstrated the effectiveness of this type of research in enabling implementation and scale-up of priority health interventions [26] , [27] . However, compared to the corresponding investment in R&D, implementation research is receiving only limited financial support; it will be important to correct this imbalance.

3. Research Domain: Health System

Health systems research addresses health system and policy questions that are not disease-specific but concern systems problems that have repercussions on the performance of the health system as a whole. It addresses a wide range of questions, from health financing, governance, and policy to problems with structuring, planning, management, human resources, service delivery, referral, and quality of care in the public and private sector. Table 2 gives a few examples of research questions that illustrate the nature of the research involved (e.g., studies on the effectiveness of different policies for attracting nurses to rural areas). Health systems issues are often highly context-specific, and many case studies try to elucidate a certain health system challenge within its specific environment.

However, with appropriate study design and planning, health systems research can not only answer policy questions relevant to the specific health system in which the research is undertaken, but can generate valuable lessons that are more amenable to adaptation and adoption in other settings. This is particularly true when a systems perspective is used, i.e., by considering all the positive and negative effects of a particular system-level intervention, this research can provide a robust and accurate understanding of health systems challenges and their potential solutions, thereby improving the utility of the findings in other settings [28] . This systems approach, in combination with stakeholder engagement, also informs the definition of priority research questions to address health systems challenges.

Health systems research by necessity is highly multidisciplinary, with a strong emphasis on social sciences, economics, and anthropological investigations, for example on community perceptions of health care. Much ongoing research consists of descriptive, comparative, and evaluation studies and secondary analytical research. Although experimental studies are less common, partly because of operational and ethical challenges in experimenting at the health system level, they can be very informative and provide convincing evidence on the benefit of innovations in health system efficiency and health impact [29] . Most research is undertaken through collaboration between academic institutions, with a major role being played by a few institutions with special expertise in health systems research or in one of its research disciplines (e.g., health economics and policy analysis). Health planners and decision-makers may contribute to defining the research questions, but are otherwise not much involved in undertaking the research itself [6] .

The research in this domain falls under the general definition by the Alliance for Health Policy and Systems Research (HPSR) as: “The production of new knowledge to improve how societies organize themselves to achieve health goals.” The Alliance for HPSR further clarifies that “the prime focus of health policy and systems research is not a specific disease or service, but rather the health system as a whole. However, health systems research sometimes adopts a disease or service specific focus” ( http://www.who.int/alliance-hpsr/en/ [17] ). More specifically, it can address any or all of the six building blocks of health systems identified by the WHO [20] : service delivery, information and evidence, medical products and technologies, health workforce, health financing, and leadership and governance. In doing so, it should explicitly acknowledge the importance of the continuous interactions between the different building blocks of the health systems and the different sectors (including non-health sectors) involved, as well as all the other characteristics of complex health systems [28] . Another definition offered by Varkevisser et al. [30] refers to health systems research as “research that enhances the efficiency and effectiveness of the health system.”

Research on health systems addresses a huge research area that has only been marginally covered to date [6] . Because of the multitude of system challenges and their complex multidimensional environment, research prioritization is essential and some recent priority-setting initiatives are seen as being timely [20] , [31] – [33] . Due to the relative scarcity of research capacity to undertake this type of research, efforts to improve the design, robustness, and applicability of the evidence generated in one setting to another would be highly desirable. Systems thinking methods and approaches can offer tremendous help and guidance on this [28] . By using a systematic, comprehensive way of examining the design and evaluation of potential health systems interventions, and ensuring involvement and ownership of all stakeholders involved, the utility and pay back from the evidence generated from this research greatly increases.

Related Research Areas and Research Terms

There are a number of related research areas that may overlap with the above research domains but are out of the scope of this paper. Some of them are briefly described here. First, monitoring and evaluation aims to track the progress of a health intervention and to determine whether it is having its intended impact. As a routine operational activity, it is usually not regarded as research by itself, although its findings are instrumental for identifying priority problems for research. However, the term evaluation can sometimes refer to the more formal evaluation designs such as process, economic, or impact evaluations, or can be used separately to answer questions related to the three research domains described in this paper [34] .

Intervention science—the development of new and improved health interventions—is another important area of research that may considerably overlap with implementation research [35] , [36] . This science is mainly concerned with biomedical research, where the early stages of intervention development are often far removed from the field. However, as the development phase is nearing its completion, evaluation of the intervention effectiveness is usually done under real-life conditions [37] . Hence, towards the end of the development process, intervention development and implementation research tend to closely overlap.

Some terms are very similar to those used to describe the three research domains but have been used in different ways in the literature. Operations research usually refers to the use of mathematical optimization methods for operational decision making, but this is sometimes also called operational research [34] , [35] , [38] .

Implementation science has been defined as the study of methods to promote the systematic uptake of research findings into routine clinical practice [39] , and as such is complementary to knowledge translation (another term with its own diversity of definitions!) [40] – [42] .

Implementation science has also been said to be similar to translational research or defined as ‘research that identifies barriers to proven interventions and that facilitates the creation of strategies to overcome them’ and in this sense implementation science is equal to implementation research as described above [15] , [43] , [44] .

The term health services research suggests research that focuses on the service component of the health system, but it is often defined more broadly and used interchangeably with health systems research [17] , [45] .

Definitions are meant to clarify. But if too many different definitions for the same terms abound, so much confusion results that they become an obstacle to progress. Our aim here is not to establish which of these definitions are correct or to launch an intense debate about definitions that would distract from the need to support the research itself and the use of research results to improve health. Instead, we seek to provide a simple framework that is easily understood by both experts in the field and the managers, policy makers, and donors working to improve health systems and deliver better health care. We have tried to map the three main research domains, the research targets, and the users, and to highlight the importance of context and study design in the subsequent utility of the research findings.

Research on operational problems, on implementation strategies, and on health system challenges all involve multidisciplinary research that tends to use the same type of quantitative and qualitative research methods. But these three research domains differ in the type of research questions they address, in the way they are designed, and in their expected outcomes. Operational research and implementation research are action-oriented, respond to operational problems or implementation challenges, and work towards developing targeted solutions. Research on health system challenges addresses more complex, systems problems and is geared towards improving the understanding of what works for whom and under what circumstances. It provides guidance on what might work better within the system as a whole.

The three research domains are not mutually exclusive, and there are large overlapping areas. Research on operational problems is about local problem solving, but not all the problems it addresses are truly local. Many occur in a similar manner in multiple locations and may represent implementation problems for specific interventions that might be efficiently tackled by implementation research, or are representative of a systems problem that could be effectively addressed through health systems research. New implementation strategies are often designed to overcome specific health system failures, e.g., how to improve access to vaccination among children who are currently not reached by immunization services or home treatment for malaria in communities where formal health systems are not able to effectively provide such treatment. In such situations, implementation research develops innovative solutions that are in effect improvements of the health system and that could be regarded as health systems research, especially when these innovations affect more than just a single intervention. Such overlap between the three research domains provides opportunities for cross-fertilization that should lead to greater consideration by operational and implementation researchers of the wider system implications of their research. Accordingly, this should encourage the expansion of study designs and an appreciation of the feasibility of experimentation with different health system solutions.

We hope the above helps clarify what research to improve health systems is seeking to achieve. In essence it is quite simple: it involves operational research on operational problems, implementation research on implementation strategies for available interventions, and research on health system challenges as the main focus of health systems research. To improve health care delivery to poor populations, all of these research domains are very much needed.

Author Contributions

ICMJE criteria for authorship read and met: RFT JHFR TA FBP CD MD CG AG JH JFKK AM MTM ZM TP RGR FZ. Agree with the manuscript's results and conclusions: RFT JHFR TA FBP CD MD CG AG JH JFKK AM MTM ZM TP RGR FZ. Conceived and designed the experiments: RFT JHFR CD MD CG JH MTM TP RGR. Wrote the first draft: RFT JHFR FBP CD MD CG JH MTM TP RGR. Wrote the paper: RFT JHFR TA FBP CD MD CG AG JH JFKK MTM ZM TP RGR FZ. Developed original concept: JHFR CD MD CG JH MTM TP RGR RFT. Commented on drafts: TA FBP AG JFKK ZM FZ.

• View Article
• Google Scholar
• 5. World Health Organization (2005) Report from the Ministerial Summit on Health Research 2004, Mexico City, Mexico. Geneva: World Health Organization.
• 7. World Health Organization (2009) Scaling up research and learning for health systems: now is the time. Report of a High Level Task Force. Geneva: World Health Organization.
• 9. The Global Fund and World Health Organization (2007) Guide to operational research in programmes supported by the Global Fund. Geneva: The Global Fund.
• 10. World Health Organization and the Global Fund (2008) Framework for Operations and Implementation Research in Health and Disease Control Programmes. Geneva: World Health Organization.
• 14. UNICEF/UNDP/World Bank/WHO Special Programme for Research and Training in Tropical Diseases (2003) Implementation Research in TDR: conceptual and operational framework. Geneva: World Health Organization. Report No.: TDR/IDE/SP/03.1.
• 17. Alliance for Health Policy and Systems Research (2007) What is health policy and systems research and why does it matter? Geneva: World Health Organization.
• 19. World Health Organization and partners (2010) The First Global Symposium on Health Systems Research (HSR) - Science to Accelerate Universal Health Coverage, Montreux, November 2010. Available: http://www.hsr-symposium.org/ . Accessed 10 September 2010.
• 20. World Health Organization (2007) Everybody's business: strengthening health systems to improve health outcomes. WHO's framework for action. Geneva: World Health Organization.
• 21. (1989) The Oxford English Dictionary. Second Edition.
• 22. World Health Organization, Department of Reproductive Health and Research (2003) Expanding capacity for operations research in reproductive health. Geneva: World Health Organization Report No.: WHO/RHR/02.18.
• 28. de Savigny D, Adam T (2009) Systems thinking for health systems strengthening. Geneva: World Health Organization, Alliance for Health Policy and Systems Research.
• 29. De Savigny D, Kasale H, Mbuya C, Reid G (2008) Fixing health systems. 2nd ed. Ottawa: International Development Research Centre.
• 30. Varkevisser CM, Pathmanathan I, A.T. B (2003) Designing and conducting health systems research projects: Proposal development and fieldwork. Ottawa: International Development Research Centre.
• 34. Last JM (2001) A dictionary of epidemiology; fourth edition. Oxford: Oxford University Press.
• 37. Edejer TT, Baltussen R, Adam T, Hutubessy R, Acharya A, et al. (2003) WHO guide to cost-effectiveness analysis. Geneva: World Health Organization.
• 38. Guiasu S (2009) Probabilistic models in operations research. New York: Nova Science Publishers.
• 50. UNICEF/UNDP/World Bank/WHO Special Programme for Research and Training in Tropical Diseases (1996) Community-directed treatment with ivermectin: report of a multi-country study. Geneva: World Health Organization Report No.: TDR/AFT/RP/96.1.

Here’s how you know

• U.S. Department of Health and Human Services
• National Institutes of Health

The Importance of Research on Health and Well-Being

Director’s Page Helene M. Langevin, M.D.

February 7, 2019

As I’ve dived into my role as Director at NCCIH, one of the things that’s made me so energized about the position is the smart, pragmatic thinking embedded within the Strategic Plan NCCIH adopted in 2016. It’s a twofold cogent recognition of: 1) the very real challenges faced daily by patients and their health care providers, and 2) the opportunities the research community has to offer much-needed evidence to inform decisions about patient care. Especially important is our third strategic plan objective , which focuses on how we can explore the potential of complementary health approaches to foster health promotion and disease prevention across the lifespan.

Why is this part of NCCIH’s strategic plan so important? Since the beginning of the 20th century, modern medicine and biomedical research have overwhelmingly focused on the study and treatment of disease. In contrast, health—and especially the return to health after an illness—has received comparatively little attention.

This emphasis on treating diseases is largely a byproduct of a very good thing—the tremendous gains yielded by researchers in finding treatments for diseases and the effectiveness of pharmacologic approaches in both treating and managing diseases. These strides in advancing human health can be seen in antibiotics to treat bacterial infections or medications to manage chronic illnesses, such as diabetes, hypertension, and rheumatic diseases.

Yet these critical successes in treating and managing disease may also mean that the often-painstaking task of helping the patient recuperate during the “convalescent” period after an acute illness, or following an exacerbation of a chronic relapsing condition, has not yet been adequately studied.

Though the treatment-focused model is dominant in our research and health care ecosystem, there has been a longstanding awareness that many chronic diseases can be prevented or better managed by incorporating nonpharmacologic interventions such as nutrition, exercise, and stress management. When these methods are incorporated into care and patients are able to make lasting behavioral changes, the end result can be more durable improvements in health. Many complementary and integrative health practices follow this model, and there’s preliminary evidence indicating that some complementary approaches may be useful in encouraging improved self-care, a better personal sense of well-being, and a greater commitment to a healthy lifestyle.

In fact, one of the research strategies within NCCIH’s current strategic plan is to advance understanding of the mechanisms through which mind and body approaches affect health, resilience, and well-being. This includes a focus on methodologically rigorous evaluations that will lead to a greater understanding of whether, when, how, and for whom such practices can have substantial impact. For example, we support research designed to understand the ways in which an integrative approach to treating chronic back pain can lead to lasting healing and improved function and well-being. This research may provide critically important new therapeutic approaches for those patients who have not found relief with surgery or pain medicines.

I’m very much looking forward to the National Advisory Council for Complementary and Integrative Health (NACCIH) meeting on Friday, February 8th. It will be my first Council meeting as the director of NCCIH. The meeting will feature a symposium, “NIH Research on Well-Being,” and I invite you to listen to the conversation on this important research topic, as well as hear updates on the Center’s activities, policies, and funding priorities during the meeting’s open session. NCCIH will livestream the open session of Council on NIH Videocast from 10:00 a.m. to 3:20 p.m. ET, and it will be archived.

Past Messages From the Director

Understanding the Real Progress Driven by Artificial Intelligence August 19, 2024

Science Shouldn’t Be This Much Fun…But It Was on the Star Talk Podcast! July 23, 2024

Bringing Research Into the Real World of Health Care July 10, 2024

More messages

Participating in Health Research Studies

What is health research.

• Is Health Research Safe?
• Is Health Research Right for Me?
• Types of Health Research

The term "health research," sometimes also called "medical research" or "clinical research," refers to research that is done to learn more about human health. Health research also aims to find better ways to prevent and treat disease. Health research is an important way to help improve the care and treatment of people worldwide.

Have you ever wondered how certain drugs can cure or help treat illness? For instance, you might have wondered how aspirin helps reduce pain. Well, health research begins with questions that have not been answered yet such as:

"Does a certain drug improve health?"

To gain more knowledge about illness and how the human body and mind work, volunteers can help researchers answer questions about health in studies of an illness. Studies might involve testing new drugs, vaccines, surgical procedures, or medical devices in clinical trials . For this reason, health research can involve known and unknown risks. To answer questions correctly, safely, and according to the best methods, researchers have detailed plans for the research and procedures that are part of any study. These procedures are called "protocols."

An example of a research protocol includes the process for determining participation in a study. A person might meet certain conditions, called "inclusion criteria," if they have the required characteristics for a study. A study on menopause may require participants to be female. On the other hand, a person might not be able to enroll in a study if they do not meet these criteria based on "exclusion criteria." A male may not be able to enroll in a study on menopause. These criteria are part of all research protocols. Study requirements are listed in the description of the study.

A Brief History

While a few studies of disease were done using a scientific approach as far back as the 14th Century, the era of modern health research started after World War II with early studies of antibiotics. Since then, health research and clinical trials have been essential for the development of more than 1,000 Food and Drug Administration (FDA) approved drugs. These drugs help treat infections, manage long term or chronic illness, and prolong the life of patients with cancer and HIV.

Sound research demands a clear consent process. Public knowledge of the potential abuses of medical research arose after the severe misconduct of research in Germany during World War II. This resulted in rules to ensure that volunteers freely agree, or give "consent," to any study they are involved in. To give consent, one should have clear knowledge about the study process explained by study staff. Additional safeguards for volunteers were also written in the Nuremberg Code and the Declaration of Helsinki .

New rules and regulations to protect research volunteers and to eliminate ethical violations have also been put in to place after the Tuskegee trial . In this unfortunate study, African American patients with syphilis were denied known treatment so that researchers could study the history of the illness. With these added protections, health research has brought new drugs and treatments to patients worldwide. Thus, health research has found cures to many diseases and helped manage many others.

Why is Health Research Important?

The development of new medical treatments and cures would not happen without health research and the active role of research volunteers. Behind every discovery of a new medicine and treatment are thousands of people who were involved in health research. Thanks to the advances in medical care and public health, we now live on average 10 years longer than in the 1960's and 20 years longer than in the 1930's. Without research, many diseases that can now be treated would cripple people or result in early death. New drugs, new ways to treat old and new illnesses, and new ways to prevent diseases in people at risk of developing them, can only result from health research.

Before health research was a part of health care, doctors would choose medical treatments based on their best guesses, and they were often wrong. Now, health research takes the guesswork out. In fact, the Food and Drug Administration (FDA) requires that all new medicines are fully tested before doctors can prescribe them. Many things that we now take for granted are the result of medical studies that have been done in the past. For instance, blood pressure pills, vaccines to prevent infectious diseases, transplant surgery, and chemotherapy are all the result of research.

Medical research often seems much like standard medical care, but it has a distinct goal. Medical care is the way that your doctors treat your illness or injury. Its only purpose is to make you feel better and you receive direct benefits. On the other hand, medical research studies are done to learn about and to improve current treatments. We all benefit from the new knowledge that is gained in the form of new drugs, vaccines, medical devices (such as pacemakers) and surgeries. However, it is crucial to know that volunteers do not always receive any direct benefits from being in a study. It is not known if the treatment or drug being studied is better, the same, or even worse than what is now used. If this was known, there would be no need for any medical studies.

• Next: Is Health Research Safe? >>
• Last Updated: May 27, 2020 3:05 PM
• URL: https://guides.library.harvard.edu/healthresearch

• CLASSIFIEDS
• Advanced search

American Board of Family Medicine

Advanced Search

The Importance of Primary Care Research in Understanding Health Inequities in the United States

• Find this author on Google Scholar
• Find this author on PubMed
• Search for this author on this site
• Info & Metrics

Eliminating health and health care inequities is a longstanding goal of multiple United States health agencies, but overwhelming scientific evidence suggests that health and health care inequities persist in the United States, despite decades of research and initiatives to alleviate them. Because of its comprehensiveness, studying health inequities in the context of primary care allows for the use of multiple paradigms and methodologic approaches to understanding almost any state of health, disease, social challenge, or societal circumstance a patient or group of patients might face. We argue in this special communication that the many features/advantages of primary care research have valuable contributions to make in reducing health inequity, and scientists, journals, and funders should increase the incorporation of primary care approaches and findings into their portfolios to better understand and end health inequity.

• Delivery of Health Care
• Health Care Disparities
• Primary Health Care

Health inequities are differences in health status or the distribution of health resources between different population groups, arising from the social conditions in which people are born, grow, live, work and age. 1 Eliminating health and health care inequities is a longstanding stated goal of multiple United States health agencies, but overwhelming evidence suggests that these inequities persist in the United States, despite decades of research and initiatives to alleviate them. This stasis has led to calls for advancement in health inequities research methods and content by several US federal organizations. In 2012, the National Institutes of Health (NIH) convened a summit calling for a broadening of approaches to address health inequities, 2 and the National Institute of Minority Health and Health Disparities (NIMHD) has led visioning exercises to identify health inequity research priority areas. 3 , 4 While these renewed calls are needed, there are still gaps to better study health inequity. Overall, US health inequities research has been frequently described as a subdiscipline of public health research, 5 and major federal health inequities initiatives have relied on surveys initially developed around the mid-20 th century. 6 While a survey-based, public health approach benefits understanding region and society-wide trends and intervention efforts to reduce inequities, definitive progress on fully understanding and eliminating health inequities remains unfulfilled. An essential avenue for understanding and addressing health care inequities may be to more directly observe how vulnerable populations interact with the US health care system. Primary care providers are the front door to this system-even in a nation without universal primary care access- to which a wide swath of the United States, including vulnerable populations, access at multiple points throughout their life. 7 , 8 The addition of primary care research perspectives, approaches, and data into health inequities research may be a crucial step toward understanding, improving, and ultimately helping end health inequity in the United States.

• The What and Why of Primary Care Research

Primary care is first contact health care that is comprehensive, continuous, and coordinated. 9 Primary care research is research done in the primary care environment, 10 therefore, involving primary care patients, practitioners, perspectives, and priorities. Because of its comprehensiveness, studying health inequities in the context of primary care allows for the use of multiple paradigms and methodologic approaches to understanding almost any state of health, disease, social challenge, or societal circumstance patients might face. Further, while most research methods can be used in primary care, some methods such as pragmatic trials, 11 , 12 dissemination and implementation research, 13 and patient-investigator partnerships 14 are especially appropriate for primary care settings. Primary care delivery will not solve inequity alone, but observational and interventional research in the primary care setting is an essential and overlooked piece of the science to understand and reduce health inequity. Research in the primary care setting is a window that displays disease and health care and a wide representation of the issues relevant to inequity: the experience of violence, poverty, addiction, racism, cultural factors, and disadvantage, among others, throughout a lifetime. 7 , 8 , 15 , 16 The beneficial relationships forged in primary care 17 , 18 may, in part, start to mitigate the effects of violence perpetrated by researchers in the past. 19 There have been calls to examine inequities over the life course, 20 and primary care disciplines, especially family medicine, are well-positioned to do this given their comprehensiveness in scope.

The Reach of Primary Care for Health Inequities Research

For the researcher interested in health inequities research, a context-specific discipline might elicit sampling concerns: does the US primary care environment contain enough patients experiencing inequities to produce meaningful understanding on these issues? Is not studying those in the US primary care environment just the study of care quality for a subpopulation with unlimited access to resources and all the health care they need? Are vulnerable people—with poor access to services and resources—represented in a context that requires access a priori ? Historically, in the United States, these questions may have resulted in caution in evaluating health inequities in primary care settings, but this is rapidly changing. Even in a society that does not have universal health care coverage, a large proportion of the population does have contact with primary care providers; in national surveys, more than 85% of US individuals, across demographic groups, have at least some usual source of care (doctor's office or clinic/health center—not the emergency department). 21 Specifically, vulnerable and marginalized populations do see primary care providers, especially in the nation's network of community health centers (CHCs). CHCs (clinics receiving federal funding to provide comprehensive primary care) serve ∼30 million patients in the United States, approximately 10% of the country, regardless of citizenship, income, insurance status, language spoken, or other socioeconomic criteria, and especially serve low-income patients and racial/ethnic minorities. 8 Whether a patient accesses a CHC or not, numerous primary care networks, many of them now interconnected, widely represent those who might experience health inequities. For instance, primary care practices nationwide are increasingly part of data-connected networks – research networks, networks with shared administrative resources, and networks that share electronic health records and their functionalities for innovation and data aggregation. 22 , 23 These networks join the existing core resource of practice-based research networks (PBRNs) in primary care. 24 Though large connected primary care networks (data networks and PBRNs) may not have the representativeness of national surveys, they contain large patient samples with richer information on objectively measured health outcomes, care utilization, and increasingly, robust social determinants of health data. 25 All this is routinely collected in primary care clinics, which is challenging to collect or subject to recall bias in public health surveys. Amid calls for the integration of social care and the evaluation of social determinants of health into health care, 26 , 27 and calls for multi-level and “complex system analysis reflective of real-world settings” 4 to better understand inequity, these reports have missed an opportunity to explicitly recommend primary care research as a viable and necessary response to these calls. The primary care setting sits at the nexus of complex system factors, is already in the “real world” and therefore may have enhanced external validity, is where most social needs are witnessed in health care, and is where research into these aims is likely to be most effective. In addition, primary care data are already multi-level and routinely collected: multiple visit observations for a patient over time, patients nested within providers, providers nested within clinics, and clinics nested in neighborhoods, cities, and states. 22 , 25

• Recommendations to Improve Health Inequity Research

Researchers interested in US health inequities should consider primary care settings as a crucial avenue for understanding the full picture of health inequity and developing real-world interventions to end this inequity. The published opportunities of the NIMHD Health Disparities Science Visioning Initiative 3 all rely on studying the primary care environment. Still, primary care is not explicitly mentioned in this list. We would continue the call for an enhanced partnership between primary care and public health in a manner that leverages the research strengths of both fields to take advantage of these opportunities optimally. This outcome would mean a concerted and longitudinal integration of national US survey data with primary care-related datasets to even more fully capture the exposures, experiences, and care of those most at risk for poor health outcomes. Second, it would mean sustained collaboration in developing and testing scalable health-related interventions that span boundaries: boundaries between regions, care settings, and between “community” and “health care” settings. In the long-term, funding agencies and health systems could invest even more in primary care centered networks to continue building data sources that have the potential to aggregate significant data on the longitudinal experience and outcomes of vulnerable populations over the entire life course. While Congress has designated the Agency for Health Care Research and Quality (AHRQ) as the “principal source of funding for primary care research,” the AHRQ's 2021 budget was 0.5% of the NIH's budget, 28 , 29 and a very small proportion of the NIH budget is awarded to disciplines in primary care research. 30 In response to all these issues, we make the following recommendations:

Funding agencies in the United States should increasingly fund research projects that utilize broad primary care settings to study health inequity.

Journal editorial boards should recognize the importance, scientific merit, and enhanced external validity of utilizing primary care settings in health inequity research. They should prioritize the inclusion of primary care researchers—especially those with experience in health equity research— on board rosters.

Researchers should consider multi-level, etiologic, and complex system analyses 4 and understand that primary care sits at a nexus of multi-level investigations into health inequity (primary care is the bridge between biology, behavior, health care, and community); researchers should utilize the existing multi-level data in primary settings and networks for observational and intervention studies.

Primary care providers treat and health inequities affect every organ, every system, every malady, in every family, and every community. Primary care researchers, along with public health researchers, may bring about understanding and intervention to end health inequity in the United States together.

• Acknowledgments

The authors would acknowledge our home institutions and the patients and staff of the OCHIN Practice-Based Research Network, who support our work in general.

This article was externally peer reviewed.

Funding: National Institute on Aging and National Institute on Minority Health and Health Disparities.

Conflict of interest: None.

To see this article online, please go to: http://jabfm.org/content/33/5/849.full .

• Received for publication February 12, 2021.
• Revision received April 12, 2021.
• Accepted for publication April 13, 2021.
• 1. ↵ World Health Organization . 10 Facts on Health Inequities and their Causes . Available from: https://www.who.int/features/factfiles/health_inequities/en/ . Published 2017 . Accessed December 2019 .
• Dankwa-Mullan I ,
• Williams K ,
• Wasserman J ,
• Palmer RC ,
• Ibrahim SA ,
• Pérez-Stable EJ
• Satcher D ,
• Higginbotham EJ
• 6. ↵ National Center for Health Statistics . Healthy People 2020 Data Issues 2015 . Available from: https://www.cdc.gov/nchs/healthy_people/hp2020/hp2020_data_issues.htm . Published November 6, 2015 . Accessed November 1, 2020 .
• Stange KC ,
• Flocke SA ,
• Miller WL ,
• Crabtree BF ,
• Zyzanski SJ
• 8. ↵ National Association of Community Health Centers . Community Health Center Chartbook . Available from: http://www.nachc.org/wp-content/uploads/2019/01/Community-Health-Center-Chartbook-FINAL-1.28.19.pdf . Published 2019 . Accessed November 1, 2020 .
• 9. ↵ World Health Organization . Primary Health Care . Available from: https://www.who.int/news-room/fact-sheets/detail/primary-health-care . Published April 1, 2021 . Accessed June 15, 2021 .
• Starfield B
• Carroll JK ,
• Dickinson LM ,
• Eichner B ,
• Michaels LAC ,
• Heintzman J ,
• Crosson J ,
• Likumahuwa S ,
• Warren NT ,
• Gaudino JA Jr . ,
• Likumahuwa-Ackman S ,
• Gillespie CF ,
• Bradley B ,
• Cottrell EK ,
• Hendricks M ,
• Dambrun K ,
• Blumenthal DME ,
• Nutting PA ,
• Goodwin MA ,
• Zyzanski SJ ,
• 19. ↵ Office of Environment, Health, Safety and Security, US Department of Energy . Chapter 12: the iodine 131 experiment in Alaska . In: Final Report of the Advisory Committee on Human Radiation Experiments . Washington, DC : US Government Printing Office ; 1995 . (Stock no. 061-000-008489) Available from: https://bioethicsarchive.georgetown.edu/achre/final/chap12_4.html . Accessed March 29, 2021 .
• Gilman SE ,
• Geronimus AT
• 21. ↵ Centers for Disease Control and Prevention . Early Release of Selected Estimates Based on Data From the National Health Interview Survey, January–September 2017 . Available from: https://www.cdc.gov/nchs/data/nhis/earlyrelease/EarlyRelease201803.pdf . Published March, 2021 . Accessed November 1, 2020 .
• Cottrell E ,
• 23. ↵ Kaiser Permanente Center for Health Research . Data & Analytics Expertise . Available from: https://research.kpchr.org/About/Capabilities/Data-Analytics-and-Expertise . Published 2018 . Accessed November 1, 2020 .
• 24. ↵ Agency for Healthcare Research and Quality . Practice-Based Research Networks . Available from: https://pbrn.ahrq.gov/about . Accessed March 25, 2021 .
• Bazemore AW ,
• 26. ↵ Lancet Editorial Staff . No health care without social care . The Lancet 2019 ; 394 : 1206 . OpenUrl
• 27. ↵ National Academy of Medicine . Integrating Social Care into the Delivery of Health Care: Moving Upstream to Improve the Nation's Health . Available from: http://nationalacademies.org/hmd/Reports/2019/integrating-social-care-into-the-delivery-of-health-care . Published September 25, 2019 . Accessed November 1, 2020 .
• 28. ↵ American Institute of Physics . Final FY21 Appropriations: National Institutes of Health . Available from: https://www.aip.org/fyi/2021/final-fy21-appropriations-national-institutes-health2021 . Published November 19, 2020 . Accessed March 25, 2021 .
• 29. ↵ Agency for Healthcare Research and Quality . Budget Estimates for Appropriations Committees, Fiscal Year 202 . Available from: https://www.ahrq.gov/cpi/about/mission/budget/2021/index.html . Published February 2021 . Accessed March 25, 2021 .
• Cameron BJ ,

In this issue

• Table of Contents
• Table of Contents (PDF)
• Cover (PDF)
• Index by author
• Back Matter (PDF)
• Front Matter (PDF)

Thank you for your interest in spreading the word on American Board of Family Medicine.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Citation Manager Formats

• EndNote (tagged)
• EndNote 8 (xml)
• RefWorks Tagged
• Ref Manager

• Tweet Widget
• Facebook Like
• Google Plus One

Jump to section

Related articles.

• No related articles found.
• Google Scholar

Cited By...

• Family Medicine Research on Health Equity, Addiction, and Eating Breakfast--Just for Starters

More in this TOC Section

• What Complexity Science Predicts About the Potential of Artificial Intelligence/Machine Learning to Improve Primary Care
• An Exploration of Professionalism in Everyday Practice
• Thinking “Green” When Treating “Pink Puffers” and “Blue Bloaters”—Reducing Carbon Footprint When Prescribing Inhalers

Similar Articles

• Research Note
• Open access
• Published: 05 October 2023

Organisational benefits of undertaking research in healthcare: an approach to uncover impact

• Judith Holliday 1 ,
• Natalie Jones 2 &
• Jo Cooke 3  

BMC Research Notes volume  16 , Article number:  255 ( 2023 ) Cite this article

1432 Accesses

19 Altmetric

Metrics details

There is increasing focus to review the societal impact of research through assessment and research excellence frameworks. These often link to financial and reputational incentives within the academic community. However, timeframes to demonstrate impact using these approaches are often long and are not designed to show benefit to service collaborators who require evidence of improvement and change to their services more immediately. Impacts that are measured this way may also miss out on unintended and positive impacts that occur as by-products of research, or through the ‘ripple effect’ that research may have on practice. Importantly, demonstrating how research makes a difference can improve the research culture in services, and motivations in service partners to become, and stay involved in research. This article describes, and provides access to, a tool called VICTOR (making V isible the I mpa CT O f R esearch) that was developed by a community of practice involving 12 NHS organisations through blending evidence from the literature, practice and service users. We describe the types of impact that have been collected by VICTOR and explore how collecting impact in this way might help research-practice partnerships and inform research methodologies and may be useful to show impacts alongside, and shortly after the research process.

Peer Review reports

Introduction

There is increasing focus within the academic establishment to review the societal impact of research through various assessment and research excellence frameworks. These often link to financial and reputational incentives in academia, for example the research excellence framework in the UK https://www.ref.ac.uk/about/ and Excellence in Research in Australia https://www.arc.gov.au/excellence-research-australia/era-2023 ). Many governments invest in both applied and basic health research for impact and benefit. The Canadian institute for Health Research (CHIR) for example, aims to develop scientific knowledge into improved health, more effective health services and products, and an effective care system http://www.cihr-irsc.gc.ca/e/37792.html . The UK based National Institute Health Research (NIHR) aims to provide health research that focuses on the needs of patients and the public [ 1 ] [ 2 ]. However, the timeframes to demonstrate impact from research findings are often very long [ 3 ], and many services want to show impact sooner than this resulting in tensions in academic- practice partnerships [ 4 ] [ 5 ]. There is emerging evidence that there are benefits for healthcare organisations to be part of research delivery in collaborations. For example, hospitals that are research active (defined in terms of linked citations in peer reviewed journals) are associated with improved mortality rates [ 6 ], and quality of care and health outcomes positively correlate with the conduct of clinical trials in NHS organisations [ 7 ]. There is also an association between research engagement of practitioners and improvements in performance and the process of care [ 8 ]. Boaz et al. [ 9 ] described these as the ‘by-products’ of research itself, but perhaps it is more than this, and may help to support motivation and engagement of services and increase collaboration with less engaged groups? There is also a growing debate that research could be more immediately beneficial to healthcare providers if conducted in a co-productive manner [ 10 ] [ 11 ] [ 12 ]. Coproduction can stimulate ‘win-win’ and mutually beneficial outcomes in the short-term [ 13 ], especially for services and service users and aids longevity of research collaborations and better reach into the healthcare system [ 14 ]. Indeed, a realist review focussing on research capacity development in health and care systems has highlighted how showing that research makes a difference can act as an important symbolic mechanism that increases research capacity and research culture in healthcare organisations [ 15 ]. Ideally these should be captured contemporaneously within the coproduction process.

Making visible the impact of conducting research in healthcare organizations: developing the VICTOR tool (making visible the ImpaCT Of research)

With this context in mind, a community of practice (CoP) that included members of Research and Development leaders in 12 NHS organisations in England completed a service development project to develop a tool that would enable the collection of case studies to uncover the immediate impact of conducting research in their organisations. This is more than a ‘by-product’ for them and contributes to quality assessment by the Care Quality Commission and establishes direct benefit to the organisation. The CoP was called ACORN (Addressing Capacity in Organisations Network) and they worked with two NIHR partnerships: The Collaboration and Leadership in Applied Health and Care for Yorkshire and Humber (CLAHRC YH) and the NIHR CRN YH.

VICTOR aimed to identify impact where it matters in the NHS, services, and people within them and to create a resource to support NHS Trusts to capture and show how applied research projects can have an impact within the organisation. Two senior NHS managers (JH and NJ) were seconded into the NIHR partnership to develop the VICTOR approach. Areas of impact were developed through collecting and organising information from a range of sources including a workshop with ACORN members to identify areas they thought were important, that made a difference to services when conducting research. The particular focus was on how undertaking research can make a difference in healthcare organisation and the wider health system.

A scoping literature review was conducted with the aim of understanding the current landscape of research impact tools and mapping out the published tools available for capturing research impact [ 16 ]. Keywords were used to systematically search the published literature to identify research, policy, and research impact tools relevant to the project. Online databases such as CINAL and Medline were iteratively searched as well as grey literature. Reports, tools and studies detailing research impact tools were exported to a reference manager so that they could be analysed. NJ and JH then screened the papers to ensure they were relevant to the project. A spreadsheet was created to list the research impact tools and extract data on the key domains of impact. NJ and JH were interested in where the research impact tools were similar, any gaps and the relevance of the tools to the NHS context.

The tools were discussed with JC. The merits of each were analysed. Findings from this review discovered gaps in the patient perspective on research impact and that many of the tools were designed for academic purposes or for contexts other than the NHS. Key tools of interest that were identified were:

Becker Medical Library Model [ 17 ].

Payback Framework [ 18 ].

Canadian Health Services Policy Research Alliance (CHSPRA) making an impact framework [ 19 .

Research Excellence Framework [ 20 .

Sarli CC, Dubinsky EK, Holmes KL. Beyond citation analysis: a model for assessment of research impact [ 21 ].

Stakeholder engagement in this project included working with ACORN which included 12 NHS organisations: three teaching hospitals; five mental health trusts; and four acute trusts. Many of these trusts also include outreach into community and public health practice. Each trust has at least two representatives in ACORN, one being a senior R&D manager, and the other a research-active or research interested practitioner. Stakeholder engagement is a powerful tool for involving those in research who have lived insights and ideas about ways to improve healthcare. [ 22 ]

Stakeholders in this project were involved in several ways:

12 ACORN NHS trusts met several times during the project to advise on progress and prototype tools.

Experts in the field were consulted about research impact domains via telephone calls.

Patient and carer representatives were consulted about prototype tools one to one and via patient research engagement groups. Feedback was also sought from a mental health charity and an older people’s charity.

Prototyping involved creating versions of the research impact tool and testing them out with stakeholders. Prototyping is a helpful way to test out a new tool in the early stages of development and design. [ 23 ]

Feedback on the prototype tools was collated by NJ and JH and used to inform the next version of the tool.

Several patient representatives tested the tool by completing the questions. They used their experiences of participation in a recent study to answer the questions. This gave the authors an understanding of whether the questions were collecting sufficient and focused information. Feedback from patient and informal carer representatives shaped the prototype tool so that the number of questions were reduced to make completing the questionnaire less onerous and the language of the tool was developed to avoid professional jargon.

In the first prototype, the domains of the tool were created by using the data extracted from the scoping review. NJ and JH extracted the key domains from other research impact tools. Information and insights from stakeholder consultation about what needed to be included in the tool were mapped onto the emerging domains. A master domain list was developed and tested out with JC and the ACORN group. Each domain had a list of criteria to define the focus for the domain for example, the ‘health benefits’ domain considers health benefits, safety and quality improvements for research participants and carers. This is that as a result of taking part in the research the participants (patient, carer or family) have improved health, a better experience of care, improved quality of life and/or more equitable access to healthcare. This domain includes the subgroups:

Health benefits such as; quality of life impacts, access to different treatments; care delivered differently; quality of information provided; health literacy; providing the same quality of care for a reduced cost.

Experience; during the study, were there any changes made to patient care that improved the experience of care for participants, carers or family as part of / as a result of being in the study for example information giving, carer support, carer interventions; health literacy.)

Patient safety; are there any examples of improved governance and/or safety for patients taking part in the study? This would include improvements to quality of research in terms of scientific quality, standards of ethics and related management aspects – set up, conduct, reporting and progression towards healthcare improvements.

Social capital; are participants / carers better connected or part of any new networks as a result of taking part in the research? This includes self-help groups, increased social networks or activities.

By socialising the draft domains we were able to gauge if there were any gaps, duplications, or areas of impact that might have been missed. Feedback shaped version 2 of the list of domains, criteria and prompts which were then used to create questions relevant to the domain criteria. Open questions were developed to elicit information from the research team members or patients [ 24 ].

The resulting areas of impact are given in Table  1 . There were six general domains of impact, with subgroups within each domain.

This framework was then used to develop a questionnaire that was modified and adapted based on two rounds of piloting within the ACORN organisations. A final VICTOR questionnaire was developed that includes 26 questions organised in six sections reflecting the impact domains and domain subgroups described in Table  1 . A Tool of four questions was developed for patients and members of the public based on consulting with service user groups. The VICTOR tool can be accessed https://www.e-repository.clahrc-yh.nihr.ac.uk/visible-impact-of-research/ )

As a service evaluation, the project does not require ethical approval through HRA however this project was conducted with the rigour and safeguards of research to protect participants’ data. The service evaluation was registered with the author’s organisation (STH) clinical effectiveness unit. Efforts were made to ensure that this project adhered to best practice guidance for service evaluation practice [ 25 ]. Consent to participate in the stakeholder consultations was through explicit verbal or written consent. Those agreeing to view the prototype tool and provide feedback were aware that their feedback data would be used in project reports and dissemination, and all data would be anonymised.

Uncovering impact: feedback from ACORN trusts through using the tool

Trusts who piloted the VICTOR tool shared their summary documents with the ACORN CoP. Many trusts reported that VICTOR had been helpful in identifying unanticipated and ‘hidden’ impacts of research, and documented changes that would otherwise have been overlooked, or not linked to research activity.

The impacts frequently cited in the pilot sites included service and workforce changes, research capacity building, and health and experiential impacts of patients and carers. Intervention studies often, but not exclusively, produced changes in workforce and services. For example, practitioners who received training as part of developing skills for new interventions frequently highlight how these skills were used in their practice more generally after the research project. These can be diverse skills, like paramedics developing better airway management techniques, or community nurses using cognitive behavioural therapy with patients who have long term conditions. Sometimes elements of the research method were then incorporated into clinical pathways, for example using screening questionnaires in radiography services, or use of autophotography in mental healthcare, where patients use photographs to express their world view or how they feel. The advantages of using such techniques were demonstrated in the research delivery and continued into everyday practice.

Many examples of impact on working practice in the healthcare system were established because of working together on a research project, for example between pharmacy and a clinical area, or between primary and secondary care. These continued to benefit the services after the research had been completed. Such stories were very insightful and meaningful to practitioners and managers, and were able to promote research in the organisation and wider community, for example in newsletters and press releases. Importantly, some patients described impacts that were not mentioned by research teams who were delivering projects, for example patients felt they were closely monitored, felt that they were making a difference, but they also had a contact person, usually the research nurse, who provided support and information about care and services. The process of collecting the information through VICTOR sometimes helped internal cohesion. Informal feedback was collected from the individuals or research teams (collated by NJ and JH) testing out the prototype tools. This suggests that using the VICTOR tool as a team facilitated reflexivity and team thinking about the benefits of the research project, and enabled teams to reflect on the successes of research together. One participant remarked “ Teams don’t usually get together after a research project ends, everyone is getting on with the next project, so it was nice to take some time together and reflect on the project”.

Another participant comments on the value of the team coming together to collaborate and completing the tool “ We collaborated across a pathway of care, medical, therapy and nursing staff, we would not normally get together to discuss the research, this was helpful as we could discuss changes and improvements in our systems and processes, applying the learning from the study”.

This strengthened relationships between research and clinical teams by recognising and documenting shared achievements and strengthened the partnerships with researchers. The process also enabled increased awareness of each other’s role and to share their views of impact.

During the prototyping notes of informal feedback suggested that it was more difficult than anticipated for the PI or research coordinator to track down members of the research team to ask them to complete a VICTOR questionnaire. This suggests that doing the feedback directly after the project was concluded could make it easier to gain feedback however this could potentially miss impacts that occur after the study 3–6 months after the project has been completed.

Outlook and conclusion

The VICTOR tool can help to describe the impact of conducting research in healthcare organisations, and it offers fertile ground for further work and debate on its wider influence. The logic for VICTOR’s development was that by uncovering impact of undertaking research ‘close to practice’, it could show immediate usefulness to clinicians, managers and patients, and stimulate a research culture, triggering a mechanism for change [ 26 ]. A report on enabling staff to do research in NHS organisations [ 27 ] highlights that feedback on research impact is an enabler to promote a research culture and encourages positive attitudes and values towards research. This may well be more beneficial in in supporting research collaborations within the wider ‘research ecosystem’, particularly in social and community care, where research capacity in needed and where immediate benefits are important and practical benefits realised [ 28 ].

There is a growing body of support and funding for long term research and practice collaborations such as the CLAHRCs in England, and the Hunter New England Population health research-practice partnerships [ 29 ]. These partnerships provide an opportunity to produce co-benefits to the researchers but currently there is not systematic evidence of how to identify immediate benefits to service partners [ 30 ] including methods to capture the intended and unintended outcomes that are context dependant [ 31 ]. VICTOR could provide a basis for this. It is argued that impact should be recognised in the eyes of the end- user and be tailored to context of where impact should occur [ 32 ] [ 33 ] and certainly we have found that hidden benefits have been uncovered through using the tool. The timeframe for VICTOR is undertaken contemporaneously, or shortly after the research and so shows immediate benefit that complements with more longer-term impacts of research collected in the academic research assessment frameworks.

VICTOR also has the potential to determine which research methods and methodologies are valuable to different care provider partners, and help to assess impact and different models of conducting research [ 30 ] [ 29 ]. Context, for example where coproduction in research is used can influence both process and outcomes [ 5 ]. VICTOR has found both stages in research can have a positive and ‘rippled effect’ on service provider organisations further down the pathways to impact and this has also been found by others [ 34 ]. Such a body of accumulated knowledge through VICTOR use might help to inform coproduction partnerships providing win-win scenarios linked to process as well as outcomes in research.

We acknowledge that this tool was coproduced with managers, practitioners, and service users in the NHS, which is both a strength and a limitation. It certainly was reported to be useful to the ACORN group and it has been downloaded by hundreds of healthcare organisations. However, it would be beneficial to see if it is useful across the health and care system, or in other countries. There may well be cultural differences in terms of benefit. This calls for more internationally work and comparison and incorporating tools like VICTOR into the research process itself. The optimum timeframe for completing VICTOR was not explored during this evaluation. We hope that by sharing our experience and access to VICTOR we can establish transferability and open dialogue with other partners and provide opportunities to explore mechanisms of impact of research in healthcare organisations.

Post development note.

The VICTOR tool and process was made available at https://hseresearch.ie/wp-content/uploads/2021/04/VICTOR-pack.pdf#:~:text=VICTOR%20enables%20engagement%20with%20research%20participants%2C%20professionals%2C%20managers,and%20help%20plan%20for%20improved%20impact%20in%20future . in Feb 2019 and to date 200 organisations have requested a pack. A web based version has been developed and is available at https://sites.google.com/nihr.ac.uk/victor/home and https://victorimpacttool.net/ . For further information on accessing the online tool please contact [email protected]

Data Availability

Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

Abbreviations

National Institute Health Research

Department of Health

Department of Health and Social Care

Making Visible the ImpaCT Of Research

Community of Practice

Addressing Capacity in Organisations to do Research Network

Applied Health and Care for Yorkshire and Humber

Clinical Research Network Yorkshire and Humber

Department of Health. 2006 Best research for best health: a new national health research strategy London:: Department of Health; Available at https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/568772/dh_4127152_v2.pdf .

Department of Health and Social Care. 2020. Best Research for Best Health: the next chapter. London: Department of Health and Social Care. Available at https://www.nihr.ac.uk/documents/about-us/best-research-for-best-health-the-next-chapter.pdf .

Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. 2011;104(12):510–20.

Article   PubMed   PubMed Central   Google Scholar  

Nutley S, Boaz A, Davies H, Fraser A. New development: what works now? Continuity and change in the use of evidence to improve public policy and service delivery. Public Money & Management. 2019;39(4):310–6.

Article   Google Scholar  

van der Graaf P, Cheetham M, Redgate S, Humble C, Adamson A. Co-production in local government: process, codification and capacity building of new knowledge in collective reflection spaces. Workshops findings from a UK mixed methods study. Health Res Policy Syst. 2021;19(1):1–13.

Google Scholar  

Bennett WO, Bird JH, Burrows SA, Counter PR, Reddy VM. Does academic output correlate with better mortality rates in NHS trusts in England? Public Health [Internet]. 2012;10.1016/j:2–5. Available from: https://doi.org/10.1016/j.puhe.2012.05.021 .

Jonker L, Fisher SJ. The correlation between National Health Service trusts’ clinical trial activity and both mortality rates and care quality commission ratings: a retrospective cross-sectional study. Public Health [Internet]. 2018;157:1–6. Available from: https://linkinghub.elsevier.com/retrieve/pii/S0033350618300015 .

Wenke RJ, Ward EC, Hickman I, Hulcombe J, Phillips R, Mickan S. Allied health research positions: A qualitative evaluation of their impact. Heal Res Policy Syst [Internet]. 2017;15(1):1–11. https://doi.org/10.1186/s12961-016-0166-4 .

Boaz A, Hanney S, Jones T, Soper B. Does the engagement of clinicians and organisations in research improve healthcare performance: a three-stage review. BMJ Open [Internet]. 2015;5(12):e009415. Available from: http://bmjopen.bmj.com/lookup/doi/ https://doi.org/10.1136/bmjopen-2015-009415 .

Rycroft-Malone J, Burton CR, Bucknall T, Graham ID, Hutchinson AM. Collaboration and Co-Production of Knowledge in Healthcare: Opportunities and Challenges. Int J Heal Policy Manag [Internet]. 2016;5(4):221–3. Available from: http://ijhpm.com/article_3152_629.html .

Greenhalgh T, Jackson C, Shaw S, Janamian T. Achieving Research Impact Through Co-creation in Community-Based Health Services: Literature Review and Case Study. Milbank Q [Internet]. 2016;94(2):392–429. https://doi.org/10.1111/1468-0009.12197 .

Castle-Clarke S, Edwards N, Buckingham H. Falling short: Why the NHS is still struggling to make the most of new innovations [Internet]. Nuffied Trust. 2017. Available from: https://www.nuffieldtrust.org.uk/research/falling-short-why-the-nhs-is-still-struggling-to-make-the-most-of-new-innovations .

Cooke J, Ariss S, Smith C, Read J. On-going collaborative priority-setting for research activity: a method of capacity building to reduce the research-practice translational gap. Health Res Policy Syst. 2015;13(1). https://doi.org/10.1186/s12961-015-0014-y .

Steens R, Van Regenmortel T, Hermans K. Beyond the research–practice gap: the development of an academic collaborative centre for child and family social work. Br J Social Work. 2018;48(6):1611–26.

Cooke J, Gardois P, Booth A. 2018. Uncovering the mechanisms of research capacity development in health and social care: a realist synthesis. Health research policy and systems , 16 (1), pp.1–22. Available from https://health-policy-systems.biomedcentral.com/articles/ https://doi.org/10.1186/s12961-018-0363-4 .

Mak S, Thomas A. Steps for conducting a scoping review. J Grad Med Educ. 2022;14(5):565–7. https://doi.org/10.4300/JGME-D-22-00621.1 . PMID: 36274762; PMCID: PMC9580325.

( https://becker.wustl.edu/impact-assessment/model ).

Donovan C, Hanney S. The ‘Payback Framework’ explained. Res Evaluation. 2011;20:181–3. https://doi.org/10.3152/095820211X13118583635756 .

Canadian Academy of Health Sciences. Making and Impact: A Preferred Framework and Indicators to Measure Returns on Investment in Health Research [Internet]. Report of the Panel on the Return on Investments in Health Research Canadian. Ottawa, Ontario. ; 2009. Available from: http://www.xn--cahsacss-3m3d.ca/ .

https://www.ref.ac.uk/ .

Sarli CC, Dubinsky EK, Holmes KL. Beyond citation analysis: a model for assessment of research impact Recommended Citation"Beyond citation analysis: a model for assessment of research impact Beyond citation analysis: a model for assessment of research impact. J Med Libr Assoc [Internet]. 2010;981(1):17–23. Available from: http://digitalcommons.wustl.edu/open_access_pubs .

Goodman MS, Ackermann N, Bowen DJ, Panel D, Thompson VS. Reaching Consensus on Principles of Stakeholder Engagement in Research. Progress in Community Health Partnerships: Research Education and Action. 2020;14(1):117–27. https://doi.org/10.1353/cpr.2020.0014 .

Article   PubMed   Google Scholar  

Lambeth G, Szebeko B. Prototyping public services. Issue November; 2011. http://www.guardianpublic.co.uk/prototyping-public-services) .

O’Cathain A, Thomas KJ. (2004) ‘Any other comments?’ Open questions on questionnaires–a bane or a bonus to research? BMC Medical Research Methodology, 4(1), 1–7).

https://arc-w.nihr.ac.uk/Wordpress/wp-content/uploads/2020/02/Full-guidelines-for-Best-Practice-in-the-Ethics-and-Governance-of-Service-Evaluation-Final02.pdf.

Cooke J, Gardois P, Booth A. Uncovering the mechanisms of research capacity development in health and social care: a realist synthesis. Health Res Policy Syst. 2018;16(1):1–22.

Dimova S, Prideaux R, Ball S, Harshfield A, Carpenter A, Marjanovic S. Enabling NHS staff to contribute to research: reflecting on current practice and informing future opportunities. Santa Monica, CA: RAND Corporation; 2018.

Book   Google Scholar  

Lorenc T, Tyner EF, Petticrew M, Duffy S, Martineau FP, Phillips G, Lock K. Cultures of evidence across policy sectors: systematic review of qualitative evidence. Eur J Pub Health. 2014;24(6):1041–7.

Wolfenden L, Yoong SL, Williams CM, Grimshaw J, Durrheim DN, Gillham K, Wiggers J. 2017. Embedding researchers in health service organizations improves research translation and health service performance: the Australian Hunter New England Population Health example. Journal of Clinical Epidemiology , 85 , pp.3–11. Available at https://www.jclinepi.com/action/showPdf?pii=S0895-4356%2817%2930254-8 .

Oliver K, Kothari A, Mays N. The dark side of coproduction: do the costs outweigh the benefits for health research? Health Res Policy Sys. 2019;17:33. https://doi.org/10.1186/s12961-019-0432-3 . https://health-policy-systems.biomedcentral.com/articles/ .

Kislov R, Wilson PM, Knowles S, Boaden R. Learning from the emergence of NIHR Collaborations for Leadership in Applied Health Research and Care (CLAHRCs): a systematic review of evaluations. Implement Sci. 2018;13(1):111. https://doi.org/10.1186/s13012-018-0805-y . https://implementationscience.biomedcentral.com/articles/ .

Reed MS. (2018) The Research Impact Handbook, 2nd edition., Fast Track Impact.

Alla K, Hall WD, Whiteford HA, Head BW, Meurk CS. 2017. How do we define the policy impact of public health research? A systematic review. Health research policy and systems, 15(1), p.84.

Jagosh J, Bush PL, Salsberg J, Macaulay AC, Greenhalgh T, Wong G, Cargo M, Green LW, Herbert CP, Pluye P. 2015. A realist evaluation of community-based participatory research: partnership synergy, trust building and related ripple effects. BMC public health , 15 (1), pp.1–11. Available at https://bmcpublichealth.biomedcentral.com/articles/ https://doi.org/10.1186/s12889-015-1949-1 .

Download references

Acknowledgements

Thanks to the ACORN group for piloting and using the VICTOR tool.

Funding for the development of the tool was provided by NIHR YH CRN and NIHR CLAHRC both hosted by Sheffield Teaching Hospitals Trust.

Author information

Authors and affiliations.

Research Department, Mid Yorkshire Teaching NHS Trust, Pinderfields Hospital, Aberford Road, Wakefield, WF1 4AL, UK

Judith Holliday

Primary Care Sheffield, Fifth Floor, 722 Prince of Wales Road, Sheffield, S9 4EU, UK

Natalie Jones

School of Health Science, University of Sheffield, 30 Regent Street, Regent Court, Sheffield, S1 4DA, UK

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed to the development of the tool; writing of the manuscript.

Corresponding author

Correspondence to Judith Holliday .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethics approval and consent to participate

Ethical approval was not required as this is a service improvement project, registered at Sheffield Teaching Hospitals Trust.

This project was registered with Sheffield Teaching Hospitals FT Trust as a service evaluation project with the clinical effectiveness unit (CEU) project number 8952 on electronic database AIMS and all methods were carried out in accordance with relevant guidelines and regulations. The project is titled ‘VICTOR- Making Visible the Impact of Research in the NHS: developing a research impact tool for clinicians.’ Natalie Jones was listed as the project lead. The sample period was 17/09/2017–07/01/2019 and the data collection period was 08/01/2018–31/12/2018. All participants in the evaluation gave informed consent to participate. They were provided with information about the project in writing and /or verbally. They had an opportunity to think about participation and whether they would like to take part before consent was taken. Findings from participants were anonymised to protect confidentiality. All relevant procedures for service evaluations in Sheffield Teaching Hospitals were adhered to and the project was supported by a co-ordinator from the clinical effectiveness unit to ensure relevant procedures were followed.

Consent for publication

Not applicable.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Holliday, J., Jones, N. & Cooke, J. Organisational benefits of undertaking research in healthcare: an approach to uncover impact. BMC Res Notes 16 , 255 (2023). https://doi.org/10.1186/s13104-023-06526-5

Download citation

Received : 10 February 2023

Accepted : 21 September 2023

Published : 05 October 2023

DOI : https://doi.org/10.1186/s13104-023-06526-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Research impact
• Coproduction
• Research collaborations
• By-products of research

BMC Research Notes

ISSN: 1756-0500

• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Evidence-based practice improves patient outcomes and healthcare system return on investment: Findings from a scoping review

Affiliations.

• 1 Helene Fuld Health Trust National Institute for Evidence-Based Practice in Nursing & Healthcare, College of Nursing, The Ohio State University, Columbus, Ohio, USA.
• 2 St. John Fisher University, Wegmans School of Nursing, Rochester, New York, USA.
• 3 Sinai Hospital, Baltimore, Maryland, USA.
• 4 Summa Health System, Akron, Ohio, USA.
• 5 The Ohio State University, College of Nursing, Columbus, Ohio, USA.
• 6 Memorial Sloan-Kettering Cancer Center, New York, New York, USA.
• 7 Family CareX, Denver, Colorado, USA.
• 8 Affiliate Faculty, VCU Libraries, Health Sciences Library, Virginia Commonwealth University School of Nursing, Richmond, Virginia, USA.
• PMID: 36751881
• DOI: 10.1111/wvn.12621

Background: Evidence-based practice and decision-making have been consistently linked to improved quality of care, patient safety, and many positive clinical outcomes in isolated reports throughout the literature. However, a comprehensive summary and review of the extent and type of evidence-based practices (EBPs) and their associated outcomes across clinical settings are lacking.

Aims: The purpose of this scoping review was to provide a thorough summary of published literature on the implementation of EBPs on patient outcomes in healthcare settings.

Methods: A comprehensive librarian-assisted search was done with three databases, and two reviewers independently performed title/abstract and full-text reviews within a systematic review software system. Extraction was performed by the eight review team members.

Results: Of 8537 articles included in the review, 636 (7.5%) met the inclusion criteria. Most articles (63.3%) were published in the United States, and 90% took place in the acute care setting. There was substantial heterogeneity in project definitions, designs, and outcomes. Various EBPs were implemented, with just over a third including some aspect of infection prevention, and most (91.2%) linked to reimbursement. Only 19% measured return on investment (ROI); 94% showed a positive ROI, and none showed a negative ROI. The two most reported outcomes were length of stay (15%), followed by mortality (12%).

Linking evidence to action: Findings indicate that EBPs improve patient outcomes and ROI for healthcare systems. Coordinated and consistent use of established nomenclature and methods to evaluate EBP and patient outcomes are needed to effectively increase the growth and impact of EBP across care settings. Leaders, clinicians, publishers, and educators all have a professional responsibility related to improving the current state of EBP. Several key actions are needed to mitigate confusion around EBP and to help clinicians understand the differences between quality improvement, implementation science, EBP, and research.

Keywords: evidence-based decision making; evidence-based practice; healthcare; patient outcomes; patient safety; return on investment.

© 2023 The Authors. Worldviews on Evidence-based Nursing published by Wiley Periodicals LLC on behalf of Sigma Theta Tau International.

PubMed Disclaimer

• Leaders and managers in nursing and healthcare are key to advancing and sustaining evidence-based practice. Melnyk BM. Melnyk BM. Worldviews Evid Based Nurs. 2023 Feb;20(1):4-5. doi: 10.1111/wvn.12626. Epub 2023 Jan 3. Worldviews Evid Based Nurs. 2023. PMID: 36594474 No abstract available.

Similar articles

• Beyond the black stump: rapid reviews of health research issues affecting regional, rural and remote Australia. Osborne SR, Alston LV, Bolton KA, Whelan J, Reeve E, Wong Shee A, Browne J, Walker T, Versace VL, Allender S, Nichols M, Backholer K, Goodwin N, Lewis S, Dalton H, Prael G, Curtin M, Brooks R, Verdon S, Crockett J, Hodgins G, Walsh S, Lyle DM, Thompson SC, Browne LJ, Knight S, Pit SW, Jones M, Gillam MH, Leach MJ, Gonzalez-Chica DA, Muyambi K, Eshetie T, Tran K, May E, Lieschke G, Parker V, Smith A, Hayes C, Dunlop AJ, Rajappa H, White R, Oakley P, Holliday S. Osborne SR, et al. Med J Aust. 2020 Dec;213 Suppl 11:S3-S32.e1. doi: 10.5694/mja2.50881. Med J Aust. 2020. PMID: 33314144
• Folic acid supplementation and malaria susceptibility and severity among people taking antifolate antimalarial drugs in endemic areas. Crider K, Williams J, Qi YP, Gutman J, Yeung L, Mai C, Finkelstain J, Mehta S, Pons-Duran C, Menéndez C, Moraleda C, Rogers L, Daniels K, Green P. Crider K, et al. Cochrane Database Syst Rev. 2022 Feb 1;2(2022):CD014217. doi: 10.1002/14651858.CD014217. Cochrane Database Syst Rev. 2022. PMID: 36321557 Free PMC article.
• The future of Cochrane Neonatal. Soll RF, Ovelman C, McGuire W. Soll RF, et al. Early Hum Dev. 2020 Nov;150:105191. doi: 10.1016/j.earlhumdev.2020.105191. Epub 2020 Sep 12. Early Hum Dev. 2020. PMID: 33036834
• Strategies for incorporating evidence-based practice into nurse residency programs: A scoping review. Sampson M, Knupp A, Chignolli H, Dhakal K, Bulkowski K, Perdue J, Warren S, Zellefrow C. Sampson M, et al. Worldviews Evid Based Nurs. 2024 Aug;21(4):407-414. doi: 10.1111/wvn.12720. Epub 2024 Mar 29. Worldviews Evid Based Nurs. 2024. PMID: 38549466 Review.
• The use of positive deviance approach to improve health service delivery and quality of care: a scoping review. Kassie AM, Eakin E, Abate BB, Endalamaw A, Zewdie A, Wolka E, Assefa Y. Kassie AM, et al. BMC Health Serv Res. 2024 Apr 8;24(1):438. doi: 10.1186/s12913-024-10850-2. BMC Health Serv Res. 2024. PMID: 38589897 Free PMC article. Review.
• Elevating Tracheostomy Care Through Data-Driven Innovation: What Can Education, Evidence-Based Practice, and Quality Improvement Learn from One Another? Brenner MJ, Pandian V. Brenner MJ, et al. Tracheostomy. 2024;1(2):1-6. Epub 2024 Jun 30. Tracheostomy. 2024. PMID: 39188761 Free PMC article.
• The Mediating Role of Students' Health Information Literacy Skills: Exploring the Relationship Between Web Resource Utilization and Health Information Evaluation Proficiency. Habiba U, Koli FS. Habiba U, et al. Health Expect. 2024 Aug;27(4):e14176. doi: 10.1111/hex.14176. Health Expect. 2024. PMID: 39148230 Free PMC article.
• Pharmacy practice and policy research in Türkiye: a systematic review of literature. Gülpınar G, Pehlivanlı A, Babaar ZU. Gülpınar G, et al. J Pharm Policy Pract. 2024 Aug 12;17(1):2385939. doi: 10.1080/20523211.2024.2385939. eCollection 2024. J Pharm Policy Pract. 2024. PMID: 39139388 Free PMC article. Review.
• Evidence-based practice knowledge, attitude, practice and barriers as predictors of stay intent among Jordanian registered nurses: a cross-sectional study. Al-Ghabeesh S, Khalifeh AH, Rayan A. Al-Ghabeesh S, et al. BMJ Open. 2024 Jul 18;14(7):e082173. doi: 10.1136/bmjopen-2023-082173. BMJ Open. 2024. PMID: 39025821 Free PMC article.
• Factors Associated with Evidence-Based Practice Competencies among Taiwanese Nurses: A Cross-Sectional Study. Cheng LC, Chen CJ, Lin SC, Koo M. Cheng LC, et al. Healthcare (Basel). 2024 Apr 26;12(9):906. doi: 10.3390/healthcare12090906. Healthcare (Basel). 2024. PMID: 38727463 Free PMC article.
• American Nurses Association. (2022). Contributing to a quality patient experience: Applying evidence-based practice to support changes in nursing dress code policies. http://www.nursingworld.org/
• American Nurses Credentialing Center. (2022). Practice model. http://www.nursecredentialing.org/Magnet.aspx
• Centers for Disease Control and Prevention. (2021). 2021 National and state healthcare-associated infections progress report. https://www.cdc.gov/hai/data/portal/progress-report.html
• Dean, J., Gallagher-Ford, L., & Connor, L. (2021). Evidence-based practice: A new dissemination guide. Worldviews on Evidence-Based Nursing, 18(1), 4-7. https://doi.org/10.1111/wvn.12489
• Eccles, M. P., & Mittman, B. S. (2006). Welcome to implementation science. Implementation Science, 1, 1-3.

Publication types

• Search in MeSH

LinkOut - more resources

Full text sources.

• Ovid Technologies, Inc.
• MedlinePlus Health Information

Miscellaneous

• NCI CPTAC Assay Portal

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Related Programs

Measurement, design, and analysis methods for health outcomes research, related articles, how health care executives can use health outcomes research for business decision making, using health outcomes research to improve quality of care, genomic medicine in the next decade, the value of applying health outcomes research to improve treatment results.

by Lisa D. Ellis

With mounting pressure on health care organizations to provide high-quality care while containing costs, there’s been an increasing reliance on using health outcomes research to identify the most effective interventions and incorporate them into clinical practice. As such, health outcomes research can provide a valuable resource to help clinicians make strategic treatment choices that will ultimately benefit patients and systems on many levels.

“Health outcomes research studies the end results of health care services, providing evidence for the value of specific medical treatments or interventions that can be used to make better decisions and improve health care,” explains Marcia A. Testa, MPH, MPhil, PhD, who serves as Senior Lecturer on Biostatistics for the Harvard T.H. Chan School of Public Health. Testa and Donald C. Simonson, MD, MBA, MPH, ScD, of the Division of Endocrinology, Diabetes and Hypertension at Brigham and Women’s Hospital and Harvard Medical School, co-direct a program called Measurement, Design, and Analysis Methods for Health Outcomes Research offered by the Harvard Chan School’s Executive and Continuing Education (ECPE) division.

Incorporating Patient-Centered Outcomes

Testa points out that some of the most effective examples of health outcomes research consider patient-centered outcomes; incorporating the patient’s lifestyle, preferences, and voice in applied research is critical, since these aspects impact treatment compliance and, therefore, outcomes. “‘Patient-reported’ outcomes are a big part of ‘patient-centeredness,’” she stresses.

Health outcomes research can also play an important role in identifying disparities among different populations and guiding clinicians on taking action to help even the playing field for patients of all socioeconomic groups and backgrounds.

Further, patient-centered outcomes, including those reported by patients themselves, enable “people and their caregivers [to] communicate and make informed health care decisions, allowing their voices to be heard in assessing the value of health care options,” according to the Patient-Centered Outcomes Research Institute’s (PCORI) website.

Utilizing the Findings

With so much to gain, it’s essential that prospective investigators understand the scope of health outcomes research and recognize how to utilize it in their efforts in the most appropriate way.

Specifically, when researchers understand how to properly design and implement health outcomes research studies and to synthesize the data, the findings can be used in a multitude of situations. For instance, the information can help practitioners to select the most appropriate treatment options for individual patients that take into account their specific needs and situations. The findings can also help to identify gaps in treatment choices for different patient populations. In addition, the data can be used to determine any interventions that are over- or under-used by population groups in order to help providers develop evidence-based treatment strategies that bring value-added care, Simonson says.

But of course, even the most well-designed and executed health outcomes research projects will only be effective if clinicians and other medical providers know how to apply the data on patients in a clinical setting.  In fact, the process of translating and using research findings is itself a relatively new field of research, called implementation science. This field studies methods to promote the adoption and integration of evidence-based practices, interventions, and policies into routine health care and public health settings. It is the natural “next step” in health outcomes research, and PCORI even offers special funding opportunities to previous PCORI research recipients for “Dissemination and Implementation” of their research results.

Bridging Challenges That Exist

To help bridge some of the challenges that exist in applying health outcomes findings, Simonson, Testa, and their colleagues are currently working on a grant project funded by PCORI titled “Benchmarking the Comparative Effectiveness of Diabetes Treatments Using Patient-Reported Outcomes and Socio-Demographic Factors,” which was recently featured at the American Diabetes Association’s (ADA) 2017 annual meeting.

With diabetes currently affecting 11.3 percent of Americans aged 21 and older, causing high blood glucose levels that can lead to a variety of serious health issues, many patients require specific dietary requirements and regular exercise, along with glucose monitoring and treatments including insulin. But while clinical trials measure the effectiveness of different treatment approaches in a general way, most physicians typically don’t alter their treatment approach to respond to these variables for each individual.

While clinical trials measure the effectiveness of different treatment approaches in a general way, most physicians typically don’t alter their treatment approach to respond to these variables for each individual.

“There are many factors that influence how well patients are able to comply with these requirements including age, education, income, and cultural and lifestyle issues,” Simonson says. Further, any side effects of treatment, or any other co-existing conditions, may also impact a person’s likelihood to follow a specific treatment regimen. Yet these aspects are largely unexplored when developing treatment strategies.

The Need for Individualized Treatment Plans

“Since diabetes patients act and respond differently to treatment due to many reasons, health care providers often cannot advise patients as to how they might respond given their personal characteristics simply because they do not have the required information,” Simonson points out.

The reason this crucial information is lacking is that “typically, no one clinical study can separate out the results by all the patient characteristics that might affect treatment,” Testa offers. “In addition, clinical drug trials do not typically measure how patients feel or how satisfied they are with their assigned treatment,” she says, adding, “In most clinical trials, the ‘true voice’ of the patient is usually silent.” To better capture this important component, Testa, Simonson, and their colleagues are pooling existing databases of diabetes patients with information gathered online and through social media.

Educating Physicians to Strengthen Treatment Efforts

“We will incorporate the results of our findings and analyses into a web-based application that will allow clinicians to predict how likely a patient will be to respond given their individual characteristics, and will allow patients and physicians to benchmark their progress against others with similar characteristics to improve the quality of care,” Testa says.

The hope is that, moving forward, more investigators and practitioners will recognize the importance of considering patients as individuals when developing treatment strategies to create a more personalized approach that will likely achieve better results.

The researchers are also using their findings to develop an online “toolkit” designed to educate physicians on how to interpret health outcomes data and put the findings into practice in the patient-care setting.

Improving the Decision-Making Process

“A diverse and growing number of groups, including employers, health care delivery organizations, insurers, pharmaceutical companies, and government agencies, currently use actionable data provided by health outcomes research to guide their decisions about different treatment options and interventions,” Testa says. This widespread use should prompt prospective investigators to make a concerted effort to consider this element in future research projects. “By using health outcomes data to guide the decision-making process across organizations, this can increase the value of every dollar spent on health care,” she adds.

Harvard T.H. Chan School of Public Health offers Measurement, Design, and Analysis Methods for Health Outcomes Research , a program focused on designing, implementing, and analyzing health outcomes studies. To learn more about this opportunity, click here .

• U.S. Department of Health & Human Services

• Virtual Tour

Staff Directory

• En Español

You are here

The National Institutes of Health (NIH), a part of the U.S. Department of Health and Human Services, is the nation’s medical research agency — making important discoveries that improve health and save lives.

dr-monica-bertagnolli-thumbnail.jpg

The NIH Director

Monica M. Bertagnolli, M.D., is the 17 th director of the National Institutes of Health, officially taking office on November 9, 2023.

bldg-one-blue-sky.jpg

NIH Leadership

NIH leadership plays an active role in shaping the agency's research planning, activities, and outlook.

orgchart.jpg

Organization

The NIH is made up of 27 different components called Institutes and Centers.

rolodex.jpg

The NIH Enterprise Directory (NED) is an electronic directory of people who work at the NIH.

nows-your-chance-to-run.JPG

Voices of the NIH Community

NIH’s project to capture an oral history of the research experience.

joseph-goldberger.jpg

For over a century, NIH scientists have paved the way for important discoveries that improve health and save lives.

Connect with Us

• More Social Media from NIH

Eight years on and not forgotten: Local Dad shares wife’s blood cancer story to highlight importance of medical research in Northern Ireland

September is Blood Cancer Awareness Month and Today is Legacy Giving Day. Giving hope for 60 years: Leukaemia & Lymphoma NI's legacy in the fight against blood cancer - As we celebrate 60 years of Leukaemia & Lymphoma NI's tireless work in the fight against blood cancer, we are honoured to share the powerful stories of those who have been directly touched by this disease. Through a series of compelling case studies, patients, family members and researchers open up about their personal journeys, offering a poignant look at the challenges they’ve faced, the resilience they’ve shown and the hope that continues to drive them forward.

• 09:34, 13 SEP 2024
• Updated 12:21, 14 SEP 2024

Stay on top of the headlines from Belfast and beyond by signing up for FREE email alerts

We have more newsletters

A local father of two has been sharing the story of his wife’s fight against blood cancer to help highlight the importance of supporting local medical research funded by Leukaemia and Lymphoma NI.

Alison and Barry Williamson and their two young children Rhys and Mya lived in Tandragee, Co Armagh . Alison was a much-loved member of her community. She worked as a classroom assistant at Tandragee Primary School and was well-known for her enthusiasm, energy and mischievous sense of humour. She had a zest for life and was fiercely positive, no matter what challenges life threw at her.

“Alison was pretty much just the perfect person,” explained Barry. “She was truly an inspiration to everyone that knew her. Her family was her life but she was always helping others.”

Towards the end of 2014, Alison began to feel unwell. She saw her weight drop rapidly.

“Alison really started behaving out of character. Often she would come home from school and need to lie down. She was constantly feeling drained and was sleeping more and more. That’s when we went to the doctor.”

Alongside the fatigue, Alison started experiencing pain in her abdomen. Eventually she underwent an operation to remove her spleen. “At that point, we thought she had started to turn a corner” said Barry, “we were hopeful that might be it and we had found the root of the problem.”

Unfortunately, in August 2015 after several more tests, the family received news of Alison’s diagnosis of Hepatosplenic T-cell Lymphoma, a rare and aggressive form of blood cancer.

“When we received the news, Alison was steadfast and determined. Whilst I sat in the corner of the room distraught, she simply said ‘Ok, I’m going to fight this.’ That’s what kind of person she was. Sharing the news with our two children was obviously a very difficult thing to have to do. But Alison was unwaveringly positive.”

After her diagnosis, Alison endured 50 days of chemotherapy and eventually was earmarked for a stem cell transplant.

“It became a race against time to find a matching donor. Everyone in our family got tested. Both Alison and I were very lucky to have such close supportive families and friends and everyone rallied round us. As always, Alison took on the news of the transplant head on.”

Alison was treated at Craigavon Hospital, was then moved to Dublin and then finally brought to Belfast City Hospital. At one stage she was taking 53 tablets a day.

“She never complained” said Barry, “it wasn’t in her nature. She was prepared to do whatever it took – undergo any treatments available – to give her the chance to spend more time with her family.”

However, on 14 May 2016, in the Intensive Care Unit of Belfast City Hospital and nine months after her initial diagnosis, Alison died. “She never gave up the fight and up until her last day I was still holding hope that she would pull through, but her body just simply couldn’t take any more.

"Alison is, without doubt, sorely missed by us and those close to her. When grief comes into your life it stays day and night. I found strength from my two children. They didn’t want to see me so sad all the time. They were grieving too and I had to make life good for them and teach them, as best I could, how to live life and find joy in things. That is why the charity and the work I try to do will make a difference to others. I take inspiration from my late wife and the fight she put up to live, and I hope she is proud of me for that.”

Since her passing, Barry, his friends and his family decided that raising money for Leukaemia & Lymphoma NI was the best way to honour her memory.

“It was Alison who started the charity work before she died, so it seemed only right to continue with it after she passed.

“Since then we’ve climbed the four highest peaks in the UK in the space of 48 hours and scaled part of the Alps, moving through three countries across three consecutive days. We’ve done glass walks, fire walks, held gala charity balls and staged mega balloon releases.

“Friends held coffee mornings and my then Rector Dean Forster and Ballymore Church allsupported me, the children and the charity in so many ways. Alison’s Mum and friend did a sky dive, the school where Alison worked staged various fundraising events and we have done so many other things with the help of family and friends, too many to mention.

“Throughout it all, we’ve felt closer to Alison – it’s kept her with us. The fundraising is something we plan to continue and we hope others will consider donating or running their own fundraising activities. It’s such a worthy cause and the charity is funding crucial research.”

The money raised by Barry and his friends and family funded the Alison Williamson PhD Studentship in 2017, leaving a lasting legacy in blood cancer research. Dr Harmony Black completed her PhD in Repair Mechanisms in 2020 and is now working as a clinical scientist in the haematology department at Belfast City Hospital where she plays a vital role in screening and analysing patient samples.

Leukaemia and Lymphoma NI recently announced a special programme of activities to mark the 60th year of the charity and to raise funds for the fight against blood cancer.

Alongside this, the charity is calling for those who have experienced Leukaemia, Lymphoma, Myeloma, or other blood cancer, and their loved ones, to share their experiences online via the LLNI website – Share your story - LLNI

The photos and extended captions gathered will form part of a special canvas presented online where members of the public can read the stories of people affected by blood cancer across the region.

Throughout September LLNI is holding a series of fundraising activities, culminating in the charity hosting a Black Tie & Diamonds Gala Ball at Titanic Belfast, where the winner of an ongoing raffle for a diamond pendant necklace will be selected at random.

Members of the public can share their story, make a donation or buy tickets for the diamond necklace raffle on the Leukaemia & Lymphoma NI website – www.llni.co.uk

Choosing to leave a gift to LLNI is a wonderful gesture to ensure that your legacy lives on through research long after you are gone. Visit www.lni.co.uk.

• Titanic Belfast
• Most Recent

• Open access
• Published: 13 September 2024

Is this really Empowerment? Enhancing our understanding of empowerment in patient and public involvement within clinical research

• Imke Schilling 1 , 2 &
• Ansgar Gerhardus 1 , 2  

BMC Medical Research Methodology volume  24 , Article number:  205 ( 2024 ) Cite this article

Metrics details

There has been a growing push to involve patients in clinical research, shifting from conducting research on, about, or for them to conducting it with them. Two arguments advocate for this approach, known as Patient and Public Involvement (PPI): to improve research quality, appropriateness, relevance, and credibility by including patients’ diverse perspectives, and to use PPI to empower patients and democratize research for more equity in research and healthcare. However, while empowerment is a core objective, it is often not clear what is meant by empowerment in the context of PPI in clinical research. This vacancy can lead to insecurities for both patients and researchers and a disconnect between the rhetoric of empowerment in PPI and the reality of its practice in clinical trials. Thus, clarifying the understanding of empowerment within PPI in clinical research is essential to ensure that involvement does not become tokenistic and depletes patients’ capacity to advocate for their rights and needs.

We explored the historical roots of empowerment, primarily emerging from mid-20th century social movements like feminism and civil rights and reflected the conceptual roots of empowerment from diverse fields to better understand the (potential) role of empowerment in PPI in clinical research including its possibilities and limitations.

Common themes of empowerment in PPI and other fields are participation, challenging power structures, valuing diverse perspectives, and promoting collaboration. On the other hand, themes such as contextual differences in the empowerment objectives, the relationship between empowerment and scientific demands, research expertise, and power asymmetries mark a clear distinction from empowerment in other fields.

PPI offers potential for patient empowerment in clinical trials, even when its primary goal may be research quality. Elements like participation, sharing opinions, and active engagement can contribute to patient empowerment. Nonetheless, some expectations tied to empowerment might not be met within the constraints of clinical research. To empower patients, stakeholders must be explicit about what empowerment means in their research, engage in transparent communication about its realistic scope, and continuously reflect on how empowerment can be fostered and sustained within the research process.

Peer Review reports

Background and problem statemen

Introduction.

There has been a growing demand from patients, researchers, research sponsors, and scientific journals to shift clinical studies from being exclusively conducted on, about, or for patients to involving patients themselves or members of the public [ 1 , 2 ]. Two primary lines of reasoning underlie active patient and public involvement (PPI):

By integrating patients’ diverse perspectives into research, the aim is to enhance the quality, appropriateness, relevance, and credibility of the research [ 3 , 4 ].

Additionally, there are normative arguments supporting PPI that revolve around moral, ethical, and rights-based considerations, primarily linked to empowering patients or the public [ 5 ]. In essence, the idea is that patients should have a say in research that directly concerns them [ 3 , 6 ]. This notion aligns with the principle of “nothing about us, without us,” which has guided movements in various contexts, including the disability rights movement [ 7 ] and Indigenous contexts [ 8 ].

By empowering patients and upholding their right to participate in research, PPI seeks to diminish social inequalities. In doing so, it aims to democratize the research process, making it more accountable and transparent to the broader population [ 2 , 3 , 4 , 5 , 9 ]. This democratization is particularly significant for marginalized groups whose perspectives are often overlooked [ 1 , 5 ].

While patient empowerment is a core objective of PPI [ 4 ], it is seldom explicitly defined within the context of PPI. Based on the etymology, the root of the term implies that ‘empowerment’ concerns matters of ‘power’. The Oxford English Dictionary offers three distinct meanings of the verb “empower” [ 10 ]. One involves granting someone legal or formal authority, another focuses on bestowing power over something, and the third pertains to strengthening an individual by providing greater control, specific attributes, or enhanced abilities. Empowerment can denote either a process or a state of being respectively an outcome.

A narrative review by Gradinger et al. revealed that in the context of public involvement, normative values are frequently referenced without clear definitions, resulting in significant variations in the understanding of empowerment [ 4 ]. While there is a general need to clarify the conceptualization of PPI to align with its intended goals [ 11 ], the emancipatory aspect of PPI remains underexplored compared to other approaches [ 12 ]. Without a precise meaning and operationalization of the term ‘empowerment’, the normative claim of PPI becomes difficult to realize and its implementation virtually impossible to assess. The lack of a shared understanding of empowerment within PPI not only fosters misinterpretation and arbitrariness in PPI practices but may also inadvertently undermine patient empowerment. From the perspective of a patient, ambiguous roles, a sense of inability to contribute, insufficient recognition of one’s contributions, or inadequate information about the benefits of involvement could potentially be rather disempowering than empowering [ 13 ]. There is a risk that the involvement may become tokenistic, and patients’ voices might be silenced when they are merely involved for show, as a formality, without genuine influence on the research. Additionally, this involvement may deplete patients’ resources and capacity to advocate for their rights and needs in potentially more effective ways [ 14 , 15 ].

In a previous study, we discovered that within the same project, patients and researchers assign varying degrees of importance to patient empowerment. While patients engaged in a patient board for a clinical trial endorsed the idea of empowerment through research participation, only one out of five researchers explicitly addressed patient empowerment as a rationale for conducting PPI [ 16 ]. Furthermore, the experiences of patients and researchers with the patient board indicated that patient empowerment is often overlooked in the implementation of PPI. Other forms of collaboration, such as open dialogues on an equal footing and providing training to enhance patients’ confidence and skills, might have proven more effective in empowering patients [ 17 ]. These findings align with those of Ives et al. [ 3 ], who also noted a potential mismatch between the stated goals of PPI and its practical execution. Ives et al. argue that the nature and conduct of PPI can vary significantly depending on who initiates it and for what purpose. For instance, if researchers involve patients primarily to enhance the quality of their research projects, the focus might be on outcome-oriented, pragmatic consultation, potentially sidelining the goal of patient empowerment. Patients may be relegated to an informational role rather than active partners in the research process. Based on these insights, we assume that empowerment does not naturally evolve from PPI and is not an automatic byproduct of it.

Aim, research interest and approach

Considering the above, it seems necessary to clarify the term empowerment within PPI in clinical research. Despite the absence of a precise understanding of empowerment in the context of PPI, the term “empowerment” has been in use across various domains for over half a century, including social work, education, corporate settings, psychology, and healthcare [ 18 ]. Therefore, this article aims to contribute to the understanding of empowerment in PPI by reflecting on the history and tradition of the term and concept of empowerment in other fields. Building on this, we aim to reflect on what lies behind the term empowerment in the context of PPI in clinical research and try to explain the disconnect between the rhetoric of empowerment in PPI and the reality of its practice in clinical trials. We have been guided by the following questions and have structured the article accordingly:

How has the concept of empowerment evolved historically?

How has empowerment been conceptualized in other fields?

To what extent does the concept of empowerment of patients through or for PPI in clinical research align with conceptual approaches to empowerment in other fields?

The article provides researchers who organize PPI with orientation on the relationship between empowerment and PPI. It offers perspectives on the possibilities and limits of empowerment in this context and invites further reflection on the topic from both researchers and patients involved in PPI.

For consistency, the term ‘patient’ is exclusively used in this article to refer to individuals who have had specific health-related experiences. However, we acknowledge that other terms, such as ‘service users’, may be more suitable and better reflect the active role that PPI strives for. This article is centered around PPI in clinical research and does not encompass reflections on PPI in other contexts, such as healthcare.

Historical development of the term empowerment

The term “to empower” has been documented since the mid-17th century, with older forms such as ‘impover,’ ‘empour,’ and ‘empowre’ [ 10 ]. In the mid-17th century, William Penn, founder of the Quaker colony of Pennsylvania, utilized the term in a religious and early democratic context. Penn’s theology of individual empowerment was based on the belief in the intrinsic dignity of all individuals, the presence of a part of God within each person (referred to as the “inward light” or “inner spirit”), and the assertion of the right to freedom of conscience. Penn’s ideas influenced the formulation of a groundbreaking constitution for Pennsylvania, serving as a model for subsequent democratic constitutions [ 19 ].

History of empowerment in the social movements

The term “empowerment,” intertwined with democracy since its inception, has evolved over time, primarily shaped by mid-20th-century social movements.

Civil Rights Movement and Black Empowerment

The civil rights movement in the 1950s and 1960s among the Black minority in the U.S. significantly influenced the idea and implementation of empowerment. Acts of civil disobedience exposed racial inequalities [ 20 ], and multiplier programs aimed to provide education and raise consciousness among the Black community [ 18 , 20 ]. Grounded in the belief in individuals’ abilities to control their lives, the movement sought to integrate the Black minority as equals with equal social rights into the democratic society. Freeing the Black minority community from oppression through collective self-organization resulted in a “new sense of somebodiness” (Martin Luther King as cited in Simon [ 19 ]).

Feminist movement

Another driver of the empowerment discourse was the second wave of the feminist movement in the 1960s and 1970s, which addressed women’s opportunities and rights for societal equality [ 21 ]. Through expanded education, improved labor conditions, economic independence, far-reaching changes in the possibilities for self-determined birth control and a developed awareness of personal (bodily) autonomy, women’s life plans became more individualized [ 22 ]. Within the movement, women found a protective framework to navigate their evolving opportunities and resulting responsibilities. It provided a social reference structure, creating spaces for self-clarification, collective articulation of devaluation, and deconstruction of internalized beliefs. This support allowed women to envision, develop, and test new life possibilities and identities, thereby fostering self-confidence [ 18 ].

Self-Help movement

A third root of the modern empowerment concept is the self-help movement, which gained importance in the 1970s in the USA and other developed countries, especially within health-related contexts [ 7 , 18 ]. As self-organized networks, self-help aimed to establish social support, explore coping strategies, and reclaim autonomy and empowerment resources. Self-help served as a counter-program to perceived disempowering state care [ 7 ], emphasizing the perspective of individuals as ‘experts on their own account’, introducing self-organized services, creating (a sense of) community and thus producing emotional ‘services’, empowering critical consumers, and representing peoples’ interests to influence socio-political decisions [ 18 ]. Key features of self-help networks included the involvement of members with a common problem, minimal professional helper involvement, emphasis on immaterial support, and goals of self- and social change achieved through equal cooperation and mutual help. Self-help groups provided critical support in niches not covered by professional care services [ 18 ].

Community action programs and community psychology

In the U.S., community-based programs aimed at empowering individuals and building networks to address social segregation [ 23 ]. These programs furnished resources and support to enable individuals and communities to take charge of their lives and implement positive changes in their community. Political initiatives, like the Equal Opportunities Act of 1965, sought to reduce inequalities and poverty, promoting “maximum feasible participation” [ 18 ]. Empowerment was considered a means of encouraging self-sufficiency and reducing dependence on government support.

In the 1970s, community action programs became linked to community psychology, viewing individuals as part of communities and collaborating to identify strengths, resources, and needs. Strategies formulated aimed to empower and promote social justice while reducing social inequalities.

The tradition of empowerment in social movements encompasses both individual self-determination and collective action against structural constraints. The primary concerns were not only about self-empowerment but also about advocating for structural changes through mass mobilization and collective efforts. In these contexts, empowerment was often pursued through independently organized groups that fostered community solidarity and collective identity. Unlike prevalent deficit-based approaches, which tend to focus on individuals’ lacks and weaknesses, empowerment in social movements nurtures and strengthens individuals’ skills and capabilities while also addressing and dismantling oppressive structures.

Today: use in various contexts

Since its emergence in mid-20th-century social movements and subsequent development in community psychology, the concept of empowerment has found application across diverse domains [ 18 , 24 ]:

Social work, encompassing individual support and collective actions.

Educational programs, such as literacy campaigns and increased pupil participation opportunities.

Development aid, representing a shift from external, top-down approaches to fostering local community capacity for participatory development and poverty reduction in developing countries.

Corporate contexts, where empowerment principles are integrated into management strategies.

Healthcare, where applications include shared decision-making and broader patient involvement.

Contemporary movements, such as racial empowerment in the “Black Lives Matter” movement and the Indigenization.

Concepts of empowerment

In this section, we explore the foundational concepts and theoretical underpinnings of empowerment.

Key concepts related to empowerment

Social scientist Barbara Bryant Solomon pioneered the conceptual foundation of empowerment in her 1976 book, “Black Empowerment: Social Work in Oppressed Communities.” Originating as a resource for students and social workers assisting Black minority clients, Solomon’s empowerment concept is based on research into the mechanisms of power and powerlessness. According to her, “empowerment refers to the reduction of an overriding sense of powerlessness to direct one’s life in the direction of meaningful personal satisfaction” [ 25 ]. At the core of this concept is the experience of powerlessness, arising from membership in a minority group subject to negative assumptions and discrimination from the majority society and its institutions [ 26 , 27 ].

While previous authors had emphasized the need to consider stigma as a factor that permeates the social situation of Black people, Solomon added that the unequal distribution of power and the experience of (structural) discrimination could affect the psyche and the negative attributions could find their way into self-perception. Thus, powerlessness of an individual means “the inability to manage emotions, knowledge, skills or material resources in a way that makes possible effective performance of valued social roles so as to receive personal gratification” [ 26 ].

At the community level, powerlessness is described as the inability to utilize resources for collective goals [ 26 ]. In short, stigma affects powerlessness, hindering access to the resources necessary for overcoming negative self-perceptions and social challenges [ 27 ]. Introducing empowerment as a method, Solomon suggested that professionals could employ it to address the powerlessness experienced by stigmatized individuals or groups. Empowerment, in her view, enables individuals to recognize their competence, perceive available opportunities for control, and ultimately enhance their self-worth and dignity [ 25 ]. In summary, Solomon’s empowerment approach is based on the belief that individuals and families have strengths and abilities and that they can be supported to use their resources more effectively for their own benefit. Solomon saw empowerment as both a process and a goal for social work in Black communities, and stated that the success of empowerment is “directly related to the degree to which the service delivery system itself is an obstacle course or an opportunity system” [ 26 ].

In 1981, community psychologist Julian Rappaport advocated for empowerment as a superior approach to paternalistic public health policies and rights-based advocacy in social work [ 28 ]. Acknowledging the diverse nature of social problems, Rappaport urged professionals to reconsider their roles in relation to clients, aligning with Solomon’s view that empowerment enhances individuals’ control over their lives.

Rappaport emphasized viewing individuals not solely as children in need or rights-bearing citizens but as complete human beings with both rights and needs. He argued that even those seemingly incompetent and in need require “[…] more rather than less control over their own lives, and fostering more control does not necessarily mean ignoring them“ [ 28 ]. Increased control is believed to positively influence psychological well-being.

Empowerment, according to Rappaport, relies on the belief that people possess or can acquire competencies, with inadequate functioning attributed to social structures or the lack of resources that prevent people from using these competencies. He advocated for competency development in real-life settings and positioned those providing help as collaborative teammates who take into account social structures and living conditions, and not as authoritative experts [ 28 ].

Furthermore, Rappaport stressed the need for diverse solutions to divergent problems, rejecting a one-size-fits-all approach in social policy. He championed a bottom-up, participatory social policy that recognizes the context-specific and varied nature of empowerment in each situation [ 28 ].

Brazilian educator and social reformer Paulo Freire expanded the concept of empowerment through his work with marginalized communities in Brazil [ 29 ]. Central to his ideas is the development of ‘critical consciousness’ through dialogic education [ 30 ]. Freire contended that oppressed individuals often lack awareness of the social and political factors sustaining their subjugation. Critical consciousness involves recognizing oppressive systems and understanding the socio-economic and political contexts fostering inequality, along with realizing one’s potential for transformation. Freire regarded the critical consciousness experience as the key to gaining strength, with education playing a fundamental role to conscientization. Freire’s dialogic teaching method, emphasizing two-way learning between teachers and students, fosters critical thinking, self-reflection, and active participation, empowers students to question and reshape their reality. Working in partnership assigns the teacher the role of a facilitator and underscores the central importance of the consumer or marginalized individuals in the process of change [ 19 , 30 ]. Complementing this, Freire’s pedagogy of questioning encourages students to critically assess the influences shaping their lives. The emphasis is not on remembering details, but on cultivating analytical skills and the capability to challenge prevailing beliefs.

Beyond individual liberation, Freire argued that true empowerment encompasses collective action and social transformation. He underscored the importance of solidarity and creating dialogic spaces for individuals to collaboratively address common experiences of oppression and work towards societal progress [ 30 ].

In summary, Freire sought to empower individuals and communities by promoting critical consciousness, dialogue, and collective action to challenge oppressive systems and foster a more inclusive and equitable society. While he placed responsibility on the oppressed for seeking their own empowerment, caution was advised to prevent reinforcing a sense of helplessness [ 29 ].

Common principles

While there is no universally agreed upon definition or concept of empowerment, some common principles can be identified, then, from what we have reviewed: Empowerment comes from a variety of sources, refers to processes and outcomes, involves both personal and collective dimensions, is based on participation, assumes that each individual has strength and capacities upon which they can build, challenges power structures with a focus on marginalized groups and the systematic inequalities they face, and must be obtained by the individuals themselves, but can be supported by third parties, e.g. professionals, who facilitate the process of empowerment in collaboration with individuals or communities [ 19 , 26 , 28 , 29 , 30 , 31 ].

As the most basic definition of empowerment, Herringer outlines: “Developmental processes over time in which individuals acquire the skills necessary to live a life that meets their own standards of ‘better’” [ 32 , translated by IS]. These processes of gaining more power or autonomy can be individual and collective [ 32 ].

Controversies

At the same time there exist some controversies around empowerment. Herringer continues his definition with the thought: “[.] what exactly constitutes a “more livable” existence is open to conflicting interpretations and ideological frameworks” [ 32 ]. Other controversies surrounding the concept of empowerment are:

Instrumentalization , tokenism and depoliticization : the concern that empowerment programs or initiatives may be implemented for instrumental, tokenistic purposes or to create the illusion of progress [ 27 ]. In such cases, empowerment becomes an empty concept without substantial impact. The adoption of empowerment concepts by the powerful (e.g. institutions or entities that hold significant structural and decision-making authority) can lead to a depoliticization of empowerment programs, as the transformative potential of such initiatives may be diminished or neutralized when circumscribed by institutional capture. This co-option of empowerment by those in power can result in a form of engagement that maintains existing power dynamics rather than challenging them.

Lack of clarity and measurement : empowerment is so diverse and open-ended that it is difficult to define in a way that its outcomes can be measured [ 24 ]. Clarity is needed regarding which aspects of empowerment are targeted. Without evaluating empowerment attempts, it is challenging to learn from experience.

Empowerment in the context of PPI

The concept of empowerment has deep roots in various social movements that sought to challenge systemic inequalities and give voice to marginalized groups. To analyze how these conceptual approaches to empowerment from social movements relate to the empowerment of patients in PPI within clinical research, we will first provide an overview of the historical development of PPI in research, followed by a recall of the relevance of empowerment in the context of PPI. We will then analyze and critically address (a) the similarities of approaches to empower patients or the public in PPI as compared with other fields, and by that get an impression how PPI in clinical research can empower patients, and (b) the distinctions and limitations of empowerment in this context, both practically and conceptually.

Evolution of PPI in research

Patient advocacy movements, gaining momentum in the mid-20th century, played a pivotal role in pushing for increased patient involvement in research [ 33 ]. These movements, which often emerged from broader social and civil rights movements, laid the foundation for what we now recognize as PPI.

For instance, the HIV-AIDS activism of the 1980s, heavily influenced by the gay civil rights movement, led to significant changes in health research by challenging the prevalent research expertise and bringing in “a ´patient perspective` to bear on institutions of health research” [ 34 ].

In the 1970s, Rose Kushner, a breast cancer patient and writer, exemplified this movement by assessing research proposals for the US National Cancer Institute, marking a notable instance of patient influence [ 33 ]. Her efforts reflected a broader movement towards giving patients a voice in research, a theme that is echoed in many PPI initiatives. The 1980s collaboration between patient organizations and the Association for Maternity Services, endorsing a randomized controlled trial on chorionic villus sampling, is another example where patient involvement began to influence research decisions directly. The 1997 international breast cancer advocacy conference organized by the US National Breast Cancer Association (NBCC) and supported by patient organizations from several countries marked a pivotal shift towards PPI, fostering dialogue on patient experiences and challenges. The conference demonstrated the NBCC’s belief that breast cancer patients should be consulted when making policies and decisions regarding research funding, and was instrumental in establishing an international advocacy movement [ 35 ].

The connection between PPI and social movements became more explicit with the establishment of organizations like INVOLVE in 1996, funded by the British government as part of their aim to create a patient-oriented healthcare system, the Canadian Institutes for Health Research in 2000, and the Patient-Centered Outcomes Research Institute (PCORI) in the United States in 2010. These organizations, drawing inspiration from social movements, emphasize the importance of involving patients and the public throughout the research process, thereby continuing the advocacy for marginalized voices in health research [ 36 , 37 , 38 ].

Globally, there is a trend toward formalized PPI approaches. Research funders, regulatory bodies, and institutions recognize the importance of involving patients and the public throughout the research process, from prioritization to dissemination [ 1 , 2 ]. At current there is still a lot of development and movement in the process.

Relevance of empowerment in PPI

As discussed, there are two arguments advocating for PPI use in research, that Ives et al. summarize [ 3 ]: (1) to improve research quality, appropriateness, relevance, and credibility (PPI as a means to an end) and (2) to use PPI to empower patients and democratize research along with its consequential impact on health(care) (PPI as an end in itself). However, empowerment through PPI should not be seen as an isolated goal, and Ives et al. phrasing as “an end in itself” might be misleading and be better put as “an end beyond narrowly instrumental goals”. PPI is a strategy that allows patients to actively shape research, thereby ensuring that the research directly addresses the practical problems they face – an argument rooted in the social movements.

PPI is essential in transforming the relationship between patients and institutions, challenging traditional power dynamics [ 34 ]. Its role is dual-faceted: it improves the quality and relevance of research while simultaneously fostering a more participatory and inclusive approach to healthcare. This dual function makes PPI a powerful tool for achieving both immediate research goals and broader societal change.

However, depending on the reasons and initiators of PPI, PPI practices can vary greatly. According to Ives et al. [ 3 ], different aims of PPI can result in distinct forms of involvement, as illustrated in Table  1 . While Ives et al. [ 3 ] seem to indicate two opposite ends of the spectrum, these “ideals” do not always play out and there are numerous intermediate forms of involvement that can exist. However, this example illustrates that the potential for empowerment in PPI, as well as its manifestations, can vary greatly depending on the approach taken.

Today PPI spans a broad range, from sporadic consultations, to ongoing collaboration between patients and researchers, and even (still rare examples of) research led by patients with support from researchers [ 39 ].

Similarity of empowerment in PPI in clinical research to earlier concepts

In the following sections we analyze and critically address the similarities and limits of empowerment in PPI in clinical research as compared with earlier concepts. Similarities of empowerment in PPI in clinical research to earlier concepts seem to be in a focus on participation, challenging power structures, valuing diverse knowledge and perspectives, and supporting collaboration.

Emphasis on participation

Active participation in decision-making processes that influence the lives of individuals and communities is a fundamental aspect of empowerment concepts across various fields [ 24 ]. In research-based PPI, facilitating the ability of patients and members of the public to have a voice, participate in decision-making processes, and contribute to research aligns with the core principles of empowerment.

Challenging power structures

Empowerment theories from different disciplines aim to reduce powerlessness and increase the power of marginalized individuals [ 25 , 28 , 29 ]. The objective of challenging power structures aligns with the concept of empowerment in PPI in research. Involving patients in planning, conducting, and communicating clinical research on a regular basis constitutes a significant shift in the power dynamics of the research landscape. Individual patients may be engaged on a one-time basis, but the collective voice of patients and the public becomes significant and co-determines research. Long-term patient involvement may be achieved through the integration of patient advisory boards in research institutions [ 40 ]. The inclusion of patient perspectives has become an expected practice, influencing power dynamics within the clinical research domain.

Recognition of diverse knowledge and perspectives

Empowerment in various fields recognizes the worth of diverse knowledge and perspectives [ 26 , 28 , 32 ]. By incorporating them, empowerment aims to challenge the conventional power structures that have systemically marginalized some voices and sustained inequality. Moreover, involving individuals with varied experiences offers exceptional insights and understandings that enhance dialogues and contribute to more thorough resolutions [ 28 ]. Similarly, patient experiential knowledge and unique insights are recognized as crucial in PPI for shaping research and complementing the specialist knowledge of clinical researchers [ 4 , 13 ]. According to the Montreal Model, patients’ experiences with illnesses, which they must manage for the rest of their lives if chronically, offer a rich source of knowledge essential for decision-making [ 41 ]. This experiential knowledge includes patients’ insights into their health issues, the trajectory of their care, and the impacts on their personal lives and those of their loved ones [ 41 ]. The involvement of patients strengthens the focus of clinical research on patients’ needs, ultimately enhancing its quality, adequacy, relevance, and credibility [ 3 , 4 ].

Collaborative relationship

Empowerment approaches typically foster collaborative relationships among various stakeholders [ 19 , 28 ]. In social work, these relationships arise between the practitioner and the client and are characterized, analogous to the idea of an alliance, by a “shared sense of urgency” (regarding the client’s problems), a “conjoint commitment to problem solving in as democratic a manner as possible”, and a “shared emphasis [.] on [the] common humanity” in the relationship [ 19 ]. Depending on the PPI approach, the concept of collaborative relationships among various stakeholders can also apply to empowering patients in research. Three involvement approaches in PPI are distinguished [ 6 ]: (1) The consultation approach achieves the lowest level of engagement and collaborative relationships, wherein patients provide advice to researchers but are not involved in decision-making. (2) Patients are partners in the research process in the collaboration approach, with their involvement in decision making and shared responsibility for the research. (3) Patients in user-led research take full responsibility for individual aspects or the whole research, with support from researchers [ 6 ]. User-led research can only be implemented to a limited extent in clinical studies, as it is subject to ethical and legal framework conditions.

To strengthen the principles of social movements in PPI, a collective approach to research, as proposed by MacDonald’s theory of civic patienthood, could provide valuable insights [ 34 ]. This theory views patients as civic actors who seek collective solutions to collective problems, shifting the understanding of patients from merely clinical subjects to engaged participants in shaping research and healthcare outcomes. This approach needs robust institutions, resources, and socialization processes to support patients’ involvement. It is particularly critical in ensuring that PPI remains genuinely democratic and is not co-opted by more powerful interests [ 34 ].

Distinctions of empowerment in PPI in clinical research to earlier concepts

While we found the heritage of social movements to inform the ethos of PPI in the principles of participation, giving people a say in decisions that affect their lives, confronting power structures—albeit on a smaller scale—, and collaborative relationships, we also found distinctions of empowerment in PPI in clinical research to earlier concepts. These seem to be in the areas of context and focus, scientific demands and ethics, expertise in research methods, and power dynamics.

Context and focus

While the goals of empowerment in other fields and PPI share similarities, there are differences in the context and focus. In social movements, empowerment refers to the process through which marginalized individuals and communities obtain power, active participation, and the ability to challenge oppressive systems [ 18 , 29 ]. These movements often aim to effect systemic changes and combat inequalities, drawing upon collective action, raising awareness, and advocacy to achieve their goals [ 29 , 30 ]. In contrast, the context of empowerment in PPI is more specific to the research process. Here, empowerment is about providing patients and the public with a voice in decision-making within that process [ 4 ]. While the influence of social movements is undeniable, the primary objective is not necessarily to address systemic inequalities on a broad scale but to enhance the quality and relevance of research by incorporating diverse perspectives. In PPI, people are empowered or given a voice “to influence research outcomes that will (or may) have a direct impact on their health status“ [ 6 ]. Though not the main objective, this involvement of diverse perspectives in research may nonetheless potentially contribute to a reduction in inequalities [ 42 , 43 ].

However, the practical implementation of PPI often faces challenges that may undermine its empowering potential. Researchers, under pressure to demonstrate measurable impact, tend to focus the conduct of involvement on substantive values such as effectiveness, quality, and validity – outcomes that are more easily quantified and aligned with traditional research goals [ 4 , 14 ]. This focus may lead to the marginalization of crucial but less easily measured normative values like empowerment, rights and accountability and process values such as partnership or respect. The demand for measurable outcomes and recommendations for the conduct of PPI that lead to rather structured and controlled PPI mechanisms shape PPI practices in ways that may suppress rather than amplify the voices of patients [ 14 ]. A more reflexive and dialogic approach to evaluating PPI might better capture its ethical and formative dimensions, ensuring that public involvement in research remains a tool for true empowerment rather than an instrument of containment [ 14 ].

Scientific demands and ethics

Empowerment in clinical research must balance patient empowerment with scientific demands and the integrity of research findings. Empowerment approaches in other fields may concentrate on personal growth and social change. However, in clinical research there is a need to find ways that respect both the methodological and ethical requirements of research and the interests of PPI. This aspect, which is specific to this context, distinguishes it from empowerment in other fields and may restrict the potential for empowerment in clinical research as well as put specific demands on the conduct of research [ 17 , 44 ]. As a result, the level of patient co-determination may be limited. For example, for methodological reasons randomization might be preferable, even if alternative methods are perceived as more appropriate by the patients involved for understandable reasons. Additionally, patients may lack a full understanding of these restrictions, causing them to suggest ideas that do not comply with the logic of scientific protocols. This encounter with limitations during interactions with scientists can potentially diminish their level of empowerment.

In addition to methodological hurdles, PPI must address ethical considerations in the pursuit of empowerment. Although it is generally assumed that patient involvement does not necessitate an ethics vote, it is nonetheless crucial to discuss with potentially involved parties regarding matters such as safeguarding their privacy and potential conflict of interests, and to furnish them with comprehensive information about the involvement’s goals and methodology [ 45 ]. The framing of the involvement, and therefore the empowerment, in this manner distinguishes it from empowerment in other fields.

Expertise in research methods

Empowering patients in research requires providing objective support and resources to enhance their comprehension of research methods and ethics [ 17 ]. Usually, patients need assistance in navigating the complexities of research processes and methodologies [ 17 , 46 ], which distinguishes empowerment in PPI from other fields. However, learning is a common aspect in any kind of empowerment. For instance, Freire’s theory of critical consciousness highlights education’s role in empowering marginalized individuals [ 30 ]. His approach centers on learners directing their own education by posing questions and emphasizes skill development over knowledge acquisition with a focus on increasing critical awareness of their circumstances.

The disparity in PPI may stem from individuals, who desire and deserve empowerment, not being the ones to decide what to learn, but from the fact that this choice is often made for them and is very factual. In terms of preparation for PPI, the learning is mostly unidirectional, whereby the researchers instruct the patients on research fundamentals [ 47 ]. However, there is a mismatch between the perception of training needs between researchers and PPI contributors (i.e. patients), both in terms of training for PPI contributors and researchers. Dudley et al. [ 47 ] found that this discrepancy leads to gaps in the support and training provided. That said, the characterization of unidirectional learning does not apply universally. For example, some PPI initiatives have employed more interactive and participatory training methods, allowing patients to engage more actively in shaping their learning experience [ 48 ].

Providing PPI support and training enables patients to acquire the necessary knowledge and skills to work alongside researchers on an equal basis, and to furnish patients with the confidence they need to challenge researchers opinions when needed [ 49 ]. Importantly, expertise in clinical research methods is not only a means of achieving empowerment but also a crucial component of enhancing the quality and relevance of research. By developing expertise, patients can contribute more meaningfully to the research process, ensuring that their perspectives and experiences are integrated in ways that improve research outcomes.

To strengthen empowerment in PPI and reduce vulnerability to co-optation by more powerful forces with different problem-solving interests, it is critical that participants have a clear understanding of the power they seek to build [ 34 ]. MacDonald’s theory of civic patienthood illustrates that socialization is central to helping patients understand their agency, role, and limitations as civic actors in PPI [ 34 ]. The design of this process can significantly impact how power and empowerment are navigated within PPI.

Power dynamics

Self-determination of the client is an essential aspect of empowerment practice in social work, and it is commonly believed that empowerment cannot be imposed upon anyone else [ 29 ]. In this regard, professionals are responsible for providing support and facilitation and it is crucial to minimize power differentials between all parties involved in order to foster relationships based on equality and partnership [ 29 ].

In research-based PPI, addressing power asymmetries between researchers and patients is critical. Researchers typically operate with institutions that have structures and established norms, facing constraints and pressures imposed by their institutions which can influence the extent of shared-decision making and the balance of power. Often, researchers have the final say in decisions [ 6 ]. These dynamics of institutional power can lead to challenges in achieving equal partnerships.

To navigate these constraints effectively, it is crucial to understand the extent to which patients are involved in the research process, how their roles are negotiated with researchers, and the level of their involvement in decision-making. Researchers must balance their own institutional limitations and the robustness of the research with the need to foster patient empowerment. This process can be challenging and at times frustrating. Promoting patient empowerment in clinical research impacts organizational processes, cultures and public relationships, requiring frameworks that recognize, address and integrate patient perspectives into research activities [ 49 ].

Discussion and implications

The goal of this article is to contribute to the understanding of empowerment in PPI in clinical research by analyzing the history and development of the concept of empowerment in earlier fields. We presented an overview of the history of empowerment in the social movements of the 20th century and outlined key concepts of empowerment from Solomon, Rappaport, and Freire. Based on this, we suggested common principles of empowerment concepts. We then presented an overview of the historical development of PPI in research, that is strongly connected to the social movements’ heritage, and reflected on the relevance of empowerment in PPI. Finally, we assessed in how far empowerment in PPI mirrors the previously developed common principles of empowerment, and analyzed similarities and distinctions.

We found the heritage of social movements to inform the ethos of PPI, as principles such as promoting participation, providing people with a say in decisions that may affect their lives, appreciating diverse knowledge, fostering respectful collaborations, and confronting power structures (even at a smaller, less existential scale) are deeply embedded in PPI practices. However, we also observed considerable distinctions in contexts and objectives: Social movement-based empowerment aimed to effect systemic changes and combat inequalities. Empowerment movements typically arose from significant inequalities and were often initiated by the oppressed. While these movements laid the groundwork for later involvement in research, the empowerment objectives in PPI are more specific to the research context. Today, the involvement process is predominantly initiated by researchers seeking to incorporate patients to increase the quality and relevance of their trials.

In the practical implementation of PPI in clinical research, empowerment may often play only a minor role, irrespective of claims made to the contrary. PPI may offer ample opportunities for fostering patient empowerment, even if the primary goal is to involve patients for the enhancement of research quality or for meeting certain requirements. Nevertheless, even in trials explicitly designed to promote patient empowerment, the level of empowerment may not satisfy each individual involved. We found that these constraints are often related to researchers’ need to adhere to institutional requirements, the duration of PPI involvement, and power imbalances in relation to researchers.

Still, we feel that tentative recommendations are warranted for facilitating empowerment in clinical trials:

Throughout the planning, execution, and dissemination of the study, close collaboration between patients and researchers is crucial. The relationship between patients and researchers should be marked by respect and mutual appreciation [ 4 ]. Both parties should value all perspectives and prioritize inclusivity in decision-making processes. MacDonalds’ model of civic patienthood offers valuable insights for strengthening patients’ voice and the power dynamics in PPI [ 34 ].

As defined by Salomon, the success of empowerment depends on “the extent to which the service delivery system functions as either an obstacle course or an opportunity system” [ 26 ]. In the case of PPI, the study and patient involvement should be designed in such a way that patients fully understand the process and its realistic limitations. It is essential to make the research accessible and transparent, with clear communication about what it can and cannot promise. Acknowledging the limitations of clinical research as a vehicle for empowerment respects patients’ capacity to understand these limitations and helps manage their expectations, fostering a more honest and trustful relationship between researchers and patients [ 16 , 17 ].

Prior to and throughout their collaboration, patients and researchers should engage in discussions about their shared objectives, expectations, and experiences [ 16 ]. These should include notions of empowerment and empowerment should be an aspect that guides the involvement.

To promote collaborative equality, patients may participate in training sessions prior to or at the beginning of their involvement. These sessions should offer a comprehensive understanding of clinical research and enhance their perspective as patients, empowering them to challenge researchers when necessary [ 3 ]. In the spirit of peer support and collective action [ 30 ], patients themselves may offer these training sessions for the benefit of their fellow patients, thereby reducing power imbalances in the learning environment.

Researchers ought to engage in training sessions for PPI [ 50 ], including instructions on how to foster empowerment.

Patients collaborating with researchers should be accompanied and supported as needed by a person who feels responsible and plays a role similar to that of a social worker in other contexts [ 29 ]. Despite time constraints in PPI, there should be opportunities for patients to share and analyze experiences, provide mutual support, and collaborate during the course of the clinical trial [ 18 , 30 ].

At the end of the participation, there should be a closing session where, among other things, the participation is reflected upon and its added value is highlighted [ 17 ]. This includes not only aspects that have changed the quality of the study, but also, for example, changes and developments at the personal level of patients and researchers. Patients who wish to continue their involvement should have opportunities to do so.

This list presents several ways for promoting empowerment within the context of PPI. It is not conclusive but rather intended to be extended and elaborated upon in further examinations of the subject. However, defining empowerment is a complex undertaking, and one may select different criteria or aspects that may lead to alternative approaches to promoting it.

Conclusions

The primary objective of clinical research is not to empower patients but to generate scientific knowledge that can improve healthcare outcomes. However, with the increasing call for involving patients in research, the concept of empowerment has become an associated goal. Our investigation sought to unpack what empowerment might mean within the context of PPI in clinical research.

Given the absence of a consensus on what empowerment in this context entails, we turned to the history and foundational concepts of empowerment from various social movements to illuminate its potential meanings and implications. We found both similarities and differences between empowerment in PPI and earlier empowerment concepts. While PPI reflects principles such as participation, challenging power structures, and valuing diverse perspectives, the empowerment it offers is often constrained by the specific context of clinical research.

Some limitations to empowerment in PPI are intrinsic to the research context itself, such as the need to adhere to rigorous scientific standards. However, other limitations are less evident and may, in fact, undermine the empowerment of patients. These include institutional power dynamics, limited opportunities for genuine decision-making, and inadequate support for patients to navigate the complexities of research processes.

To address these challenges, it is crucial for those involved in PPI to be explicit about what they mean by empowerment and to consider whether and how it is valued in their research endeavors. Transparency regarding both external and internal limitations is essential. This includes an explicit exchange between researchers and patients about the realistic scope and potential of patients’ involvement, as well as ongoing reflection and dialogue about how empowerment can be fostered and sustained within the research process. By doing so, PPI can move closer to fulfilling its promise of genuinely empowering patients, rather than merely using the term as a rhetorical tool.

Data availability

No datasets were generated or analysed during the current study.

Abbreviations

Patient and Public Involvement

Greenhalgh T, Hinton L, Finlay T, Macfarlane A, Fahy N, Clyde B, et al. Frameworks for supporting patient and public involvement in research: systematic review and co-design pilot. Health Expect. 2019;22(4):785–801.

Article   PubMed   PubMed Central   Google Scholar  

Domecq J, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14(1):89.

Ives J, Damery S, Redwod S. PPI, paradoxes and Plato: who’s sailing the ship? J Med Ethics. 2012;39(3):181–5.

Article   PubMed   Google Scholar  

Gradinger F, Britten N, Wyatt K, Froggatt K, Gibson A, Jacoby A, et al. Values associated with public involvement in health and social care research: a narrative review. Health Expectations: Int J Public Participation Health care Health Policy. 2015;18(5):661–75.

Article   Google Scholar  

Boote J, Baird W, Beecroft C. Public involvement at the design stage of primary health research: a narrative review of case examples. Health Policy. 2010;95(1):10–23.

Boote J, Telford R, Cooper C. Consumer involvement in health research: a review and research agenda. Health Policy. 2002;61(2):213–36.

Charlton J. Nothing about us without us: disability oppression and empowerment. Oakland: University of California Press; 1998.

Book   Google Scholar  

Came H, Gifford H, Wilson D. Indigenous public health: nothing about us without us! Public Health. 2019;176:2–3.

Article   CAS   PubMed   Google Scholar  

Baxter L, Thorne L, Mitchell A. Small voices, big noises. Lay involvement in Health Research: lessons from other Fields. Exester: Washington Singer; 2001.

Google Scholar  

Oxford English Dictonary. Oxford: Oxford University Press; 2021. empower, v.

Esmail L, Moore E, Rein A. Evaluating patient and stakeholder engagement in research: moving from theory to practice. J Comp Eff Res. 2015;4(2):133–45.

Rowland P, McMillan S, McGillicuddy P, Richards J. What is the patient perspective in patient engagement programs? Implicit logics and parallels to feminist theories. Health. 2016;21(1):76–92.

Staley K. Exploring Impact: Public Involvement in NHS, Public Health and Social Care Research. National Institute for Health Research (NIHR), INVOLVE, editors. Eastleigh2009 28.10.2009.

Russell J, Fudge N, Greenhalgh T. The impact of public involvement in health research: what are we measuring? Why are we measuring it? Should we stop measuring it? Res Involv Engagem. 2020;6(1):63.

Komporozos-Athanasiou A, Fudge N, Adams M, McKevitt C. Citizen Participation as Political Ritual: towards a sociological theorizing of ‘Health Citizenship’. Sociology. 2016;52(4):744–61.

Schilling I, Behrens H, Hugenschmidt C, Liedtke J, Schmiemann G, Gerhardus A. Patient involvement in clinical trials: motivation and expectations differ between patients and researchers involved in a trial on urinary tract infections. Res Involv Engagem. 2019;5(15).

Schilling I, Behrens H, Bleidorn J, Gagyor I, Hugenschmidt C, Jilani H, et al. Patients’ and researchers’ experiences with a patient board for a clinical trial on urinary tract infections. Res Involv Engagem. 2019;5:38.

Herriger N, Spurensuche. Eine Kurze Geschichte Des Empowerment-Konzeptes. In: Herriger N, editor. Empowerment in Der Sozialen Arbeit. Stuttgart: Kohlhammer; 2020. pp. 22–56.

Chapter   Google Scholar  

Simon B. The empowerment tradition in American Social Work: a history. New York: Columbia University; 1994.

Weisbrot R. Freedom bound: a history of America’s civil rights movement. New York: Plume; 1991.

West G, Blumberg RL, editors. Women and social protest. Oxford: Oxford University Press; 1990.

Beck-Gernsheim E. Vom „Dasein für andere zum anspruch auf ein Stück „eigenes Leben: Individualisierungsprozesse Im Weiblichen Lebenszusammenhang. In: Wilz SM, editor. Geschlechterdifferenzen - Geschlechterdifferenzierungen. Volume 1. Wiesbaden: VS Verlag für Sozialwissenschhaften; 2008. pp. 19–62.

Schutz A, Miller M, editors. People Power. The Community Organizing tradition of Saul Alinsky. Nashville: Vanderbilt University Press; 2015.

Pankofer S. Empowerment - Eine Einführung. In: Tilly M, Sabine P, editors. Empowerment konkret. Berlin, Boston: De Gruyter Oldenbourg; 2016. pp. 7–22.

Solomon B, Empowerment. Social work in oppressed communities. J Social Work Pract. 1987;2(4):79–91.

Solomon B. Black empowerment: social work in oppressed communities. New York: Columbia University; 1976.

Blank B. Empowerment - Ein Leitkonzept Der Sozialen Arbeit in Der Migrationsgesellschaft? In: Blank B, Gögercin S, Sauer KE, Schramkowski B, editors. Soziale Arbeit in Der Migrationsgesellschaft: Grundlagen – Konzepte – Handlungsfelder. Wiesbaden: Springer Fachmedien Wiesbaden; 2018. pp. 327–40.

Rappaport J. In praise of paradox: a social policy of empowerment over prevention. Am J Community Psychol. 1981;9(1):1–25.

Boehm A, Staples L, Empowerment. The point of View of consumers. Families Society: J Contemp Social Serv. 2004;85:270–80.

Freire P. Education for critical consciousness. 2013 ed. London/New York: Bloomsbury Academic; 1974.

Cattaneo LB, Chapman AR. The process of empowerment: a model for use in research and practice. Am Psychol. 2010;65(7):646–59.

Herriger N. Begriffliche Annäherungen: Vier Zugänge zu Einer Definition Von empowerment. In: Herriger N, editor. Empowerment in Der Sozialen Arbeit. Stuttgart: Kohlhammer; 2020. pp. 13–21.

Thornton H. Patient and public involvement in clinical trials. BMJ: Br Med J. 2008;336(7650):903–4.

Macdonald GG. Civic patienthood: a critical grounded theory of how patients transform from clinical subjects to civic actors. University of British Columbia; 2023.

Liberati A. Consumer participation in research and health care. BMJ. 1997;315(7107):499.

Article   CAS   PubMed   PubMed Central   Google Scholar  

INVOLVE, Resources. 2019 [ https://www.invo.org.uk/resource-centre/

Frank L, Basch E, Selby JV. The PCORI perspective on patient-centered outcomes research. JAMA. 2014;312(15):1513–4.

Canadian Institutes of Health Research (CIHR). Strategy for Patient-Oriented Research 2023 [ https://www.cihr-irsc.gc.ca/e/41204.html

Schilling I, Herbon C, Jilani H, Rathjen KI, Gerhardus A. Aktive Beteiligung von Patient*innen an klinischer Forschung – Eine Einführung. Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen. 2020:56–63.

Engler J, Kuschick D, Tillmann J, Kretzschmann C, Wallacher S, Kersting C, et al. [Patient and Public Involvement in Family Medicine Research]. ZFA (Stuttgart). 2022;98(5):178–83.

PubMed   Google Scholar  

Pomey M-P, Flora L, Karazivan P, Dumez V, Lebel P, Vanier M-C, et al. The Montreal model: the challenges of a partnership relationship between patients and healthcare professionals. Santé publique (Vandoeuvre-lès-Nancy. France). 2015;27:S41–50.

NHS England. Working in partnership with people and communities: Statutory guidance. 2023. https://www.england.nhs.uk/long-read/working-in-partnership-with-people-and-communities-statutory-guidance/

INVOLVE. Diversity and inclusion: what’s it about and why is it important for public involvement in research? Eastleigh: INVOLVE; 2012.

Staniszewska S, Jones N, Newburn M, Marshall S. User involvement in the development of a research bid: barriers, enablers and impacts1. Health Expect. 2007;10(2):173–83.

Jilani H, Rathjen KI, Schilling I, Herbon CM, Scharpenberg M, Brannath W, et al. Handreichung Zur Patient*innenbeteiligung an klinischer Forschung. Bremen: Universität Bremen; 2020.

Forsythe LP, Ellis LE, Edmundson L, Sabharwal R, Rein A, Konopka K, et al. Patient and Stakeholder Engagement in the PCORI Pilot projects: description and lessons learned. J Gen Intern Med. 2015;31(1):13–21.

Dudley L, Gamble C, Allam A, Bell P, Buck D, Goodare H, et al. A little more conversation please? Qualitative study of researchers’ and patients’ interview accounts of training for patient and public involvement in clinical trials. Trials. 2015;16(1):190.

Clausen J. Partizipative Forschung in Der Deutschen Rheuma-Liga — Inhaltliche und praktische Umsetzung Der Partizipativen Forschung in Einer Patientenorganisation. Zeitschrift für Evidenz Fortbildung Und Qualität Im Gesundheitswesen. 2020;155:64–70.

Hickey G, Brearley S, Coldham T, Denegri S, Green G, Staniszewska S, et al. Guidance on co-producing a research project. Southampton: INVOLVE; 2018.

Hickey G, BMC. On Medicine [Internet]. BMC:. 2018. https://blogs.biomedcentral.com/on-medicine/2018/12/07/global-patient-public-involvement-network-vision-mission/

Download references

Acknowledgements

Not applicable.

Imke Schilling was awarded a post-doctoral scholarship by the University of Bremen.

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

Department for Health Services Research, Institute of Public Health and Nursing Research, University of Bremen, Grazer Straße 4, 28359, Bremen, Germany

Imke Schilling & Ansgar Gerhardus

Health Sciences Bremen, University of Bremen, 28359, Bremen, Germany

You can also search for this author in PubMed   Google Scholar

Contributions

Imke Schilling: Conceptualization, Methodology, Investigation, Resources, Data Curation, Writing – Original Draft, Writing – Review and Editing, Project administration, Funding acquisition, Ansgar Gerhardus: Conceptualization, Methodology, Writing – Review and Editing, Supervision, Funding acquisition.

Corresponding author

Correspondence to Imke Schilling .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Declaration of Generative AI and AI- assisted technologies in the writing process

During the preparation of this work the authors, as non-native speakers, used Deepl Write and ChatGPT in order to eliminate grammatical or spelling errors and to conform to correct scientific English within the article. After using these tools, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Schilling, I., Gerhardus, A. Is this really Empowerment? Enhancing our understanding of empowerment in patient and public involvement within clinical research. BMC Med Res Methodol 24 , 205 (2024). https://doi.org/10.1186/s12874-024-02323-1

Download citation

Received : 11 December 2023

Accepted : 27 August 2024

Published : 13 September 2024

DOI : https://doi.org/10.1186/s12874-024-02323-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Patient and public involvement
• Empowerment
• Clinical Research

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

• Open access
• Published: 17 April 2017

Understanding relevance of health research: considerations in the context of research impact assessment

• Mark J. Dobrow 1 ,
• Fiona A. Miller 1 ,
• Cy Frank 2 &
• Adalsteinn D. Brown 1  

Health Research Policy and Systems volume  15 , Article number:  31 ( 2017 ) Cite this article

15k Accesses

25 Citations

9 Altmetric

Metrics details

With massive investment in health-related research, above and beyond investments in the management and delivery of healthcare and public health services, there has been increasing focus on the impact of health research to explore and explain the consequences of these investments and inform strategic planning. Relevance is reflected by increased attention to the usability and impact of health research, with research funders increasingly engaging in relevance assessment as an input to decision processes. Yet, it is unclear whether relevance is a synonym for or predictor of impact, a necessary condition or stage in achieving it, or a distinct aim of the research enterprise. The main aim of this paper is to improve our understanding of research relevance, with specific objectives to (1) unpack research relevance from both theoretical and practical perspectives, and (2) outline key considerations for its assessment.

Our approach involved the scholarly strategy of review and reflection. We prepared a draft paper based on an exploratory review of literature from various fields, and gained from detailed and insightful analysis and critique at a roundtable discussion with a group of key health research stakeholders. We also solicited review and feedback from a small sample of expert reviewers.

Conclusions

Research relevance seems increasingly important in justifying research investments and guiding strategic research planning. However, consideration of relevance has been largely tacit in the health research community, often depending on unexplained interpretations of value, fit and potential for impact. While research relevance seems a necessary condition for impact – a process or component of efforts to make rigorous research usable – ultimately, relevance stands apart from research impact. Careful and explicit consideration of research relevance is vital to gauge the overall value and impact of a wide range of individual and collective research efforts and investments. To improve understanding, this paper outlines four key considerations, including how research relevance assessments (1) orientate to, capture and compare research versus non-research sources, (2) consider both instrumental versus non-instrumental uses of research, (3) accommodate dynamic temporal-shifting perspectives on research, and (4) align with an intersubjective understanding of relevance.

Peer Review reports

Various levels of government in Canada collectively invest multiple billions of dollars in health-related research per annum, above and beyond investments in the management and delivery of healthcare and public health services. In recognition of this sizeable collective commitment, much work has focused on the impact of health research to explore and explain the consequences of these investments and inform strategic planning. Relevance is tacit in the increased attention to the usability and impact of health research. Additionally, research funders increasingly engage in relevance assessment as an input to decision processes; yet, it is unclear whether relevance is a synonym for or predictor of impact, a necessary condition or stage in achieving it, or a distinct aim of the research enterprise. Therefore, the main aim of this paper is to improve our understanding of research relevance as it relates to research quality and research impact, with specific objectives to (1) unpack research relevance from both theoretical and practical perspectives, and (2) outline key considerations for the assessment of research relevance.

Globally, there has been increasing critical assessment of the value of health research investments [ 1 – 3 ], with growing interest in research impact assessment (RIA) in the health sector [ 4 – 6 ]. RIA focuses on understanding how research activity can directly and indirectly advance knowledge, influence decision-making, and effect health and socio-economic outcomes, with a small but growing body of work seeking to develop better measures to evaluate (and ideally attribute) the returns on health research investments [ 6 ]. The Canadian Academy of Health Sciences (CAHS) released a comprehensive report on the subject in 2009 that presented a call for action, with a number of recommendations including establishing collaborative efforts among Canadian research funders to advance frameworks and sets of indicators and metrics for health research impact [ 4 ]. The CAHS impact framework [ 4 ], which drew on the Buxton and Hanney [ 7 ] ‘payback model’, among others, has provided a thoughtful starting point for considering the impact of health research in Canada. Subsequent work by Alberta Innovates – Health Solutions (AIHS) on a Research to Impact Framework (described in Graham et al. [ 8 ]) provides further insights on operationalising RIA frameworks for health research in Canada.

These initiatives are part of a broadly discussed shift in approaches to knowledge production, from an emphasis on investigator-initiated, curiosity driven work judged and guided by scientists, to expanded approaches to knowledge production, drawing on a wider set of actors and approaches, and emphasising relevance and usability. This shift from science produced by and for scientists to knowledge production that is “ socially distributed, application-oriented, trans-disciplinary, and subject to multiple accountabilities ” [ 9 ] has been characterised as a shift from ‘mode 1’ to ‘mode 2’ knowledge regimes. In the language of mode 2, interest in research ‘impact’ expresses a concern for application or consequence, and – in the economic language of return on investment – a concern that the yield is at least equal to the investment in the research itself. Extending this reasoning, interest in research ‘relevance’ may reflect a concern for accountability – linking research to the actor(s) for whom the research is performed and who will, ideally, put it to use.

In Canada, interest in research impact and relevance appears to have been felt most forcefully in the context of health services and policy research, which has long been encouraged to orient to the needs of policymakers, health system planners and related decision makers. More recently, there has been increased attention to ensuring that all forms of health research are ‘patient oriented’ – that is, that the research is prioritised, conducted and applied in ways that are accountable to this important end user. This call has been picked up on several fronts, including by the Canadian Institutes of Health Research (CIHR), which released its Strategy for Patient-Oriented Research (SPOR) in 2011. The SPOR vision “…is to demonstrably improve health outcomes and enhance patients’ health care experience through integration of evidence at all levels in the health care system ” [ 10 ]. In some respects, it represents a fundamental re-orientation for the primary funder of health research in Canada.

Though relevance is tacit in attention to research impact and the wider concern with mode 2 knowledge production, explicit attention to the meaning or measurement of research relevance is limited. The CAHS and AIHS frameworks, for example, acknowledge ‘relevance’ of health research but do not clearly define the term nor describe approaches for assessing it [ 4 , 8 ]. Rather, these frameworks emphasise the role of broad stakeholder engagement approaches and feedback mechanisms as methods for addressing relevance. For example, the AIHS framework notes the challenge of, and need to, move “ …beyond the collection of traditional scientific indicators […] to include measures of greater interest to the broader stakeholder community… ” [ 8 ] without stating explicitly how “ greater interest ” or related concepts such as relevance should be judged. As currently constructed, these RIA frameworks provide important advances in how we think about the impact of health research, but they were not intended to provide guidance specifically to the assessment of the relevance of health research.

Despite this lack of specific guidance on research relevance from a scholarly or measurement perspective, attention to it as a practical component of health research funding and organisation is evolving. There is, for example, growing use of ‘relevance assessment’ by research funders. The Canadian Health Services Research Foundation, in particular, was an innovator in incorporating relevance review into its applied research funding programmes, including promoting partnerships and knowledge translation (KT) with health system stakeholders [ 11 ]. Current applications for funding from the Institute of Gender and Health at CIHR go through ‘relevance review’ ( http://www.cihr-irsc.gc.ca/e/45212.html ). Similarly, applications for Ontario’s Health System Research Fund are judged based on ‘internal review of relevance and impact’ ( http://www.health.gov.on.ca/en/pro/ministry/research/cihr.aspx ). However, given the lack of conceptual clarity on research relevance, and in particular, how relevance assessment aligns with and differs from impact assessment, there is a critical gap in our understanding that has implications for both its contemporary and ongoing application and our ability to make sound research investment decisions.

This work was commissioned by the Ontario SPOR SUPPORT (Support for People and Patient-Oriented Research and Trials) Unit (OSSU) – one of several units established at provincial and regional levels across Canada to work with CIHR in pursuing the SPOR. Like other research organisations, OSSU saw the need to consider the relevance of the research it supported, and it established both scientific and relevance advisory committees as part of its original governance structure [ 12 ], tasking the latter to “ …develop a measure, or small set of strategic measures, that serves to inspire the Ontario research, implementation, provider and patient communities to come together to make a difference for patients ” [ 12 ]. In the spirit of research and scholarship, OSSU then asked what exactly this commitment to research ‘relevance’ entailed.

Our approach to answering this question involved the scholarly strategy of review and reflection. As with the early investigations into research impact assessment, we were surprised to find so little reflexive attention to the topic within the health research community [ 13 ]. We prepared a draft paper based on an exploratory review of literature from various fields, and gained from detailed and insightful analysis and critique at a roundtable discussion with a small group of key health research stakeholders. We also solicited review and feedback from a small sample of expert reviewers.

The structure of our paper is as follows. First, to ‘unpack’ the concept of relevance, we review theoretical literature and then consider practical work both from within and outside the health sector, to ask what has been argued and concluded about the nature of relevance and its appropriate assessment. Next, we outline a series of forward-looking considerations for assessing research relevance and conclude with reflections on how research relevance assessment fits with evolving interest in RIA.

Unpacking relevance

Theoretical perspectives.

Before considering the relevance of health research, we need to step back and consider what we mean by the term ‘relevance’. A range of descriptors is often used to define relevance, including ‘pertinent to…’, ‘bearing upon…’, ‘connected with…’, or ‘appropriate to…’, ‘…the matter at hand’, as well as ‘germane’, ‘apropos’, ‘material’, ‘applicable’ and ‘satisfactory’. A large body of dedicated theoretical work on relevance, drawn from many fields and perspectives, such as computer science, information science, statistics/probability theory, artificial intelligence, cognitive science, epistemology, linguistics and jurisprudence [ 14 ], reflects its importance but also the challenge for establishing a common understanding of the term [ 14 , 15 ]. For example, Gärdenfors [ 16 ], in his discussion on the logic of relevance, noted that “ …relevance ought to be a central concept in the philosophy of science… ” given the position that “ …it is only relevant information that is of any importance… ” (p. 351). However, from a ‘research’ relevance perspective, the theoretical work on relevance has been linked to ‘information’, ‘evidence’, ‘reasoning’, ‘argument’ and ‘decision’ [ 15 – 18 ], each presenting variable framing that impedes practical definition or consistent comprehension of the term. Floridi [ 14 ] recently suggested that existing theories are “ …utterly useless when it comes to establish the actual relevance of some specific piece of information ” (p. 69), and goes on to advance a ‘subjectivist’ interpretation, with relevance judged by the questioner. While a subjectivist approach to relevance is intuitively appealing, its contribution to the assessment of research relevance presents particular challenges that we will discuss later in the paper.

Another approach to unpacking relevance is to consider the theoretical model behind the broad-based research strategies that have governed research investments and policies in high-income countries since the end of the Second World War. For the better part of the 20th century, a linear model was the dominant conceptual framework, whereby basic research was viewed as a necessary input for applied research, which then led to development and production [ 19 , 20 ]. In the late 1990s, an alternate thesis was introduced when Stokes proposed a new model for broad-based research strategy – known as Pasteur’s Quadrant – that highlighted the conceptual relationship between the ‘quest to understand’ and ‘practical needs’ [ 21 ]. While some research is clearly focused on advances in basic research (e.g. Niels Bohr’s foundational research on atomic structure and quantum theory), and some research is clearly focused on applied problems (e.g. Thomas Edison’s practical inventions), Stokes emphasised the potential for use-inspired basic research (e.g. Louis Pasteur’s foundational research on microbiology that addressed contemporaneous population health challenges). Pasteur’s Quadrant invokes consideration of ‘relevance’ with some commentators framing the two-by-two relationship as the relevance for advancement of basic knowledge and the relevance for immediate application [ 22 ]. Stokes’ model adds conceptual insight on the role of relevance when considering the value of research to society, however, it was not intended to specifically conceptualise the term and does not distinguish it from other related concepts such as research impact or value. Therefore, to provide further insights, we next consider relevance in practical settings.

Health sector perspectives

In the health sector, the idea that research should be ‘relevant’ is commonplace. Commitments to ‘knowledge translation’ and the ‘knowledge to action cycle’ [ 23 ] emphasise issues of relevance and provide considerable insight into approaches to ensuring research usability and use. At the same time, the health research community has given disproportionate attention to issues of research quality, with an emphasis on internal validity that may downplay external validity and suggest some tension between rigour and relevance. Thus, though the concept of relevance is of central importance to the health research enterprise, the failure to unpack it or explore it both theoretically and practically leaves room for misunderstanding and misapplication.

In the health sector, research relevance often arises as a practical question of the ‘fit’ between a body of knowledge or research approach and a specific field or issue (e.g. public health, primary healthcare, healthcare access, genomics, alternative healthcare, healthcare reform in rural areas). The results of two recent International Society for Pharmacoeconomics and Outcomes Research task forces take this approach. The task forces developed questionnaires to assess the relevance and credibility of research other than randomised controlled trials (e.g. observational research, meta-network analysis) to inform healthcare decision-making [ 24 , 25 ]. Both make similar observations about relevance, reinforcing the subjectivist approach noted earlier, and can be summarised by the following statement by Berger et al. [ 24 ]:

“ Relevance addresses whether the results of the study/apply [sic] to the setting of interest to the decision maker. It addresses issues of external validity similar to the population, interventions, comparators, outcomes, and setting framework from evidence based medicine. There is no correct answer for relevance. Relevance is determined by each decision maker, and the relevance assessment determined by one decision maker will not necessarily apply to other decision makers. Individual studies may be designed with the perspective of particular decision makers in mind (e.g. payer or provider) ” (p.148, emphasis added).

Research relevance in health is also noted in discussion and debate regarding the value of qualitative research relative to the more established forms of quantitative health research. For example, Mays and Pope [ 26 ] suggest that qualitative research can be assessed “… by two broad criteria: validity and relevance ”. Their further discussion provides some insight into the several ways that research might be relevant, suggesting that:

“[r] esearch can be relevant when it either adds to knowledge or increases the confidence with which existing knowledge is regarded. Another important dimension of relevance is the extent to which findings can be generalised beyond the setting in which they were generated ” [ 27 ].

The work of Mays and Pope positions research relevance amidst the longstanding tension between internal and external validity. This tension reflects opposing foci on internal validity as the quality/rigour of research methodology and external validity as the applicability/transferability of research to other settings or contexts. While external validity is not the only measure of relevance – as research may remain relevant to some contexts even when not generalisable to others – it is an important component, and one that has not always attracted sufficient attention. For example, the Canadian health research community has focused considerable practical attention on internal validity as a critical component of evidence for clinical and health policy decisions. Evidence-based medicine, the Cochrane Collaboration, the Canadian and United States task forces on preventive healthcare/services and a long list of aligned groups have developed and established many tools to assess the quality of research evidence (e.g. GRADE [ 28 ]), with a predominant focus on issues of internal validity, and an emphasis on evidence hierarchies that is sometimes seen to be incompatible with ‘real world’ relevance. The relative lack of similar approaches or tools that focus on external validity in health research is notable, though movements to marshal evidence in support of sound public policy, such as the Campbell Collaboration, have attended to issues of external validity in other areas of health and social policy [ 29 ]. Further, there are emerging approaches and tools for documenting the external validity of health research and facilitating its use [ 30 ]. For example, WHO has supported the development of workbooks to contextualise health systems guidance for different contexts [ 31 ] and the field of local applicability and transferability of research has emerged to facilitate the adaptation of interventions from one setting to another, including the development of some well-documented tools like RE-AIM [ 32 ].

Alongside these emerging approaches and tools sits the established field of KT. KT has a strong history in Canada with a distinctive feature being a reliance on stakeholder engagement to support a commitment to improve research relevance. For example, the AIHS framework relies heavily on KT and stakeholder engagement approaches as part of its RIA, describing the mobilisation of knowledge through “ …a process of interactions, feedback, and engagement using a variety of mechanisms (e.g. collaborations, partnerships, networks, knowledge brokering) with relevant target audiences (i.e. actors and performers) across the health sector ” ([ 8 ] p. 362). Experience in stakeholder engagement, particularly with clinical, management and policy decision-makers, has become fairly extensive and there is now increased attention on engaging patients as core stakeholders in health research. If relevance is truly subjective, then KT efforts (including engagement, dissemination, promotion, communication) would appear to represent reasonable approaches for articulating, conveying and improving research relevance. However, if there are underlying elements of relevance that are more universal, then there is a risk that KT efforts – and subjectivist approaches to ensuring relevance – are akin to commercial marketing or communication strategies where the aim is to ‘sell’ more product and/or generate more influence that may not align with a more objective lens.

In sum, the health research community in Canada has a longstanding history of critically appraising research quality based on study design and research methodology, with greater emphasis on internal rather than external validity. As the same time, there is established expertise in KT, emphasising engagement with research users and adaptation to settings or contexts of use – approaches that may imply a subjectivist interpretation of relevance. Thus, while relevance is an important concept for the health research enterprise, its use is largely tacit and taken for granted.

Non-health sector perspectives

To unpack relevance further we consider some non-health sector perspectives that give attention to the term, often with formal definitions or taxonomies established. Examples include the legal, financial accounting, education and web search (information retrieval) sectors, each of which are briefly described below.

From a legal perspective, relevance has a specific meaning that relates to the admissibility of evidence in terms of its probative value (i.e. the extent to which evidence contributes to proving an important matter of fact) [ 33 ]. For example, a common objection to legal testimony or evidence is that it is ‘irrelevant’ [ 34 ]. Legal processes for considering the admissibility or legal-relevance of evidence are firmly established, requiring explicit declaration of evidentiary sources and direct consideration of that evidence as it relates both to a specific case and related historical precedents, something that is undeveloped in the health sector [ 35 ]. It is the formality, explicitness and retrospective nature of this process, which is directly associated with a specific case (or decision), that is characteristic of the consideration of relevance in the legal context.

Financial accounting provides another perspective on relevance. In this field, relevance is viewed as a fundamental component of generally accepted accounting principles. Relevance and materiality are emphasised such that accountants and auditors focus on financial information that meets the decision-making needs of users and is expected to affect their decisions. In financial accounting, ‘value relevance’ provides a more focused perspective on relevance, defined as “ …the ability of information disclosed by financial statements to capture and summarise firm value. Value relevance can be measured through the statistical relations between information presented by financial statements and stock market values or returns ” [ 36 ]. Similar to the legal perspective, the financial accounting perspective on relevance is set with a formal context, where the focal point (i.e. financial performance) is clear and principles (i.e. generally accepted accounting principles) and processes (i.e. financial reporting and auditing) are clearly established and monitored.

Education provides a slightly more expansive approach to operationalising relevance, given the more general aim of the enterprise. In the United States, the Glossary of Education Reform [ 37 ] notes that “ …the term relevance typically refers to learning experiences that are either directly applicable to the personal aspirations, interests, or cultural experiences of students (personal relevance) or that are connected in some way to real-world issues, problems, and contexts (life relevance) ”. They further state that “ personal relevance occurs when learning is connected to an individual student’s interests, aspirations, and life experiences ”, while “ life relevance occurs when learning is connected in some way to real-world issues, problems, and contexts outside of school ”. A similar framing of relevance in this context suggests that it “…extends the learning beyond the classroom by teaching students to apply what they are learning to real world situations ” [ 38 ]. While the education sector also makes numerous references to a ‘rigour and relevance’ dyad [ 39 ] in contrast to the dominance of the internal validity focus in healthcare, it is the prominent dual focus on ‘personal’ relevance (with its subjectivist orientation) and ‘life’ or ‘real world’ relevance (with its more universal orientation) that seems to most clearly define the education sector’s perspective on relevance.

One of the most intensive and competitive sectors focusing on relevance is the web search (or information retrieval) field. This includes dominant search engines such as Google and Bing, as well as a wide range of commercial and social media sites such as Amazon, eBay, Facebook and LinkedIn, that compete either directly or indirectly on their ability to identify relevant information in response to user queries. Therefore, the ability of these organisations to advance the theory and practice related to relevance is fundamental to their success. For example, Google was built upon the effectiveness of its search algorithm, which is in a constant state of evolution. Both explicit and implicit approaches to assess relevance are used to contribute to search algorithm refinements [ 40 ]. The explicit approach focuses on ‘relevance ratings’, whereby evaluators (e.g. human raters) are contracted to assess the degree of ‘helpfulness’ of search results paired to specific search queries [ 41 ]. The implicit approach to assess relevance monitors and aggregates search behaviour of millions of users who are likely unaware that their behaviour is being assessed. Google has more recently advanced ‘personalised relevance’, which uses past individual search behaviour to personalise/tailor future search results for the same individual. Pariser has critiqued this concept as “ the filter bubble ” [ 42 ], warning that Google’s intent to optimise search algorithms for personal relevance creates a “ …personal ecosystem of information… ” that limits the diversity of search results and promotes insularity. This personal relevance is situated within the pervasiveness of social media, which facilitates the advancement of ‘social relevance’. Personal and social relevance highlight two important orientations towards relevance – one built on increasingly detailed understanding of individual preferences and the other reflecting the growing power and increasing accessibility of crowd-sourced perspectives. Overall, web search has made important contributions to how we understand and operationalise relevance, including the use of increasingly sophisticated explicit and implicit feedback mechanisms and the ability to draw upon and analyse big data sets. Web search has also exposed the contrasting orientations of personal and social relevance that underscore the challenges of combining or integrating different relevance assessments.

These non-health sector perspectives on relevance highlight several considerations. First, they reinforce general findings that point to perspective, decision context, timeliness and precision of focus or ‘fit’ as key elements of relevance. Additionally, they highlight a few distinctive considerations. The formalistic contexts of financial accounting and law emphasise issues such as precedent and legitimacy, implying that relevance in a research sense might require the demonstration of some legitimate or credible association between research and its use or user, among other considerations. Further, the complex consumerist world of social media highlights some of the challenges of a purely subjectivist definition of relevance. Whereas the International Society for Pharmacoeconomics and Outcomes Research guidance takes a subjectivist stance in suggesting that, “[t] here is no correct answer for relevance ” [ 24 ], the “ filter bubble ” criticised by Pariser [ 42 ] suggests otherwise. Relevance solely to the personally-perceived interests of a research user is unlikely to adequately serve the collective commitments to health and health equity that are especially germane to the health research enterprise.

Forward-looking considerations for assessing the relevance of health research

To this point, we have endeavoured to unpack relevance from theoretical and practical perspectives. In light of these insights and in the context of persistent interest in research impact assessment and evolving interest in research relevance, we now turn to some specific forward-looking considerations for research relevance assessment (RRA).

Relevance of research versus everything else

The first consideration for RRA is the acknowledgement that research is only one of many sources of insight to inform the needs or actions of research users. A research user is influenced by a wide range of political, legal, media, economic and other contextual information, interactions and experiences, as well as prevailing organisational governance, leadership, culture and values that all serve to complement (and often dominate) any insights that might be derived from research [ 43 ]. This reality implies that ‘relevance’ has a different meaning for researchers and research users. Researchers are typically interested in the relevance of a specific research product or activity for identifiable actions of (potentially) multiple research users; relevance is here judged relative to both the perceived needs of research users, and the extent and content of other related research. In contrast, research users are typically focused on identifying multiple relevant inputs to guide a specific action, only some of which may be research; relevance is here judged relative to both the research user’s needs and the form and content of the other inputs.

Given these distinct orientations to research relevance, RRA needs to be explicit about its comparative lens. Clear distinctions should be made between relevance based on the merits of the research product or activity (researcher lens) and relevance based on the relative value of research compared to other research and non-research sources (research user lens). RRA provides an opportunity to build more robust ways to characterise and assess the contribution of research to research users, including a more systematic and transparent articulation of anticipated research uses (akin to the Research Councils UK’s ‘Pathways to Impact’ [ 44 ] or descriptions of planned study design and methodological approach published in study protocols/registrations for randomised controlled trials or systematic reviews).

Beyond instrumental uses of research

The considerations noted above rely heavily on instrumental uses of research. Theoretically derived definitions of relevance, such as Floridi’s [ 14 ], tend to focus on the response to a specified question. This suggests a direct and tangible connection between research and its ‘use’. However, as Weiss [ 43 ] and others have observed, most types of research use are not instrumental, where use is documented and explicitly addresses a specific query or challenge for a research user. Rather, research use tends to be more conceptual, where use is indirect and evolves over time, or symbolic, where use may be politically or tactically motivated [ 43 ]. Research may also create externalities or unintended effects. For example, general research activity might support an engaged learning environment, interactive research relationships, and additional research-related discourse that provides benefits that are not attributable to any specific research product or activity. This has important ramifications for how research is funded and the role that relevance can play in that assessment. Ultimately, RRA needs to go beyond a singular understanding of research use as instrumental use, to develop better methods for capturing and assessing the relevance of the many non-instrumental uses of research.

The temporal factor

Another closely related consideration for RRA is the temporal context. Almost all research is conducted in a temporally defined period. Yet, while the quality of research is typically characterised by its methodology, which is a static feature typically not subject to temporal variation (e.g. the assessed quality of a randomised controlled trial should be consistent over time), relevance of research can be considered at any time (e.g. prior to the initiation of a research study or at different points in time post-completion) and is therefore subject to dynamic perceptions as they pertain to evolving action or decision contexts. Cohen [ 15 ] suggests that “ …relevance, like reasoning, has a prospective dimension as well as a retrospective one. It helps prediction as well as explanation ” (p. 182). The important insight is that, in contrast to research quality, the relevance of a specific research product can change over time, making assessment of research relevance more challenging.

This requires RRA to acknowledge the temporal factor and its associated implications for research relevance. At minimum, RRA should specify the temporal context as either pre-research (e.g. proposal/funding stage) or post-research (e.g. after research results have been produced). RRA at the pre-research stage focuses on proposed inputs and hypothetical outputs and outcomes, and may be more likely to overestimate instrumental research use and underestimate non-instrumental use. RRA at the post-research stage focuses mainly on the importance and value of actual outputs and tangible results, and may capture more non-instrumental research use. The pre-research stage is clearly aligned with research funding/investment processes, while the post-research stage can contribute to retrospective return-on-investment calculations and more general research impact assessment. However, employing this simple temporal categorisation should not lead us to lose sight of the dynamic, iterative nature of research relevance and the opportunity to assess it at interim and ongoing stages that captures re-interpretations or re-applications of research findings over time.

Moving from a subjective to an intersubjective understanding of relevance

An underlying theme in our review of relevance is subjectivity. Consider the broad scientific paradigms of positivism and interpretivism that are typically respectively aligned with research quality and research relevance. Research quality can be viewed as relating to characteristics or features that are assessed objectively, while research relevance may be seen as subjectively adjudicated. The subjective focus emphasises the variability of different perspectives and contexts and the suggestion that anyone can have a different take on the relevance of a specific research product or activity. For RRA, this reinforces a user-centred orientation to relevance assessment that privileges the judgment of the interrogator and raises the key question regarding who is positioned as the main arbiter of research relevance.

However, while relevance may never be characterised as universal, it could be argued that it is not purely subjective either. Rather, relevance may be more consistent with an intersubjective understanding that emphasises the extent of agreement or shared understanding among individual subjective perspectives representing a way to bridge the personal and the universal. The intersubjective view, while not presenting an objective approach to measuring relevance, does provide a road towards a meaningful and structured assessment of research relevance. It also emphasises the importance of representation in forging the intersubjective judgments that guide the research enterprise.

This paper has unpacked research relevance from different perspectives and outlined key considerations for its assessment. Alongside research impact assessment, research relevance seems increasingly important in justifying research investments and guiding strategic research planning. Indeed, judgments of ‘relevance’ are becoming a key component of the health research enterprise. However, consideration of relevance has been largely tacit in the health research community, often depending on unexplained interpretations of value, fit and potential for impact. Reviewing the various uses of relevance in health research, the concept is sometimes used as a synonym for research impact or positioned as a reliable predictor of later consequence. In many ways, research relevance seems a necessary condition for impact – a process or component of efforts to make rigorous research usable. However, relevance is not a necessary or sufficient condition to achieve impact. We expect that research that is relevant, and thus accountable to specific and legitimate users, will be impactful, but this may not necessarily be the case where other factors intervene. Additionally, we may expect that research that is impactful will be appropriately accountable – but again, this is not necessarily the case. Ultimately, relevance stands apart from research impact. Like rigour, relevance is a complementary but distinctive dimension of what it is that ensures ‘the good’ in health research.

While ‘relevance’ is ever-present, understanding of the concept in terms of health research is emergent and not well codified. To improve our understanding, this paper outlines four key considerations, including how research relevance assessments (1) orientate to, capture and compare research versus non-research sources, (2) consider both instrumental versus non-instrumental uses of research, (3) accommodate dynamic temporal-shifting perspectives on research, and (4) align with an intersubjective understanding of relevance. We believe careful and explicit consideration of research relevance, guided by transparent principles and processes is vital to gauge the overall value and impact of a wide range of individual and collective research efforts and investments. We hope this paper generates more discussion and debate to facilitate progress.

Abbreviations

Alberta Innovates – Health Solutions

Canadian Academy of Health Sciences

Canadian Health Services Research Foundation

Canadian Institutes of Health Research

knowledge translation

Ontario SPOR SUPPORT Unit

research impact assessment

research relevance assessment

Strategy for Patient-Oriented Research

Kleinert S, Horton R. How should medical science change? Lancet. 2014;383(9913):197–8.

Article   PubMed   Google Scholar  

Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, Howells DW, Ioannidis JPA, Oliver S. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383(9912):156–65.

Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–9.

Panel on the Return on Investments in Health Research. Making an impact: a preferred framework and indicators to measure returns on investment in health research. Ottawa: Canadian Academy of Health Sciences; 2009.

Google Scholar  

Banzi R, Moja L, Pistotti V, Facchini A, Liberati A. Conceptual frameworks and empirical approaches used to assess the impact of health research: an overview of reviews. Health Res Policy Syst. 2011;9:26.

Article   PubMed   PubMed Central   Google Scholar  

Milat AJ, Bauman AE, Redman S. A narrative review of research impact assessment models and methods. Health Res Policy Syst. 2015;13:18.

Buxton M, Hanney S. How can payback from health services research be assessed? J Health Serv Res Policy. 1996;1:35–43.

CAS   PubMed   Google Scholar  

Graham KER, Chorzempa HL, Valentine PA, J M. Evaluating health research impact: Development and implementation of the Alberta Innovates – Health Solutions impact framework. Res Eval. 2012;21:354–67

Nowotny H, Scott P, Gibbons M. Introduction: ‘Mode 2’ Revisited: The New Production of Knowledge. Minerva. 2003;41(3):179–94.

Article   Google Scholar  

Canadian Institutes of Health Research. Strategy for Patient-Oriented Research. 2013. http://www.cihr-irsc.gc.ca/e/41204.html . Accessed 27 Jul 2014.

Lomas J. Preface: The first ones over the barricade. In: Potvin L, Armstrong P, editors. Shaping Academia for the Public Good: Critical Reflections on the CHSRF/CIHR Chair Program. Toronto: University of Toronto Press; 2013.

Ontario Ministry of Health and Long-Term Care – Community and Health Promotion Branch. Ontario Support for People and Patient-Oriented Research and Trials (SUPPORT) Unit: Business Plan. Toronto, ON: MOHLTC; 2013.

Buxton M. The payback of ‘Payback’: challenges in assessing research impact. Res Eval. 2011;20(3):259–60.

Floridi L. Understanding epistemic relevance. Erkenntnis. 2008;69(1):69–92.

Cohen LJ. Some steps towards a general theory of relevance. Synthese. 1994;101(2):171–85.

Gärdenfors P. On the logic of relevance. Synthese. 1978;37(3):351–67.

Schlesinger GN. Relevance Theoria. 1986;57(1):57–67.

Keynes JM. A treatise on probability. London: MacMillan and Co. Limited; 1921.

Bush V. Science - the endless frontier. A report to the President on a program of postwar scientific research. Washington: National Science Foundation; 1945.

Dudley JM. Defending basic research. Nat Photonics. 2013;7:338–9.

Article   CAS   Google Scholar  

Stokes DE. Pasteur's quadrant - basic science and technological innovation. Washington: Brookings Institution Press; 1997.

Tushman M, O'Reilly C. Research and relevance: implications of Pasteur's Quadrant for doctoral programs and faculty development. Acad Manag J. 2007;50(4):769–74.

Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson N. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26(1):13–24.

Berger ML, Martin BC, Husereau D, Worley K, Allen JD, Yang W, Quon NC, Mullins CD, Kahler KH, Crown W. A questionnaire to assess the relevance and credibility of observational studies to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. Value Health. 2014;17(2):143–56.

Jansen JP, Trikalinos T, Cappelleri JC, Daw J, Andes S, Eldessouki R, Salanti G. Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. Value Health. 2014;17(2):157–73.

Mays N, Pope C. Qualitative research in health care: assessing quality in qualitative research. BMJ. 2000;320(7226):50–2.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Pope C, Ziebland S, Mays N. Qualitative research in health care: analysing qualitative data. BMJ. 2000;2000(320):114–6.

Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schunemann HJ, for the GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924–6.

Campbell Collaboration. Campbell Collaboration Systematic Reviews: Policies and Guidelines (Version 1.2). 2016. https://www.campbellcollaboration.org/library/campbell-collaboration-systematic-reviews-policies-and-guidelines.html . Accessed 27 Mar 2017.

Burchett H, Umoquit M, Dobrow M. How do we know when research from one setting can be useful in another? A review of external validity, applicability and transferability. J Health Serv Res Policy. 2011;16(4):238–44.

World Health Organization. WHO recommendations: Optimizing health worker roles to improve access to key maternal and newborn health interventions through task shifting. Annex 8 Contextualizing the guidelines – workbook. 2012. http://www.who.int/reproductivehealth/publications/maternal_perinatal_health/Annex_8_Contextualizing_Workbook.pdf?ua=1 . Accessed 27 Sep 2016.

Re-aim.org. What is RE-AIM. 2017. http://re-aim.org/about/what-is-re-aim . Accessed 27 Mar 2017.

Center for the Study of Language and Information, Stanford University. Stanford Encyclopedia of Philosopy: The legal concept of evidence. 2015. http://plato.stanford.edu/entries/evidence-legal/ . Accessed 27 Sep 2016.

Cornell University Law School. Rule 402. General admissibility of relevant evidence. 2016. https://www.law.cornell.edu/rules/fre/rule_402 . Accessed 27 Sep 2016.

Giacomini M. One of these things is not like the others: the idea of precedence in health technology assessment and coverage decisions. Milbank Q. 2005;83(2):193–223.

Karğın S. The Impact of IFRS on the Value Relevance of Accounting Information: Evidence from Turkish Firms. Int J Econ Finance. 2013;5(4):71–80.

Abbott S. Relevance. In: Abbott S, editor. The glossary of education reform. Portland: Great Schools Partnership; 2013.

Pearson. Rigor and relevance: an overview. 2014. http://www.californiareading.com/media/pdf/rigor_and_relevance.pdf . Accessed 12 Oct 2014.

International Center for Leadership in Education. The Rigor Relevance Framework. 2016. http://www.leadered.com/our-philosophy/rigor-relevance-framework.php . Accessed 27 Sep 2016.

Google. Google Inside Search – Algorithms. 2016. http://www.google.ca/insidesearch/howsearchworks/algorithms.html . Accessed 27 Sep 2016.

Google. Search Quality Evaluator Guidelines. 2016. Accessed 28 Mar 2016.

Pariser E. The Filter Bubble: What the Internet is Hiding from You. New York: Penguin Group Inc.; 2011.

Weiss C. The many meanings of research utilization. Public Adm Rev. 1979;39(5):426–31.

Research Councils UK. Pathways to Impact. 2014. http://www.rcuk.ac.uk/innovation/impacts . Accessed 22 Feb 2017.

Download references

Acknowledgements

We acknowledge and appreciate the contributions of participants of a roundtable discussion to gather feedback on an earlier version of this paper. Participants included Simon Denegri, National Director for Public Participation and Engagement in Research, National Institute for Health Research (NIHR) UK, and Chair of INVOLVE, UK; Lee Fairclough, Vice-President, Quality Improvement, Health Quality Ontario; Michael Hillmer, Director, Planning, Research and Analysis Branch, Ontario Ministry of Health and Long-Term Care; John McLaughlin, Chief Science Officer and Senior Scientist, Public Health Ontario; Allison Paprica, Director, Strategic Partnerships, ICES; Michael Schull, President and CEO, ICES; and Vasanthi Srinivasan, Executive Director, Ontario Strategy for Patient-Oriented Research (SPOR) SUPPORT Unit (OSSU). We also want to thank John Lavis of the McMaster Health Forum for his very helpful comments on an earlier draft. Though we owe these individuals and organisations many thanks for their insights and support, we alone are responsible for the final product.

This work was commissioned by the Ontario SPOR Support Unit (OSSU). The executive director of the OSSU was one of the participants in a roundtable discussion to gather feedback on an earlier version of this paper, but beyond that, the OSSU did not have any role in the design of the study, collection, analysis or interpretation of the data, or writing of the manuscript.

Availability of data and materials

Not applicable. No datasets were generated or analysed during the development of the article.

Authors’ contributions

ADB acquired funding for the study. MJD, FAM and ADB conceptualised the study. MJD, FAM, CF and ADB participated in the review and writing of the manuscript. MJD, FAM and ADB participated in the roundtable discussion. MJD, FAM and ADB reviewed and approved the final version of the manuscript (CF passed away prior to submission of the manuscript).

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and affiliations.

Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, 155 College Street, 4th Floor, Toronto, ON, M5T 3M6, Canada

Mark J. Dobrow, Fiona A. Miller & Adalsteinn D. Brown

Alberta Innovates - Health Solutions, Edmonton, Alberta, Canada

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Mark J. Dobrow .

Additional information

This article is dedicated to the memory of Dr Cy Frank, our co-author and esteemed colleague, whose untimely death occurred midway through development of this work. Among his many interests, Dr Frank was a champion for improving understanding of research impact assessment and provided many insights on the concept of research relevance, some of which we expand upon in this article. His many contributions to the health sector will live on, but he will be greatly missed.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Dobrow, M.J., Miller, F.A., Frank, C. et al. Understanding relevance of health research: considerations in the context of research impact assessment. Health Res Policy Sys 15 , 31 (2017). https://doi.org/10.1186/s12961-017-0188-6

Download citation

Received : 27 September 2016

Accepted : 07 March 2017

Published : 17 April 2017

DOI : https://doi.org/10.1186/s12961-017-0188-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Health research systems
• Research relevance
• Research impact

Health Research Policy and Systems

ISSN: 1478-4505

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• For authors
• Browse by collection
• BMJ Journals

You are here

• Volume 14, Issue 9
• Evaluating the impact of malaria rapid diagnostic tests on patient-important outcomes in sub-Saharan Africa: a systematic review of study methods to guide effective implementation
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0001-9521-624X Jenifer Akoth Otieno 1 ,
• Lisa Malesi Were 1 ,
• http://orcid.org/0000-0002-7316-3340 Caleb Kimutai Sagam 1 ,
• Simon Kariuki 1 ,
• http://orcid.org/0000-0002-7951-3030 Eleanor Ochodo 1 , 2
• 1 Centre for Global Health Research, Kenya Medical Research Institute , Kisumu , Kenya
• 2 Centre for Evidence-Based Health Care, Division of Epidemiology and Biostatistics, Faculty of Medicine and Health Sciences , Stellenbosch University , Cape Town , South Africa
• Correspondence to Ms. Jenifer Akoth Otieno; jenipherakoth15{at}gmail.com

Objective To perform critical methodological assessments on designs, outcomes, quality and implementation limitations of studies evaluating the impact of malaria rapid diagnostic tests (mRDTs) on patient-important outcomes in sub-Saharan Africa.

Design A systematic review of study methods.

Data sources MEDLINE, EMBASE, Cochrane Library, African Index Medicus and clinical trial registries were searched up to May 2022.

Eligibility criteria Primary quantitative studies that compared mRDTs to alternative diagnostic tests for malaria on patient-important outcomes within sub-Sahara Africa.

Data extraction and synthesis Studies were sought by an information specialist and two independent reviewers screened for eligible records and extracted data using a predesigned form using Covidence. Methodological quality was assessed using the National Institutes of Health tools. Descriptive statistics and thematic analysis guided by the Supporting the Use of Research Evidence framework were used for analysis. Findings were presented narratively, graphically and by quality ratings.

Results Our search yielded 4717 studies, of which we included 24 quantitative studies; (15, 62.5%) experimental, (5, 20.8%) quasi-experimental and (4, 16.7%) observational studies. Most studies (17, 70.8%) were conducted within government-owned facilities. Of the 24 included studies, (21, 87.5%) measured the therapeutic impact of mRDTs. Prescription patterns were the most reported outcome (20, 83.3%). Only (13, 54.2%) of all studies reported statistically significant findings, in which (11, 45.8%) demonstrated mRDTs’ potential to reduce over-prescription of antimalarials. Most studies (17, 70.8%) were of good methodological quality; however, reporting sample size justification needs improvement. Implementation limitations reported were mostly about health system constraints, the unacceptability of the test by the patients and low trust among health providers.

Conclusion Impact evaluations of mRDTs in sub-Saharan Africa are mostly randomised trials measuring mRDTs’ effect on therapeutic outcomes in real-life settings. Though their methodological quality remains good, process evaluations can be incorporated to assess how contextual concerns influence their interpretation and implementation.

PROSPERO registration number CRD42018083816.

• INFECTIOUS DISEASES
• Tropical medicine

Data availability statement

Data are available upon reasonable request. Our reviews’ data on the data extraction template forms, including data extracted from the included studies, will be availed by the corresponding author, JAO, upon reasonable request.

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See:  https://creativecommons.org/licenses/by/4.0/ .

https://doi.org/10.1136/bmjopen-2023-077361

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

We conducted a robust literature search to get a recent representative sample of articles to assess the methodology.

In addition to the methodology of studies, we evaluated the implementation challenges that limit the effect of the tests.

We only included studies published in English which might have limited the generalisability of study findings, but we believe this is a representative sample to investigate the methods used to assess the impact of malaria rapid diagnostic tests.

Introduction

The malaria burden remains high in sub-Saharan Africa despite several interventions deployed to control. 1 Interventions include but are not limited to the adoption of parasitological confirmation of malaria infection using malaria rapid diagnostic tests (mRDTs) and effective treatment using artemisinin-based combination therapies. 2 3 In 2021, there were 247 million cases of malaria reported globally, an increase of 2 million cases from 245 million cases reported in 2020. 4 This estimated increase in 2021 was mainly reported in sub-Saharan Africa. 4 Of all global malaria cases in 2021, 48.1% were reported in sub-Saharan Africa—Nigeria (26.6%), the Democratic Republic of the Congo (DRC) (12.3%), Uganda (5.1%) and Mozambique (4.1%). 4–6 Similarly, 51.9% of the worldwide malaria deaths were reported in sub-Saharan African—Nigeria (31.3%), the DRC (12.6%), the United Republic of Tanzania (4.1%) and Niger (3.9%). 4–6

Following the 2010 WHO’s policy on recommending parasitological diagnosis of malaria before treatment, the availability and access to mRDTs have significantly increased. 7 For instance, globally, manufacturers sold 3.5 billion mRDTs for malaria between 2010 and 2021, with almost 82% of these sales being in sub-Saharan African countries. 4 In the same period, National Malaria Control Programmes distributed 2.4 billion mRDTs globally, with 88% of the distribution being in sub-Saharan Africa. 4 This demonstrates impressive strides in access to diagnostic services in the public sector but does not effectively reveal the extent of test access in the private and retail sectors. Published literature indicates that over-the-counter (OTC) malaria medications or treatment in private retail drug stores are often the first point of care for fever or acute illness in African adults and children. 7–9 Using mRDTs in private drug outlets remains low, leading to overprescribing antimalarials. Increased access to mRDTs may minimise the overuse of OTC medicines to treat malaria.

Universal access to malaria diagnosis using quality-assured diagnostic tests is a crucial pillar of the WHO’s Global Technical Strategy (GTS) for malaria control and elimination. 4 10 11 Assessing the role of mRDTs in achieving the GTS goals and their impact on patient-important outcomes is essential in effectively guiding their future evaluation and programmatic scale-up. 12 Rapidly and accurately identifying those with the disease in a population is crucial to administering timely and appropriate treatment. It plays a key role in effective disease management, control and surveillance.

Impact evaluations determine if and how well a programme or intervention works. If impact evaluations are well conducted, they are expected to inform the scale-up of interventions such as mRDTs, including the cost associated with the implementation. Recent secondary research (systematic reviews on the impact of mRDTs on patient-important outcomes) 13 is only based on assessing mRDTs’ effect and does not consider how well the individual studies were conducted. Odaga et al conducted a Cochrane review comparing mRDTs to clinical diagnosis. They included seven trials where mRDTs substantially reduced antimalarial prescription and improved patient health outcomes. However, they did not assess the contextual factors that influence the effective implementation of the studies. There is a need to access the methodological implementation of studies that evaluate the impact of mRDTs. To our knowledge, no study has investigated the implementation methods of studies evaluating the impact of mRDTs.

We aimed to perform critical methodological assessments on the designs, outcomes, quality and implementation limitations of studies that evaluate the impact of mRDTs compared with other malaria diagnostic tests on patient-important outcomes among persons suspected of malaria in sub-Saharan Africa. We defined patient-important outcomes as; characteristics valued by patients which directly reflect how they feel, function or survive (direct downstream health outcomes such as morbidity, mortality and quality of life) and those that lie on the causal pathway through which a test can affect a patient’s health, and thus predict patient health outcomes (indirect upstream outcomes such as time to diagnosis, prescription patterns of antimalarials and antimicrobials, patient adherence). 14

We prepared this manuscript according to the reporting guideline: Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA-2020) 15 ( online supplemental files 1; 2 ). The protocol is registered with the International Prospective Register of Systematic Reviews and was last updated in June 2022. The protocol is also available as a preprint in the Open Science Network repositories. 12

Supplemental material

Patient and public involvement, criteria for including studies in this review, study designs.

We included primary quantitative studies published in English. We included observational and experimental studies in either controlled or uncontrolled settings. We did not limit trials to the unit of randomisation (individual or cluster). We extracted qualitative data from quantitative studies on implementation limitations. We excluded studies, which only provided test accuracy statistics without evaluating the tests’ impact on patient-important outcomes and modelling studies. We also excluded editorials, opinion pieces, non-research reports, theoretical studies, secondary quantitative studies, reports, case studies, case series or abstracts with insufficient information or no full texts available, as the methodology of the studies could not be fully appraised.

Population and setting

We defined our population as people suspected of having malaria infection caused by any of the four human malaria parasites ( Plasmodium falciparum, P. malariae, P. ovale and P. vivax ) who reside in any sub-Saharan African country, regardless of age, sex or disease severity.

Intervention

We restricted studies for inclusion to those assessing mRDTs, regardless of the test type or the manufacturer.

We included studies comparing mRDTs to microscopy, molecular diagnosis (PCR) or clinical/presumptive/routine diagnosis.

We included studies reporting on at least one or more patient-important outcomes. We adopted the conceptual framework for the classification of these outcomes as described by Schumacher et al . 16 Further details regarding the classification are available in our protocol. 12

Measures of the diagnostic impact that indirectly assess the effect of mRDTs on the diagnostic process, such as time to diagnosis/turn-around time and prediagnostic loss to follow-up.

Measures of the therapeutic impact that indirectly assess the effect of mRDTs on treatment decisions, such as time to treatment, pretreatment loss to follow-up, antimalarial/antibiotics prescription patterns and patient adherence to the test results.

Measures of the health impact that directly assess the effect of mRDTs on the patient’s health, such as mortality, morbidity, symptom resolution, quality of life and patient health costs.

Search methods for identifying studies

Electronic searches.

Given the review’s purpose to assess the methodology of existing studies, we searched the following electronic databases for a representative sample till May 2022; MEDLINE, EMBASE, Cochrane Library and African Index Medicus. We also searched clinical trial registries, including clinicaltrials.gov, the meta-register of controlled trials, the WHO trials register and the Pan African Clinical Trials Registry. We applied a broad search strategy that included the following key terms: “Malaria”, “Diagnosis”, “Rapid diagnostic test”, “Impact”, “Outcome” and their associated synonyms. The full search strategy is provided in online supplemental file 2 .

Other searches

We searched reference lists and citations of relevant systematic reviews that assessed the impact of mRDTs on patient-important outcomes. We checked for searches from conference proceedings within our search output.

Study selection

Two reviewers independently screened the titles and abstracts of the search output and identified potentially eligible full texts using Covidence—an online platform for systematic reviews. 17 We resolved any differences or conflicts through discussion among the reviewers or consulting a senior reviewer.

Data extraction

Two reviewers independently extracted data from studies included using a predesigned and standard data extraction form in Covidence. 17 We piloted the form on two potentially eligible studies before its use and resolved any differences or conflicts through a discussion among the reviewers or consulting a senior reviewer. The study information that was extracted included the following:

General study details include the first author, year, title, geographical location(s), population, target condition and disease seasonality.

Study design details such as the type of study, intervention, comparator, prediagnostic, pretreatment and post-treatment loss to follow-up, outcome measures and results for outcome measures (effect size and precision). Study design issues were also considered, including sample size, study setting, inclusion criteria and study recruitment.

The quality assessment of the included studies was also performed using the National Institute for Health (NIH) quality assessment tools 18 ( online supplemental file 3 ).

The implementation challenges, as reported by study authors in the methods and the discussion sections, were extracted according to the four main domains of the Supporting the Use of Research Evidence (SURE) framework for identifying barriers and enablers to health systems: recipient of care, providers of care, health system constraints and sociopolitical constraints 19 ( online supplemental file 4 ).

Quality assessment

We assessed the methodological quality of included studies in Covidence. 17 We adopted two NIH quality assessment tools 18 for experimental and observational designs. Two reviewers independently assessed the methodological quality of studies as stratified by study design. We resolved any differences or conflicts by discussing among the reviewers or consulting a senior reviewer. Our quality evaluation was based on the number of quality criteria a study reported about its internal validity. The overall score was used to gauge the study’s methodological quality. We did not exclude studies based on the evaluation of methodological quality. Instead, we used our assessment to explain the methodological issues affecting impact studies of mRDTs.

We did not pool results from included individual studies, but we conducted descriptive statistics by synthesising our results narratively and graphically, as this was a methodological review. All included studies were thereby considered during narrative synthesis.

Quantitative data

We started our analysis by listing and classifying identified study designs and patient-important outcomes according to similarities. Stratified by study design, we used descriptive statistics for summarising key study characteristics. Descriptive analysis was done using STATA V.17 (Stata Corp, College Station, TX).

Qualitative data

We used the thematic framework analysis approach to analyse and synthesise the qualitative data to enhance our understanding of why the health stakeholders thought, felt and behaved as they did. 20 We applied the following steps: familiarisation with data, selection of a thematic framework (SURE), 19 coding themes, charting, mapping and interpreting identified themes.

A summary of our study selection has been provided in figure 1 . Our search yielded 4717 records as of June 2022. After removing 17 duplicates, we screened 4700 studies based on their titles and abstracts and excluded 4566 records. After that, we retrieved 134 full texts and screened them against the eligibility criteria. We excluded 110 studies. The characteristics of excluded studies are shown in online supplemental file 5 . Therefore, we included 24 studies in this systematic review.

• Download figure
• Open in new tab
• Download powerpoint

Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 flow diagram showing the study selection process.

General characteristics of included studies

Study characteristics have been summarised in online supplemental file 6 . Studies included in this review were done in Ghana (7, 29.2%), Uganda (7, 29.2%), Tanzania (6, 25%), Burkina Faso (3, 12.5%), Nigeria (2, 8.3%) and Zambia (1, 4.2%). Most studies (16, 66.7%) were done on mixed populations of children and adults, while the remaining (8, 33.3%) were done on children alone. All studies (24, 100%) tested mRDTs as the intervention. Most studies (18, 75%) compared mRDTs to presumptive treatment/clinical diagnosis/clinical judgement, while the remaining (7, 29.2%) had microscopy and routine care (1, 4.2%) as their comparator. No study reported on PCR as a control.

Of all included studies, (17, 70.8%) were carried out in rural areas within government-owned facilities, (7, 29.2%) in urban areas and (2, 8.3%) in peri-urban areas. Few studies (6, 25%) were conducted in privately owned propriety facilities. Most studies (15, 62.5%) were conducted in health facilities and only (9, 37.5%) were within the communities. Studies conducted within health centres were (9, 37.5%), while those conducted in hospitals were (7, 29.2%). Most studies (15, 62.5%) were conducted during the high malaria transmission season, (9, 37.5%) during the low malaria season and (4, 16.7%) during the moderate malaria season. P. falciparum was the most common malaria parasite species (21, 87.5%)

We included multiple-armed studies with an intervention and a comparator ( online supplemental file 6 ). Of the 24 studies, (15, 62.5%) were experimental designs in which, (10, 41.7%) were cluster randomised controlled trials (4, 16.7%) were individual randomised controlled trials and (1, 4.2%) was a randomised crossover trial. Of the remaining studies, (5, 20.8%) were quasi-experimental designs (non-randomised studies of intervention) in which (4, 16.7%) were pre-post/before and after studies and (1, 4.2%) was non-randomised crossover trials. The remaining studies (4, 16.7%) were observational where, (3, 12.5%) were cross-sectional designs and (1, 4.2%) was a cohort study.

Patient-important outcomes

Patient-important outcome measures and individual study findings are summarised in online supplemental file 7 . Of the 24 included studies, (21, 87.5%) measured the therapeutic impact of mRDTs, while (13, 54.2%) evaluated its health impact and only (1, 4.2%) assessed its diagnostic impact. Only (13, 54.2%) of all studies reported statistically significant findings.

Measures of therapeutic impact

Of the included studies, (20, 83.3%) reported on either antimalarials or antibiotics prescription patterns. The patient’s adherence to test results was reported by (3, 12.5%) studies, and the time taken to initiate treatment was reported by (2, 8.3%). In contrast, the pretreatment loss to follow-up was reported by (1, 4.2%) study. Studies reporting statistically significant findings on prescription patterns were (12, 50%), in which (11, 45.8%) demonstrated mRDTs’ potential to reduce over-prescription of antimalarials. In contrast, (1, 4.2%) study reported increased antimalarial prescription in the mRDT arm. Other statistically significant findings were reported by two studies where (1, 4.2%) reported that patients’ adherence to test results was poor in the malaria RDT arm. In contrast, the other (1, 4.2%) reported that mRDTs reduced the time to offer treatment.

Measures of health impact

Of the included studies, (6, 25%) reported on mortality, while (5, 20.8%) reported on symptom resolution. Patient health cost was reported by (4, 16.7%) studies, while patient referral and clinical re-attendance rates were reported by (2, 8.3%) each. Few (3, 12.5%) studies reported statistically significant findings on measuring the health impact that mRDTs improved the patient’s health outcomes by reducing morbidity.

Measures of diagnostic impact

Time taken to diagnose patients with malaria was reported by (1, 4.2%) study where diagnosis using mRDTs reduced the time to diagnose patients, but the findings were not statistically significant.

Implementation challenges

The themes identified among included studies according to the SURE framework 19 are presented in table 1 . Most themes (n=7, 50%) emerged from the health system constraints domain while only one theme was reported under the domain, social and political constraints. Two themes, human resources and patient’s attitude were dominant. Lack of qualified staff in some study sites and patient’s preference for alternative diagnostic tests other than mRDTs hindered effective implementation of five studies.

• View inline

Implementation challenge reported by the included studies

Methodological quality of included studies

The methodological quality of the included studies is summarised in figures 2 and 3 . All studies assessed their outcomes validly and reliably and consistently implemented them across all participants. Some studies did not provide adequate information about loss-to-follow-up. Overall, (17, 70.8%) were of good methodological quality in which (11, 45.8%) were experimental, (3, 12.5%) were quasi-experimental and (3, 12.5%) were observational studies; however, blinding was not feasible. Concerns regarding patient non-adherence to treatment were reported in some studies. Sample size justification which is crucial when detecting differences in the measured primary outcomes was poorly reported among most studies. A detailed summary of each study’s performance is available in online supplemental files 8 and 9 .

Quality assessment of controlled intervention study designs. NIH, National Institute for Health.

Quality assessment of observational study designs. NIH, National Institute for Health.

In this methodological systematic review, we assessed the designs, patient-important outcomes, implementation challenges and the methodological quality of studies evaluating the impact of mRDTs on patient-important outcomes within sub-Saharan Africa. We found evidence of mRDTs’ impact on patient-important outcomes came from just a few (six) from Western, Eastern and Southern African countries. Few studies were done on children, while most enrolled mixed populations in rural settings within government-owned hospitals. Few studies were conducted within the community health posts. Included studies assessed mRDTs’ impact compared with either microscopy/clinical diagnosis, with a majority being carried out during the high malaria transmission seasons in areas predominated by P. falciparum . Studies included were primary comparative designs, with experimental designs being the majority, followed by quasi-experimental and observational designs.

While most studies evaluated the therapeutic impact of mRDTs by measuring the prescription patterns of antimalarials/antibiotics, few assessed the test’s health and diagnostic impact. Few studies reported statistically significant findings, mainly on reduced antimalarial prescription patterns due to mRDTs. Most studies were of good quality, but quality concerns were lack of adequate information about loss-to-follow-up, inability to blind participants/providers/investigators, patient’s poor adherence to treatment options provided as guided by the predefined study protocols and lack of proper sample size justification. Key implementation limitations included inadequate human resources, lack of facilities, patients’ unacceptability of mRDTs, little consumer knowledge of the test and the providers’ low confidence in mRDTs’ negative results.

Schumacher et al conducted a similar study focusing on the impact of tuberculosis molecular tests, but unlike ours, they did not focus on implementation challenges. Similar to our results, Schumacher et al 16 identified that evidence of the impact of diagnostic tests comes from just a small number of countries within a particular setting. 16 Likewise, most studies evaluating the impact of diagnostic tests are done in health facilities like hospitals rather than in the community. 16 Our finding that the choice of study design in diagnostic research is coupled with trade-offs is in line with Schumacher’s review. 16 In the same way, experimental designs are mostly preferred in assessing diagnostic test impact, followed by quasi-experimental studies—majorly pre-post studies—conducted before and after the introduction of the intervention. 16 Our findings also agree that observational designs are the least adopted in evaluating diagnostic impact. 16 Similarly, our review’s finding concur with Schumacher et al that it may be worthwhile to explore other designs 16 that use qualitative and quantitative methods, that is, the mixed-methods design, as this can create a better understanding of the test’s impact in a pragmatic way.

Our findings that studies indirectly assess the impact of diagnostic tests on patients by measuring the therapeutic impact rather than the direct health impact agree with Schumacher et al . 16 However, in this systematic review, the ‘prescription patterns’ were most reported in contrast to Schumacher et al , where the ‘time to treatment’ was by far the most common. 16 Similar to our finding, Schumacher et al determined that there is a trade-off in the choice of design and the fulfilment of criteria set forth to protect the study’s internal validity. 16 While Schumacher et al investigated the risk of bias, our review focused on methodological quality. 16

Diagnostic impact studies are complex to implement despite being crucial to any health system seeking to roll-out the universal health coverage programmes. 21 Unlike therapeutic interventions that directly affect outcomes, several factors influence access to and effective implementation of diagnostic testing. 22 While it is easier to measure indirect upstream outcomes to quantify mRDTs’ impact on diagnosis and treatment options, it is crucial to understand the downstream measures such as morbidity (symptom resolution, clinical re-attendance and referrals), mortality, patient health costs 22 are key to improving value-based care. Contextual factors such as the provider’s lack of trust in the test’s credibility can negate the positive effects of the test, such as good performance. This is a problem facing health systems that are putting up initiatives to roll out mRDTs as the providers often perceive that negative mRDTs’ results are false positives. 16 22 Consequently, lacking essential facilities and human resources can hinder the true estimation of the value mRDTs contribute to the patient’s health in resource-limited areas.

Strengths and limitations

We conducted a robust literature search to get a recent representative sample of articles to assess the methodology. In addition to the methodology of studies, we evaluated the implementation challenges that limit the effect of the tests. Although we only included studies published in English which could affect generalisability of these findings, we believe this is a representative sample. Included studies were just from a few countries with sub-Sahara which could limit generalisability to other countries within the region. Since the overall sample size may not be an adequate representative of the entire population, the findings presented herein should be interpreted with caution. Additionally, considerations of the limited diversity in terms of study populations, interventions and outcome measures due to the few countries represented in the review should be included when interpreting our findings.

Health system concerns in both anglophone and francophone countries in sub-Saharan Africa are similar. 23 Studies did not report on blinding, but this did not affect their methodological quality since prior knowledge of the test and the intervention itself calls for having prior knowledge of the test. Our study was limited by reporting of study items such as randomisation and blinding of participants, providers and outcome assessors. This limited our quality assessment in quasi-experimental studies. Therefore, authors are encouraged to report the study findings according to the relevant reporting guidelines. 24 Most studies did not justify their sample sizes which could have compromised the validity of findings by influencing the precision and reliability of estimates. In cases where the sample size is inadequate, the reliability and generalisability of the findings becomes limited due to imprecise estimates with broad CIs. Studies reported poor adherence to protocols which could have reduced the sample size and the overall statistical power which could limit validity.

Implications for practice, policy and future research

Controlling the malaria epidemic in high-burden settings in sub-Saharan Africa will require the effective implementation of tests that do more than provide incremental benefit over current testing strategies. Contextual factors affecting the test performance need to be considered a priori and factors introduced to mitigate their effect on implementing mRDTs. Process evaluations 25 can be incorporated into quantitative studies or done alongside quantitative studies to determine whether the tests have been implemented as intended and resulted in certain outputs. Process evaluations 25 can be incorporated into experimental studies to assess contextual challenges that could influence the design. Process evaluations can help decision-makers ascertain whether the mRDTs could similarly impact the people if adopted in a different context. Therefore, not only should process evaluations be performed but they should also be performed in a variety of contexts. It is prudent that patient-important outcomes be measured alongside process evaluations to better understand how to implement mRDTs. It may be worthwhile to focus on methodological research that guides impact evaluation reporting, particularly those that consider contextual factors. Future studies on the impact of mRDTs could improve by conducting mixed-methods designs which might provide richer data interpretation and insights into implementation challenges. Future studies could also consider providing clear justification for the sample size to ensure there is enough power to detect a significant difference.

Most studies evaluating mRDTs’ impact on patient-important outcomes in sub-Saharan Africa are randomised trials of good methodological quality conducted in real-life settings. The therapeutic effect of mRDTs is by far the most common measure of mRDTs’ impact. Quality issues include poor reporting on sample size justification and reporting of statistically significant findings. Effective studies of patient-important outcome measures need to account for contextual factors such as inadequate resources, patients’ unacceptability of mRDTs, and the providers’ low confidence in mRDTs’ negative results, which hinder the effective implementation of impact-evaluating studies. Process evaluations can be incorporated into experimental studies to assess contextual challenges that could influence the design.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

Acknowledgments.

We also acknowledge the information search specialist Vittoria Lutje for designing the search strategy and conducting the literature searches.

• Oladipo HJ ,
• Tajudeen YA ,
• Oladunjoye IO , et al
• Soniran OT ,
• Anang A , et al
• Bruxvoort KJ ,
• Leurent B ,
• Chandler CIR , et al
• World Health Organization
• ↵ Malaria facts and statistics: Medicines for Malaria Venture , 2023 . Available : https://www.mmv.org/malaria/malaria-facts-statistics [Accessed 30 Mar 2023 ].
• Karisa B , et al
• Chandler CIR ,
• Hall-Clifford R ,
• Asaph T , et al
• Schellenberg D ,
• World health organization
• Otieno JA ,
• Caleb S , et al
• Hopkins H ,
• Cairns ME , et al
• Ochodo EA ,
• Schumacher S , et al
• McKenzie JE ,
• Bossuyt PM , et al
• Schumacher SG ,
• Qin ZZ , et al
• ↵ Covidence -better systematic review management 2022 . Available : https://www.covidence.org/ [Accessed 17 Feb 2023 ].
• National Institute of Health (NIH)
• Wakida EK ,
• Akena D , et al
• Schildkrout B
• Sinclair D ,
• Lokong JA , et al
• Oleribe OO ,
• Uzochukwu BS , et al
• Equator Network
• Skivington K ,
• Matthews L ,
• Simpson SA , et al
• Batwala V ,
• Magnussen P ,
• Bonful HA ,
• Adjuik M , et al
• Webster J , et al
• Warsame M , et al
• Reyburn H ,
• Mbakilwa H ,
• Mwangi R , et al
• Mbonye AK ,
• Lal S , et al
• Bisoffi Z ,
• Sirima BS ,
• Angheben A , et al
• Ikwuobe JO ,
• Faragher BE ,
• Alawode G , et al
• Bruxvoort K ,
• Kalolella A ,
• Nchimbi H , et al
• Yeboah-Antwi K ,
• Pilingana P ,
• Macleod WB , et al
• Narh-Bana S ,
• Epokor M , et al

X @AkothJenifer, @sagamcaleb1

Contributors Concept of the study: EO. Drafting of the initial manuscript: JAO. Intellectual input on versions of the manuscript: JAO, LMW, CKS, SK, EO. Study supervision: SK, EO. Approving final draft of the manuscript: JAO, LMW, CKS, SK, EO. Guarantor: JAO.

Funding EO is funded under the UK MRC African Research Leaders award (MR/T008768/1). This award is jointly funded by the UK Medical Research Council (MRC) and the UK Foreign, Commonwealth & Development Office (FCDO) under the MRC/FCDO Concordat agreement. It is also part of the EDCTP2 programme supported by the European Union. This publication is associated with the Research, Evidence and Development Initiative (READ-It). READ-It (project number 300342-104) is funded by UK aid from the UK government; however, the views expressed do not necessarily reflect the UK government’s official policies. The funding organisations had no role in the development of this review.

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

Numbers, Facts and Trends Shaping Your World

Read our research on:

Full Topic List

Regions & Countries

• Publications
• Our Methods
• Short Reads
• Tools & Resources

Read Our Research On:

In Tied Presidential Race, Harris and Trump Have Contrasting Strengths, Weaknesses

2. issues and the 2024 election, table of contents.

• Other findings: An uncertain election outcome, the more critical candidate, Trump and the 2020 election
• Voting preferences among demographic groups
• Support for Harris, Trump among voters and nonvoters in recent elections
• How Harris and Trump supporters see their vote
• Do voters think it’s clear who will win?
• Most voters cite several issues as very important to their vote
• Changes in confidence in candidates on issues, following Biden’s departure from race 
• Do voters see the candidates as ‘too personally critical’?
• Do the candidates make you feel proud, hopeful, uneasy or angry?
• How the candidates make Harris and Trump supporters feel
• How men and women view the impact of the candidates’ genders
• Views of the candidates’ races and ethnicities
• Views of the candidates’ ages among younger and older voters
• Views of the values and goals of the other candidate’s supporters
• Should the president work with the opposing party in Congress?
• Top economic concerns: Food and consumer prices, housing costs
• Acknowledgments
• The American Trends Panel survey methodology

As concerns around the state of the economy and inflation continue, about eight-in-ten registered voters (81%) say the economy will be very important to their vote in the 2024 presidential election.

While the economy is the top issue among voters, a large majority (69%) cite at least five of the 10 issues asked about in the survey as very important to their vote.

There are wide differences between voters who support Harris and Trump when it comes to the issues.

Among Trump supporters, the economy (93%), immigration (82%) and violent crime (76%) are the leading issues. Just 18% of Trump supporters say racial and ethnic inequality is very important. And even fewer say climate change is very important (11%).

For Harris supporters, issues such as health care (76%) and Supreme Court appointments (73%) are of top importance. Large majorities also cite the economy (68%) and abortion (67%) as very important to their vote in the election.

Most voters cite several issues as very important to their vote this November. Very few – just 5% – say only one issue or no issues are highly important.

Majorities of both Harris supporters (71%) and Trump supporters (69%) say at least five of 10 issues included in the survey are very important to their vote.

Harris supporters are more likely than Trump supporters to say most of the issues included are very important. About a third of Harris supporters (32%) say at least eight of 10 issues are very important, compared with 17% of Trump supporters.

Top voting issues: 2020 versus 2024

While the economy has long been a top issue for voters – and continues to be one today – other issues have become increasingly important for voters over the past four years.

Immigration

About six-in-ten voters (61%) today say immigration is very important to their vote – a 9 percentage point increase from the 2020 presidential election and 13 points higher than during the 2022 congressional elections.

Immigration is now a much more important issue for Republican voters in particular: 82% of Trump supporters say it is very important to their vote in the 2024 election, up 21 points from 2020.

About four-in-ten Harris supporters (39%) say immigration is very important to their vote. This is 8 points higher than the share of Democratic congressional supporters who said this in 2022, but lower than the 46% of Biden supporters who cited immigration as very important four years ago.

In August 2020, fewer than half of voters (40%) said abortion was a very important issue to their vote. At the time, Trump voters (46%) were more likely than Biden voters (35%) to say it mattered a great deal.

Following the Supreme Court’s decision to overturn Roe v. Wade , opinions about abortion’s importance as a voting issue shifted. Today, 67% of Harris supporters call the issue very important – nearly double the share of Biden voters who said this four years ago, though somewhat lower than the share of midterm Democratic voters who said this in 2022 (74%). And about a third of Trump supporters (35%) now say abortion is very important to their vote – 11 points lower than in 2020.

Confidence in Harris and Trump on top issues

Voters have more confidence in Trump than Harris on economic, immigration and foreign policies. Half or more voters say they are at least somewhat confident in Trump to make good decisions in these areas, while smaller shares (45% each) say this about Harris.

In contrast, voters have more confidence in Harris than Trump to make good decisions about abortion policy and to effectively address issues around race. Just over half of voters have confidence in Harris on these issues, while 44% have confidence in Trump on these issues.

Trump holds a slight edge over Harris for handling law enforcement and criminal justice issues (51% Trump, 47% Harris). Voters are equally confident in Harris and Trump to select good nominees for the Supreme Court (50% each).

Fewer than half of voters say they are very or somewhat confident in either candidate to bring the country closer together (41% are confident in Harris, 36% in Trump). And voters express relatively little confidence in Trump (37%) or Harris (32%) to reduce the influence of money in politics.

Since Biden dropped out of the presidential race in July , there has been movement on how confident voters are in the candidates to address issues facing the country.

Abortion policy

In July, 48% of voters were confident in Biden to make good decisions about abortion policy. Today, 55% of voters are confident in Harris to do the same.

Harris currently has an 11-point advantage over Trump on voters’ confidence to handle abortion policy decisions.

Immigration policy

Voters also express more confidence in Harris to make wise decisions about immigration policy than they did for Biden before he withdrew from the race. Today, 45% are confident in Harris on this issue; in July, 35% said this about Biden.

While Trump’s advantage over Harris on immigration policy is less pronounced than it was over Biden, he continues to hold a 7-point edge. Voters are as confident in his ability to make wise decisions about immigration policy as they were in July (52%).

Foreign and economic policies

Harris has also improved over Biden in voters’ confidence to make good decisions about foreign and economic policies. Currently, 45% of voters are confident in Harris on each of these issues.

In July, 39% had confidence in Biden to make good foreign policy decisions, while a similar share (40%) had confidence in him on economic policy.

Trump holds an edge over Harris on both of these issues, though both are somewhat narrower than the advantage he had over Biden on these issues in July.

Sign up for our weekly newsletter

Fresh data delivery Saturday mornings

Sign up for The Briefing

Weekly updates on the world of news & information

• Donald Trump
• Election 2024
• Kamala Harris
• Political Issues
• Political Parties
• Voter Demographics

The Political Values of Harris and Trump Supporters

As robert f. kennedy jr. exits, a look at who supported him in the 2024 presidential race, harris energizes democrats in transformed presidential race, many americans are confident the 2024 election will be conducted fairly, but wide partisan differences remain, joe biden, public opinion and his withdrawal from the 2024 race, most popular, report materials.

• September 2024 Presidential Preference Detailed Tables
• Questionnaire

901 E St. NW, Suite 300 Washington, DC 20004 USA (+1) 202-419-4300 | Main (+1) 202-857-8562 | Fax (+1) 202-419-4372 |  Media Inquiries

Research Topics

• Email Newsletters

ABOUT PEW RESEARCH CENTER  Pew Research Center is a nonpartisan, nonadvocacy fact tank that informs the public about the issues, attitudes and trends shaping the world. It does not take policy positions. The Center conducts public opinion polling, demographic research, computational social science research and other data-driven research. Pew Research Center is a subsidiary of The Pew Charitable Trusts , its primary funder.

© 2024 Pew Research Center

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Int J Prev Med

Qualitative Methods in Health Care Research

Vishnu renjith.

School of Nursing and Midwifery, Royal College of Surgeons Ireland - Bahrain (RCSI Bahrain), Al Sayh Muharraq Governorate, Bahrain

Renjulal Yesodharan

1 Department of Mental Health Nursing, Manipal College of Nursing Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, India

Judith A. Noronha

2 Department of OBG Nursing, Manipal College of Nursing Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, India

Elissa Ladd

3 School of Nursing, MGH Institute of Health Professions, Boston, USA

Anice George

4 Department of Child Health Nursing, Manipal College of Nursing Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, India

Healthcare research is a systematic inquiry intended to generate robust evidence about important issues in the fields of medicine and healthcare. Qualitative research has ample possibilities within the arena of healthcare research. This article aims to inform healthcare professionals regarding qualitative research, its significance, and applicability in the field of healthcare. A wide variety of phenomena that cannot be explained using the quantitative approach can be explored and conveyed using a qualitative method. The major types of qualitative research designs are narrative research, phenomenological research, grounded theory research, ethnographic research, historical research, and case study research. The greatest strength of the qualitative research approach lies in the richness and depth of the healthcare exploration and description it makes. In health research, these methods are considered as the most humanistic and person-centered way of discovering and uncovering thoughts and actions of human beings.

Introduction

Healthcare research is a systematic inquiry intended to generate trustworthy evidence about issues in the field of medicine and healthcare. The three principal approaches to health research are the quantitative, the qualitative, and the mixed methods approach. The quantitative research method uses data, which are measures of values and counts and are often described using statistical methods which in turn aids the researcher to draw inferences. Qualitative research incorporates the recording, interpreting, and analyzing of non-numeric data with an attempt to uncover the deeper meanings of human experiences and behaviors. Mixed methods research, the third methodological approach, involves collection and analysis of both qualitative and quantitative information with an objective to solve different but related questions, or at times the same questions.[ 1 , 2 ]

In healthcare, qualitative research is widely used to understand patterns of health behaviors, describe lived experiences, develop behavioral theories, explore healthcare needs, and design interventions.[ 1 , 2 , 3 ] Because of its ample applications in healthcare, there has been a tremendous increase in the number of health research studies undertaken using qualitative methodology.[ 4 , 5 ] This article discusses qualitative research methods, their significance, and applicability in the arena of healthcare.

Qualitative Research

Diverse academic and non-academic disciplines utilize qualitative research as a method of inquiry to understand human behavior and experiences.[ 6 , 7 ] According to Munhall, “Qualitative research involves broadly stated questions about human experiences and realities, studied through sustained contact with the individual in their natural environments and producing rich, descriptive data that will help us to understand those individual's experiences.”[ 8 ]

Significance of Qualitative Research

The qualitative method of inquiry examines the 'how' and 'why' of decision making, rather than the 'when,' 'what,' and 'where.'[ 7 ] Unlike quantitative methods, the objective of qualitative inquiry is to explore, narrate, and explain the phenomena and make sense of the complex reality. Health interventions, explanatory health models, and medical-social theories could be developed as an outcome of qualitative research.[ 9 ] Understanding the richness and complexity of human behavior is the crux of qualitative research.

Differences between Quantitative and Qualitative Research

The quantitative and qualitative forms of inquiry vary based on their underlying objectives. They are in no way opposed to each other; instead, these two methods are like two sides of a coin. The critical differences between quantitative and qualitative research are summarized in Table 1 .[ 1 , 10 , 11 ]

Differences between quantitative and qualitative research

Qualitative Research Questions and Purpose Statements

Qualitative questions are exploratory and are open-ended. A well-formulated study question forms the basis for developing a protocol, guides the selection of design, and data collection methods. Qualitative research questions generally involve two parts, a central question and related subquestions. The central question is directed towards the primary phenomenon under study, whereas the subquestions explore the subareas of focus. It is advised not to have more than five to seven subquestions. A commonly used framework for designing a qualitative research question is the 'PCO framework' wherein, P stands for the population under study, C stands for the context of exploration, and O stands for the outcome/s of interest.[ 12 ] The PCO framework guides researchers in crafting a focused study question.

Example: In the question, “What are the experiences of mothers on parenting children with Thalassemia?”, the population is “mothers of children with Thalassemia,” the context is “parenting children with Thalassemia,” and the outcome of interest is “experiences.”

The purpose statement specifies the broad focus of the study, identifies the approach, and provides direction for the overall goal of the study. The major components of a purpose statement include the central phenomenon under investigation, the study design and the population of interest. Qualitative research does not require a-priori hypothesis.[ 13 , 14 , 15 ]

Example: Borimnejad et al . undertook a qualitative research on the lived experiences of women suffering from vitiligo. The purpose of this study was, “to explore lived experiences of women suffering from vitiligo using a hermeneutic phenomenological approach.” [ 16 ]

Review of the Literature

In quantitative research, the researchers do an extensive review of scientific literature prior to the commencement of the study. However, in qualitative research, only a minimal literature search is conducted at the beginning of the study. This is to ensure that the researcher is not influenced by the existing understanding of the phenomenon under the study. The minimal literature review will help the researchers to avoid the conceptual pollution of the phenomenon being studied. Nonetheless, an extensive review of the literature is conducted after data collection and analysis.[ 15 ]

Reflexivity

Reflexivity refers to critical self-appraisal about one's own biases, values, preferences, and preconceptions about the phenomenon under investigation. Maintaining a reflexive diary/journal is a widely recognized way to foster reflexivity. According to Creswell, “Reflexivity increases the credibility of the study by enhancing more neutral interpretations.”[ 7 ]

Types of Qualitative Research Designs

The qualitative research approach encompasses a wide array of research designs. The words such as types, traditions, designs, strategies of inquiry, varieties, and methods are used interchangeably. The major types of qualitative research designs are narrative research, phenomenological research, grounded theory research, ethnographic research, historical research, and case study research.[ 1 , 7 , 10 ]

Narrative research

Narrative research focuses on exploring the life of an individual and is ideally suited to tell the stories of individual experiences.[ 17 ] The purpose of narrative research is to utilize 'story telling' as a method in communicating an individual's experience to a larger audience.[ 18 ] The roots of narrative inquiry extend to humanities including anthropology, literature, psychology, education, history, and sociology. Narrative research encompasses the study of individual experiences and learning the significance of those experiences. The data collection procedures include mainly interviews, field notes, letters, photographs, diaries, and documents collected from one or more individuals. Data analysis involves the analysis of the stories or experiences through “re-storying of stories” and developing themes usually in chronological order of events. Rolls and Payne argued that narrative research is a valuable approach in health care research, to gain deeper insight into patient's experiences.[ 19 ]

Example: Karlsson et al . undertook a narrative inquiry to “explore how people with Alzheimer's disease present their life story.” Data were collected from nine participants. They were asked to describe about their life experiences from childhood to adulthood, then to current life and their views about the future life. [ 20 ]

Phenomenological research

Phenomenology is a philosophical tradition developed by German philosopher Edmond Husserl. His student Martin Heidegger did further developments in this methodology. It defines the 'essence' of individual's experiences regarding a certain phenomenon.[ 1 ] The methodology has its origin from philosophy, psychology, and education. The purpose of qualitative research is to understand the people's everyday life experiences and reduce it into the central meaning or the 'essence of the experience'.[ 21 , 22 ] The unit of analysis of phenomenology is the individuals who have had similar experiences of the phenomenon. Interviews with individuals are mainly considered for the data collection, though, documents and observations are also useful. Data analysis includes identification of significant meaning elements, textural description (what was experienced), structural description (how was it experienced), and description of 'essence' of experience.[ 1 , 7 , 21 ] The phenomenological approach is further divided into descriptive and interpretive phenomenology. Descriptive phenomenology focuses on the understanding of the essence of experiences and is best suited in situations that need to describe the lived phenomenon. Hermeneutic phenomenology or Interpretive phenomenology moves beyond the description to uncover the meanings that are not explicitly evident. The researcher tries to interpret the phenomenon, based on their judgment rather than just describing it.[ 7 , 21 , 22 , 23 , 24 ]

Example: A phenomenological study conducted by Cornelio et al . aimed at describing the lived experiences of mothers in parenting children with leukemia. Data from ten mothers were collected using in-depth semi-structured interviews and were analyzed using Husserl's method of phenomenology. Themes such as “pivotal moment in life”, “the experience of being with a seriously ill child”, “having to keep distance with the relatives”, “overcoming the financial and social commitments”, “responding to challenges”, “experience of faith as being key to survival”, “health concerns of the present and future”, and “optimism” were derived. The researchers reported the essence of the study as “chronic illness such as leukemia in children results in a negative impact on the child and on the mother.” [ 25 ]

Grounded Theory Research

Grounded theory has its base in sociology and propagated by two sociologists, Barney Glaser, and Anselm Strauss.[ 26 ] The primary purpose of grounded theory is to discover or generate theory in the context of the social process being studied. The major difference between grounded theory and other approaches lies in its emphasis on theory generation and development. The name grounded theory comes from its ability to induce a theory grounded in the reality of study participants.[ 7 , 27 ] Data collection in grounded theory research involves recording interviews from many individuals until data saturation. Constant comparative analysis, theoretical sampling, theoretical coding, and theoretical saturation are unique features of grounded theory research.[ 26 , 27 , 28 ] Data analysis includes analyzing data through 'open coding,' 'axial coding,' and 'selective coding.'[ 1 , 7 ] Open coding is the first level of abstraction, and it refers to the creation of a broad initial range of categories, axial coding is the procedure of understanding connections between the open codes, whereas selective coding relates to the process of connecting the axial codes to formulate a theory.[ 1 , 7 ] Results of the grounded theory analysis are supplemented with a visual representation of major constructs usually in the form of flow charts or framework diagrams. Quotations from the participants are used in a supportive capacity to substantiate the findings. Strauss and Corbin highlights that “the value of the grounded theory lies not only in its ability to generate a theory but also to ground that theory in the data.”[ 27 ]

Example: Williams et al . conducted a grounded theory research to explore the nature of relationship between the sense of self and the eating disorders. Data were collected form 11 women with a lifetime history of Anorexia Nervosa and were analyzed using the grounded theory methodology. Analysis led to the development of a theoretical framework on the nature of the relationship between the self and Anorexia Nervosa. [ 29 ]

Ethnographic research

Ethnography has its base in anthropology, where the anthropologists used it for understanding the culture-specific knowledge and behaviors. In health sciences research, ethnography focuses on narrating and interpreting the health behaviors of a culture-sharing group. 'Culture-sharing group' in an ethnography represents any 'group of people who share common meanings, customs or experiences.' In health research, it could be a group of physicians working in rural care, a group of medical students, or it could be a group of patients who receive home-based rehabilitation. To understand the cultural patterns, researchers primarily observe the individuals or group of individuals for a prolonged period of time.[ 1 , 7 , 30 ] The scope of ethnography can be broad or narrow depending on the aim. The study of more general cultural groups is termed as macro-ethnography, whereas micro-ethnography focuses on more narrowly defined cultures. Ethnography is usually conducted in a single setting. Ethnographers collect data using a variety of methods such as observation, interviews, audio-video records, and document reviews. A written report includes a detailed description of the culture sharing group with emic and etic perspectives. When the researcher reports the views of the participants it is called emic perspectives and when the researcher reports his or her views about the culture, the term is called etic.[ 7 ]

Example: The aim of the ethnographic study by LeBaron et al . was to explore the barriers to opioid availability and cancer pain management in India. The researchers collected data from fifty-nine participants using in-depth semi-structured interviews, participant observation, and document review. The researchers identified significant barriers by open coding and thematic analysis of the formal interview. [ 31 ]

Historical research

Historical research is the “systematic collection, critical evaluation, and interpretation of historical evidence”.[ 1 ] The purpose of historical research is to gain insights from the past and involves interpreting past events in the light of the present. The data for historical research are usually collected from primary and secondary sources. The primary source mainly includes diaries, first hand information, and writings. The secondary sources are textbooks, newspapers, second or third-hand accounts of historical events and medical/legal documents. The data gathered from these various sources are synthesized and reported as biographical narratives or developmental perspectives in chronological order. The ideas are interpreted in terms of the historical context and significance. The written report describes 'what happened', 'how it happened', 'why it happened', and its significance and implications to current clinical practice.[ 1 , 10 ]

Example: Lubold (2019) analyzed the breastfeeding trends in three countries (Sweden, Ireland, and the United States) using a historical qualitative method. Through analysis of historical data, the researcher found that strong family policies, adherence to international recommendations and adoption of baby-friendly hospital initiative could greatly enhance the breastfeeding rates. [ 32 ]

Case study research

Case study research focuses on the description and in-depth analysis of the case(s) or issues illustrated by the case(s). The design has its origin from psychology, law, and medicine. Case studies are best suited for the understanding of case(s), thus reducing the unit of analysis into studying an event, a program, an activity or an illness. Observations, one to one interviews, artifacts, and documents are used for collecting the data, and the analysis is done through the description of the case. From this, themes and cross-case themes are derived. A written case study report includes a detailed description of one or more cases.[ 7 , 10 ]

Example: Perceptions of poststroke sexuality in a woman of childbearing age was explored using a qualitative case study approach by Beal and Millenbrunch. Semi structured interview was conducted with a 36- year mother of two children with a history of Acute ischemic stroke. The data were analyzed using an inductive approach. The authors concluded that “stroke during childbearing years may affect a woman's perception of herself as a sexual being and her ability to carry out gender roles”. [ 33 ]

Sampling in Qualitative Research

Qualitative researchers widely use non-probability sampling techniques such as purposive sampling, convenience sampling, quota sampling, snowball sampling, homogeneous sampling, maximum variation sampling, extreme (deviant) case sampling, typical case sampling, and intensity sampling. The selection of a sampling technique depends on the nature and needs of the study.[ 34 , 35 , 36 , 37 , 38 , 39 , 40 ] The four widely used sampling techniques are convenience sampling, purposive sampling, snowball sampling, and intensity sampling.

Convenience sampling

It is otherwise called accidental sampling, where the researchers collect data from the subjects who are selected based on accessibility, geographical proximity, ease, speed, and or low cost.[ 34 ] Convenience sampling offers a significant benefit of convenience but often accompanies the issues of sample representation.

Purposive sampling

Purposive or purposeful sampling is a widely used sampling technique.[ 35 ] It involves identifying a population based on already established sampling criteria and then selecting subjects who fulfill that criteria to increase the credibility. However, choosing information-rich cases is the key to determine the power and logic of purposive sampling in a qualitative study.[ 1 ]

Snowball sampling

The method is also known as 'chain referral sampling' or 'network sampling.' The sampling starts by having a few initial participants, and the researcher relies on these early participants to identify additional study participants. It is best adopted when the researcher wishes to study the stigmatized group, or in cases, where findings of participants are likely to be difficult by ordinary means. Respondent ridden sampling is an improvised version of snowball sampling used to find out the participant from a hard-to-find or hard-to-study population.[ 37 , 38 ]

Intensity sampling

The process of identifying information-rich cases that manifest the phenomenon of interest is referred to as intensity sampling. It requires prior information, and considerable judgment about the phenomenon of interest and the researcher should do some preliminary investigations to determine the nature of the variation. Intensity sampling will be done once the researcher identifies the variation across the cases (extreme, average and intense) and picks the intense cases from them.[ 40 ]

Deciding the Sample Size

A-priori sample size calculation is not undertaken in the case of qualitative research. Researchers collect the data from as many participants as possible until they reach the point of data saturation. Data saturation or the point of redundancy is the stage where the researcher no longer sees or hears any new information. Data saturation gives the idea that the researcher has captured all possible information about the phenomenon of interest. Since no further information is being uncovered as redundancy is achieved, at this point the data collection can be stopped. The objective here is to get an overall picture of the chronicle of the phenomenon under the study rather than generalization.[ 1 , 7 , 41 ]

Data Collection in Qualitative Research

The various strategies used for data collection in qualitative research includes in-depth interviews (individual or group), focus group discussions (FGDs), participant observation, narrative life history, document analysis, audio materials, videos or video footage, text analysis, and simple observation. Among all these, the three popular methods are the FGDs, one to one in-depth interviews and the participant observation.

FGDs are useful in eliciting data from a group of individuals. They are normally built around a specific topic and are considered as the best approach to gather data on an entire range of responses to a topic.[ 42 Group size in an FGD ranges from 6 to 12. Depending upon the nature of participants, FGDs could be homogeneous or heterogeneous.[ 1 , 14 ] One to one in-depth interviews are best suited to obtain individuals' life histories, lived experiences, perceptions, and views, particularly while exporting topics of sensitive nature. In-depth interviews can be structured, unstructured, or semi-structured. However, semi-structured interviews are widely used in qualitative research. Participant observations are suitable for gathering data regarding naturally occurring behaviors.[ 1 ]

Data Analysis in Qualitative Research

Various strategies are employed by researchers to analyze data in qualitative research. Data analytic strategies differ according to the type of inquiry. A general content analysis approach is described herewith. Data analysis begins by transcription of the interview data. The researcher carefully reads data and gets a sense of the whole. Once the researcher is familiarized with the data, the researcher strives to identify small meaning units called the 'codes.' The codes are then grouped based on their shared concepts to form the primary categories. Based on the relationship between the primary categories, they are then clustered into secondary categories. The next step involves the identification of themes and interpretation to make meaning out of data. In the results section of the manuscript, the researcher describes the key findings/themes that emerged. The themes can be supported by participants' quotes. The analytical framework used should be explained in sufficient detail, and the analytic framework must be well referenced. The study findings are usually represented in a schematic form for better conceptualization.[ 1 , 7 ] Even though the overall analytical process remains the same across different qualitative designs, each design such as phenomenology, ethnography, and grounded theory has design specific analytical procedures, the details of which are out of the scope of this article.

Computer-Assisted Qualitative Data Analysis Software (CAQDAS)

Until recently, qualitative analysis was done either manually or with the help of a spreadsheet application. Currently, there are various software programs available which aid researchers to manage qualitative data. CAQDAS is basically data management tools and cannot analyze the qualitative data as it lacks the ability to think, reflect, and conceptualize. Nonetheless, CAQDAS helps researchers to manage, shape, and make sense of unstructured information. Open Code, MAXQDA, NVivo, Atlas.ti, and Hyper Research are some of the widely used qualitative data analysis software.[ 14 , 43 ]

Reporting Guidelines

Consolidated Criteria for Reporting Qualitative Research (COREQ) is the widely used reporting guideline for qualitative research. This 32-item checklist assists researchers in reporting all the major aspects related to the study. The three major domains of COREQ are the 'research team and reflexivity', 'study design', and 'analysis and findings'.[ 44 , 45 ]

Critical Appraisal of Qualitative Research

Various scales are available to critical appraisal of qualitative research. The widely used one is the Critical Appraisal Skills Program (CASP) Qualitative Checklist developed by CASP network, UK. This 10-item checklist evaluates the quality of the study under areas such as aims, methodology, research design, ethical considerations, data collection, data analysis, and findings.[ 46 ]

Ethical Issues in Qualitative Research

A qualitative study must be undertaken by grounding it in the principles of bioethics such as beneficence, non-maleficence, autonomy, and justice. Protecting the participants is of utmost importance, and the greatest care has to be taken while collecting data from a vulnerable research population. The researcher must respect individuals, families, and communities and must make sure that the participants are not identifiable by their quotations that the researchers include when publishing the data. Consent for audio/video recordings must be obtained. Approval to be in FGDs must be obtained from the participants. Researchers must ensure the confidentiality and anonymity of the transcripts/audio-video records/photographs/other data collected as a part of the study. The researchers must confirm their role as advocates and proceed in the best interest of all participants.[ 42 , 47 , 48 ]

Rigor in Qualitative Research

The demonstration of rigor or quality in the conduct of the study is essential for every research method. However, the criteria used to evaluate the rigor of quantitative studies are not be appropriate for qualitative methods. Lincoln and Guba (1985) first outlined the criteria for evaluating the qualitative research often referred to as “standards of trustworthiness of qualitative research”.[ 49 ] The four components of the criteria are credibility, transferability, dependability, and confirmability.

Credibility refers to confidence in the 'truth value' of the data and its interpretation. It is used to establish that the findings are true, credible and believable. Credibility is similar to the internal validity in quantitative research.[ 1 , 50 , 51 ] The second criterion to establish the trustworthiness of the qualitative research is transferability, Transferability refers to the degree to which the qualitative results are applicability to other settings, population or contexts. This is analogous to the external validity in quantitative research.[ 1 , 50 , 51 ] Lincoln and Guba recommend authors provide enough details so that the users will be able to evaluate the applicability of data in other contexts.[ 49 ] The criterion of dependability refers to the assumption of repeatability or replicability of the study findings and is similar to that of reliability in quantitative research. The dependability question is 'Whether the study findings be repeated of the study is replicated with the same (similar) cohort of participants, data coders, and context?'[ 1 , 50 , 51 ] Confirmability, the fourth criteria is analogous to the objectivity of the study and refers the degree to which the study findings could be confirmed or corroborated by others. To ensure confirmability the data should directly reflect the participants' experiences and not the bias, motivations, or imaginations of the inquirer.[ 1 , 50 , 51 ] Qualitative researchers should ensure that the study is conducted with enough rigor and should report the measures undertaken to enhance the trustworthiness of the study.

Conclusions

Qualitative research studies are being widely acknowledged and recognized in health care practice. This overview illustrates various qualitative methods and shows how these methods can be used to generate evidence that informs clinical practice. Qualitative research helps to understand the patterns of health behaviors, describe illness experiences, design health interventions, and develop healthcare theories. The ultimate strength of the qualitative research approach lies in the richness of the data and the descriptions and depth of exploration it makes. Hence, qualitative methods are considered as the most humanistic and person-centered way of discovering and uncovering thoughts and actions of human beings.

Financial support and sponsorship

Conflicts of interest.

There are no conflicts of interest.