Notes
Article history
The contractual start date for this research was in June 2017. This article began editorial review in May 2023 and was accepted for publication in May 2024. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The Global Health Research editors and publisher have tried to ensure the accuracy of the authors’ article and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this article.
Permissions
Copyright statement
Copyright © 2024 Dickens et al. This work was produced by Dickens et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.
2024 Dickens et al.
Introduction
There is increasing recognition and expectation that health research should be locally prioritised to ensure best use of available resources, maximise the likelihood of research leading to action (policy and practice) and enhance relevance to the needs of the potential beneficiaries. 1 Prioritisation with locally relevant stakeholders should therefore be the first step in the research cycle, although it is often neglected. Identifying research priorities is nevertheless difficult, due to differing stakeholder perspectives,2,3 competing potential research topics, multiple outcomes and uncertainties in predicting and measuring impact. 4 Good practice recommends a comprehensive and structured approach,5 with clear criteria and methods for prioritisation, and recruitment of stakeholder participants to ensure inclusiveness of expertise, gender, ethnicity, religion, educational level and region. In health research, this commonly involves inclusion of patients, the public, health providers, managers and policymakers.
There are several established approaches for prioritising health research. 6–9 In a recent literature review, approximately one-quarter of recently published health research prioritisation exercises used either the metrics-based Child Health and Nutrition Research Initiative (CHNRI) method6 or the consensus building Delphi method,7 while one-fifth reported using a combination of expert panel interview and focus group discussion. 4 Other common tools include the James Lind Alliance method,8 the Combined Approach Matrix (CAM),9 combinations of literature reviewing and questionnaires, and use of online surveys. The CHNRI, James Lind Alliance and CAM methods have clear criteria and scoring processes, offering more transparency and replicability than the Delphi and other approaches, but are complex and time-consuming,10 especially for non-researchers. The whole process can last many months and can require several activities/meetings, making it less accessible for many participants and therefore less representative. Some prioritisation exercises may be carried out independently, with little discussion, relying on written descriptions and the understanding of participants. A simpler approach which adapts and combines the strengths of several methods could be useful,4 especially in settings where research prioritisation is not commonplace, researchers and participants are unfamiliar with such processes and time is limited.
Research prioritisation in low- and middle-income countries (LMIC) is uncommon,10 and the potential beneficiaries of health research infrequently consulted. 11 There is growing recognition of the need to implement better consultation of local stakeholders in LMIC research planning, especially when funding arises from high-income countries. 12 In this paper, we report the development and application of a simple, rapid research prioritisation approach (RAPID-RP) which was specifically designed to elicit the views of relevant patients, clinicians, managers and policy-makers in prioritising the most important research questions within a global health research programme on chronic obstructive pulmonary disease (COPD) in primary care. The approach used and the lessons learnt will have application in other similar settings and are transferable across research areas.
Aim
To develop and implement a novel rapid approach to prioritise research questions within a multi-country global health research programme conducted in LMICs.
Prioritisation methods
Setting: the Breathe Well programme
The Breathe Well programme (NIHR Global Health Research Group on Global COPD in Primary Care, University of Birmingham) was funded by the UK National Institute for Health and Care Research (NIHR) in 2017 to build research capacity in four LMICs on the prevention, diagnosis and management of COPD in the primary care setting. We developed a collaboration between UK researchers at the University of Birmingham with expertise in respiratory health, the Chief Executive and Chair of the International Primary Care Respiratory Group (IPCRG) and primary care clinical researchers in Brazil, China, Georgia and North Macedonia (LMIC partner countries) to form the core co-investigator team. Together, we provided training and mentoring to research teams in our partner LMIC countries, supporting them to design and deliver research studies in primary care relevant to the needs of their local populations. The programme had the following aims:
-
strengthen local research capacity in the partner countries in community-based COPD research and generic population research methods
-
develop and test community-based and culturally appropriate approaches for identifying undiagnosed COPD in low-resource countries
-
adapt evidence-based behavioural approaches for management of COPD according to cultural needs and the local healthcare infrastructure, and assess the feasibility of their implementation
-
build a robust platform for future collaborative research within the partner countries and other similar settings.
An important part of the programme was to identify local research needs, involving key stakeholders to prioritise research questions for evaluation within the programme and beyond.
Stakeholder prioritisation process
Our approach (Figure 1) was designed pragmatically during several informal discussions between lead research investigators, largely adapted from the CHNRI method6 but also drawing from other approaches. We created a simple process where the main stakeholder consultation could be conducted in 3 to 4 hours, particularly when stakeholders are unfamiliar with research prioritisation.
FIGURE 1.
Research prioritisation model.
Defining the context and identifying potential research topics
We held an initial scoping workshop with 26 participants, including a multidisciplinary group of UK academics with expertise in respiratory health research (epidemiologists, primary and secondary care physicians, physiologist, sociologist, health economist, biostatistician), respiratory-focused primary care clinical researchers from Brazil, China, Georgia and the Republic of North Macedonia, as well as the Chief Executive of the International Primary Care Respiratory Group (IPCRG). The workshop was used to identify important topic areas within the four LMICs. The group members were selected to include experts form a wide range of disciplines, research managers and leading researchers with local country knowledge, including those with experience of working with more disadvantaged populations. Results from the workshop were consolidated to identify common themes, which were then aligned with the remit of the funded programme, resulting in a shortlist of 10 potential research study topics (Table 1). A short evidence-based lay summary vignette was prepared for each topic (see Appendix 1), then refined in discussion with other members of the research team. Each vignette was a paragraph providing a brief background, what the knowledge gap was and how the findings from a study on the topic could be used. The selected topics were initially phrased as questions but rephrased as study titles to improve clarity for participants in response to feedback on the first prioritisation event.
| Study | Study title | Short title |
|---|---|---|
| Prevention | ||
| 1 | A study to explore the views of patients and healthcare professionals on the use of e-cigarettes for quitting smoking. | E-cigarettes |
| 2 | A study to assess the effectiveness of communicating ‘lung age’ to adult smokers in primary care to promote smoking cessation. | Lung age |
| Diagnosis/screening | ||
| 3 | A study to find the most efficient screening test strategy for identifying new cases of COPD. | Screening tests |
| 4 | A study to assess the effectiveness of treatment for people with newly diagnosed COPD (through case-finding). | Early treatment |
| Management | ||
| 5 | A study to explore the views of patients, healthcare professionals and policy-makers on the barriers and enablers to good-quality COPD management. | Quality management |
| 6 | A study to assess the feasibility of a new pulmonary rehabilitation programme (PR), adapted to local needs in low-resource settings. | PR feasibility |
| 7 | A study to assess whether an adapted pulmonary rehabilitation programme improves outcomes in patients with case-found and mild COPD, in settings where medication is suboptimal. | PR outcomes |
| 8 | A study to assess whether a weight management programme improves dyspnoea and quality of life among COPD patients. | Weight management |
| 9 | A study to assess whether a simple behavioural intervention to promote handwashing and reduce transmission of infections can reduce exacerbations among COPD patients. | Handwashing |
| 10 | A study to explore how to improve clinical education about COPD for primary health care professionals (HCP) | HCP education |
Identification and recruitment of participants
Research teams from each of the four countries identified and invited 6–10 relevant participants with a range of characteristics from each of three stakeholder groups (to ensure 4–6 attended): patients (range of severity of disease, gender, age and social group), clinicians (primary, secondary and tertiary care) and policy-makers/managers (local, national, non-government organisations). Research teams chose whether to hold a combined meeting or to arrange them separately, according to local context.
Prioritisation meetings
Meetings were conducted in each country between October and December 2017. Each group session lasted for 3 to 4 hours, with refreshment breaks as appropriate. Venues were chosen for each meeting to ensure accessibility and including separate rooms for group sessions. Meeting material, including the 10 research study topics and the accompanying evidence vignette for each, was translated and sent to participants 1 week before the meeting in case they wished to read it in advance. Locally appropriate compensation was provided, for example small gifts and refreshments.
These meetings were to guide research activity and therefore considered stakeholder consultations. As such, no ethics approval was sought for them. Nevertheless, as meeting discussions were audio-recorded and analysed, all participants were informed about this in advance and asked to sign consent forms on arrival, agreeing to the discussion being recorded and analysed, and for the data to be used in publications. There was also a short explanation of the purpose of the meeting and reassurance that all expressed views would be valued and respected. Participants were then divided into stakeholder groups with two facilitators from the local research team allocated to each. The lead facilitator explained the process and guided the discussion and prioritisation, and the second supported through timekeeping and note-taking. All facilitators were trained by the lead research team to ensure a standardised approach across all countries. Discussions were conducted in the local language. Data from the group discussion were transcribed in the local language and the transcript translated to English by a professional translator. Data from the transcripts were used to explore the rapid prioritisation process and provide learning for its future application. To do this, the transcripts were primarily analysed to provide contextual information to explain the final list of priorities. In addition, the data were analysed to explore the group process and identify aspects which worked well or could be improved.
Rating and ranking research topics
The prioritisation process involved two main stages: rating each topic individually, before finally ranking their importance relative to each other. Minor revisions were made to the model following piloting with our patient advisory group and comments from our independent scientific advisory committee, for example incorporating a traffic light system for rating importance.
The facilitator summarised the first study and evidence vignette, before asking open questions to seek participants’ views on the research study and to explore their understanding of the relevant health problem. The facilitator then guided discussion so that participants considered the study in relation to three domains: local importance (burden, priority, likely impact), feasibility (deliverability, affordability, acceptability) and the extent to which the study addressed equity issues (accessibility, reach, impact on reducing inequalities).
After considering these issues, participants were asked to independently rate the study overall as either ‘very important’, ‘unsure’ or ‘not important’ using a traffic light system (see Appendix 2). They were then encouraged to discuss reasons for their ratings, to explore factors influencing the decision. This process was repeated for each research study topic, with participants rating each one before moving on to the next. There were approximately 15–20 minutes of discussion per study.
After all of the 10 research study topics were individually rated and discussed, there was a short break where facilitators summarised the ratings across the group. The summary scores were then presented to participants, who were asked to reflect on these through discussion, and then independently rank the 10 study topics in order of priority: 1 = highest priority, 10 = lowest priority (see Appendix 3). There was further discussion to gain insight into the decision-making process, and facilitators also encouraged suggestions for additional topic areas that had not been covered or had emerged during the group discussion.
Prioritisation by research team
In addition to the stakeholder prioritisation process, LMIC research team members (directly after the meeting) independently scored the 10 topics in relation to six domains: importance, feasibility, equity, international novelty, potential for future funding, and capacity building. Each domain was scored on a scale of 0 (not at all) to 5 (very much) (see Appendix 4). Research teams then independently ranked the 10 topics in order of priority (see Appendix 5).
Analyses
Summarising topic ratings
In each country setting, the ratings for groups of patients, clinicians and policy-makers/managers were summarised using bar charts, presenting the percentage of stakeholders rating each topic as ‘important’, ‘unimportant’ or ‘unsure’. Ratings from research team members were presented as star diagrams, using the average score for each domain. Wider star shapes represented research study topics with higher average scores, while stars clustered towards the centre had lower ratings (see Figure 2).
FIGURE 2.
Star diagram (by LMIC country) summarising research team ratings for each of 10 research topics across 6 domains. Research study key: 1 = E-cigarettes; 2 = Lung age; 3 = Screening tests; 4 = Early treatment; 5 = Quality management; 6 = PR feasibility; 7 = PR outcomes; 8 = Weight management; 9 = Handwashing; 10 = HCP education.
Summarising topic rankings
The sum of the rankings for each topic was calculated for each stakeholder group (patients, clinicians, policy-makers/managers and researchers) and then a rank order presented for each group, and overall.
Qualitative analysis
Meeting transcripts were analysed using the Framework method for the analysis of qualitative data. 13 The analysis was undertaken by Dickens with the help of a research student, supervised by Gale. Emerging themes were arranged in an analytical framework, allowing comparison of how the meetings were conducted in different stakeholder groups and different countries. The framework covered facilitation, engagement and interaction (to allow exploration of the prioritisation process and what aspects worked well or needed to be modified), research topics and research literacy, culture and society, and reasoning (for/against a topic), to provide insights into why topics were prioritised (Table 2). All meeting transcripts were anonymised prior to being analysed. However, extra steps were taken to anonymise team member identities when discussing facilitation styles, to avoid the risk of identifying specific individuals.
| 1. Facilitation, engagement and interaction | 3. Culture and society |
| 1.1 Questioning (facilitator) | 3.1 Laws/regulations |
| 1.2 Further discussion (facilitator) | 3.2 Politics and policies |
| 1.3 Managing contributions (facilitator) | 3.3 Economic context |
| 1.4 Active listening/reflecting (participants) | 3.4 Demographic/social differences |
| 1.5 Agreement/disagreement (participants) | 3.5 Health literacy and education |
| 1.6 Reflections on prioritisation methodology (facilitator/participants) | 3.6 Trust/respect for doctors and health professionals |
| 1.7 Reflections on engagement/experience of the process (facilitator/participants) | 3.7 Role of family and social networks |
| 1.8 Facilitator- vs. participant-led | 4. Reasoning (for/against a study) |
| 1.9 Confusions vs. clarity | 4.1 Level of potential impact/benefit |
| 2. Research topics and research literacy | 4.2 Likelihood of eventual uptake/routinisation |
| 2.1 Confusion/clarification/understanding of research topic | 4.3 Absence/presence of evidence |
| 2.2 Practicalities of conducting study | 4.4 Role in patient–doctor consultations |
| 2.3 Experiences of dealing with issue in own practice/experience | 4.5 Time efficiency |
| 2.4 Focus (study-specific vs. general topic) | 4.6 Cost or cost-effectiveness |
| 2.5 Other suggestions for research | 4.7 Balance primary/secondary care settings |
Finally, we asked the research teams to write a short reflective report on their experience of conducting the prioritisation meetings. These were also analysed thematically to identify learning for how to optimise the process.
Results
Patient, practitioner and policy/manager participants
Meetings were attended by four to eight members of each external stakeholder group in each of the countries, representing a range of demographic characteristics (with the exception of China, where all patients were male and clinicians were all general practitioners) (Table 3). Research team ratings were completed by four to six researchers per country.
| Country | Patients | Clinicians | Policy-makers/managers | |||
|---|---|---|---|---|---|---|
| Age group (years) | Sex | Role | Work place | Role | Work place | |
| China | 70–75 | M | General practitioner | Suburban community hospital | Department director | Urban university hospital |
| 70–75 | M | General practitioner | Suburban community hospital | Department director | Urban community health service centre | |
| 65–70 | M | General practitioner | Suburban community hospital | Department director | Urban county health service centre | |
| 50–55 | M | General practitioner | Suburban community hospital | Director | Health commission | |
| Assistant dean | Suburban community hospital | |||||
| Brazil | 75–80 | M | Nurse | Basic health unit | Department director | Primary care |
| 75–80 | M | Nurse | Basic health unit | Department director | Secondary care | |
| 60–65 | M | Nurse | Specialty centre | Technical support | Secondary care | |
| 75–80 | M | Physiotherapist | Basic health unit | Hospital superintendent | Tertiary care | |
| 55–60 | F | Physiotherapist | Specialty centre | Medical co-ordinator | Urgency and emergency | |
| 60–65 | F | Family physician | Basic health unit | Care line manager | Office of the Secretary of Health | |
| Family physician | Basic health unit | Nursing co-ordinator | Urgency and emergency | |||
| Pharmaceutical | Basic health unit | |||||
| Georgia | 65–70 | F | Family doctor | Primary care | Non-communicable disease department | National Centre for Disease Control and Public Health |
| 65–70 | F | Family doctor | Primary care | Non-communicable disease department | National Centre for Disease Control and Public Health | |
| 65–70 | F | Family doctor | Primary care | Non-communicable disease department, head | National Centre for Disease Control and Public Health | |
| 65–70 | F | Family doctor, head of family medicine | Primary care | Advocacy in tobacco control | NGO | |
| 75–80 | M | Family doctor | Primary care | |||
| 75–80 | M | General practitioner | Primary care | |||
| 70–75 | M | General practitioner | Multi-profile hospital | |||
| 70–75 | M | General practitioner | Multi-profile hospital | |||
| Macedonia | 65–70 | M | Head of department | University clinic | Director, pharmaceutical department | Government agency |
| 60–65 | M | Head of department | Secondary care institute | Executive director | NGO | |
| 65–70 | F | Family physician | Primary care centre | National contact person | Government agency | |
| 50–55 | F | Head of department | General hospital | Head of department for primary care | Government agency | |
| 50–55 | F | Hospital doctor | General hospital | Junior collaborator | Government agency | |
| 60–65 | F | Hospital doctor | Hospital | Financial manager | Government agency | |
| Family physician | Primary care centre | Manager of private health institution | Primary care centre | |||
| Advisor to the director | Government agency | |||||
Ratings
Research topics that were rated highly across all countries, by the majority of stakeholder participant groups, included testing of lung age to aid smoking cessation, evaluating adapted pulmonary rehabilitation programmes, exploring how to improve clinical education for primary care staff and identifying efficient COPD screening test strategies (see Figure 4, Appendix 6). The majority of the remaining topics had less consistent ratings. In Brazil and Georgia, evaluation of handwashing to reduce exacerbations was rated highly by patients whereas policy-makers gave it low importance, and in North Macedonia, the impact of weight management on dyspnoea and quality of life was rated highly by patients and policy-makers, but most clinicians rated it as having low importance. Evaluation of the use of e-cigarettes for quitting smoking had low overall ratings across most countries.
Variability in ratings between stakeholders was lowest in Brazil, with the majority of research topics being considered as very important by all stakeholders. China stakeholders demonstrated relatively consistent ratings for most topics, with the most variability seen for the handwashing and weight management topics, where clinicians rated these lower. Greater variation between stakeholder groups was observed in Georgia and North Macedonia, where less than half of the topics had reasonable agreement. In Georgia, patients were most likely to rate topics as very important and the policy-maker group more likely to rate topics as not important. In general, most stakeholders across all four countries rated research on e-cigarettes as not important or were unsure, although clinicians in Georgia and Brazil rated it more highly.
Research team participant scores were similar to the external stakeholders in terms of topics rating highest and lowest. Across the settings, the topics with the highest ratings (widest stars) included improving clinical COPD education for primary care staff, communication of lung age for smoking cessation, feasibility of an adapted pulmonary rehabilitation programme and efficiency of COPD screening strategies (Figure 2). The topics concerning e-cigarettes and handwashing received low ratings from the research teams in all four countries. Overall, the larger number of domains used in the research teams’ ratings did not identify substantial differences, with topics tending to have similar ratings across all six domains. The study regarding the use of e-cigarettes for smoking cessation was an exception to this pattern, being rated as novel but relatively unimportant. In Brazil, across all research topics, the likelihood of attracting future funding was rated lower than the other five domains.
Rankings
Overall, in three out of four countries, the top ranked topic was identifying efficient COPD screening test strategies (Figure 3), and it was ranked second in the fourth country (North Macedonia). Clinical education for primary care staff was second ranked in China and Brazil and third ranked in Georgia. Evaluations of lung age to aid smoking cessation was highest ranked in North Macedonia, and research on the feasibility of locally adapted pulmonary rehabilitation was second ranked in Georgia, and third in Brazil and North Macedonia. Topics concerning e-cigarettes, weight management and handwashing were ranked lowest across all countries. Differences between stakeholder groups were observable, both within and between countries. Among patients, exploring how to improve clinical education for primary care staff was ranked as most important in China, but relatively low in Georgia and North Macedonia. Evaluation of lung age to aid smoking cessation was ranked highly by patients in China, but not by those in Brazil. In contrast, clinicians and policy-makers in Brazil ranked this highly. The study with the most discrepant rankings across countries concerned barriers and enablers to good-quality COPD management. With the exception of Georgia, there was between a five- and seven-point difference in rankings between stakeholder groups for this particular study, with no consistent pattern by stakeholder group. Among the researcher group, patterns were similar to the overall summary, although the study exploring views on the barriers and enablers to good-quality COPD management was more prominent.
FIGURE 3.
Stakeholder and research team rankings of the 10 research topics, by country. 1 = E-cigarettes; 2 = Lung age; 3 = Screening tests; 4 = Early treatment; 5 = Quality management; 6 = PR feasibility; 7 = PR outcomes; 8 = Weight management; 9 = Handwashing; 10 = HCP education.
Qualitative analysis from transcripts to explain and learn from RAPID-RP
Participant engagement
Levels and nature of the engagement with each of the projects varied across the different countries and types of stakeholders. This was in part related to the experiences and research literacy of the stakeholder participant and in part a reflection on the experience and background of the facilitator.
Although evidence vignettes were provided to accompany each research study, participants usually needed further clarification about the meaning of specific research topics. Redirection by the facilitators was also required if participants were focused on general or unrelated topic areas rather than the study-specific topic (e.g. smoking cessation rather than e-cigarettes), or they were unnecessarily concerned about possible difficulties in methodologies. These concerns sometimes dominated other aspects and could have adversely influenced ratings/rankings.
While discussing research topics, stakeholders often reflected on their own experiences to support their perspective. Patients, for example, talked extensively about their experiences of smoking cessation and e-cigarette use, citing family pressure to quit smoking and discussing the cost of e-cigarettes. Some clinicians commented that, in their experience, multicomponent smoking cessation interventions appeared more successful than simple interventions such as e-cigarettes, while some secondary care clinicians discussed their view of the importance of improving clinical education to avoid inappropriate referrals and misdiagnoses.
Box 1 provides recommendations for engagement drawn from our learning.
-
Ensure research topics are articulated clearly with a clear message about potential importance/impact.
-
Ensure participants have the opportunity to ask questions to help them understand the problem, relevance and potential impact of each research topic before full group discussion.
-
Ensure participants are reminded about the three domains (importance, feasibility and equity) to consider when discussing and rating each topic.
Additional research questions
Participants made few suggestions for additional research topics beyond the 10 proposals, although those which arose naturally evolved from the discussions. The relatively fixed structure of meetings may be responsible for this, with facilitators focusing mainly on the prescribed content.
Facilitator style
The structured nature of the discussions, ordered around the 10 proposals and the ranking/rating process, meant that overall the process was facilitator-led. However, within that broad structure, a range of facilitative styles were noted, with some facilitators being more effective at engaging all participants (see Box 2 for examples of prompts used) and generating debate and discussion, while others were more directive. In all cases, participants were able to share their views on the possible research projects to some extent.
-
‘Can you elaborate on this?’ (China patient group, researcher 1)
-
‘Do you see any negative reactions? How likely is this study to be cost-effective?’ (Georgia doctors/policy-maker group, researcher 1)
-
‘Does anyone have a slightly different opinion?’ (Macedonia patient group, researcher 1)
-
‘Someone else want to share their opinion, or should I ask you some questions?’ (Macedonia patient group, researcher 1)
-
‘Can we please hear your opinion?’ (Macedonia patient group, researcher 2)
Facilitation style seemed to be more closely related to the facilitators themselves than to the stakeholder group, for example open facilitators adopted the same approach whether it was healthcare professionals, policy-makers or patients in the group. We did not observe differences by stakeholder group in terms of engagement or interaction.
Key challenges for facilitators included managing participant misunderstanding about the process, and difficulties in encouraging non-contributors to participate, or disagreements to be voiced in the group. There was little evidence of participant disagreement across all stakeholder groups, and therefore a key suggestion is to ensure more in-depth facilitator training before conducting the prioritisation exercise.
Participant views on prioritisation process
Overall, participants were supportive of the prioritisation process, acknowledging the importance of identifying topics that could have an impact on health in different countries. It appeared that some clinicians and policy-makers had previously participated in similar processes, with varied opinions about their success. However, such involvement was a novelty for patients in all countries, who were very enthusiastic about these meetings and the potential to inform future research.
Contextual issues affecting decision-making
The rationale for decisions about ratings and rankings for the topics varied across the different stakeholder groups and countries. However, it was clear from discussions that decisions were shaped (as intended) by demographic, organisational, socio-cultural, economic and other contextual issues, allowing the team to gain insight into differences across settings. For example, some participants considered that informing smokers of their lung age would likely encourage smoking cessation while others thought that information alone would be insufficient, and the respective ratings/rankings reflected these differing opinions. Overall, the perceived acceptance, feasibility and impact of interventions, were they to be implemented in real life, seemed to be the strongest factor influencing ranking/ratings. Within this, themes arising from the discussions that influenced decisions are summarised below and illustrated by selected translated quotes (Box 3).
-
Socio-cultural: ‘It is social culture and sometimes cigarettes are indispensable in social situations, it is true … At home, … I do not smoke, but … guests may say that, you did not even prepare a cigarette for me when I visit you? … So, as for quitting smoking, even if I quit, I can’t live without cigarettes. Because wherever I go, I have to give other people cigarettes’ (China patient group)
-
‘Maybe quitting with an electronic cigarette can work in urban areas, but in the rural even if they are free of charge people will not use them. Not from the perspective that is something new, but because of a long tradition we have, 500 years under the Ottoman Empire in these areas they use rolled tobacco cigarettes as a ritual and would not be acceptable and there would be no effect as one that we are looking for’n (Macedonia policy-maker group)
-
Practical barriers: ‘People in Tbilisi … know this but you should see what’s going on in less developed regions of Georgia. They don’t even have water to wash hands’ (Georgia patient group)
-
Organisational: ‘I think it’s [clinical education on COPD] important, because it will make it easier for the doctor himself to treat the patient’ (Brazil patient group)
Practical barriers
Participants did not prioritise topics that they deemed to have little chance of being accepted or adhered to long term. For example, non-availability of clean water in some rural areas was considered a barrier to interventions to promote handwashing in Georgia and Brazil. Exploration of views on e-cigarettes for smoking cessation was opposed in Brazil, China and Georgia as participants discussed that they were either not available or were too expensive. Some participants acknowledged practical barriers to other topics, such as lack of provision of pulmonary rehabilitation in small hospitals/cities.
Regional factors
Others alluded to attitudinal and socio-cultural factors influencing implementation and acceptability in different regions of their countries. Participants discussed research needs being heterogeneous, with some topics being more relevant or important in some subgroups of the population or in certain geographical areas. For example, urban populations were perceived as more accepting of new initiatives, while smoking cessation interventions were considered unlikely to succeed in rural areas as smoking was ingrained in the culture. Patients in more deprived or rural areas were considered to have lower health awareness and worse health education, leading to poor self-management of their COPD and delayed consultations with clinicians.
Organisational factors
Healthcare organisational issues were considered in participants’ decision-making. For example, research on clinical education of primary care staff was anticipated to facilitate expansion of access to medical treatment by giving patients confidence to visit doctors in rural locations rather than relying on hospital doctors. Conversely, the poor reputation of primary care clinicians’ ability and knowledge in several of the countries caused patients to have a clear preference to consult secondary care doctors or those in major hospitals, rather than those in primary care or smaller hospitals. The divide between primary and secondary care was also apparent in clinician group discussions, with secondary care doctors in North Macedonia citing misdiagnoses and inappropriate referrals from primary care counterparts. Although ranked highly, interventions such as lung age for smoking cessation and screening for undiagnosed COPD were thought by some to potentially impact negatively on patient consultations due to the perceived increased time required, and interventions to promote handwashing could harm doctor patient relationships by causing offence.
Socio-cultural factors
Cultural issues informed some decisions, such as the pervasiveness of tobacco use in some communities like Macedonia and China, making smoking cessation interventions more challenging to accept.
Economic factors
Economic factors influenced discussion around various research topics, particularly among clinicians and policy-makers, with the test accuracy study prioritised due to the perceived need to identify cost-effective strategies for identifying undiagnosed COPD in resource-limited settings. Cost implications were discussed in relation to the future implementation of interventions, with clinicians and policy-makers in several countries citing costs as a barrier for purchasing required equipment, and there was an acknowledgement that patients in more deprived areas would not be able to afford some interventions. While affordability was a key reason for the e-cigarette study receiving poor ratings, legal and regulatory issues were also cited, due to licensing and regulation not being in place in most countries.
Reflections of research teams
The prioritisation activity was new for the research teams, but all found this a successful approach for deciding on research priorities. Attendance of relevant stakeholders was secured by flexibility and direct invitations, including personal invitation of patients from their own doctors. Feedback from patients after attendance was extremely positive, some citing this as the only time they had been asked for their opinions.
In order to mitigate against potential power imbalances, patient stakeholders were consulted in a separate group, but research teams emphasised parity of all stakeholders’ contributions, with one team giving patients name cards at the prioritisation meeting, naming them ‘patient experts’ to demonstrate that they were valued members of the meeting.
All research teams reflected on the importance of good facilitation during the meeting, in terms of rephrasing study material using lay terminology to optimise understanding, ensuring engagement of all participants, and refocusing discussion to the specific research topics in question.
Outcomes
The prioritisation process was successful in identifying the top research priorities in each country, which both informed the successfully delivered Breathe Well research programme14–17 and provided the rationale for subsequent and future projects. Informed by country-specific priorities, the Breathe Well programme included topics to identify the most efficient screening strategies for undiagnosed COPD in Brazil and China, a study to adapt and evaluate the feasibility of delivering pulmonary rehabilitation in Georgia, and a trial to evaluate the effectiveness of providing information on lung age on quitting among smokers in Macedonia. Subsequently, two teams explored developing topics to improve education on respiratory disease for primary care staff. Aside from the selection of research topics, the prioritisation process had additional beneficial impacts for the teams, including being a springboard for patient and stakeholder involvement that continued throughout the research topics, increasing awareness of the respiratory research programme within the locality and influencing national policy. 18
Discussion
We developed a rapid prioritisation process which was successfully implemented across four distinct settings with stakeholder groups that had no prior experience of inputting to research ideas. Local research teams successfully convened and conducted this rapid prioritisation exercise with minimal support. Four to eight members of each stakeholder group (individual groups of patients, clinicians and managers/policy-makers, or combined where locally appropriate) participated in a 3- to 4-hour meeting, prioritising the research questions most important to their local context.
Our simple metrics-based approach was pragmatic, accessible and acceptable, and of particular value in LMIC settings where stakeholder involvement in prioritisation is a relatively novel concept and patient organisations for either peer support or inputting to service or research activities rarely exist. It provided an objective approach to prioritisation based on majority ranking but also allowed us to highlight differing perspectives among stakeholders. Using qualitative analysis of the discussions, we gained insights into factors influencing prioritisation decisions.
While the summary ratings produced variable results across stakeholders and between countries, rankings were more consistent, more easily interpretable and likely to better represent the discussions. The ratings and subsequent discussion probably allowed people to reflect about the topics before deciding on the final rankings. There were five research topics which were consistently highly ranked (identifying effective COPD screening test strategies, exploring how to improve clinical education for primary care staff, evaluating adapted pulmonary rehabilitation programmes, testing of lung age to aid smoking cessation, and exploring barriers and enablers to good-quality COPD management). The ordering varied a little across the countries, generally reflecting the final selected research projects, but screening for undiagnosed COPD was the lead priority in Brazil, China and Georgia and the second priority in North Macedonia. In Georgia, priorities were more consistent between the different stakeholder groups than in the other three countries but overall there were no particular patterns of note. With a simple descriptive process and limited data, it was not possible to unpick any clear trends of differences between groups. There were three research questions which were consistently at the bottom of the rankings (evaluations of handwashing to reduce exacerbations, the impact of weight management on dyspnoea and quality of life, and the use of e-cigarettes for quitting smoking).
Observations and reflections about the rapid process of prioritisation revealed important learning. Researchers and participants engaged well and gave very positive feedback about the process. Patients in particular were enthusiastic about the opportunity to inform future research on their own illness. Having a structured approach was helpful to ensure adequate discussion of the 10 proposed topics. The evidence briefs provided a useful starting point, although facilitators needed to further clarify the topics or context during the meeting, especially for patient participants. Facilitator style varied, and it was important to maintain a balance between avoiding being too directive and ensuring focus on the topics and domains of relevance. Participants found equity to be the most difficult domain to consider, and feasibility sometimes dominated conversations, although eventually decisions were often based on likely research impact. Facilitators were able to find prompts to encourage participants to give their views but recognised the need for good communication skills and more training in this area.
It was clear from both the differences in priorities and the recorded discussions that this process truly reflected local needs, that context was influential during decision-making and that the perceived acceptance and impact of interventions were the strongest factors influencing the decision-making. Economic factors were also important for clinicians and managers/policy-makers.
Comparison with other literature
Our method brings together and adapts the most useful aspects of previously published prioritisation methods to provide an approach suitable for rapid prioritisation with a range of different stakeholders in LMIC settings. The CHNRI method6 requires research ideas to be suggested by expert participants or the management team based on current evidence and these are scored using five standard criteria: answerability, equity, impact, deliverability and effectiveness. Our tool also uses research ideas generated by experts, but uses a simpler framework with only three criteria, which may be simpler for participants. Another advantage over the CHNRI process is the ability for discussion and clarification of the research synopsis provided while still maintaining independent grading by participants. This type of social process can add value over a purely technical approach, as observed in other prioritisation approaches such as the Socio-Technical Allocation of Resources (STAR) process for distributing funding for patient treatments because it recognises that successful resource allocation processes depend on transparent stakeholder engagement and discussion that elicits their different weighting of judging criteria. 18,19 The addition of discussion and reflection prior to the final ranking allowed some degree of consensus building.
Our research teams considered six criteria, but this did not add particular value over the three criteria used by the other participants. In contrast with the James Lind Alliance approach,8 an extensive systematic review of the evidence was not required; we used topic and methodology experts’ knowledge of the literature, and local practitioner input of their context, to create accessible vignettes to discuss with participants. All consensus building processes were completed in less than one day, which is much faster than the multistep, time-consuming process required by other methods. 6,7,9
Strengths and limitations
To our knowledge, this rapid, pragmatic approach is novel, especially when applied to LMIC settings. Other researchers have adapted the James Lind Alliance method to a more rapid approach,20 mainly focusing on efficient literature review methods, but our approach was closer to an adaptation of the CHNRI method, bringing in aspects of consensus building. We relied on the knowledge of topic, methodology and local practitioner experts all working in respiratory health to identify topics for prioritisation. We assumed that their expertise and knowledge of the literature was sufficient to identify relevant research gaps within the context of the project remit. However, it is possible that a more thorough literature review would have identified additional topics.
While some local LMIC research teams had previously collaborated with clinicians and policy-makers, particularly those with policy roles, no teams had prior experience of involving patients in this way. With no established patient support groups in these countries (nor in many other LMICs), recruiting and involving appropriate patients was a new challenge. Although a range of patients were recruited successfully through personal invitations, this approach may have limited the type of patients who participated and doctors’ subconscious biases may have influenced patient selection. Nevertheless, this pragmatic approach resulted in a range of patient views to be considered, and benefit to the involved patients was also reported. Anecdotal evidence as well as qualitative data demonstrated that all participants engaged in the process, valued the contribution of all stakeholders and were willing to participate in such initiatives in the future.
All research teams were able to recruit sufficient numbers and choose appropriate participants from all stakeholder groups and were able to implement the prioritisation process in their localities with minimal support. The meetings were relatively quick and cheap to organise, suggesting it could be a sustainable model to use in the future. We tested our process in four diverse LMICs with different demography, social and political situations and healthcare structures, which suggests that our approach is transferable to multiple settings. Learning could also be tested for transferability to high-income settings where a pragmatic rapid approach is required.
Despite efforts to obtain participants’ perceptions of the selected research topics, it is possible that the facilitators’ own views may have influenced the discussions or decisions of the groups. While facilitator intervention was required on occasions to refocus discussion or clarify meanings, it is possible that their input will have influenced the discussions.
As participant recruitment took place in one geographical area of each country, the results of the prioritisation process are not necessarily representative and the results therefore need interpreting with caution. The process therefore could be repeated in localities with different socio-demographic characteristics, to capture the views of the wider society.
Implications for future practice
While we gave brief training to all facilitator teams in advance of the meetings, and this was sufficient for delivering the process, our findings suggest that additional targeted training would have been beneficial, focusing on how to orientate participants at the start of the session regarding the purpose, how best to facilitate active and inclusive discussion, and clarifying the meaning and purpose of each study proposal. In addition, the importance of a supporting role for the facilitator team was noted, that is an external person attending the meetings, with a clear awareness of the overall process, with an ability to generate good working relationships with facilitators and to discuss the practicalities of running the process.
Specific lessons learnt include:
-
Plan timings and location to optimise attendance and participation.
-
Personal invitation especially from doctors is helpful to engage patients, although this should be balanced by ensuring inclusivity of a diverse range of patients.
-
Local context is important in terms of determining group membership and whether stakeholder groups can be combined or remain separate. Potential power imbalances should also be considered.
-
Simple criteria for driving discussions are more easily understood than multiple domains; equity was found to be a difficult concept.
-
A simple traffic light system was good for ratings, but rankings were eventually the most discriminatory.
-
Hold sufficient training sessions before the prioritisation meeting to ensure all facilitators are fully prepared.
-
Build in time and prompts to allow for additional topics to be contributed.
-
Double-check translation to maximise clarity and understanding of the process and vignettes.
Recommendations for future research
Although we successfully adapted and simplified the prioritisation process, this needs replication in other contexts. Based on our experience, there are several aspects of the rapid prioritisation process that need further exploration to allow validation and potentially further simplification. These include:
-
The research topic identification was led by expert knowledge and gaps filled by checking the list against knowledge of other stakeholders during the prioritisation workshops. A comparison of this approach against a formal systematic literature review is needed to validate it.
-
We used a summary of rankings from different stakeholder groups to decide on priorities. The ranking was undertaken after a group discussion and reflection on each topic. A comparison of this approach against independent ranking (i.e. with no prior group discussion) would allow us to understand to what extent the discussion process contributes to consensus building.
-
Rating against different domains by researchers showed little discrimination between domains and all participants struggled with equity as a domain. Future research should tease out whether and to what extent explicit rating against domains is helpful or whether a simpler single rating with implicit consideration of domains would be sufficient.
-
We held meetings with only one group of each type of stakeholder in each country. Further validation should be obtained by assessing replicability of rankings in other identity-specific stakeholder groups with different compositions, or by asking another set of stakeholders to check and comment on the rankings obtained.
Conclusion
We developed an efficient, comprehensive approach for health research priority setting within a time-limited research programme that was successfully implemented in four countries. Despite some limitations, it was well-received by participating stakeholders and enabled comparison of views from different stakeholder groups. The process allowed researchers and policy-makers to focus on research that would be of most value to each country, although stakeholder participation was geographically limited to the centres where local researchers were based, and may not be fully representative of the needs of everyone. Due to the minimal resources required and the brevity of the process, it may be attractive to other investigators in LMICs, and would also be transferable to higher income settings.
Additional information
Contributions of authors
Andrew P Dickens(https://orcid.org/0000-0002-7591-8129) (Epidemiologist Research Fellow) conceptualised and designed the process and trained the research teams; contributed to identification of the research topics for prioritisation and the development of the prioritisation process; qualitative analyses; wrote the first draft of the paper; approved the final draft.
Nicola Gale(https://orcid.org/0000-0001-5295-8841) (Health Sociologist and Qualitative Expert) conceptualised and designed the process and trained the research teams; contributed to identification of the research topics for prioritisation and the development of the prioritisation process; qualitative analyses (supervision); wrote the first draft of the paper (supervision); approved the final draft.
Peymane Adab(https://orcid.org/0000-0001-9087-3945) (Co-PI, Public Health Physician) conceptualised and designed the process and trained the research teams; contributed to identification of the research topics for prioritisation and the development of the prioritisation process; wrote the first draft of the paper (supervision); approved the final draft.
KK Cheng(https://orcid.org/0000-0002-1516-1857) (Professor of Primary Care and Epidemiology) contributed to identification of the research topics for prioritisation and the development of the prioritisation process; approved the final draft.
Chunhua Chi(https://orcid.org/0000-0003-3566-5582) (Professor of Family and Respiratory Medicine) led the delivery of the stakeholder engagement in each country; contributed to identification of the research topics for prioritisation and the development of the prioritisation process; approved the final draft.
Jaime Correia de Sousa(https://orcid.org/0000-0001-6459-7908) (Primary Care Professor) contributed to identification of the research topics for prioritisation and the development of the prioritisation process; approved the final draft.
Alexandra Enocson(https://orcid.org/0000-0002-4415-0989) (Breathe Well Programme Manager and Respiratory Practitioner) contributed to identification of the research topics for prioritisation and the development of the prioritisation process; approved the final draft.
Amanda Farley(https://orcid.org/0000-0002-4370-0264) (Epidemiologist) contributed to identification of the research topics for prioritisation and the development of the prioritisation process; approved the final draft.
Kate Jolly(https://orcid.org/0000-0002-6224-2115) (Professor of Public Health) contributed to identification of the research topics for prioritisation and the development of the prioritisation process; approved the final draft.
Sue Jowett(https://orcid.org/0000-0001-8936-3745) (Health Economist) contributed to identification of the research topics for prioritisation and the development of the prioritisation process; approved the final draft.
Mariam Maglakelidze(https://orcid.org/0000-0001-6513-0509) (Public Health Clinician) led the delivery of the stakeholder engagement in each country; contributed to identification of the research topics for prioritisation and the development of the prioritisation process; approved the final draft.
Tamaz Maghlakelidze(https://orcid.org/0009-0005-1237-8558) (Respiratory Consultant) led the delivery of the stakeholder engagement in each country; contributed to identification of the research topics for prioritisation and the development of the prioritisation process; approved the final draft.
Sonia Martins(https://orcid.org/0000-0001-5405-5064) (Primary Care Respiratory Physician) led the delivery of the stakeholder engagement in each country; contributed to identification of the research topics for prioritisation and the development of the prioritisation process; approved the final draft.
Zihan Pan(https://orcid.org/0000-0003-4502-1107) (Primary Care Clinician) led the delivery of the stakeholder engagement in each country (support); contributed to identification of the research topics for prioritisation and the development of the prioritisation process; approved the final draft.
Alice Sitch(https://orcid.org/0000-0001-7727-4497) (Medical Statistician) contributed to identification of the research topics for prioritisation and the development of the prioritisation process; approved the final draft.
Katrina Stavrikj(https://orcid.org/0000-0002-8319-4554) (Director of Centre for Family Medicine) led the delivery of the stakeholder engagement in each country; contributed to identification of the research topics for prioritisation and the development of the prioritisation process; approved the final draft.
Alice Turner(https://orcid.org/0000-0002-5947-3254) (Professor of Respiratory Medicine) contributed to identification of the research topics for prioritisation and the development of the prioritisation process; approved the final draft.
Siân Williams(https://orcid.org/0000-0002-0527-2254) (Public Health Manager and Chief Executive of the International Primary Care Respiratory Group) contributed to identification of the research topics for prioritisation and the development of the prioritisation process; approved the final draft.
Rachel E Jordan(https://orcid.org/0000-0002-0747-6883) (PI, Epidemiologist) conceptualised and designed the process and trained the research teams; contributed to identification of the research topics for prioritisation and the development of the prioritisation process; wrote the first draft of the paper (supervision); approved the final draft.
Acknowledgements
The investigator and collaborative team includes: Rachel E Jordan, Peymane Adab (CIs), Andrew P Dickens, Nicola Gale, Alexandra Enocson, Brendan G Cooper, Alice Sitch, Sue Jowett, Rafael Stelmach, Rachel Adams, Kar Keung Cheng, Chunhua Chi, Jaime Correia de Sousa, Amanda Farley, Kate Jolly, Mariam Maglakelidze, Tamaz Maghlakelidze, Zihan Pan, Katrina Stavrikj, Alice Turner and Siân Williams. We are grateful for support from our International Scientific Advisory Committee (Professor Debbie Jarvis, Faculty of Medicine, National Heart & Lung Institute, Imperial College London [Chair]; Dr Semira Manaseki-Holland, Institute of Applied Health Research, University of Birmingham; Professor David Mannino, College of Medicine, University of Kentucky; Professor Niels Chavannes, Department of Public Health and Primary Care, Leiden University Medical Center). We acknowledge work done by Jihong Kim (sociology master’s student) who conducted the initial data analyses, supervised by Dickens and Gale. We especially want to thank the stakeholders who participated in the described meetings and the research teams for their assistance, without whom this work would not have been possible. We gratefully acknowledge International Primary Care Respiratory Group (IPCRG) for introducing us to the primary care networks involved in this study and for its continued facilitation of clinical engagement. We would also like to acknowledge Radmila Ristovska (1955–2020), also involved in the initiation of this study.
Data-sharing statement
All available data are held securely and enquiries for access can be made through the corresponding author.
Ethics statement
This report is based on the description of a process of stakeholder consultation to establish research priorities, rather than a research study. As such, no ethical approval was sought.
Information governance statement
The University of Birmingham is committed to handling all personal information in line with the UK Data Protection Act (2018) and the General Data Protection Regulation (EU GDPR) 2016/679. Under the Data Protection legislation, The University of Birmingham is the Data Controller, and you can find out more about how we handle personal data, including how to exercise your individual rights and the contact details for our Data Protection Officer here: www.birmingham.ac.uk/documents/university/legal/university-of-birmingham-data-protection-policy.pdf
Disclosure of interests
Full disclosure of interests: Completed ICMJE forms for all authors, including all related interests, are available in the toolkit on the NIHR Journals Library report publication page at https://doi.org/10.3310/CTHF1385.
Primary conflicts of interest: Peymane Adab declares grant funding paid to her institution from NIHR, MRC and Colt Foundation, funding to her institution and to cover expenses as Chair of NIHR Public Health Research Funding Committee, Deputy Director of NIHR School for Public Health Research, funding to cover expenses as Member of Wellcome Trust Early Career Advisory Group in Population and Public Health, Output Assessor for Panel A, Subpanel 2 in 2021 Research Excellence Framework, and unfunded contributions as Chair for several NIHR funded TSCs, Member of the MRC funded Natural Experiments Evaluations Project Oversight Group, Member of Obesity Health Alliance Independent Obesity Strategy Working Group, Member of NIHR Palliative and End of Life Care Research Partnerships Call Panel, NIHR/UKRI Long COVID Funding Call Panel, NIHR COVID-19 Recovery and Learning Funding Committee; Jaime Correia de Sousa declares grant funding to his institution from AstraZeneca and GSK, advisory board and consulting fees paid to himself from Boheringer Ingelheim, GSK, AstraZeneca, Bial and Medinfar, payment for lectures from GSK, AstraZeneca and Sanofi Pasteur, support for attending meetings from Mundipharma and Mylan, leadership role for International Primary Care Respiratory Group (IPCRG); Amanda Farley declares grant funding paid to her institution from NIHR HTA, NIHR EME, MRC and Ethicon (Johnson and Johnson), membership on DMEC for NIHR funded e-cigarette trial (no honorarium), personal funding for leadership role for International Primary Care Respiratory Group (IPCRG) and as expert funding panel member for Cancer Research, UK;; Kate Jolly declares sub-committee chair of NIHR Programme Grants for Applied Health Research (payments to institution); Rachel E Jordan declares grant funding to her institution from NIHR, membership of Boehringer Ingelheim Primary Care Advisory Board, unfunded leadership role for International Primary Care Respiratory Group (IPCRG) – research sub-committee, membership of NIHR Global Health Group panel and RfPB COPD Highlight panel; Sue Jowett declares unfunded membership of the HTA Funding Committee Policy Group (formerly CSG) and the HTA Clinical Evaluation and Trials Committee; Alice Sitch declares grant funding to her institution from NIHR Birmingham BRC and AstraZeneca; Alice Turner declares grant funding to her institution from AstraZeneca and Chiesi, payment of honoraria from GSK and Boehringer, support for attending meetings and/or travel from AstraZeneca and Chiesi; Siân Williams declares her institution has received conference sponsorship and independent educational grants from Pfizer Global Medical Grants, AstraZeneca, GSK, Boehringer Ingelheim, Vitalograph, Caire Diagnostics and Thorasys; KK Cheng, Chunhua Chi, Andrew P Dickens, Alexandra Enocson, Nicola Gale, Tamaz Maghlakelidze, Mariam Maglakelidze, Sonia Martins, Katrina Stavrikj and Zihan Pan have no conflicts to declare.
Department of Health and Social Care disclaimer
This publication presents independent research commissioned by the National Institute for Health and Care Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, MRC, NIHR Coordinating Centre, the Global Health Research programme or the Department of Health and Social Care.
This article was published based on current knowledge at the time and date of publication. NIHR is committed to being inclusive and will continually monitor best practice and guidance in relation to terminology and language to ensure that we remain relevant to our stakeholders.
Funding
This research was funded by the National Institute for Health and Care Research (NIHR) Global Health Research programme (NIHR award ref: 16/137/95) using UK aid from the UK Government to support global health research. The views expressed in this publication are those of the author(s) and not necessarily those of the NIHR or the UK government.
This article reports on one component of the research award NIHR Global Health Research Group on Global COPD in Primary Care, University of Birmingham. Other articles published as part of this thread are: [LINKS to other articles]. For more information about this research please view the award page (https://www.fundingawards.nihr.ac.uk/award/16/137/95)
About this article
The contractual start date for this research was in June 2017. This article began editorial review in May 2023 and was accepted for publication in May 2024. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The Global Health Research editors and publisher have tried to ensure the accuracy of the authors’ article and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this article.
Copyright
Copyright © 2024 Dickens et al. This work was produced by Dickens et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaption in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.
List of abbreviations
- CAM
- Combined Approach Matrix
- CHNRI
- Child Health and Nutrition Research Initiative
- COPD
- chronic obstructive pulmonary disease
- HCP
- primary health care professionals
- IPCRG
- International Primary Care Respiratory Group
- LMIC
- low- and middle-income countries
- NIHR
- National Institute for Health and Care Research
- PR
- pulmonary rehabilitation programme
- RAPID-RP
- rapid research prioritisation approach
- STAR
- Socio-Technical Allocation of Resources
References
- Fleurence RL, Torgerson DJ. Setting priorities for research . Health Policy 2004;69:1-10. https://doi.org/10.1016/j.healthpol.2003.11.002.
- Crowe S, Fenton M, Hall M, Cowan K, Chalmers I. Patients’, clinicians’ and the research communities’ priorities for treatment research: there is an important mismatch. Res Invol Engag 2015;1.
- Tallon D, Chard J, Dieppe P. Relation between agendas of the research community and the research consumer. Lancet 2000;355:2037-40.
- Yoshida S. Approaches, tools and methods used for setting priorities in health research in the 21st century. J Glob Health 2016;6.
- Viergever RF, Olifson S, Ghaffar A, Terry RF. A checklist for health research priority setting: nine common themes of good practice. Health Res Policy Syst 2010;8.
- Rudan I, Gibson JL, Ameratunga S, El Arifeen S, Bhutta ZA, Black M, et al. Child Health and Nutrition Research Initiative . Setting priorities in global child health research investments: guidelines for implementation of CHNRI method. Croat Med J 2008;49:720-33.
- Dalkey N, Helmer O. An experimental application of the Delphi method to the use of experts. Manage Sci 1963;9:458-67.
- The James Lind Alliance . The James Lind Alliance Guidebook 2021. www.jla.nihr.ac.uk/jla-guidebook/ (accessed 18 March 2024).
- Ghaffar A, Collins T, Matlin SA, Olifson S. The 3D Combined Approach Matrix: An Improved Tool for Setting Priorities in Research for Health. Geneva: Global Forum for Health Research; 2009.
- Kapiriri L, Schuster-Wallace C, Chanda-Kapata P. Evaluating health research priority-setting in low-income countries: a case study of health research priority-setting in Zambia. Health Res Policy Syst 2018;16.
- Pickler RH, Tubbs-Cooley HL. Patient-centered outcomes research: a ‘new’ research agenda. J Pediatr Health Care 2014;28:101-4.
- Tembo D, Hickey G, Montenegro C, Chandler D, Nelson E, Porter K, et al. Effective engagement and involvement with community stakeholders in the co-production of global health research. BMJ 2021;372.
- Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 2013;13.
- Martins SM, Dickens AP, Salibe-Filho W, Albuquerque Neto AA, Adab P, Enocson A, et al. Accuracy and economic evaluation of screening tests for undiagnosed COPD among hypertensive individuals in Brazil. NPJ: Prim Care Resp Med 2022;32.
- Maglakelidze M, Kurua I, Maglakelidze N, Maglakelidze T, Chkhaidze I, Gogvadze K, et al. Feasibility of a pulmonary rehabilitation programme for patients with symptomatic chronic obstructive pulmonary disease in Georgia: a single-site, randomised controlled trial from the Breathe Well Group. BMJ Open 2022;12.
- Pang H, Pan Z, Adams R, Duncan E, Chi C, Kong X, et al. Community lung health service design for COPD patients in China by the Breathe Well group. NPJ: Prim Care Resp Med 2022;32.
- Pan Z, Dickens AP, Chi C, Kong X, Enocson A, G Cooper B, et al. Accuracy and cost-effectiveness of different screening strategies for identifying undiagnosed COPD among primary care patients (≥40 years) in China: a cross-sectional screening test accuracy study: findings from the Breathe Well group. BMJ Open 2021;11.
- Fernandes G, Williams S, Adab P, Gale N, de Jong C, de Sousa JC, et al. Engaging stakeholders to level up COPD care in LMICs: lessons learned from the ‘Breathe Well’ programme in Brazil, China, Georgia, and North Macedonia. BMC Health Serv Res 2024;24.
- Airoldi M, Morton A, Smith JAE, Bevan G. STAR – people-powered prioritization: a 21st-century solution to allocation headaches. Med Decis Mak: Int J Soc Med Decis Mak 2014;34:965-75.
- Cowan K, Fulop NJ, Harshfield A, Ng PL, Ntouva A, Sidhu M, et al. Rapid prioritisation of topics for rapid evaluation: the case of innovations in adult social care and social work. Health Res Pol Syst 2021;19.
Appendix 1 Example of a lay summary vignette
A study to find the most efficient screening test strategy for identifying new cases of COPD
‘Chronic obstructive pulmonary disease (or COPD) is a very common condition across the world. It is a lung disease caused mainly by smoking or exposure to indoor or outdoor air pollution. Patients with COPD do not recover and many suffer ill health and many admissions to hospital. However, we also know that at least half of people with the disease do not know they have it, and therefore we could try to identify them earlier and give them appropriate care. We could offer screening, or testing, to many people to try to find those affected who don’t yet know it. There are several different tests we can use, such as offering different symptom questionnaires and simple breathing tests on a large scale to people who might be at risk. But we do not know which test or combination of tests is the best, or the best value for money, particularly in lower resource countries. We could plan a study to compare all of these test. With the results, we could identify the best and most cost-effective approach to use to identify new patients.’
Appendix 2 Participant rating form
Breathe Well research prioritisation: rating
Appendix 3 Participant ranking form
Breathe Well prioritisation: ranking
| Name: | Please circle: patient/clinician/policy-maker |
| Job role: | Date: |
In the previous session, you were asked to rate the importance of each proposed research question.
Now, with the previous exercise in mind, please rank all studies in priority order for research. Mark the highest priority with a ‘1’ and the lowest priority with a ‘10’.
| Please rank all studies (1 = highest priority, 10 = lowest priority) |
|---|
| Prevention |
| 1. A study to explore the views of patients and healthcare professionals on the use of e-cigarettes for quitting smoking 2. A study to assess the additional effectiveness of communicating ‘lung age’ to adult smokers in primary care to promote smoking cessation |
| Diagnosis/screening |
| 3. A study to find the most efficient screening test strategy for identifying new cases of COPD 4. A study to assess the effectiveness of treatment for people with newly case-found COPD |
| Management |
| 5. A study to explore the views of patients, healthcare professionals and policy-makers on the barriers and enablers to good-quality COPD management 6. A study to assess the feasibility of a new pulmonary rehabilitation programme adapted to local needs in low-resource settings |
| 7. A study to assess whether an adapted pulmonary rehabilitation programme improves outcomes in patients with case-found and mild COPD, in settings where medication is suboptimal 8. A study to assess whether a weight management programme improves dyspnoea and quality of life among COPD patients |
| 9. A study to assess whether a simple behavioural intervention to promote handwashing and reduce transmission of infections can reduce exacerbations among COPD patients 10. A study to explore how to improve clinical education about COPD for primary care staff |
Appendix 4 Research team rating form
Breathe Well research prioritisation: research team scoring (Part 1)
| Name: | Group session facilitated (please circle): Patients Clinicians Policy-makers Clinicians/policy-makers (combined) |
| Date: |
Please score each of the proposed research studies in three different categories: local importance, feasibility and equity. The table below gives some guidance on what the categories could include from the viewpoint of patients, clinicians and policy-makers:
| Important: In an ideal world, how important is this? Is this a priority? | Feasible: In the real world, how feasible/acceptable is this idea? | Equitable: Will this benefit all patient groups? Will it be accessible to all? | |
|---|---|---|---|
| Patients | Would this make your quality of life better? Is this important to you? Is this a short- or long-term priority? | Would you realistically be able to participate in this kind of project? Or in the longer term would you be able to participate in the proposed service? | Would you and other patients all be able to access this service and participate in the project? Would it be of benefit to all patients who took part? |
| Clinicians | Would you feel you could provide a better service? Would it improve your professional satisfaction? | Would this be technically feasible? How complex would implementation be? Would this work within your relationship with patients? | Could you set up this project and the service in locations which would reach all types of patients? |
| Policy-makers | How does this compare in importance to other international, national and local goals? | Are the resources available for this? Are the knowledge and skills available for this? Is the workforce available for this? | Will this lead to benefit for all eligible patients? Will it help to reduce health inequalities? |
Please rate the below research studies on a score from 0 to 5, where 0 = not at all and 5 = very much for each category.
| Research study | 0 = Not at all, 5 = Very much | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Important | Feasible | Equitable | ||||||||||||||||
| Prevention | ||||||||||||||||||
| 1. A study to explore the views of patients and healthcare professionals on the use of e-cigarettes for quitting smoking | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| 2. A study to assess the additional effectiveness of communicating ‘lung age’ to adult smokers in primary care to promote smoking cessation | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| Diagnosis/screening | ||||||||||||||||||
| 3. A study to find the most efficient screening test strategy for identifying new cases of COPD | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| 4. A study to assess the effectiveness of treatment for people with newly case-found COPD | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| Management | ||||||||||||||||||
| 5. A study to explore the views of patients, healthcare professionals and policy-makers on the barriers and enablers to good-quality COPD management | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| 6. A study to assess the feasibility of a new pulmonary rehabilitation programme adapted to local needs in low-resource settings | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| 7. A study to assess whether an adapted pulmonary rehabilitation programme improves outcomes in patients with case-found and mild COPD, in settings where medication is suboptimal | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| 8. A study to assess whether a weight management programme improves dyspnoea and quality of life among COPD patients | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| 9. A study to assess whether a simple behavioural intervention to promote handwashing and reduce transmission of infections can reduce exacerbations among COPD patients | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| 10. A study to explore how to improve clinical education about COPD for primary care staff | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| Please write any comments here (continue overleaf): | ||||||||||||||||||
Breathe Well research prioritisation: research team scoring (Part 2)
| Name: | Group session facilitated (please circle): Patients Clinicians Policy-makers Clinicians/policy-makers (combined) |
| Date: |
Please score each of the proposed research studies in three different categories:
-
Novelty of research study internationally
-
Potential for future funding (local or international)
-
Capacity building.
Please rate the below research studies on a score from 0 to 5, where 0 = not at all and 5 = very much for each category.
| Research study | 0 = Not at all, 5 = Very much | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Novelty | Future funding | Capacity building | ||||||||||||||||
| Prevention | ||||||||||||||||||
| 1. A study to explore the views of patients and healthcare professionals on the use of e-cigarettes for quitting smoking | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| 2. A study to assess the additional effectiveness of communicating ‘lung age’ to adult smokers in primary care to promote smoking cessation | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| Diagnosis/screening | ||||||||||||||||||
| 3. A study to find the most efficient screening test strategy for identifying new cases of COPD | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| 4. A study to assess the effectiveness of treatment for people with newly case-found COPD | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| Management | ||||||||||||||||||
| • A study to explore the views of patients, healthcare professionals and policy-makers on the barriers and enablers to good-quality COPD management | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| 6. A study to assess the feasibility of a new pulmonary rehabilitation programme adapted to local needs in low-resource settings | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| 7. A study to assess whether an adapted pulmonary rehabilitation programme improves outcomes in patients with case-found and mild COPD, in settings where medication is suboptimal | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| 8. A study to assess whether a weight management programme improves dyspnoea and quality of life among COPD patients | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| 9. A study to assess whether a simple behavioural intervention to promote handwashing and reduce transmission of infections can reduce exacerbations among COPD patients | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| 10. A study to explore how to improve clinical education about COPD for primary care staff | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 | 0 | 1 | 2 | 3 | 4 | 5 |
| Please write any comments here (continue overleaf): | ||||||||||||||||||
Appendix 5 Research team ranking form
Breathe Well prioritisation: research team ranking
| Name: | Group session facilitated (please circle): Patients Clinicians Policy-makers Clinicians/policy-makers (combined) |
| Date: |
In the previous session, you were asked to rate the importance of each proposed research question.
Now, with the previous exercise in mind, please rank all studies in priority order for research. Mark the highest priority with a ‘1’ and the lowest priority with a ‘10’.
| Please rank all studies (1 = highest priority, 10 = lowest priority) |
|---|
| Prevention |
| 1. A study to explore the views of patients and healthcare professionals on the use of e-cigarettes for quitting smoking |
| 2. A study to assess the additional effectiveness of communicating ‘lung age’ to adult smokers in primary care to promote smoking cessation |
| Diagnosis/screening |
| 3. A study to find the most efficient screening test strategy for identifying new cases of COPD |
| 4. A study to assess the effectiveness of treatment for people with newly case-found COPD |
| Management |
| 5. A study to explore the views of patients, healthcare professionals and policy-makers on the barriers and enablers to good-quality COPD management |
| 6. A study to assess the feasibility of a new pulmonary rehabilitation programme adapted to local needs in low-resource settings |
| 7. A study to assess whether an adapted pulmonary rehabilitation programme improves outcomes in patients with case-found and mild COPD, in settings where medication is suboptimal |
| 8. A study to assess whether a weight management programme improves dyspnoea and quality of life among COPD patients |
| 9. A study to assess whether a simple behavioural intervention to promote handwashing and reduce transmission of infections can reduce exacerbations among COPD patients |
| 10. A study to explore how to improve clinical education about COPD for primary care staff |
| Please write any comments here: |
Appendix 6
FIGURE 4.
Stakeholder ratings, by country. 1 = e-cigarettes; 2 = lung age; 3 = screening tests; 4 = early treatment; 5 = quality management; 6 = PR feasibility; 7 = PR outcomes; 8 = weight management; 9 = handwashing; 10 = HCP education.
