An evidence base to optimise methods for involving patient and public contributors in clinical trials: a mixed-methods study

Article history

The research reported in this issue of the journal was joint funded by the HS&DR programme or one of its preceeding programmes and INVOLVE as project number 10/2001/29. The contractual start date was in October 2011. The final report began editorial review in June 2014 and was accepted for publication in March 2015. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Dr Hanley reports personal fees from the Medical Research Council and personal fees from User Involvement in Voluntary Organisations Shared Learning Group, outside the submitted work.

Copyright statement

© Queen’s Printer and Controller of HMSO 2015. This work was produced by Gamble et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

What is patient and public involvement?

Public involvement in research is described as research being carried out ‘with’ or ‘by’ members of the public rather than ‘to’, ‘about’ or ‘for’ them. 1 The role of patients and the public therefore extends far beyond that of a research ‘subject’ or participant. Increased recognition that patients and public are stakeholders in research has led to increasing calls that they be represented within that research process, resulting in a growth of patient and public involvement (PPI) in health research both nationally and internationally, and also within the peer review process of that research. 2–4 This includes the USA, where it is known as stakeholder engagement, and Australia, where it is termed consumer and community participation.

It has been suggested that clinical trials are particularly likely to benefit from PPI. 5,6 Health research funding bodies strongly encourage researchers to implement PPI at every stage of the research process and specifically to include PPI contributors on Trial Steering Committees (TSCs). 7–12 Assimilation of PPI into grant applications is therefore becoming commonplace, with clinical trial funding bodies requiring that plans for PPI be submitted by investigators to ensure trial participants’ needs are respected, and to maximise research quality and relevance. 7,12–15

When should patient and public involvement start?

Patient and public involvement can start at various stages of a trial and may influence many aspects. The ability to have an impact on a trial has to be considered in line with the opportunity to exert influence. Staniszewska et al. 16 discuss the importance of PPI in the design of research to optimise its impact and relevance. During the design stages of a trial, many decisions are made that determine the relevance and conduct of the proposed research: the precise specification of the research question including the outcomes to be measured; visit schedules; methods of data collection; and recruitment and consent procedures. Fudge et al. 17 suggest that decisions made by professional researchers at the outset of a study have a cumulative and significant influence on the potential for PPI to have an impact on a study and that involvement is more difficult to achieve once studies are under way. It is therefore important to consider the stage of the trial at which PPI begins alongside the process of involvement when identifying impact.

Boote et al. 18 reviewed published case examples that focused solely on PPI at the design stage of primary health research; they identified just six peer-reviewed journal articles reporting on PPI in the development of a clinical trial. The PPI methods entailed group discussions. Although the methods of PPI may not be considered representative of the diverse nature of PPI, suggesting the presence of selective reporting, the key contributions identified have been reported from other models of involvement: review of patient information sheets and consent procedures; suggestion of outcome measures; review of acceptability of data collection procedures; and recommendations on the timing of potential participants being approached for the study and timing of follow-up. 5 To understand how frequently these contributions occur and the factors associated with their occurrence there is a need to investigate PPI in an unselected cohort of trials.

The UK Department of Health guidelines, Best Research for Best Health,7 state that PPI must be included in all stages of the research process including priority setting, defining research outcomes, selecting research methodology, patient recruitment, interpretation of findings and dissemination of results.

The National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme has encouraged PPI, previously asking researchers to consider the benefits of its incorporation and now requesting evidence of PPI from researchers submitting their proposals. Oliver and Gray19 assessed the impact of public involvement in the HTA research-commissioning programme; however, the impact of public involvement in the research funded by the HTA programme is yet to be assessed.

How should patient and public involvement be implemented?

Challenges to the realisation of plans for PPI include debate regarding its purpose, lack of evidence regarding the impact of PPI, complexities in researchers and contributors sharing power, and difficulties in ensuring sufficient resources for PPI. 5,15,18,20–22 Alongside such challenges are uncertainties regarding how best to plan PPI. Guidance drawing on the opinions and experiences of those involved in PPI activity within trials is available21,23 and a 2011 review examined case studies of PPI in the design and conduct of trials. 6 However, the evidence base is limited in terms of the range of trials, researchers and patients that have informed this previous work, and there has been no systematic evaluation of the extent to which triallists’ intentions for PPI are put into practice.

Little is known about how or when researchers incorporate PPI in clinical trials, or what impact may stem from that involvement. There are indications that PPI can have favourable impacts upon every stage of the research process5,24–28 by helping to ensure that research funds are appropriately prioritised and that research evidence is relevant to patients, by improving recruitment and retention rates and by supporting the uptake of research in practice. Indications that PPI may have unfavourable impacts upon research24,29 or no impact at all30 have also appeared. In this intensely moral and political arena, the rarity of such reports has raised concerns that the benefits of PPI have been selectively reported. 5,31 Concerns have been expressed that the existing literature is selectively reported to make the case for or against PPI, with many reports aiming to make the case or convince the sceptics about PPI, and these concerns have led to questions about the quality of the evidence base for PPI in trials. Furthermore, as much reporting has involved single case studies, generalisability of the PPI literature is limited and may provide a misleading account of how PPI is implemented and its impact. 31 Crucially, these problems make it difficult to predict what type of involvement is most effective and where.

The relationship between the nature of involvement and the control of PPI contributors in the decision-making of the research process has been debated, with higher levels of control often being considered as of higher quality, and lower levels of control being criticised as tokenistic. 32 However, this approach has been critiqued. 33 For example, while it is often assumed that approaches that are described as limited, tokenistic or of low quality are ineffective, they may still achieve valuable impacts; conversely, those considered to be better models of involvement might not.

Who can be a patient and public involvement contributor?

The NIHR states:34

We need people with everyday experience of health, education, social care or services delivered in your home or near where you live. We often look for people who have experience of specific health services as a patient or carer, and who have an interest in research. We welcome individuals and representatives of voluntary organisations and patient groups.

However, there is debate about whether or not it is acceptable for individuals employed within a medical or research capacity to provide PPI. Current NIHR HTA guidance states that ‘To achieve the aim of bringing fresh eyes to the work of the HTA programme a patient or member of the public should not normally be a health practitioner, manager or researcher’. 35 Concerns have also been raised about PPI contributors becoming professionalised. 36 This may happen either as a result of contributing across a number of separate research projects or as a result of their role in facilitating or supporting PPI contributors within the remit of their employment, and may be extended to those undertaking leading roles in charities and patient organisations. The debate on the professionalisation of PPI contributors also has implications for training provisions.

Patient and public involvement contributors may be selected because their attributes or experience are considered to strengthen their ability to contribute to their role in the research. Others may be selected based on the perception of their ability to ‘represent’ the wider population of interest. Little is known about how PPI contributors are identified or the selection process used by researchers. This may affect the training and support needs of the PPI contributor in fulfilling their role.

Whereas the evidence on PPI activity in research is expanding, PPI training has received little research attention. Training demands time and resource, and also has potential to shape the future conceptualisation, implementation and impact of PPI in research. INVOLVE, the UK-based advisory body on PPI in health and social care research, reports that most PPI training courses have been developed within particular organisations or in the context of individual research projects. 37 It defines training broadly as any activity ‘that aims to help members of the public and researchers develop their knowledge, skills and experience to prepare them for public involvement in research’ (p. 5). 37 An examination of training and educational provision of PPI in research confirms the diversity of aims, content and delivery of training. For example, education and training for PPI contributors ranges from a year-long formally assessed and certificated course on the discovery, testing and evaluation of medical products and technologies,38 to one-day informal workshops to help contributors identify suitable research roles and build confidence. Examples of training for researchers are almost as variable, ranging from formal modules on the theory, policy and current practice of PPI within accredited master’s courses39,40 to single ‘awareness raising’ workshops on the aims and implementation of PPI in research. 41

Although this diversity of training may be appropriate, it raises questions about how to ensure training is fit for purpose. INVOLVE proposes that training be provided for both PPI contributors and researchers,37,42 tailored to their needs and roles, delivered on an ongoing basis and in ways that allow contributors and researchers to learn from each other. 37 These principles were drawn from consultations with over 30 stakeholders who had direct experience of PPI training either as providers or recipients. However, few details of the methods of consultation are available and little is known about the perspectives of those researchers and PPI contributors who have not participated in training. A key consideration for any training is that it engages with the diversity of learners’ needs and is meaningful from their perspective. 43 Insights on how members of the clinical trials community perceive PPI training, regardless of whether or not they have had prior experience of training, will help to ensure its relevance and uptake.

Should we assess impact of patient and public involvement?

Robust evidence on the effectiveness of PPI in research is absent. It has been argued that PPI in research is ‘the right thing to do’ and should occur irrespective of impact. 44 However, incorporating PPI within research requires time and resource,5 so we should be expected to learn from both the positive and negative experiences of researchers and PPI contributors to determine facilitators and barriers to impact for the benefit of future research. 45

For PPI contributors, getting involved in research has been reported to lead to ‘personal development’ such as boosting confidence, empowerment and a sense of purpose. 46 Similarly, there can be personal benefits for researchers, who have reported that their attitudes, values and beliefs about the worth of PPI had been heightened as a result of such involvement. 20 However, as well as being a vehicle for improving research validity, there are indications that ‘patient influence’ can pose a potential threat to the validity of research if it is not drawn upon appropriately. 14 For example, PPI in technical decisions may result in worse as opposed to improved project outcomes. 47

It is important that accounts of researchers and PPI contributors be accessed in establishing an evidence base to guide future approaches to the implementation of PPI. Each brings different perspectives and, consequently, the two parties may differ in their views of how PPI impacts on trials. This position is supported by the evaluation of PPI in the UK Clinical Research Collaboration (UKCRC). 48 Indeed, much of the existing literature looks at the experience of PPI representatives or advisory groups, but there is less research on the experiences of professional researchers and clinical trials units (CTUs). An examination of the experience of PPI from all perspectives is needed to strengthen understanding and develop a more robust evidence base for future implementation.

A national questionnaire survey on the role of PPI in designing, conducting and interpreting randomised trials managed by clinical trial coordinating centres concluded that PPI was still uncommon. 49 Since the publication of this survey there have been many changes in the clinical research environment, including those brought about by the establishment of the UKCRC in 2004 and the UKCRC Registered Clinical Trials Units (RCTUs) in 2007. 50 RCTUs are assessed as having the expertise necessary to ensure high-quality, successful and timely trials, and to meet regulatory and governance requirements. There is limited knowledge about the engagement of RCTUs with PPI contributors, and challenges to early PPI for trials competing for public funding have been identified. These include the short time frame for completion of applications, the lack of resources to support PPI and difficulties in identifying appropriate PPI contributors. It is expected that a growing proportion of publicly funded clinical trials will be co-ordinated via a RCTU. Therefore, there is an important need for new work to be conducted within the network of RCTUs to explore the current role they have in determining the process and quality of PPI, and to aid strategic planning for future practices drawing strength across RCTUs.

How should the impact of patient and public involvement be assessed?

The assessment of impact is difficult because of the complexity of PPI. Problems with the conceptualisation and measurement of the impact of PPI have also been identified,51 and few studies have accessed the perspectives of both PPI contributors and researchers. Moreover, much of the literature on the impact of PPI in research has not focused specifically on randomised trials, although these are regarded as particularly likely to benefit from PPI. 5

In assessing impact stemming from PPI in clinical trials, there is a need to consider the empirical evidence on how PPI was actually implemented in its broadest form. For example, direct impact may be observed in terms of suggested improvements to patient information sheets, logistics or the visit schedule, which in turn may be thought to lead to improved recruitment. Direct impact may be observed in relation to the choice of outcomes to be measured, either by suggestions to include outcomes otherwise considered unimportant by health professionals or by suggestions to improve the likely completion rate of participant questionnaires. In turn, these may lead respectively to research that patients are more likely to use to help them make decisions and to research of higher quality. Impact may also be less direct, for example, with PPI providing an opportunity for dialogue between researchers and PPI contributors; the improved awareness of patient perspectives may not only help to guide researchers’ attention towards clinical problems that are most relevant to patients, but also drive their motivation to address these problems.

Despite its importance, there is a lack of well-accepted approaches to the assessment of the impact of PPI in health research. 5,52,53 Staniszewska54 commented that the varied methods used to assess and report the impact of involvement caused difficulties when trying to synthesise evidence across studies. More recently a Public Involvement Impact Assessment Framework (PiiAF)55 has been developed to help researchers at the beginning to identify the issues that could affect the impacts public involvement can have on their research and to develop an approach to assessing these impacts during their research.

Aim

The overall aim of this project, known as EPIC (Evidence base for Patient and public Involvement in Clinical trials), is to increase knowledge of PPI within randomised trials by:

1. establishing an unselected evidence base of how PPI has been implemented within randomised trials

2. identifying associated impact to inform the future optimisation of PPI by systematically describing and critically evaluating the process, challenges and impact of PPI from the perspectives of the PPI representative, chief investigator (CI) and CTU staff.

Establishing the cohort

The cohort was identified as randomised trials that were actively receiving funding from the NIHR HTA programme between 2006 and 2010. The cohort included randomised trials at different stages, from recently funded applications to randomised trials that had reached the final report stage, providing data on PPI across all stages of the research process.

The NIHR HTA programme has a two-stage application process (Figure 1). In summary, the outline application is considered by the funding board and applicants are asked to address feedback from the board if a full application is requested. The full application is sent for external peer review and considered by the board to determine if it should be rejected or any changes made prior to funding. The full application consists of a completed application form and a detailed project description.

FIGURE 1.

National Institute for Health Research HTA programme application process.

We requested all available documentation relating to the application process. This comprised outline applications; the minutes of the board meetings at which the outline applications were considered and which contained feedback for the applicants to consider in submitting the full application; the full application form and the detailed project description; external referee reports; and the minutes of the board in which the final decision on funding was made. Prior to the release of these documents the NIHR HTA programme contacted the CIs of the trials involved, informing them of the intention to release their names, which were readily available within the public domain. We signed a confidentiality agreement and the NIHR HTA programme redacted sensitive information regarding budget information and names of coapplicants (not available within the public domain) before releasing the documentation.

Phase 1: documentation, data extraction and coding

A Microsoft Access^® version 2010 (Microsoft Corporation, Redmond, WA, USA) database was developed to assist in extracting and analysing the information on PPI within the applications. Each application was given a unique identifier within the database and this identifier, rather than the NIHR unique identifier, is used to maintain confidentiality throughout. A PPI advisory group of five members with experience of providing PPI in randomised trials was established for this project (see Appendix 1). The advisory group commented upon the data extraction tool and made recommendations for changes. Data were extracted to characterise the cohort and to describe PPI activity within the two-stage application process and plans for involvement once trial funding was secured. Trial characteristics linked to trial complexity, or thought to be barriers or facilitators to recruitment,56 were also extracted. In brief, the extracted data included characteristics of the trial design and setting, disease or condition under study, type of intervention, participant characteristics, recruitment setting and any text that described or was relevant to PPI. Data were extracted by three reviewers (LD, JP, CG). Extracted text was anonymised by replacing any identifying details with a general term in brackets [term], or using [. . .] to indicate removed text.

Text extracts describing PPI were examined to determine the stage of its actual or planned initiation; for example, the outline application (submitted in the first stage of the application process) may have specified that PPI was planned to occur during the development of the full application (submitted in the second stage of the application process). The stages of initiation were the development of the outline application; the development of the full application; and following a positive funding decision or during the trial.

The text was also examined to determine the role of the PPI contributor’s input. This was categorised as managerial, responsive or oversight. We categorised PPI contributors as managerial if they were described as coapplicants or involved in the management of the trial, for example a member of the Trial Management Group (TMG). We categorised PPI contributors as having a responsive role if descriptions of their input were largely confined or targeted towards a particular aspect of the application or trial, or if PPI contributions were on an ‘as required’ basis. An oversight role was defined by appointment as an independent member of either the TSC or the Data Monitoring Committee (DMC). Descriptions of PPI in the application documentation were often limited, and coding ‘rules’, informed by our knowledge of grant application development and clinical trial implementation processes, were devised to categorise the descriptions. The codes were developed by CG and LD after reading the PPI descriptions and then reviewed by JP. CG and LD independently categorised the PPI descriptions. Disagreements between CG and LD were discussed and agreement reached by referring back to the documentation. All categorisations were cross-checked by JP. The coding rules (Box 1) and the classifications were reviewed by the PPI advisory group and no changes were suggested.

The external referee assessment forms that were completed by each referee began requesting referees to comment on any aspect of the proposal that they considered relevant from the perspectives of patients or service users. In addition, within a section on resources and feasibility, referees were asked to consider whether or not there is ‘appropriate representation from all relevant groups in the research team (this might include consumers, researchers from different disciplines, managers and professionals) and is the role of each collaborator/co-applicant clear (particularly important for multi-centre studies)?’ All areas of the form were examined for references to referee assessment of PPI.

Comments from external reviewers and the funding board were coded as positive, negative or factual. Positive comments implied that the referee or board considered the PPI to be satisfactory or sufficient, and did not contain suggestions for adding to the existing PPI plans. Negative comments were those which indicted that PPI was considered weak or unclear, or gave direction on strengthening PPI from that proposed within the application. A comment may indicate the absence of PPI in an area, such as membership of TMG or TSC, but not specifically indicate that this was required. Where this occurred it was interpreted as a negative comment in that it was aiming to highlight a gap in the approach to PPI. Factual statements were those which outlined the importance of service users without commenting on the plans proposed, or identified PPI plans without any indication of the respondent’s view on their appropriateness.

Phase 1 analysis

In considering trends over time, the year the outline application was submitted was used. The cohort was identified as randomised trials that were in receipt of funding from the NIHR HTA programme between 2006 and 2010, but the year the outline applications were made ranged from 2003 to 2008.

The following specific conditions were selected to consider in more detail: mental health, pregnancy and childbirth, human immunodeficiency virus/acquired immunodeficiency syndrome, cancer, stroke, paediatrics, diabetes, and dementias and neurodegenerative diseases. These were identified either by their strong history of PPI or by the establishment of a NIHR clinical research network for a specific condition. 57

Categorical data were summarised using descriptive statistics with numbers and percentages. Chi-squared tests or Fisher’s exact tests were used as appropriate.

Phases 2 and 4: surveys of chief investigators, patient and public involvement contributors, and UK Clinical Research Collaboration Registered Clinical Trials Units

Three surveys were planned as part of the EPIC project targeting CIs, PPI contributors and the network of RCTUs. Each survey was web-based and developed in SurveyMonkey (www.surveymonkey.com), with portable document format (PDF) versions available from the EPIC website. Each survey consisted of both closed and open questions to avoid constraining responses. In addition, further free text was collected when ‘other’ was selected as a closed response. The surveys were initially developed by LD and CG and sent for comments to BY, JP and PRW prior to being considered by the PPI Advisory Group. Surveys were piloted within the Clinical Trials Research Centre, University of Liverpool, prior to being finalised. The CI and PPI survey questions targeted opinions and motivations about PPI, methods of engagement, areas of contribution and level of impact within the cohort of trials. The RCTU survey questions focused on the experience and processes of the RCTUs across trials rather than within the cohort.

The link for the CI survey was emailed to the CI of each trial within the cohort. The e-mail addresses of each CI were obtained from the funding application but were checked against web searches on the CI names to ensure that they were up to date. The invitation e-mail described EPIC in brief, explained why they were being contacted and referred to the initial contact made by the NIHR HTA programme about the project. E-mails to the CIs also contained a link to a website which contained PDF versions of the survey and information sheets. The CI surveys were sent out on 13 March 2013 and two reminder e-mails were sent to non-responders on 17 April 2013 and 28 June 2013. Non-responders known to members of the EPIC research team were contacted to encourage completion.

Names of the PPI contributors for each trial were not available in the public domain. To obtain their contact details we requested the CI of each trial to contact their PPI contributor(s) asking them to contact the EPIC team so that we could send them information about the project and the PPI survey. An additional e-mail was sent to the trial CIs reminding them to contact their PPI contributors about EPIC. In addition an advert was drafted by JP and sent to the PPI Advisory Group for comments. The advert was placed on the websites of Involving People and North West People in Research. Finally we contacted the NIHR HTA programme to ask if it could contact the PPI contributors to inform them about EPIC; however, PPI contact details were not held by the NIHR HTA programme. The NIHR HTA programme contacted the chair of each of the TSCs asking them to contact any PPI contributors known to them.

The directors of the 46 RCTUs were contacted by e-mail requesting them to complete the CTU survey. The names of the fully and provisionally registered CTUs were obtained from the UKCRC website58 following the publication of the results of the 2012/13 Review Process. Websites for each RCTU were identified and contact details of the directors obtained. The survey could have been circulated on our behalf by the UKCRC across the directors using the group UKCRC directors e-mail list, but it was hoped a personal e-mail would encourage response. The initial e-mail was sent on 27 March 2013. The e-mail contained a brief summary of EPIC and the purpose of the survey along with links to the EPIC website, which contained a PDF version of the survey. The RCTU directors were asked to complete or delegate completion of the survey, which targeted PPI processes across trials in their units rather than focusing on individual trials. Reminder e-mails were sent on 29 April 2013 and, if it was unclear whether or not contact had been made, we used their websites to identify an alternative senior person within the RCTU.

Analysis of phase 2 and phase 4 surveys

All surveys were analysed using descriptive statistics, and chi-squared tests were used for cross-tabulations between questions. Recurring themes were identified within the free-text responses and used to group responses provided.

Phase 3: interviews

Within the CI and PPI surveys, respondents indicated if they were willing to be contacted to take part in an interview to further explore their experiences of PPI within the trial.

We initially sampled CIs for maximum diversity based on their survey responses, although we eventually invited all but three of the CIs who had responded to the survey and indicated their willingness to be interviewed. We invited for interview all PPI contributors who returned a survey response and indicated their willingness to take part. Additionally, we invited TMs for all trials for which the CI or PPI contributor had been interviewed. We obtained contact details for TMs from CTUs, trial websites and protocols, or via CIs.

We contacted all potential informants by e-mail and provided an information leaflet inviting them to contact the EPIC research associate to arrange an interview. Non-responders were sent one reminder e-mail. We expected that some PPI contributors might access their e-mail accounts infrequently so we subsequently telephoned those who had not responded. All informants provided signed or audio-recorded verbal consent before being interviewed.

A psychologist, LD, conducted audio-recorded semistructured telephone interviews with informants between April 2013 and November 2013. Before starting interviews, she explained that study data would be anonymised and kept confidential. Interviewing was conversational to allow informants to voice their views and experiences of PPI freely. In order to minimise the risk of idealised accounts, LD adopted a neutral stance in her interviewing. This was to avoid creating a sense that informants had to justify or defend their approach to PPI, which might have inhibited or coloured their accounts. LD familiarised herself with each of the documents for each trial before interviews to tailor questions to specific aspects of the trial. Nevertheless, we used topic guides to steer the interviews (see Appendix 2, Topic guides). We developed three versions to ensure interviews were appropriate for each of the three informant groups (CI, PPI contributor and TM), although the topic guides mirrored one another to ensure core topic areas were explored. Table 1 provides summary topic guides for researchers and PPI contributors. Topic guides were informed by the previous literature, reviewed by EPIC team members and the PPI advisory group, and developed in the light of the ongoing data analysis. In addition, the PPI advisory group read transcripts from early interviews and fed back on the interview with implications for the topic guide. Interviewing paralleled the analysis and continued until theoretical saturation had been reached,59 and additional data ceased contributing to the analysis. Interviews were transcribed using an ‘efficient’ verbatim style that involved transcribing the content of informants’ accounts, rather than detailed features of speech such as subvocalisations and duration of pauses and hesitations. All transcripts were checked for accuracy and anonymised.

This qualitative workstream of EPIC allowed us to access CIs’ and PPI contributors’ accounts of PPI in their own words and to analyse them inductively. Given the moral and political expectations surrounding PPI, we thought it was particularly important to adopt an interpretive approach60,61 and consider how informants talked about PPI. Therefore, we focused on the language informants used to describe PPI and on the aspects of PPI they gave little emphasis to in their interviews, as well as what they emphasised. Before each interview we reviewed the documents for each trial on their PPI plans, in order to tailor our questions and identify particular lines of enquiry to pursue. The interviews enabled us to seek clarification and prompt informants to elaborate on their experiences and perspectives. Similarly, informants were able to seek clarification from us, to elaborate on their perspectives and to raise topics that they considered important which we had not foreseen.

Phase 3 analysis

Analysis was informed by the principles of the constant comparative method62,63 with elements of content analysis. 64 We used procedures to support rigour in qualitative research65 and, as we note above, our approach was interpretive. To ensure a contextualised analysis we referred to transcripts as a whole as well as to particular data segments. We initially analysed CI and PPI contributor transcripts at the informant group level for evidence of their beliefs and experiences about the process and impact of PPI as well as their views and experiences of training and support for PPI. Subsequently, we triangulated CI and PPI contributor transcripts within each trial, before comparing them with the TM transcripts within each trial. Where trials did not have a full data set (i.e. did not include all three groups of informants), we compared the two available accounts. Where only one account was available, analysis was at the informant group level.

The analysis was led by LD, who read CI and PPI contributor transcripts several times before developing open codes. BY also read multiple transcripts, and she and LD met regularly to compare interpretations of the data and review the ongoing analysis. Open coding took place at multiple levels, from line-by-line coding of detailed descriptions to the general stance informants took towards PPI. Open codes were grouped into categories and organised into a framework. Coding and indexing of data was assisted by NVivo 9 software (QSR International, Warrington, UK) and we continually compared categories with new data and amended them to ensure that they reflected the data while accounting for deviant cases. For TM transcripts we conducted some open coding of sections relevant to the categories emerging from the CI and PPI contributor analysis. Subsequent discussion and review of detailed analysis reports by other members of EPIC team including DB, who led an analysis of the same data set on the implementation of PPI, helped to refine the analysis and corroborate the findings. To evidence our interpretations we present illustrative extracts from the data. Extract codes indicate informant group (CI; PPI contributor 1 or 2, where more than one were interviewed for the same trial; TM) and trial identification numbers.

To compare what PPI actually happened in the trial with that planned within the application (implementation of PPI), the following primary and secondary data sources were used.

Primary sources of data were trial documentation (full application forms, reviewer comments, detailed project descriptions and study protocols), from which we extracted data about trial teams’ plans for PPI; and CI and PPI contributor interview transcripts, from which we determined whether or not the documented plans were implemented.

Secondary sources of data were outline application forms, CI survey responses and TM interview transcripts. We used the secondary sources in cases of ambiguity, that is where it was unclear from the primary sources whether or not aspects of a particular set of plans had been implemented. We also used the secondary sources to elucidate the illustrative examples that we present in the results.

The implementation of PPI analysis, led by DB, used a thematic analysis approach to analyse the interview data regarding the implementation of plans for PPI, alongside data extracted from trial documentation about written plans for PPI. Thematic analysis is a useful method for identifying, analysing and reporting patterns (themes) within data. 66 DB familiarised herself with the interview data by reading the transcripts several times, and then drew on the Framework technique,67 which is a manual method to develop and apply open codes to the interview data. Codes were grouped into broader categories within the framework and compared with data extracted from the documented plans. Other members of the EPIC team, who were familiar with the interview transcripts and trial documentation, examined the early stages and ongoing refinements of the descriptive coding framework, as well as the tabulated comparisons of planned and implemented PPI, thus providing confidence in the credibility and ‘confirmability’ of the findings. 68

Cohort documentation

One hundred and eleven randomised trials were identified as being in receipt of funding from the NIHR HTA programme between 2006 and 2010, and were therefore eligible for inclusion within the cohort. Complete documentation for each trial was requested. The initial batch of documentation was provided by the NIHR HTA programme and then supplemented by LD visiting the NIHR HTA offices to access missing documentation. All included trials had at least one of the outline application form, the full application form or the detailed project description available, from which we could assess PPI plans.

Cohort summary

Table 2 summarises the completeness of trial documentation available across the cohort. Of the 111 trials eligible for inclusion, 110 were required to submit an outline application. The dates of submission of the associated outline applications were between 2003 and 2008.

Table 3 provides a summary of the trial characteristics. Trials were funded across a wide range of clinical conditions, with the most common area of study being trials in mental health (16%). The majority of trials (79%) were aimed at treatment of a condition, with 17% working on prevention. The trials used a wide range of interventions. Over one-third investigated a medicinal product and just under one-fifth each evaluated behavioural interventions (18%), surgical techniques (15%) and devices (16%).

Table 4 describes the characteristics of trial participants and features of the trial designs. Three-quarters of the trials recruited adults only, with paediatric trials accounting for 18% of the cohort. The majority of the trials were not gender specific and approximately a quarter recruited participants at the time of diagnosis. Trial recruitment was most commonly conducted within secondary care (61%). Just under one-quarter of the trials (25 of 111) involved blinding of the treating clinical team or the participants, with 28 of 111 blinding the outcome assessor only. Just over 15% of trials used a placebo, and all participants received an active intervention in one-third of these, indicating the use of a double dummy design.

To develop understanding of PPI within the earliest stages of clinical trial development we placed an emphasis on the outline application process. The specific objectives were to identify if, and how, PPI is described within the early development of a grant application for funding; examine how PPI contributions within the development of the outline application were reviewed by the funding board; consider how applicants describe their proposed PPI plans for the development of the full application and the trial once funded; and describe variations in PPI in relation to time of funding and trial characteristics.

Patient and public involvement within the first stage of the application process

Outline applications were available for 90 of the 111 randomised trials in the cohort. The trial and participant characteristics restricted to these 90 trials are summarised elsewhere. 69 Of these 90 outline applications, 49 (54%) provided some level of detail on PPI. Table 5 summarises the stage of initiating PPI and role of PPI across the trials. The first row of Table 5 shows that there were 19 applications in which the text provided within the outline described PPI occurring at all three stages (within the outline application, in the full application and once the trial was funded). Of these 19 applications the role was managerial at each stage in 13. In the remaining six there was variation in the roles across the stages, or it was unclear, when a statement indicated that PPI would occur but no details were provided to allow classification. An example of a description that we were unable to categorise is ‘Investigators have, and will continue to, collaborate with service users.’

Twenty-nine per cent (26 of 90) specified a level of involvement within the development of the outline application. This was managerial in 13, on a responsive basis in 7, unclear in 2 and other approaches used in 4 (e.g. a patient survey or pilot feedback). Within the ‘other’ approaches it was difficult to determine conclusively whether this was PPI or they were examples of data collection aimed at ascertaining public opinion. In the three applications that specified use of a survey, the extent of the distribution of the survey was unclear in two. PPI was planned to occur within the full application for 32 trials (36%). This was managerial in 18, responsive in 9 and unclear in 5. Forty-three (48%) applications indicated that PPI was planned after the trial was funded. This was as managerial in 22, responsive in 6, a member of the TSC in 8, unclear in 5 and other in 2. The numbers of outline applications by year with and without details of PPI are displayed in Figure 2, with Figure 3 showing the percentage of applications with PPI. These figures show a general trend for an increasing number of funded applications; however, the proportion of those containing PPI fluctuates. The proportion ranges from approximately half to two-thirds (see Figure 3), with the exception of 2003, for which only one application was available.

FIGURE 2.

Number of outline applications by year in which the application was made.

FIGURE 3.

Percentage of outline applications containing PPI details by the year in which application was made. The number of trials included within each year is indicated at the top of each bar.

Figure 4 displays the data by year for specific conditions. Both of the diabetes trials were in children and were coded as paediatric, and there were no human immunodeficiency virus/acquired immunodeficiency syndrome trials within the cohort with outline applications available. Figure 4 suggests declining rates of PPI in paediatric and mental health, with other areas, including the general ‘other’ category, demonstrating an increase over time. However, as shown in Table 6, the numbers in some categories were small and therefore limit conclusions based on disease areas. Table 7 suggests an absence of association between specification of PPI within the outline application and disease area (n = 0.51).

FIGURE 4.

Cumulative percentage of outline applications by disease/condition.

Table 8 shows the associations between trial design characteristics, or characteristics of the condition under study, and consideration of PPI within the outline application.

Of the intervention aims, numbers were too small to draw conclusions about involvement in diagnostic studies, but prevention trials described PPI more frequently than treatment trials. Trials involving educational, behavioural or physical interventions were more likely to have provided details of PPI in the outline application than trials involving drugs or devices. Surgical trials were significantly less likely to have provided details of PPI.

The settings for trial recruitment did not appear to affect specification of PPI, whereas recruiting participants at the point of diagnosis was associated with less PPI.

Forty-three trials were described as being blind and these trials were associated with increased frequency of describing PPI. Of these trials 23 involved blinding of the outcome assessor only. Only 14 trials involved a placebo and, of these, participants in five trials all received an active intervention, with the placebo used as a double dummy. The allocation of a placebo only to one arm of the trial was significantly associated with greater frequency of PPI detail.

Only two trials were in rare conditions. Both of these trials provided details of PPI within the outline application. Although there appeared to be some increase in describing PPI when the condition was long term, there did not appear to be any influence based upon impact on life expectancy.

Board comments on outline applications

National Institute for Health Research HTA Board minutes were available for 70% (77 of 110) of the outline applications. Only nine (12% of 77) board minutes gave feedback on PPI and two of these did so indirectly (Table 9). One comment was supportive of the PPI described in the outline, which involved a PPI contributor as a coapplicant.

Of the 77 trials for which board minutes were available, the corresponding outline applications were also available for 64 (84%). Of these applications 39% (25 of 64) gave no information about PPI.

Eight of the nine sets of board comments that were made about PPI expressed the need for applicants to increase PPI. Six of these contained no detail about PPI within the application. Of these six, two were drug trials, three were exercise interventions and one was comparing an invasive with a non-invasive intervention. Two were recruiting participants with addiction (smoking and alcohol), two were recruiting elderly participants, one was recruiting infants and one was recruiting participants with a long-term, chronic, debilitating condition.

In the two outline applications which had given some details on PPI, comments from the board were to increase PPI. In one application the PPI contributors were employed within a NIHR condition-specific network in roles relating to PPI. These individuals also had relevant experience as carers or patients of the condition. In the other application the PPI contributor was a coapplicant with an unrelated clinical background (midwife) and was the mother of a child with the condition being studied.

Patient and public involvement within the second stage of the application process

Documentation within the second stage of the NIHR HTA application process requires submission of a full application form and a detailed project description. Ninety-five per cent (106 of 111) of the full applications were available and 89% (99 of 111) of detailed project descriptions. The level of information about PPI within the full application forms was limited, with greater detail generally provided within the detailed project descriptions.

Changes in patient and public involvement between the first and second stages of application

There were 80 trials in the cohort for which we had the outline application, the full application and the detailed project description.

Referee comments on the full applications

There were 515 sets of referee comments for the 111 trials in the cohort. The minimum number of referees for a trial was one, the maximum was nine and the median was five.

Across all referee comments only 41% (211 of 515) gave a comment in relation to the PPI. The median number of referee comments relating to PPI per trial was two, with a minimum of zero (occurring for 11 applications) and a maximum of six.

Thirty-four per cent (72 of 211) of comments were positive, 51% (107 of 211) were negative and 15% (32 of 211) were factual. Sixty-three applications had more than one set of referee comments that contained text relating to PPI; after factual comments were removed, 52 applications remained, of which 56% (29 of 52) had referee assessments that disagreed in their assessment of PPI. Table 12 provides examples of conflicting referee comments.

Board comments on the full application

Board comments on the full application were available for 90% (100 of 110) of the cohort. Of these, only 14% (14 of 100) made a comment about PPI (Table 13).

Of these 14 comments, nine (64%) were negative, stating that PPI should be increased, but gave no indication of how this should be achieved or why existing plans were insufficient; four were neutral, with two of these providing some direction for PPI if appropriate; and one was positive.

There were no clear reasons why these 14 applications received comments on their PPI plans in comparison with the other applications within the cohort.

Key points

• Within this cohort there was no trend evident to suggest that PPI is increasingly being sought within randomised trials.

• There was no association identified between specification of PPI activity within the first stage of the application and disease area.

• There was some association between trial characteristics and frequency of PPI.

• Only half of first-stage applications provided some detail on PPI, with only one-quarter describing PPI activity in their development.

• A third of first-stage applications described PPI activity plans in the development of the second stage of the application process and half had plans for PPI activity once the trial was funded.

• The majority of first-stage applications which stated plans for PPI activity in the development of the second-stage application implemented their plans, with over 70% being consistent with the plans they had outlined.

• Half of the first-stage applications that did not describe plans for PPI in the development of the second stage did obtain PPI in its development.

• There was inconsistency in the second stage of the application process between the full application form and detailed project description regarding PPI activity, planned or completed, possibly suggesting that applicants were confused about where this should be incorporated, or that it was a consequence of space constraints.

• Few applications indicated that the approach to incorporating PPI in the development of the application would remain the same during the funded trial.

• Qualitative research methods were described as an approach to deliver PPI-associated objectives; however, it was often unclear that this would be conducted by people with the necessary skills.

• Board feedback to applicants rarely commented on PPI.

• Only two-fifths of referees commented on PPI, and referees often disagreed on the acceptability of PPI plans described.

Chief investigators survey

The survey was sent to 111 CIs, of whom 81 (73%) responded.

To consider the potential for respondent bias, CI response was considered against the inclusion of details about PPI within their outline application. This was chosen as an indicator of baseline motivation and positivity towards PPI. The proportion of CIs responding was higher for those who included PPI in the outline application [PPI detail included, 82% (40 of 49); PPI detail not included, 66% (27 of 41); outline application not available, 67% (14 of 21)] however, the result did not reach statistical significance (p = 0.19).

The results of the survey are provided in Appendix 3. Fifty-two per cent (42 of 81) of the respondents thought that PPI should always be incorporated in a research study, 43% (35 of 81) felt that PPI could be beneficial but was not always necessary and 5% (4 of 81) were not convinced of its benefits. When stratified against availability of the outline application and the absence or presence of detail about PPI within it, there were no clear differences in the proportions who thought it should always be incorporated [PPI detail included, 55% (22 of 40); PPI detail not included, 52% (14 of 27); outline application not available, 43% (6 of 14)]. Of the four who responded they were not convinced of the benefits of PPI, the outline applications were available for two, neither of which provided any detail of PPI.

Forty respondents had text about PPI in their outline application. Of those that felt PPI should always be incorporated, 45% (10 of 22), 73% (16 of 22) and 91% (20 of 22) respectively provided detail within the outline application about PPI occurring in the development of the outline application, the full application and once the trial was funded, compared with 67% (12 of 18), 61% (11 of 18) and 72% (13 of 18) of those who felt it not always necessary.

Conversely, those who thought PPI should always be incorporated were more likely to report they considered PPI immediately (66.7%, 28 of 42) (question 3) than those who considered it not always necessary (45.7%, 16 of 35) (p = 0.006). Prompting by the CTU occurred more frequently among the group who considered it not always necessary (20%, 7 of 35, vs. 9.5%, 4 of 42).

Ten of the 11 (91%) who were prompted by the CTU provided detail about PPI in the outline application compared with six of the nine (67%) who reported being prompted by the application form (outline application form not available for one respondent to question 3 category 3). This may suggest that CTUs are an effective mechanism for initiating PPI.

The majority (65.8% 52 of 79) of respondents reported more than one motivation for including PPI (question 5). Requirement for funding was the sole reason for four CIs, compared with being the right thing to do for 14 and previous experience of its benefits for nine.

Of the 55 who indicated that their motivation was due to PPI being the right thing to do, 34 (62%) indicated their personal view was that PPI should always be incorporated, with 21 (38%) indicating PPI can be beneficial but not always necessary (cross-tabulation of questions 2 and 5).

Patients, carers and parents were the most common PPI contributors involved within the cohort. Of those studies that involved a charity member, only five reported this as the only PPI contributor involved, whereas 16 also involved patients, 13 involved carers and/or parents, and five involved medical staff.

Of the 11 that reported medical staff as PPI contributors, they all also included patients, parents or carers as contributors, with five also including charity members. An open question (question 7) asked for the reasons behind the selection of PPI contributors. Multiple reasons could be provided within a single response and the coded categories are provided in Table 14. Where it was not clear from the statement provided, information from question 6 (‘Which PPI representatives did you involve?’) was used to support the coding. It was often difficult to determine whether the previous experience referred to related to being a research participant or to providing PPI, so no distinction is made. The most common reason for selection was that the demographics of the PPI contributors were believed to make them representative of the patient population. Other leading factors were their experience and knowledge of providing PPI, experience from an existing or previous role considered to be beneficial, having previously worked with them or their connection to charities or organisations.

The most common way of identifying PPI contributors was opportunistic, for example by patients, parents or carers known or by previous involvement. Charities, patient support groups and voluntary organisations were also frequently used. Advertising and use of the research networks or NHS Patient Advisory Liaison were uncommon.

The majority (54 of 76, 71%) felt that they had provided a clear description to the PPI contributor(s) at the time they joined the trial (question 9). However, the level of detail they had provided to the PPI contributor was not captured by this question and it is likely the level of detail varied considerably from a short introductory e-mail to specific terms of reference.

Over 80% of the CIs indicated they had a PPI contributor on their TSC (question 10). This is specified within the NIHR HTA constitution of TSCs. 11

Thirty-three (43%) CIs reported a single approach to PPI contributions, 26 (34%) reported two, eight (11%) reported three, six (8%) reported four and two (3%) reported five.

In 79% (26 of 33) of the trials where a single approach was used, this was including a member on the TSC. This could suggest ticking the funder’s requirement box or taking the funder’s lead on how to implement PPI.

Considering the number of approaches used in obtaining PPI in relation to the CIs’ personal opinion of PPI (Fisher’s exact test p = 0.005), those who thought PPI should always be included were more likely to have more than one approach to obtaining it (Table 15).

The number of PPI contributors per trial varied (Table 16), as did the frequency of contact between researchers and contributors (Table 17).

Eight described how the frequency of meetings varied with the stage of the trial, with greater frequency occurring during the early stages of the trial:

During study set up and enrolment every couple of months. In follow up once every 6 months.

Initially once per month. Less frequently during the study. More frequent involvement towards the end.

Variation in frequency of contact was also associated with role. Meetings with members of the TSC were most commonly reported to occur every 6 months (71%, 45 of 63); 24% (15 of 63) met more frequently than this and 3% (2 of 63) less frequently; one informant commented only that the frequency varied and did not provide an average.

Of the 20 coapplicants, 40% (8 of 20) met once a month, 50% (10 of 20) met every 6 months and the remaining 10% (2 of 20) specified a frequency greater than once a month.

Of the 23 with TMG members, 43% (10 of 23) met monthly, 22% (5 of 23) met every 6 months, three met three times a year and three met once every week or two.

Of the 10 DMC members, four met monthly, four every 6 months, one yearly and one twice a month. A high frequency of meetings (more often than 6-monthly) is unlikely for a DMC unless it is a high-risk trial, suggesting the potential for confusion among respondents between the different committee titles and remits.

Of the 16 involved as PPI advisory groups, three met monthly, six met every 6 months and the remaining seven described variable frequencies.

The majority (59%, 45 of 76) did not experience any problems related to including PPI in the trial (question 13). The most commonly reported problem was inability of the PPI contributors to attend meetings (24%, 18 of 76). Other problems were related to finding suitable contributors, the contributors not fully undertaking their role, and training and support for the PPI contributors.

Of interest is that 80% of CIs (60 of 76) felt that researchers should be trained to help them support PPI contributors (question 14).

Patient and public involvement by trial stage is provided in Table 18. Few reported involvement in analysis and just over one-third reported it for dissemination.

The most common areas of PPI contribution during trial setup were in considering the burden of patient participation and designing or commenting on the patient information sheets (question 16). Thirteen provided free-text comments describing ‘other’ areas of contribution. These are provided in Table 19 along with the closed response selections.

The most common areas of PPI activity during trial conduct (question 19) were troubleshooting recruitment, attending meetings and raising the trial profile. Of the nine respondents who reported PPI in the analysis, four stated they were yet to reach that stage of the trial. The free-text comments of the remaining five are provided below:

Emerging findings were presented to [patient advisory group] whose feedback helped us interpret study results and disseminate study findings.

They read and commented on the analysis, discussed whether the correct assumptions had been made about the data. Helped understand what the data meant.

Was involved in the close out meeting and gave feedback on data quality assurance processes.

Commenting on what they thought the analysis meant.

As a co-applicant.

The CI opinion of impact of PPI in each of the areas of PPI contributions is summarised in Table 20. High impact occurred more frequently in trial setup, with low or no impact being more common during trial conduct, analysis and dissemination.

The free-text responses provided to questions on impact (questions 17, 20, 22 and 24) are provided in Appendix 3 along with the level of impact selected.

Patient and public involvement contributors survey

Thirty-two PPI contributors from 28 trials completed the survey (two respondents each for four trials). Of these, 31 were contacted about the survey via the CI of the study and one was identified as a result of the NIHR HTA programme letter to TSC chairs. All PPI contributors who were successfully contacted to complete the survey did so.

As identification of PPI contributors could largely be achieved only via the CI, we considered respondent bias by whether or not there was an association between a PPI response and the CI opinion on PPI as categorised by CI response to question 2 on the CI survey. The results are provided in Table 21 and no association was evident (p = 0.93).

The results of the survey are provided in Appendix 3, Table 30. The majority of contributors were approached personally by a member of the research team (66%). Just over half were involved during the development of the application for funding, a percentage largely consistent with the results of phase 1. The most frequently indicated motivations behind undertaking the role were personal experience of the disease, general interest in the topic or research, wanting to help and involvements with charities. One-quarter of respondents cited their previous experience of providing PPI as a motivating factor (question 4), with 41% indicating that they had previous experience.

Half of the respondents indicated that they were the only person providing PPI (question 6). This is not consistent with the results of the CI survey but may be explained by the level of interaction between PPI contributors in different roles. The majority of respondents (72%) indicated being on the TSC (question 11), with approximately one-fifth of respondents being on TMGs or being coapplicants. Contact (question 12) was once every 6 months for 41%, a frequency consistent with membership on a TSC. Only four respondents indicated receiving any training to assist them in their role (question 15). One-quarter indicated receiving peer support from other PPI contributors (question 18), with 41% indicating this came from a member of the research team. Sixty-nine per cent were not aware of any resources available to them (question 19) and 59% received feedback from the research team on their involvement (question 21).

Over 60% of respondents felt they had the right level of involvement (question 22). No one indicated they would have preferred to be less involved, but 16% wanted increased involvement and 22% wanted a targeted approach to areas of involvement.

The various areas of involvement are summarised within Table 22 for ease of comparison across the areas respondents were interested in undertaking before the trial started (question 13) and during the trial (question 14), the areas in which they would have liked to have received training (question 17) and areas where they felt their contribution made a difference (question 20). There was a general consistency within each area between interest before and during the trial, and where they felt their contribution made a difference. Of interest is that fewer than 60% indicated an interest in outcomes and patient information sheets, areas commonly associated with PPI. A higher proportion of PPI contributors felt their contribution in piloting questionnaires or assessments made a difference, despite the lower proportion who indicated it as an area of interest. This was reversed for data collection, where nearly one-fifth indicated it as an area of interest before the trial but only 3% felt it was an area in which they had made a difference. Although this was also the case for dissemination, a number of respondents had not yet had the opportunity to make a difference because of the stage of the trial.

Respondents were asked what they would change about their experience of providing PPI within this trial (question 28). Of the 29 responses, nine stated that they would not make any changes. Three commented on logistical factors that could be improved, for example ‘Have the meeting later in the day given that they are held 80 miles away from my home’. Three would have had earlier involvement:

I would have been more use had I joined the trial at the outset. I feel I am more useful in the new trial, because I have been able to help guide patient information from the start, rather than coming in half way through when documents had already been prepared.

Two commented on increasing the level of involvement: ‘It would have been nice to have been more involved and to feel that this was wanted’. Five suggested training and support: ‘Greater awareness of my role within the trial and some training to aid this’. One individual commented on an aspect related to their ability to ‘represent’ trial participants:

In some ways I was a ‘proxy’ for the voice of the [participant demographic] this trial was targeting. To be genuinely participatory, it would have been good to have [someone within the demographic] as a PPI rep, but that may have involved a change of methodology for the group. I felt I had to fit in with the way the steering group was designed to run, rather than the group being adapted to accommodate PPI.

One commented on financial reimbursement and one on ensuring inclusion in circulation of finalised documentation. Three responses provided were not applicable to the question.

The majority (78%) felt engaged in the research project and valued as a member of the team (question 23), with all respondents recommending the role to others (question 29).

Key points

Impact

None of our informants identified PPI as having an unfavourable impact on the trial. Of the 21 CIs interviewed, 14 described PPI as having an impact and seven explicitly stated that they felt PPI had not made an impact. Of the 17 PPI contributors, 11 reported that they felt their input had made a difference to the trial, three explicitly indicated that they had made no impact and three could not identify impacts arising from their input. Of the 10 TMs, five described PPI as having made an impact and five did not identify any impacts of PPI. When we triangulated the accounts of researchers and PPI contributors from the same trial, they were largely in agreement about the perceived impact of PPI. Divergences between the groups could largely be attributed to informants being unable to remember specific PPI contributions because a trial had started ‘a long time ago’.

Based on the accounts of informants who perceived PPI as having an impact, we distinguished two main types: focused and diffuse (Table 23). Focused impact comprised PPI contributors’ input that, from the perspective of the informant, changed or influenced an aspect of the trial, whereas diffuse impact comprised PPI contributions that influenced the way researchers thought or felt about the trial. In addition to the examples in Table 23, focused impact included PPI contributors helping to choose the primary outcome for the trial and increase recruitment through their contacts and networks. As Table 23 indicates, diffuse impact largely entailed interactions between researchers and PPI contributors that helped to reassure the research team and increase or maintain their confidence and motivation for the trial. For example, both researchers and PPI contributors described how a PPI contributor’s presence kept the research ‘grounded’ and reminded them ‘what it’s about.’

Informants who reported no impact either explicitly stated that they felt PPI had not had an impact, or were unable to identify a PPI contribution that had led to a change in the trial or influenced the researchers:

Within that TSC I can’t remember them making any particular contribution that changed the way we run the, ran the study. And if they had made a substantial contribution I would remember.

CI 10

She reviewed all the paperwork, she came to the meetings, but did she actually change the trial in a meaningful way? Well no um probably not.

CI 7

Informants who explicitly reported no impact from PPI spoke only of the lack of focused impact. None of these informants referred to diffuse impact when speaking about the absence of PPI influence on the trial, whereas many informants who reported that PPI had made an impact identified both the focused and diffuse types and their accounts did not deem one more important than the other. Therefore, we use the term ‘impact’ to refer to both types.

Influences on impact

We identified two main influences on whether or not informants perceived PPI as having an impact: whether or not CIs expressed any personal goals and plans for PPI aside from its perceived role in leveraging funding; and the quality of the relationship between the PPI contributors and the researchers.

Training for patient and public involvement contributors in clinical trials

Key points

• Well over half of the informants indicated that PPI had made a difference to the trial or influenced the trial team, and none reported unfavourable impacts from PPI.

• Chief investigators who described goals for PPI and planned its implementation in the light of these goals tended to report impact, whereas those whose goals for PPI did not extend beyond meeting perceived funding requirements usually reported little or no impact from PPI.

• Researchers whose accounts indicated they did not value or have goals for PPI tended to implement it mainly in one way: by including PPI contributors on TSCs.

• Patient and public involvement contributors who spoke of having a good relationship, particularly in terms of feeling part of the team, also tended to report impact from PPI, and both researchers and PPI contributors pointed to the importance of implementing PPI before seeking funding.

• Whether or not CIs valued PPI seemed to be linked to the goals they described and how they implemented PPI. CIs who expressed scepticism about PPI focused mainly on using PPI to meet funding requirements, whereas those who valued PPI often described in detail how it was of benefit within their trials.

• Some researchers seem to accord little value to PPI. It also raises the possibility that this may become a self-perpetuating cycle, with such researchers implementing PPI in ways that may provide little opportunity for it to benefit randomised controlled trials and then concluding that PPI made little difference to their trials.

• Informants had reservations about the need for training in PPI, particularly training for PPI contributors. Very few contributors had received training and many were reluctant to engage in it. Researchers shared this lack of enthusiasm for training PPI contributors, although both groups of informants welcomed informal induction ‘conversations’ to help contributors to understand their roles.

• Induction seemed to provide little scope for contributors to negotiate their roles, and support for contributors was largely implicit and focused on practical arrangements rather than on helping contributors to function in their roles.

• Rather than training contributors, researchers used their networks and others’ recommendations to identify and select individuals who already possessed attributes perceived as important for the role. Therefore, informants tended to see training PPI contributors as redundant because, through the way they had been selected, contributors were believed to possess the necessary attributes.

• Informants were also concerned that training and cumulative experience in PPI roles overprofessionalised contributors and limited their ability to provide an authentic patient perspective. Researchers described a tension between needing contributors who could provide an authentic patient perspective and needing contributors who could function in oversight and managerial roles (e.g. as members of TSCs and TMGs respectively).

• Some commented that this tension could be resolved by selecting particular PPI contributors for particular roles within a trial. Informants were more receptive to training researchers in PPI than training PPI contributors, and most researchers had either received training or indicated that they would find it helpful. Nevertheless, a sizable minority pointed to how it was sufficient to learn about PPI on the job or that evidence to inform training was lacking. Contributors also saw a fairly limited role for training researchers in PPI, although some pointed to the use of plain English and clarity about PPI contributor roles as areas in which researchers could benefit from training.

Intentions to actions

As shown in Table 25, all but three of the 28 trials had documented plans for PPI in their grant application. These documents varied greatly regarding the extensiveness of PPI activity planned and the precision with which plans were described, from vague references to activities that hinted at PPI – ‘We will make use of two primary care research networks and an [intervention-specific] research network’ (trial 115) – to statements that were quite precise: ‘The [society] confirmed their willingness to represent their members through steering committee membership [. . .] and to help in the construction of the MREC [Multicentre Research Ethics Committee] application and patient information leaflets’ (trial 102). Based on informants’ interview accounts, all trials subsequently incorporated some form of PPI and it was clear from the interviews that documented plans were fully implemented in most (20 of 25) instances regardless of whether the plans were vague or precise, minimal or extensive. The three trials without documented plans did proceed to include some PPI activity, perhaps prompted, to an extent, by comments from peer reviewers, who had remarked on the lack of PPI plans in each case. A further three trials expanded on documented plans, giving a total of six trials which had seen addition or expansion of plans for PPI.

Despite informants indicating that most of the documented plans for PPI had been implemented, some revealed no personal expectations for PPI and spoke of using it as a means of ‘ticking the right boxes’. This raises questions about the motivations behind the PPI plans in some grant applications.

Based on informants’ accounts it appeared that six trials largely confined PPI to an oversight mode of involvement, although some had hinted at other modes in their applications. We begin by examining what happened in these trials.

Contributors on the challenges of patient and public involvement, and suggestions for improvement

Most PPI contributors mentioned challenges or difficulties linked to their involvement in the trial which may inform future research teams in planning and implementing PPI. Some of the contributors’ challenges paralleled CIs’ accounts whereas others were unique to the contributors (see Table 26). Whereas researchers referred to problems they had experienced in their communication with contributors, a prominent issue exclusively mentioned by contributors related to the problems they experienced with ‘jargon’ and the technical language that was used in trials, such as statistical or medical terminology and acronyms. Several contributors suggested remedies such as supplying a list of acronyms or a booklet of research terms, or simply ‘if they’re going to use jargon, explain it’ (PPI 64). A further idea was that the person chairing meetings could try to ensure that discussion about statistical issues or other areas of technical expertise was translated and summarised adequately. Contributors talked about difficulties in interacting with researchers, including not always feeling listened to by everyone. One contributor who had been invited by her consultant and had previous experience of PPI implied that ‘some doctors’ were unwilling to understand the perspectives of patients (PPI2 27). Another felt that female researchers were more understanding than males regarding problems with travelling or feelings of insecurity, and a further contributor alluded to how in meetings the team sometimes talked about patient experiences in a ‘dispassionate’ way and, although this was not a problem for that particular contributor, she felt it might be for others (PPI1 27).

Some of the challenges that contributors described echoed those raised by the CIs. These included lack of clarity about roles, and the difficulties contributors experienced in attending meetings, for instance because of a health condition. Such practical difficulties could give rise to additional complexities. For one contributor, infrequent meetings meant ‘not much to build a relationship on’ and, while academics worked closely together, she had to ‘work quite hard to keep up’ (PPI 16). Contributors also talked about wanting to be more involved in between annual meetings, to take part in ‘shaping the bid’ (PPI 20) so that it was less focused on the primary clinical outcome, to see the intervention itself and to have initial briefing meetings at the outset of their involvement. Finally, one contributor described it as a ‘downfall’ that he was not receiving feedback or ‘thank yous’ and commented on how important it was to make PPI contributors ‘feel valued’ (PPI 34).

Key points

• Most triallists are putting their plans into action, although in some cases the plans were minimal and relatively easy to execute.

• Many trials implemented multiple modes of PPI, which is both surprising and encouraging given that PPI was less prominent when the proposals for the trials in this cohort were being developed.

• Regardless of statements about PPI in their funding application, some triallists had no expectations of what PPI might achieve, and their only motivation for including PPI was a belief that it was necessary or would help to secure funding for their trial.

• Chief investigators encountered complications from which they learnt valuable lessons, suggesting that it is perhaps necessary to learn by experience in this area, including what to expect of PPI. Difficulties in finding and retaining suitable contributors, and engaging in PPI ‘too little too late’, led triallists to say they would do things differently in future, including seeking involvement from a more diverse source such as patient panels or focus groups.

• Patient and public involvement contributors themselves mentioned that becoming involved after the trial had begun, or infrequently, resulted in missed opportunities for them to contribute. Some referred to uncertainty about their role and many struggled with jargon, an enduring problem despite the availability of apparently straightforward solutions.

Key points

• UK Clinical Research Collaboration Registered Clinical Trials Units are proactively considering PPI requirements within their trials portfolios, as demonstrated by the significant proportions that engage in identifying contributors and the increasing numbers that are developing standard operating procedures.

• The implications of the level of activity within CTUs to support PPI need to be considered in line with core funding requirements.

• UK Clinical Research Collaboration Registered Clinical Trials Units are determining PPI requirements within individual trials based on the trial characteristics. This presents an opportunity to consider whether or not a formal risk assessed approach to PPI could be developed and evaluated within this network.

• Targeted engagement between INVOLVE and the UKCRC network of RCTUs should be considered to benefit PPI activity from both researcher and contributor perspectives.

Summary of main findings

Results in the context of previous research

To our knowledge this is the largest study of PPI in randomised trials to date. Several of our observations receive support from previous studies and reviews of PPI, although most of this work has not concentrated on PPI in randomised trials. Compared with other forms of health and social care research, randomised trials are highly structured and intensively regulated entities. This limits the relevance of PPI studies conducted outside the context of a randomised trial for understanding how PPI can make a difference within trials. For example, it will usually be harder to change aspects of a randomised trial after it has started than other types of studies.

Our findings are timely given the 2014 announcement71 of a strategic review of PPI in research within the UK and the increased emphasis on stakeholder involvement internationally.

Study strengths and limitations

Our study had some limitations and our findings should be regarded carefully, particularly as PPI is a field where policy has tended to outpace evidence. We used a historical cohort of trials that had been funded between 2006 and 2010. Even in the short time since then, the emphasis on PPI has grown and our findings may not reflect the planning and implementation of PPI in trials funded more recently.

Our sample was limited to trials within one UK-based research funding stream, the NIHR HTA programme. Although this may limit the transferability of our findings, as one of the world’s leading funders of health research, NIHR’s research activity is substantial.

To be eligible for inclusion within the cohort, applicants were required to have received funding for the trial between 2006 and 2010. Documentation could not be made available for unsuccessful applications. Consequently this study does not link details of PPI within the application to the successful award of the grant. However, while the HTA programme encouraged PPI during this period, via its guidance notes to applicants and web information, and allowed it to be budgeted for, PPI was not mandatory within the period of the cohort, and that on its own would not have led to a decision that the outline application should not progress. As the cohort was identified by receipt of funding during the period rather than on the year when the outline application was submitted for consideration, fewer trials were available dating from the beginning and the end of the period.

The period of the cohort should be considered when generalising the findings to the present day, given revisions to the guidance for applicants and application forms in relation to PPI. From 2012, HTA boards have started to include PPI membership, and the standard NIHR application form has been introduced, with revised sections requiring applicants to define their PPI involvement clearly, and guidance notes to applicants clearly stating that there is now an ‘expectation’ of ‘active involvement’ of patients and the public in the research it supports. 87 This may lead to increased descriptions but may not be associated with better involvement: a view which has some support from the current guidance, which states, ‘Whilst patient and public involvement (PPI) may not always be needed for all types of research, it is always relevant for HTA trials. Many PPI sections on the application are unconvincing to our consumer referees and Board members’. 88

Some of the trials in our sample were also initiated and completed some time before the interviews. However, this limitation is offset somewhat by the inclusion of ongoing trials in which PPI activity was more recent and therefore easier to recollect. In some cases informants clearly struggled to recall events for trials which had ended several years previously or where researchers were involved in a number of trials simultaneously. We explored with informants how PPI contributors were involved in the trials but did not directly quiz CIs about why certain plans within their application were not implemented. This was intentional, as we did not want to pose questions which might have seemed accusatory and had a detrimental impact on the rapport between informant and interviewer, or risk informants becoming defensive. Whereas some triallists seem to have expanded on their plans for PPI once the trial was under way there may, conversely, have been instances in which plans were not fully documented within the grant application.

Like most other studies exploring the impact of PPI in research, it was limited to investigating researchers’ and PPI contributors’ reports of their views and experiences. 5 Objective techniques for evaluating impact and its influences remain elusive in a process that is inherently relational, subjective and socially constructed. 45 For example, some informants reported that PPI contributors’ input helped to improve response rates by reducing the length of questionnaires, yet there is the possibility that valuable information was lost in the process. Participants in Barber et al. ’s45 mixed-methods Delphi survey and qualitative interview study questioned the feasibility of objectively evaluating the impact of PPI on most research processes and outcomes. In this regard, a strength of our study is that we triangulated the accounts of multiple informants in half the sampled trials. Linked to our study’s retrospective design, however, informants struggled to recall particular examples of PPI input. In addition, as others have noted,45 there are inherent difficulties in attributing impact to the contributions of particular individuals, when the actions to address many difficulties within trials are likely to be the product of a series of complex interactions among research team members.

Although our informants were drawn from a cohort of trials, we could interview only those who opted to do so. The response rate to the CI survey, which was our main route for accessing interview informants, was high (73%) whereas the response rate for the CI interviews was lower (51%). In addition, all of the PPI contributors interviewed were involved in managerial or oversight roles, as we were unable to access those in responsive roles, because most researchers did not hold contact details for contributors in such roles. Our access to the information about PPI was limited in some cases; for example, where the only informant from a trial was the PPI contributor in an oversight role, we were unable to ascertain the other types of PPI within that trial.

Because our interview sample was drawn from a cohort study and survey we are able to provide more information about our sample than is typical for qualitative studies. Although we aimed to purposively sample survey respondents to access a diversity of views, we eventually invited almost all of those who indicated willingness to be interviewed. The survey responses of the subsample who were interviewed were more favourable in their views of PPI than the wider sample of surveyed CIs, although, as we report, interviewed CIs expressed a diversity of views about PPI and some were sceptical about its value. Our access to PPI contributors was limited to those whose contact details were provided by CIs responding to the survey and one identified by the chair of a TSC.

The survey responses cannot be taken as a fixed or true point from which to assess the adequacy of our interview sample. 69 In their interviews participants described their experiences in detail and we were able to consider how they talked about PPI, as well as what they said about it. In this context, it is notable that, while all of the CIs in the survey reported some impact from PPI, the interview accounts of CIs told a rather different story, with one-third describing PPI as having no impact. This indicates the usefulness of qualitative approaches for investigating complex processes such as PPI, which are subject to moral, reputational and other sanctions. The diversity of perspectives that we accessed also suggests that we had some success in minimising the selectivity that has been a difficulty in some previous work on PPI.

To our knowledge this is the first study to report the accounts of researchers and PPI contributors who have not previously engaged with PPI training as well as those who have. Our study provides insights about how PPI training can be developed to enhance its relevance from the perspective of both groups. By exploring training needs in the context of informants’ wider experiences of PPI, our study has also identified some potential topics for training beyond those articulated by our informants. As this illustrates, individuals are not necessarily able to identify potential deficits in their own knowledge and skills,89 and a limitation of our study is that we did not formally assess informants’ knowledge and understanding of PPI.

Future research

Given the difficulties for some informants in recalling PPI contributions, future research in this area that takes a prospective approach would be valuable. PPI is an area of rapidly evolving practice, and prospective research would also be valuable to explore how such changes are influencing how PPI is interpreted and implemented. In view of the difficulties for informants in attributing impact, and the relational and subjective nature of PPI activity, ethnographic research that combines observation and multi-informant interviews is likely to be informative. Many will also regard future prospective investigation of the impact of PPI on trial outcomes, such as recruitment, retention and participant experience of trials, as essential to further optimise PPI. The role of funders and the UKCRC network of registered trials units in monitoring change and subsequent impact should be explored.

The research community needs to give further consideration to processes for selecting PPI contributors and models of implementing PPI. Randomised trials may benefit from a diversity of patient perspectives and have the potential for benefit from ‘professionalised or experienced’ PPI contributors as well as those who are research naive.

However, one or two specially selected individuals cannot represent the perspectives of diverse groups of patients. Indeed, there was evidence that PPI contributors were selected for their atypical characteristics that facilitated their ability to provide input and reduced the need for training. PPI via patient advisory groups or panels enables the voices of multiple and diverse groups of patients to be heard. The findings we report indicate that this type of PPI is more powerful in terms of its perceived impact on research than oversight and managerial PPI. However, when one or two individuals are selected from such groups the process or mechanism for how they will actively engage with the population of interest about the trial needs to be clear. Effective mechanisms for achieving this, including social media, should be explored, as should the role of qualitative researchers with the skills necessary to explore and summarise diverse opinions.

Achieving consensus on essential and desirable attributes, skills and experience of PPI contributors in relation to trial-specific roles may facilitate researchers in selection processes, thereby helping PPI contributors to achieve impact. Further consideration should be given to training requirements, ensuring they are evidence based and evaluated in relation to implementing PPI and subsequent impact. Further research is needed on the role and value of jointly training researchers and PPI contributors in ‘how to do PPI’ rather than separately training PPI contributors on research and researchers on ‘how to do PPI’. In addition, the RCTU network, INVOLVE and funders should consider developing agreed packages of materials to be distributed to CIs, CTUs and PPI contributors that could be used to assist them in developing and supporting PPI activity.

This project has highlighted the strength of the funders in shaping the approach to PPI. However, further changes are needed to the approach funders use to request information about PPI in their application forms. In a bid to move away from a tick box approach to PPI it is a paradox to include a section of tick boxes within this section of the application form. Currently a list of tick boxes is provided to identify the areas of PPI activity. Requesting specific goals for PPI, and assessing the methods and costs for achieving such goals, may promote more considered approaches and improve the ability of reviewers and funders to assess such plans.

Further consideration is needed on how funds can be made available to researchers to support development of PPI plans and activity prior to grant submission. There was some evidence to suggest variation in practice relating PPI activity to some trial characteristics. Many trial activities are now considered in a risk-proportionate approach. Evidence from this project suggests that informally this may be being applied to PPI activity in randomised trials. The acceptability, applicability and cost-effectiveness of such an approach should be considered along with identification of pertinent factors.

Implications and tips for the trials community

We have used the insights of informants to generate practical tips which may help future triallists and PPI contributors (Box 3). We envisage that these be considered alongside previously published guidance for PPI in trials21,23 and consensus principles for PPI in health research. 90,91 The tips generated from evidence in our study cover the importance of early planning, of timely and flexible PPI, and of communication and clarification of roles. They also stress the need to consider the difficulties posed by the use of ‘jargon’, and problems contributors experience in understanding certain aspects of the research process. The difficulties contributors experience with specialist or technical terminology have been widely reported. 16,18,77 Our data suggest that this problem has existed for some considerable time, and we outline the practical solutions suggested by PPI contributors. The tips in Box 3 could be used to inform PPI training and could be helpful in other types of health research. Given that the usefulness of the points in Box 3 depends on researchers’ willingness to engage genuinely with PPI, the tips we present might also assist funding bodies and grant reviewers in determining whether or not submitted plans are fit for purpose.

A study of the UK health and social care research community has recently informed the development of a PiiAF, which emphasises the value of well thought-through planning before implementing PPI as well as the subsequent evaluation of its impact,77 and INVOLVE21 has emphasised the importance of clear guidance about roles. However, researchers also need some scope for flexibility and contingency in planning PPI: our finding that some triallists expanded their sometimes already detailed plans supports the need for flexible and iterative approaches to PPI in order to accommodate the unexpected and respond to opportunities and difficulties as they arise.

Our findings add fuel to recent drives and initiatives to promote the assessment and reporting of PPI processes3,8,92,93 including the GRIPP (Guidance for Reporting Involvement of Patients and Public) checklist. 94 The CONSORT (Consolidated Standards of Reporting Trials) Statement, which was established specifically to encourage adequate reporting of randomised trials, does not cover PPI. We suggest that consideration be given to incorporating advice on reporting of PPI in the main CONSORT checklist, so that reference to PPI is incorporated within the main reports of trials, alongside separate detailed reports on PPI, in line with the GRIPP checklist. If, in planning their PPI, triallists are prepared to consider and report its outcomes in terms of not only what happened and how, but also how this matched the needs of the trial, whether or not any complications arose or adaptations were made, and what lessons were learnt, then the evidence base will grow and the research community as a whole can learn. The EPIC project has highlighted the value of listening to the accounts of PPI contributors as well as researchers, and this should feed into the evaluation and reporting of PPI.

Implications for funders

Many researchers believed that funding would not be forthcoming unless they included PPI. Although this might be regarded as indicating the success of policies to promote PPI, it was clear that some circumvented these policies by adopting a minimal approach to PPI. In the light of our findings, research funders might want to consider how their policies could be refined to address this difficulty. Our study points to the inadvisability of applying ‘one size fits all’ methods to the implementation and evaluation of PPI in research, and underlines the importance of funding panels conducting nuanced assessments of a research team’s goals for PPI in the context of a particular trial. This might encompass scrutiny of a research team’s account of how their proposed trial stands to benefit from PPI and assessing the suitability of their plans in the light of these goals. It might also involve accepting that PPI should be proportionate to the needs of a particular trial and that a minimal approach to PPI may be legitimate in some cases. Researchers who can adequately justify such an approach should not fear that their chances of funding success will automatically be jeopardised by being candid about this. A sizable minority of informants did not report any impact from PPI. Although our findings point to problems in the implementation of PPI as contributing to this lack of perceived impact, it is conceivable that some trials will have little to gain from extensive and elaborate forms of PPI.

Our findings endorse recent revisions to the NIHR’s standard application form, which now require applicants to clearly define their proposed PPI activity. Asking researchers to specify the type of involvement is a step in the right direction. However, we would suggest further changes should be implemented as described within the further research section, as the risk of strategic minimalism remains if plans are not afforded careful, context-specific consideration by funders and reviewers. Equally, there is a risk of inadvertent PPI profligacy, that is the encouragement of elaborate plans for PPI that are disproportionate to the needs of a trial. Ticking several boxes rather than just one box could equally be a token gesture, as well as an expensive one. Therefore, researchers might be encouraged to think just as much about why, how and when PPI will be useful as about what and how much PPI.

Although funding body policies support PPI, this support was not usually evident in HTA Board feedback to applicants at the outline stage. This may be because of the difficulties in assessing PPI given the lack of detail provided within applications. When feedback about PPI was given, this did not provide any guidance on how it should be addressed; that aspect is important given that it is dominated by opinion rather than evidence. Statements about the need to improve PPI should be supported with guidance on what PPI contributions would be appropriate within the trial being considered. Peer reviewers and board members who are asked to comment on PPI should be supported in doing so. Adoption of critical appraisal guidelines may be beneficial in achieving this. 95

Funding is available to support pre-application PPI; for example, the UK-based NIHR Research Development Service offers very small grants, which others have found to be helpful. 73,74 However, these grants are not easily or quickly accessible, particularly for those working to the typically tight deadlines of funding calls. Paradoxically, this renders pre-application PPI the most difficult to implement, even though our findings indicate that PPI is often most useful at this stage. Innovative organisations that involve patients at a metatrial level in research priority setting96 and in schemes such as COMET (Core Outcome Measures in Effectiveness Trials),97 which promotes the involvement of patients in developing ‘core outcome sets’, are providing knowledge and resources that individual trials can use. However, at the level of individual trials, infrastructural support for early PPI is also needed. Although there have been innovations in this area – for example, the US-based Patient-Centred Outcomes Research Institute has recently announced a number of ‘Pipeline to Proposals’ Engagement Awards8 – such moves are relatively novel, and similar steps by other organisations would be beneficial.

As well indicating the need for structures and resources to support PPI, our findings point to the importance of PPI that is fit for purpose, realistic and proportionate. We found that triallists who fully implemented a primarily oversight mode of PPI perceived little value in this involvement. Although oversight PPI seemed limited in terms of its practical impact, arguably it may serve important ethical and moral functions. However, in order to avoid inadvertently promoting PPI that is devoid of any function for both researchers and contributors, as we note above, funders should take full account of any PPI which has taken place prior to funding applications as well as encourage applicants to justify future plans for involvement. The NIHR HTA programme states: ‘While patient and public involvement (PPI) may not always be needed for all types of research, it is always relevant for HTA trials’. 98 Even if there is consensus that PPI is relevant for all trials, it may not be relevant at all stages of all trials. Equally, funders may wish to contemplate how ‘contingency’ resources could be made available for those trials that encounter unexpectedly intensive needs for PPI over the course of their implementation.

Accessing the perspectives of both PPI contributors and researchers is a strength of the EPIC project; however, this was limited by the difficulty in contacting PPI contributors. For future prospective assessments of PPI, both researchers and PPI contributors should be involved in its independent evaluation. Problems with the conceptualisation and measurement of the impact of PPI have been identified making meaningful evaluation problematic. However, funders could help by leading on prospective evaluation of PPI, utilising their strength to document and evaluate PPI processes rather than in the specification of the approach that should be used. Such specification may discourage researchers from considering what PPI is really needed or even act as a constraint.

Funders need to establish a register of contact details for PPI contributors to ensure contact is not required to be via CIs. In addition, progress reports requested by funders during trial conduct now request information on what PPI has occurred since previous reports. Requesting such progress reports from PPI contributors as well as researchers would be beneficial and allow funders to consider both researcher and contributor perspectives and provide communication between contributors and those funding their participation.

Conclusion

We conclude that if researchers, PPI contributors and research funders wish to enhance PPI in trials they should consider how PPI can inform or benefit a trial and plan PPI to suit these goals, work to develop good relationships between PPI contributors and researchers, involve PPI contributors at an early stage, and favour responsive and managerial roles for PPI contributors in preference to roles that involve only oversight.

Effective mechanisms to obtain diversity of PPI contributors need to be explored. Selection of contributors has been identified as a training need and the use of mixed models has been suggested, to allow benefit from experienced contributors on oversight or trial management committees and research-naive contributors on responsive groups. However, where the aim of PPI is to gain a wide spread or diverse opinions, the role of qualitative researchers to support PPI in delivering such goals should be considered.

We recommend that funders remove PPI tick box sections from their forms and instead request a PPI-specific protocol separately requesting goals, methods and costs of PPI; that approach should enable reviewers to appraise the relevance and appropriateness of such plans. We would also advise funders against specifying the nature of PPI activity, to avoid minimalistic approaches intended solely to comply with funder requirements. We recommend increased availability and levels of funding to support pre-application PPI, and the identification of contingency funds to support PPI in response to unplanned need.

We also recommend that PPI contributors be enabled to report on their activities directly to the funders, and that the UKCRC formalise requirements for registered CTUs to support PPI activity. CTUs are ideally placed to lead on the development of a risk-based approach to PPI and of resources to evaluate PPI. They would also be central to encouraging greater peer support between PPI contributors both within and between clinical trials.

Collaboration between funders, INVOLVE and the UKCRC network of registered CTUs should be increased to ensure that they are aware of each other’s available resources, expectations and constraints. Such collaboration could be used to identify core materials that should be packaged for CTUs to provide to researchers and PPI contributors engaging on a trial to enable role negotiation, manage expectations and identify training needs to enable PPI contributors to function in their role.

Department of Health disclaimer

This report presents independent research commissioned by NIHR. The views and opinions expressed by the interviewees in this publication are those of the interviewees and do not necessarily reflect those of the authors, the NHS, NIHR, the Medical Research Council, Central Commissioning Facility, NIHR Evaluation, Trials and Studies Coordinating Centre, the Health Services and Delivery Research programme or the Department of Health.

Contribution of authors

Carrol Gamble, professor of medical statistics, conceived the idea for the research and led the development of the grant application and project. She developed the data extraction tool, developed the database, extracted data, analysed data, developed and analysed the surveys, developed the interview schedules, commenting on the ongoing analysis and cowrote the report.

Louise Dudley, psychology research assistant, was involved in the development of the data extraction tool, completed data extraction, developed the surveys and was involved in developing the interview schedules, recruiting informants to the study, conducting interviews, conducting qualitative data analysis and interpretation, and writing the report.

Alison Allam, Philip Bell, Heather Goodare and Alison Walker formed the Public Advisory Group. A full description of the Public Advisory Group activities is provided in Appendix 1.

Deborah Buck was involved in conducting qualitative data analysis and interpretation, and writing the report.

Bec Hanley, director, TwoCan Associates, contributed to the project specification, commented on the data extraction items, surveys and manuscript, and co-led the co-ordination of the Public Advisory Group.

Jennifer Preston, consumer liaison manager, contributed to the project specification, commented on the data extraction items, extracted data, commented on the surveys and manuscript, and co-led the co-ordination of the Public Advisory Group.

Paula R Williamson, professor of medical statistics, contributed to the project specification, commented on the data extraction items and the surveys, and provided comments on each phase of the report.

Bridget Young, professor of psychological sciences, contributed to the project specification, commented on the data extraction items and surveys, led the qualitative phase of the report, including the interview schedules and qualitative data analysis and interpretation, and co-wrote the report.

Publications

Buck D, Gamble C, Dudley L, Preston J, Hanley B, Williamson PR, et al. From plans to actions in patient and public involvement: qualitative study of documented plans and the accounts of researchers and patients sampled from a cohort of clinical trials. BMJ Open 2014:4:e006400.

Gamble C, Dudley L, Allam A, Bell P, Goodare H, Hanley B, et al. Patient and public involvement in the early stages of clinical trial development: a systematic cohort investigation. BMJ Open 2014;4:e005234

Dudley L, Gamble C, Preston J, Buck D, Hanley B, Williamson P, et al. What difference does patient and public involvement make and what are its pathways to impact? Qualitative study of patients and researchers from a cohort of randomised controlled trials. PLOS ONE 2015;10:e0128817.

Dudley L, Gamble C, Allam A, Bell P, Buck D, Goodare H, et al. A little more conversation please? Qualitative study of researchers’ and patients’ interview accounts of training for patient and public involvement in clinical trials. Trials 2015;16:190.

Data sharing statement

The report is based on data extracted from NIHR HTA reports, survey responses and qualitative interview data. Anonymised data is available on request from the author for survey responses and HTA report extracts. Data from qualitative interviews is not available because of the difficulty in making interviews anonymous. We have provided relevant anonymised data excerpts from qualitative interview transcripts in the main body of the report.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health.

The EPIC advisory group

Topic guides

The topic guides for PPI contributors and TMs mirrored that for the CIs.

Chief investigator