ReseArch with Patient and Public invOlvement: a RealisT evaluation the RAPPORT study

Normalisation Process Theory

In attempts to help understand why it is that seemingly effective innovations often fail to flourish when rolled out in practice (context), NPT addresses two important problems that face service improvers as they try to get innovations into practice:

• process problems – about the implementation of new ways of thinking, acting and organising

• structural problems – about the integration of new systems of practice into existing organisational and professional settings.

Because PPI in research is not an entirely ‘new’ innovation, intervention, programme or phenomenon, but also is not yet fully normalised within contemporary research practice, it has been suggested15 that NPT offers unique potential for helping theorise how PPI is/is not routinely operationalised (embedded) and sustained (or not) in practice.

As an ‘action’ theory, NPT also sharpens our analytical focus on the actual work that people performing PPI do, or do not do, rather than people’s attitudes (how they feel about what they do) or their intentions (what they say they are going to do). 134 NPT distinguishes action according to a set of four constructs. Each of these represents a generative mechanism of social action. That is, each construct represents different kinds of work that people do in orienting themselves to a set of practices of PPI in research. These constructs and their components (domains) (Table 3) address the ways that people make sense of the work entailed in implementing and integrating PPI (coherence); how they engage with this work (cognitive participation); how they enact it (collective action); and how they appraise its effects and modify it (reflexive monitoring).

Normalisation Process Theory has been developed for using in a flexible and dynamic way and can both direct and sensitise the practical and analytic trajectories of a research project. There is currently no published review of its empirical application, but it has been seen to have been applied to good effect in a number of social and health-care settings. 121,134,135 Of relevance in the RAPPORT study, and discussed in the following sections, is that case study methods have been seen to be more robust when theoretical propositions to guide data collection and analysis have been previously developed. 136 We have used NPT constructs and domains both to direct our data collection and to guide our analysis, particularly within our case study work. In effect, these domains map across to the RE, and are most significant in theorising PPI mechanisms and outcomes, and to some extent the context, but, as discussed below, were combined with additional questioning and coding to inductively explore other emerging significant themes.

We also drew on previous work focused on the architecture and infrastructure29,137 required for PPI to be operationalised.

Sample

Our sample here included current studies or those completed in the previous 2 years in the six exemplar areas and being undertaken in England. To ensure relevance to the NHS and scientific robustness, studies were limited to those registered with the UKCRN portfolio, excluding commercially funded studies. To ensure inclusion of studies most likely to have been designed since the embedding of PPI in the research governance framework,1 we excluded studies older than 2 years (end date of recruitment before 1 September 2009). We also excluded studies funded by commercial organisations because, despite current criticisms of the lack of transparency in drug trials,140,141 access to the study documents required was likely to be limited.

Data collection

Studies were identified using the using the search terms in Table 4.

Details of each study were electronically searched via funding body databases. Relevant documentation such as reports, abstracts and protocols was reviewed for the type of research (e.g. laboratory based or qualitative) and any evidence of the nature and extent of PPI. Where information was not available electronically, the funding body was contacted and requested to provide further relevant details of the study. Indeed, one of our findings from this initial scoping exercise is that there is minimal direct open access to such study documentation. Significantly, even when the protocol was available on the funder’s or study’s website, there was little information about PPI. Therefore, we also contacted study teams directly or through collaboration with the Association of Medical Research Charities (AMRC) for PPI information.

Scoping response

In total, 1464 studies in the six broad topic/exemplar areas were downloaded from the UKCRN database. One hundred and two were excluded as not in the specific topic areas, 263 as more than 2 years old and 261 as commercially funded. A total of 838 studies were included (Table 5). These were then divided into the 478 included in the scoping, and 360 for the online survey. All of the latter group were within the four selected regions of England (Figure 4).

FIGURE 4.

Flow diagram of scoping and survey.

In the UK, clinical research networks have been established in each of the four UK nations funded by their health departments. These national networks form UKCRN. The UKCRN portfolio of studies was used as one single database to identify relevant research studies, and information including study title, sample size, end date of recruitment and contact details was downloaded into an Excel^® (version 14, Microsoft Corporation, Redmond, WA, USA) spreadsheet.

For studies not covered by the online survey, that is running outside the four selected geographical regions, including studies in Scotland, Wales and Northern Ireland, direct requests were made. Documents or e-mail replies were then reviewed by two team members independently for background study information and any evidence of the nature, extent and activities of PPI. The scoping exercise obtained information about PPI from 182 studies out of a possible total of 478 (38%) (Table 6). Documents received in response to requests for PPI information included 93 protocols, 12 journal articles, 64 e-mail replies, 10 interim/final reports and 3 Integrated Research Application System (IRAS) forms. Other documents (such as grant applications, reports to funders and lay summaries) were also provided. Some studies provided more than one source of information. The studies in the scoping had sample sizes ranging from 12 to 250,000, with a median of 250. The two main funders were NIHR (29%) and charities (29%). The remaining funders were a mix of government departments, the European Union, research councils and others. Forty-two per cent of the scoping studies were clinical trials, 33% were mixed methods (qualitative and quantitative non-clinical trials), 10% were basic science and 15% were tissue bank/database.

Analysis

Variations in PPI were mapped against the type of research, topic area and funding body. Drawing on the conceptual framework of Oliver et al. 132 and the PIRICOM systematic review,29 a scoping framework was used to assess the stages of the research at which PPI took place, whether involvement was of lay groups or individuals and where it was located on the continuum of PPI from user-led to minimal PPI. We also assessed evidence of the ‘architecture of involvement’:29 those processes needed to enable PPI in research, which included a budget for PPI, defined roles for lay people, training and support, and means of communication between researchers and lay people. This was used to evidence the range of resources supporting PPI. The results from this analysis are presented in Chapter 5.

Sample

Four geographical regions in England were purposively selected for the survey to ensure maximum variation: the South West has a relatively large rural population and a long history of PPI;143 the East of England has established PPI networks144 with rural and urban areas; there is less evidence of any long-established PPI history in the North East, which has one major research hub and socioeconomic factors affected by a heavily industrialised past; and London is the most densely clustered in terms of research centres and has a relatively high population of people from black and minority ethnic (BME) groups.

Data collection

Drawing on the findings of the PIRICOM study,29 a validated online survey tool (Bristol Online Surveys, University of Bristol, Bristol, UK) was designed to collect data to assess how PPI is operationalised and its perceived impact on each study. Although the PIRICOM study found little evidence of PPI being theoretically conceptualised, there has been some work to develop a consensus on the principles and indicators of successful PPI,133 and this was used to underpin the survey design. The research team’s previous experience of using online survey tools suggested that good response rates are achieved when the survey remains focused, it takes no longer than 15 minutes to complete and rapid feedback is promised to participants. There was close working with the appropriate comprehensive local research network, specialty networks and local specialty groups to publicise and disseminate information about the study. The survey was electronically distributed to lead investigators across the four diverse regions and its findings also provided the test-bed for stage 3. Participants received a summary of findings.

Survey response and study characteristics

From the UKCRN portfolio 360 studies across the four regions were identified and an e-mailed invitation and link to complete the survey was sent out to all named chief investigators (CIs) or researchers. From the first e-mail there was a 17% response rate and following the reminder there was a total of 101 responses to the survey (28% response rate) (Table 7).

The 101 studies surveyed ranged in sample size from 5 to 300,000, and grants awarded ranged from less than £50,000 (8% of sample) to 30 over £1M (30% of sample). Funding was split evenly between NIHR (40%) and charities (40%). The survey included 40% clinical trials, 33% quantitative and qualitative, 12% basic science and 7% tissue/bank database and a further 7% were coded ‘other’ (as insufficient information to classify).

Analysis

The focus of the stage 2 survey was to describe researchers’ experiences and perceptions of positive and negative outcomes of PPI, and benefits and challenges of PPI for researchers. The focus of analysis in this stage was therefore the use of summary statistics alongside qualitative analysis of open-text responses. Using an electronic survey tool enabled rapid access to data that were readily transferred to SPSS (version 20, IBM SPSS Statistics, Armonk, NY, USA) for analysis. This analysis was mapped against the results of the scoping in stage 1 to identify any recurring patterns within and between types of research, stages in research process where PPI occurred and topic areas. The findings from these two phases were used to develop a sampling frame for the final case study phase. The survey findings were then used as one of the ways to identify research teams likely to be interested in taking part in stage 3.

Sample

To ensure typicality and sufficient variation across topic areas, types of research and geographical regions, a sampling frame (Figure 5) developed from the previous stages was used to purposively select up to 20 case studies where the case is a single research study. This enabled maximum variation, but also allowed comparison within and between region, topic area and type of research.

FIGURE 5.

Sampling frame. Arth, arthritis; Dem, dementia; Diab, diabetes mellitus; PH, public health.

Data collection and recruitment

To ensure a range of studies that varied in their stage of research implementation, several recruitment strategies were employed:

1. Potential studies were identified from the survey.

2. Suitable early stage studies that had been awarded funding but had not yet been adopted on the UKCRN portfolio were identified by local Research Design Services (RDSs), by relevant charitable funding organisations from the AMRC, and by networks such as the Primary Care Research Network from their futures list. The RDSs, AMRC organisations and local research networks such as the Primary Care Research Network sent out information about the RAPPORT study and investigators interested in participating were asked to contact the RAPPORT study team.

Data were collected from each case study to inform the three stages of realist enquiry – context, generative mechanisms and outcomes119 – using the methods of documentary analysis, semistructured interviews with key informants from case studies, tracking and semistructured interviews with research funders and networks as follows:

Overview of the analytic approach

Semistructured interview schedules for this study were informed by NPT, as was the initial coding frame, but, in line with guidance given regarding coding data using NPT, this did not preclude us from exploring other avenues in the data or limit our thinking.

If you are doing some form of coding via NPT constructs and components, you always need a code that is ‘other’, as in, issues and topics that do not clearly fit within the NPT. 151

Inductive codes were also developed.

Thematic content analysis was used first to identify key themes and common experiences as site specific and then to make cross-case comparisons. This established how different contexts (e.g. topic area, type of research and setting) affected processes (e.g. training and support), mechanisms (e.g. relationships and communication) and outcomes of PPI (e.g. evidence of changes, responsiveness to PPI input and impact on dissemination). Research team data analysis meetings provided a productive environment for generating themes, collective sense-making and enriched understanding through all stages of the study.

Utilising a similar approach to those of Schierhout et al. 152 and Rycroft-Malone et al. ,153 the analysis involved the following steps:

1. A subset sample of initial interview transcripts of interviews with both researchers and PPI representatives as conducted by various team members were coded independently (manually) by team members (PW, EM, JK, MC and DM). This was done both deductively, using an initial adapted version of the 16 NPT components, and inductively, capturing unforeseen themes which emerged within the data relating to our research questions. 154

2. A meeting was then held to discuss the coding of each sample interview, to address inconsistencies in the application of the coding categories and to refine collective understanding of them (increasing inter-rater reliability). We added several subcategories; for example, under the category ‘7. legitimation – participants do/do not buy in to the intervention (PPI)’ we added the subcodes ‘a) keen to do the research’ and ‘b) keen to do PPI’ to reflect distinctions within the accounts given by participants. We also further adapted the NPT coding frame to distinguish between where participants were discussing potential (hypothetical) and actual outcomes of PPI within case studies. In addition, given that NPT is an action theory, we ensured that coding categories captured where participants were reported not to have something as well as where they did have it, and that such instances were not labelled based on judgements or assumptions. Indeed, minor revisions were made throughout (e.g. the addition of inductive codes such as ‘background to project’).

3. All data relating to the identified categories were highlighted within the first round of interview transcripts and documents collated for each case study, using NVivo 9 (QSR International, Warrington, UK).

4. Data generated through interview were triangulated (including with study documents) to corroborate, elaborate, contradict or complement other data,155 in order to interpret PPI processes.

5. An initial summary of the NPT analysis and a RE were then written for each case study and shared within the team to check consistency in usage of the coding categories.

6. The Economic and Social Research Council-funded web-based NPT toolkit was then used151 to develop radar plots for each case study. These plots provided a temporal snapshot: a visual representation of the embeddedness of PPI within a case study within the four constructs and 16 domains. It must be noted that the expression of embeddedness is, of course, a function of the PPI design on different projects: the fuller the radar plots, the more fully embedded the PPI. Spikier plots illustrate internal variance in the degree of embeddedness (Figure 6).

FIGURE 6.

Levels of PPI embeddedness illustrated through NPT radar plots.

Radar plots supported our understanding of changes within case studies over time and ongoing comparison between them. Both the RE summaries and radar plots acted as aides-memoires for the research team throughout the research and also helped to maintain the specificity and wholeness of each case study. Summaries were updated and radar plots were re-evaluated at 18 months or the end of that study to capture/reflect any changes in PPI processes.

1. We repeatedly read, viewed and discussed and compared data from each case study at regular team meetings to develop our analysis and to identify emerging features of CMO across case studies.

2. We then tested CMO configurations across the data set as a whole, accepting, rejecting or refining CMO configurations through discussion.

3. Context, mechanism and outcome configurations were further refined through discussions with case study respondents, the reference groups and the RAPPORT advisory group, and four regional events conducted at the end of the study (see Appendix 5).

The steps above were initially undertaken in the order shown, although most were returned to repeatedly and iteratively.

A finalised NPT coding frame was devised and subcodes were added to the NPT constructs outlined earlier, and further inductive codes (such as PPI model description and personal narrative) were added. Analysis was informed by the research questions and RE, with methodological triangulation occurring during analysis and interpretation of results. 156

Coapplicants

Two members of the University of Hertfordshire PIRG were coapplicants on the RAPPORT study.

Co-researchers

There were three co-researchers: the two coapplicants plus a mother of a child with CF. These three were involved throughout the study, but were particularly involved in case study data collection (interviewing) and analysis. The two lay coapplicants were members of the research team (out of a total of 11 members), which met every 3–4 months. Team meetings were held face to face every 6 months, with a teleconference in between. The research team commented on all study documents, findings and publications.

Reference groups and representation

The project had reference links who were consulted during the progress of the study to add insight on PPI in each topic area. Some links were via the research team itself: one co-researcher had a child with CF, one had diabetes, one had arthritis, and two had cared for a family member with dementia and had been on other studies involved with dementia. Where links did not exist, reference groups were set up: an IDD group and a CF group of young people. The IDD reference group met face to face during the project to facilitate communication (including Easy Read materials); the CF reference group ran on a virtual basis.

Project advisory group

The project advisory group (a total of 22 members) met in person every 6 months and was well attended by the five lay representatives and the lay chair. The lay representatives consisted of two members from PPIRes, the two co-researchers from the University of Hertfordshire PIRG and one lay representative from the RCGP. In addition, the advisory panel included representatives linked to reference panels and lay groups: the Learning Disabilities Research and Development Manager (NHS North Essex) and the National Cancer Research Network Consumer Involvement Lead, who was later replaced by the Mental Health Research Network PPI Lead.

As for all advisory group members, PPI representatives reviewed all documentation concerning the research, and commented on emergent findings and study progress; meetings were minuted in detail so PPI contributions were included. The topic of PPI in the RAPPORT study was also an additional standing agenda item at each meeting, to ensure that feedback was welcomed and queries or concerns raised openly; this added to the reflective material utilised to assess impact within the study itself.

Research design

Four members of the University of Hertfordshire PIRG submitted initial ideas for the research proposal; their contribution actively shaped and informed the proposal design. From the beginning, two research team/PIRG members actively contributed to the development of the full proposal, which was also discussed at regular PIRG meetings.

Data collection and data analysis

Three co-researchers were heavily involved in stage 3: case studies. The agreed main purpose of their input was to enable engagement with those playing PPI roles in some of the case studies: a peer-to-peer interview, as used in previous studies. 88–90 This was expected to add value to the study by allowing themes to emerge that were different from or additional to those captured by other means, even though the co-researchers were also involved in reflective analysis of the whole data set, not just their own interviews.

In recognition of the co-researchers’ needs, the whole research team developed a training session on interview techniques and the use of digital recorders, and co-researchers also helped develop the interview schedules. The peer interviews (with PPI representatives on case studies) were allocated according to interest, availability and background, sampling a total of eight case studies. Co-researchers were supported in carrying out the telephone interviews (the majority of interviews were by phone), either at the University of Hertfordshire or in the co-researchers’ homes, and carried out a total of 14 first interviews and three tracking interviews. The numbers of peer interviews were lower than originally planned because of difficulties in finding mutually convenient times for co-researchers and respondents. In reality, co-researchers had less availability and flexibility to accommodate interviews that other members of the research team.

Once the interviews were transcribed, 11 data analysis sessions (including training) were held with the researchers and the co-researchers (either individually or as a team). Only one further data analysis meeting was held without the co-researchers when they were unable to attend. The initial meetings were used to agree on coding structures and the use of NPT and to compare individually coded transcripts. Subsequently, data analysis meetings tested out the CMO configurations within and across the various case studies. PPI co-researchers also individually coded interview transcripts.

Dissemination

Study development and protocol

Two quotes from the report of 2010 show that there was lay involvement as soon as the funding call came out and in developing the proposal:

Details of the call [for funding] were circulated to the PIRG for comment and initial ideas; four members responded. Responses were fed into the pool of initial ideas about the proposal. Ideas were fed back to the PIRG as they developed and after two weeks of on-line discussion PIRG members were asked what level of continuing activity they would like to undertake in the proposal development. Two members volunteered as co-applicants, the other two wished to contribute informally to the development. Additionally, the developing proposal was presented at the regular PIRG meetings to enable the whole group to have an input.

Early drafts of the outline proposal were circulated to the 2 PIRG members as part of the co-applicant team. They actively contributed to the development of the outline proposal by:

Data collection

Interviews carried out by PPI representatives (peer interviews) had some additional perceived benefits. PPI members often had more knowledge about a particular health condition than the researcher and could better relate to the interviewees:

I felt her [PPI’s] warm personality and humour shone through, and this made it very easy to build up a rapport with her quickly, and therefore make more effort to contribute. There was careful and gentle questioning, never intrusive, and she encouraged me carefully to think of other things I might have missed. Throughout the interviews I have had with her, I felt I was a real person and not an object of research.

RAPPORT PPI respondent CS17PPI01, e-mail exchange with respondent

[I had a] lot in common with her . . . think the connection we made was important, both had daughter with [health condition] . . . it felt more like you were having a chat as opposed to doing an interview.

Co-researcher03, 26 November 2012

The relationship between co-researcher and respondent also developed with time:

Yes, you know, it was like . . . she was asking about my daughter, I told her she might be going to hospital, she asked how was she and she said you have my number if you want to call me.

Co-researcher03, 26 November 2012

However, this benefit of peer interviewing was not considered to have the same impact across data collection. Not all experiences are shared: ‘I’m not sure user involvement made much difference in this interview as she was only [in her twenties] and not closely related to my experience’ (Co-researcher03, 18 October 2012).

One of the researchers commented that ‘re-designing the interview schedule so that it was more informal and chatty, involving people with real experience in talking to PPI reps in similar positions, helped make interviews more relaxed, open and reciprocal’ (reflections July 2014) (Box 4).

Data analysis

The RAPPORT team, including the co-researchers, used NPT as the candidate programme theory and also used it to shape the interview schedule and data analysis. One co-researcher found framing data analysis through NPT constructs ‘intellectually stimulating’ (Co-researcher02), while another found using interpretive approaches more generally (and NPT’s abstractions in particular) to be very challenging. This co-researcher devised her own set of inductive codes and analysed the transcripts using her own coding system. The use of this separate coding process added to the rigour and validity of data analysis as similar themes emerged:

I also think that having co-researchers independently coding transcripts added rigour to the process – I remember concerns from the 3 core researchers about whether there was data forcing into the NPT constructs. [X] went off, did free inductive coding and when we compared the same findings emerged – quite a relief!

Research team member reflections July 2014

Impact of patient and public involvement on representatives

As found within the RAPPORT study, the co-researchers gained personally from carrying out the research:

stimulating to get me back to using my brain a bit more and think about research.

Co-researcher03, 27 April 2012

However, this did not apply to the same extent to those who were not co-researchers:

I found this quite difficult to do, as I want to do more than you are asking me to do all the while.

Lay advisory group member, reflective meeting, 15 January 2014

Discussion of emergent findings

Patient and public involvement representatives on the research and advisory teams continually reminded the researchers of what is important, such as the current debate around PPI and professionalisation. The research team was able to test out the findings from the survey/scoping and case studies with PPI representatives as stakeholders who are aware of the current climate of PPI.

Coherence

Roles and expectations of PPI representatives. The original project’s assumptions about the different roles, expectations and capacities of the various PPI representatives within the RAPPORT study needed revisiting over time: ‘Being on an advisory group I found quite limiting’. After a discussion at the end of a project meeting about whether or not being a PPI member of an advisory group was superfluous, it was then acknowledged that ‘they have a real role of accountability and PPI need to play a part in that’. There was also positive feedback about the improvements over time: ‘advisory groups have a bad reputation in PPI as having been tokenistic, this project did not feel like that’ (all three comments from the advisory group reflective meeting, 15 January 2014).

One co-researcher (Co-researcher02) was ‘disappointed’ that she was not able to do more interviews and had expectations of doing more than the research team were able to organise. However, she acknowledged this was also because of her own problems with availability of dates and different topic areas. She also felt she had ‘let the team down by not keeping to deadline’ (this was not seen as a problem by the research team). This co-researcher also reflected on the ‘difficulty of keeping sometimes, a lay role and not getting carried away with all the theoretical and methodological aspects . . . and a recognition of my own bias’. This comment resonates with the debate about ‘professionalisation’ of lay participants, and perhaps shows that there will not be one long-lasting ‘coherent’ view of PPI – rather a need for renegotiation and flexibility.

Participation

There was feedback that having support for personal and practical queries improved inclusiveness and contribution. Having a local person to co-ordinate PPI travel, expenses and other practical matters was found to be very important, as was accommodating preferences for written/hard copy materials rather than electronic communications:

paying attendance makes me feel valued, my time feel valued and respected, it is a big thing for me, not just paying me enough for the day, they are paying me, respecting me and my time.

Lay representative, advisory group reflective meeting, 15 January 2014

Collective action

Relationships. Replicating the findings from the RAPPORT study, the importance of relationships emerged as central in this study. Two of the co-researchers had a history of working with the CI and the team based at the University of Hertfordshire: there was prior shared understanding and trust, and this developed during the time of the project. The two PPIRes members on the advisory group already had relationships with coapplicants at the University of East Anglia but not with the University of Hertfordshire or other research team members, and new relationships across institutions developed over time. All advisory meetings started with lunch, allowing time to socialise. With hindsight, it was acknowledged that more time for the RAPPORT team to meet informally would have been beneficial, but lengthy journeys and budget constraints would have made this difficult. One co-researcher commented on:

the challenge of keeping everyone in PPI in ‘the loop’ being a task – with our other PPI colleagues being at a distance, but there was continued involvement and commitment to keeping all in touch and involved.

Co-researcher02,15 January 2014

Relationships had to develop with new members of the advisory group and with the reference groups, and these took time to establish. Explicit emphasis on this might have increased input and facilitated early project processes:

Usually I am approaching people to be on the advisory group that I have already have some relationship with or worked with before . . . it was quite different to have people recommended that I had not met in depth before, I think I had been quite naive before . . . if you haven’t got that relationship . . .

Team researcher reflections, 15 January 2014

Flexible provision for access. Respect for and accommodation of PPI representatives’ different skill mixes and requirements was an issue. For example, two of the co-researchers had different qualitative data analysis skills, two of the PPI representatives did not use computers and preferred to receive and annotate paper copies and one PPI representative on the advisory group asked for contact to be made by text.

Health/illness. Over a 2.5-year study, illness has meant that at times it was difficult for some PPI representatives to travel long distances or into London and be as involved in the RAPPORT study as they would have liked. Not all activities could be undertaken as expected, and the team worked around to find solutions. Every effort was made to be inclusive, by holding meetings locally wherever possible, organising convenient parking at the university, providing lifts, accompanying representatives on public transport, paying for taxis and so on.

Reflexive monitoring

Both lay advisory group representatives and research team members were encouraged to record formative reflections upon PPI and were provided with prompt questions and notebooks. However, as the study progressed, this proved difficult to maintain, with lay representatives reporting they did not do it immediately and then forgot to do it, so opportunities were instead created to prompt and audio-record reflections informally as conversations before or after certain meetings throughout the study. Two of the co-researchers, however, did provide the paper records of their formative thoughts at the end of the project.

A more formal summative reflective session on PPI in the RAPPORT study was conducted to capture the impact of PPI within the study. Two face-to-face meetings were held towards the end of the project: the first was with members of the research team (facilitated by two external facilitators) and a second for members of the advisory group. Each session was recorded and summary notes produced. An ‘ALPS’ [Amander’s Levels of Patient and Public Involvement Significance, a reflective tool developed by A. Wellings, a lay member of the advisory group, and reproduced with permission (Amander Wellings, PPIRes panel, 2014, personal communication)] was sent to all members before the reflective session and provided a stimulus for this discussion (see Appendix 6).

Coherence

A surprising finding was the existence of a dual role. It was not clear whether this role was deliberately created or was due to a lack of understanding in the difference between participation and involvement.

Cognitive participation

Enthusiasm or buy-in to PPI was often evident in the free-text responses; however, the extent of this buy-in must be questioned when 17% of studies from the survey reported having no PPI.

Collective action

It was also surprising that the relational work to embed PPI was so prominent. For basic science and tissue banks reporting PPI, this work appeared particularly important in an area of research where PPI was less commonly found. The necessary tasks for PPI did appear to have some slight differences in topic areas, but the scoping and survey were unable to capture these nuances in detail.

Reflexive monitoring

Free-text responses clearly highlighted some positive appraisal of PPI. While there was some evidence of systematic evaluation, overall evidence for this was sparse.

Summary of key findings

This is one of the first studies to contribute significantly to our understanding of PPI as a concept, through the analysis of a large number of case studies, selected to capture varying levels of PPI embeddedness, in order to understand how it may change over time during the research process.

In testing whether or not topic area had an influence on the embedding of PPI, we found similarity between the nature of the clinician–patient relationship and the way PPI was enacted. This was further influenced by the topic area’s history of PPI and the tacit models that had evolved. For example, we found PPI in IDD studies to be framed by an empowerment model and resonant of the long history of partnership working between practitioners and service users. Well-established PPI structures in some topic areas such as dementia provided a pragmatic source for the lay perspective, but we found unintended outcomes for this model, including the outsourcing of PPI to external parties with little researcher engagement. It also exemplified a ‘one-off’ model whereby PPI was limited to a single task.

Researcher perceptions of what lay representatives could be expected to understand or be interested in, coupled with a lack of effort to communicate in an accessible way, sometimes led to limiting lay involvement in parts of the research process, potentially limiting impact. However, we also found in areas such as basic science and tissue banks, where PPI has traditionally been seen as having a limited role, that research teams were enabling PPI, leading to improved relevance of research priorities and questions.

While still challenging for a number of studies and funders, recruiting a diverse range of lay people was enhanced when the study setting interfaced with a diverse population. Particular PPI structures helped, including an outreach model with either a specific lay person or a co-ordinator acting as a bridge to a broader community. The quality of such connections was important to create the context for involvement. The resulting impact was culturally sensitive, participant-focused research designs.

In contrast, researchers’ descriptions of ‘usual suspects’ as lay representatives reflected a perception of concerns about the professionalisation of PPI. However, our findings suggested this was at odds with reality and with the advantages of the experienced lay representative being able to develop a number of skills that helped their role, and enabling a career trajectory for the lay person within PPI.

An important finding in both the survey and the case studies was the dual role, whereby the lay advisor was also a study participant. Although this had the benefit of real-time feedback about the impact of the research design and enhanced a sense of ownership of the study for lay respondents, there was also evidence of conflicts of interest and a blurring of roles. In some case studies the dual role was a deliberate PPI design; however, in others it clearly demonstrated that involvement was not understood as being different from participation, and highlighted the level of PPI drift away from INVOLVE definitions. It was also indicative of a lack of training or experience in PPI, particularly in novice researchers. Training or on-the-job experience not only helped researchers to differentiate between involvement and participation, but also enhanced an understanding of what both parties bought to the table. Development of positive relationships underpinned by trust was the minimal requirement for any form of PPI; this led to improved self-worth for lay people, and increased researchers’ confidence in their work. Breakdown in this relationship had a profound effect on levels of PPI embeddedness over time.

It was clear from the case studies that PPI was most likely to be embedded when hosted in well-resourced research centres with a dedicated budget and PPI co-ordinator. PPI groups and panels had continued engagement with the research host organisation, were reimbursed equitably and had social interaction with the research teams; these are crucial to the sustaining of positive relationships. Resourcing PPI in such a way also enabled a fully entwined model of PPI, in which lay people and researchers worked seamlessly together, with research priorities and research questions created by the lay people involved. This generated some research programmes that would not have been run without lay involvement. However, embedding PPI to this level required not only resources but also time to develop.

We found that funder requirements played a key role in influencing levels of reimbursement. In addition, how PPI was operationalised in individual studies could be traced back to the funder’s ethos of PPI. Within funding organisations there was a growing interest in evaluating processes and outcomes of PPI within studies. However, even with funder expectations, only a limited number of case studies were reflecting on and evaluating PPI in an ongoing way to improve the quality of PPI processes and outcomes.

Enabling contexts

The findings suggested there were eight enabling contexts for the mechanisms (Box 5). Five of these (funder, topic area and design, resources, organisation of PPI and research host) were expected, based on the literature, and shaped our sampling framework to ensure a range of case studies as a test-bed. The other three (training, positive relationships and positive experience) became visible during data collection and analysis.

Context–mechanism–outcome configurations

As described in Chapter 3, through an iterative process we tested candidate CMO configurations within each case study and discussed our findings with case study respondents in the tracking interviews, with our reference groups and finally at the four regional events. We identified six CMO configurations that explained salient case study actions, potential contexts, plausible mechanisms and potential outcomes, both intended and unintended. CMO configurations focus on immediate outcomes within a study. It was not possible to include longer-term outcomes of a study, as these evolve over a longer period of time. Tables of each CMO are presented and then illustrated by data and case study exemplars. Within exemplars, NPT radar plots will be shown, and any change in PPI embeddedness over time will be illustrated through tracking radar plots.

Funder requirements

Researchers were particularly aware that when applying for specific grants there were even stronger imperatives to have PPI as a funding requirement. PPI was recognised as a part of the contemporary research landscape, and some reflected that having formalised PPI as a requirement was beneficial, as experience of good PPI usually leads to more in the future:

more researchers are willing to engage, whether that’s because they want to or whether because they have to because of the grant application, I think there’s a bit of both there, but then again, that’s fine because those that . . . engage because they have to kind of start to begin to see the worth of it anyway.

CS07Res01-01

Overall, the funder respondents clearly asserted that PPI both in research and at funder and system level had developed significantly over the previous 5 years, describing a journey along a continuum of learning and understanding of PPI, despite noting that there was still some way to go until it was fully embedded as normal practice:

I think organisationally that’s been a sea-change . . . Externally with researchers I sense there has broadly been a change but there’s still a lot to do, but certainly the quality of the PPI anecdotally that we’re getting in our applications is on the up, possibly to do with . . . everything being reviewed by a public contributor.

Funder01

Structures and processes for PPI within funder organisations themselves inevitably varied and strongly reflected the influence of their core funding source, mission, scale and length of service. PPI structures had developed either quite systematically or more recently and organically as part of the overall PPI movement.

Working closely with INVOLVE, two overarching programme management organisations and two research programmes that are funded by central government had, over a number of years, developed similar, embedded structures for PPI within their organisations. However, the sheer volume of applications to process could potentially result in the PPI representatives having to work with a conveyor belt approach. The industrial scale of PPI as metaphor was also picked up by one of the lay representatives:

I think, having started from a low base, it’s [PPI] now in danger I think of almost becoming an industry; that is that everybody concerned with health care has to have some public involvement, and that goes for GP practices, hospitals, and very much in research, all areas of research . . . It sometimes strikes me that it’s a kind of bandwagon. Don’t get me wrong, I think it’s one of the most useful things in the health economy, but it seems to have made jobs for an awful lot of people, let’s put it that way.

Funder07(lay rep.)

The other funding bodies taking part in the RAPPORT study had only recently started to implement PPI strategies in their organisations. A government-funded research council had endorsed its PPI strategy only a year before and there was now a strategic vision around PPI. The structure of this funding organisation was less complex and was led by a member of staff responsible for both public involvement and engagement. This organisation had a PPI panel of around 20 members who were offered the opportunity to be involved in particular funded projects; alternatively, the organisation would ask specific medical charities to nominate lay representatives for particular condition-specific projects. Another large national funder, financed through endowments, had a different stance on PPI, focusing more on public engagement:

patient and public involvement doesn’t just have to be people in a focus group or a patient representative on a steering committee, it can, you can use kind of more engagement approaches that can still inform and can kind of be part of the research process . . . sometimes when I talk to people in the patient and public involvement sphere there’s quite a narrow definition of how they see patient and public involvement, and so yeah, I can see it a bit wider than that.

Funder08

Public engagement was seen as a key policy driver within this organisation, and strategically it focused on facilitating research teams to improve its public engagement, with PPI being seen as operationalised at research team level.

We also interviewed two representatives from condition-specific medical research charities. Their PPI strategies had gradually developed over recent years:

we’re currently looking at expanding it, so it is something that’s definitely on the radar of people within the charity, it’s just the technicalities. I think, you know, everyone’s definitely willing to do it and I know that our Medical Director is very keen on PPI, and I think it is almost evolving organically . . .

Funder01(CharityA)

A major finding from the interviews with funder representatives was that their stances or values (and thus PPI structures and processes) were actively shaped by the origin of their funding and their histories. The funding organisations that received government public funding as part of a national research agenda clearly articulated the reasons for PPI in methodological terms:

It [PPI], you know, is likely to improve recruitment. It makes sure that it meets patient needs so that when it gets out the other end if you like into the implementation phase that it can actually have a possibility of having impact on an individual with a said condition.

Funder04

In contrast, the medical charities reliant on fundraising spoke of PPI in terms of an ethical rationale:

we get our money from the public, from people with the condition, so it’s really important that we include viewpoints of people with the condition in our decision.

Funder02(CharityB)

Hence, funders influenced PPI in subtly different ways depending on their organisational values. Case studies funded by NIHR funding streams were required to have PPI as a different way of working within the actual research process because of the emphasis on the methodological importance of PPI. In contrast, those funded by charities tended to have less focus on a requirement for PPI in the actual research process, with structures in their organisation concentrated more on ensuring PPI in funding decisions.

Structures for patient and public involvement

Host organisations, and in particular CLAHRCs, were instrumental in providing a PPI infrastructure. This often involved having a designated lead for PPI. Six of the case studies had a dedicated PPI co-ordinator who was separate from the research team, and a further case study had a research team member who took on the role of the PPI co-ordinator as part of their duties. Case study 21 (Box 6) had a colocated co-ordinator who, as a non-researcher, was seen to bridge the gap between researchers and the public:

BOX 6 - Case study 21: IDDs

Evaluation of peer-led interventions for parents of children with intellectual and developmental disabilities: systematic review Description of patient and public involvement

The research centre hosts a large PPI group made up of parents.

A project-specific ‘expert reference group’ was established, made up of the PPI representatives, and reviewed the data collected.

Normalisation Process Theory

Coherence: roles and expectations of PPI clearly defined by all. Coherence in describing the approach and philosophy of PPI in the organisation and research team.

Participation: key person drives it forward. Model adapted from elsewhere and now underpins the entire centre’s work.

Collective action: roles clearly described and training given. Relationships seen as key; trust and making it fun.

Reflexive monitoring: no formal evaluative tool used but it is actively piloting tools developed within the CLAHRC. All describe how they appraise PPI and modify their actions subsequently.

Radar plots

Patient and public involvement impact and outcomes

Research question setting: the group are encouraged to put forward research questions and this idea originated from one of the parents.

Design of the study: sensitivity of statements on questionnaires.

Recruitment: recruitment improved through postcard-size reminders.

Outcome measures: increased relevance of outcome measures; parents set the inclusion criteria for the systematic review.

Dissemination: conference presentations by lay reps and lay summary.

Impact for PPI representatives: increased knowledge of health services (parents), increased self-worth (parents), respite from caring (parents).

she’s become a great interface between us and the families because she helps, she’s . . . she knows enough about research to be able to explain what we do, but equally doesn’t know enough that she can challenge me or others in the team to explain more clearly what we’re asking of families. And I think that’s probably the greatest reason for our success has been the sort of creation of that role and the development of it.

CS21Res01-01

Case study 21 exemplified fully embedded PPI which was enabled by all eight identified contexts. It also demonstrated an ‘entwined’ model of PPI in which research and PPI were wrapped together much like a double helix. It was hosted within a research centre primarily funded by a charity (C1) with a topic area (children with severe cognitive disabilities) (C3) where active involvement of parents was seen as a prerequisite of meaningful research. The study design (systematic review) (C3), compared with the ongoing qualitative research being undertaken in the programme of work, required more effort from the research team to ensure that parents were involved and the review was rigorous. The research centre (C7) had an established PPI system (C6) including a funded co-ordinator who supported parents and undertook extensive outreach work. The centre was also part of a CLAHRC with a vibrant PPI community and resources for PPI (C4), and parents were reimbursed for their time and received training (C2) for their role. Resources also included the time researchers needed to work with parents on the systematic review. This level of resources was necessary for an entwined model of PPI (see Chapter 7). There was a strong focus on positive relationships (C5) that were developed over time and enhanced by social activities. This led to a positive experience of PPI (C8) for all involved and the sustaining of a virtuous cycle whereby PPI became increasingly embedded. At the start of data collection PPI was already embedded in CS21, so there was no change in the tracking radar plots over time.

The need for time to develop a PPI infrastructure that could support embedded PPI was also clear in CS22 (Box 7). The topic area of CF had little history of established PPI structures, and the study designs are more commonly towards basic science. This case study captured the early days of study set-up and developing PPI, so radar plots were less full. After a year and despite focused work by a PPI champion, the radar plots remained unchanged, suggesting that embedding PPI requires a significant amount of time: at least 5 years. The limited impact of PPI may reflect the very early stages of a research study; however, as other case studies show significant impact in the early stages of the research cycle, it may also be indicative of a neophyte PPI structure which is not enabling a fuller impact.

BOX 7 - Case study 22: CF

Randomised controlled clinical trial Description of patient and public involvement

Parents of children with CF (one member of charity). Experience of committees.

Stages: steering committee, four afternoons a year. One PPI representative also gives up about 3 days a month, as they sit on a monthly committee.

Recruitment of PPI: known to researchers (through clinic or other committees).

Normalisation Process Theory

Coherence: PPI has recently become a new way of working for PPI reps, researchers and PPI co-ordinators. Those interviewed have similar ideas about PPI but report some of the scientists/researchers in the hospital are not so supportive or understanding of PPI.

Participation: PPI should be part of work, but not all clinicians see it that way. PPI representative wants to find a cure in their child’s lifetime. Disagreement between researchers that PPI could understand all the science (this is reported evidence). PPI representative was unsure about research progress over time.

Collective action: training has been provided on research and governance, more recent training provided by charity – researchers and PPI representatives trained together. There has been a PPI co-ordinator for 3 years. There is a formal procedure to include PPI in research proposals.

Reflexive monitoring: PPI representatives get feedback that proposals have been successful, how their views were taken on board and how their suggestions on the protocol have been taken up.

Radar plots

Patient and public involvement impact and outcomes

Recruitment: improved patient leaflets.

Outsourcing patient and public involvement

For case studies not hosted by organisations with existing PPI structures, an alternative was to recruit an ad hoc, project-specific group of people or individuals to support the study as PPI. However, another possibility was for research teams to access external PPI groups or panels. Topic area (C3) was an enabling factor and respondents from two national networks covering dementia and diabetes mellitus both held a list of names of interested PPI representatives. One network had a formal registration process on its website (which was updated annually and had around 3000 names) and the other spoke of a more informal, dynamic list based on personal relationships. Both networks had ready access to around 150–250 regularly involved and engaged PPI representatives; had been approached by researchers (in this topic area and beyond) for PPI in protocol development, funding before the application stage, help with PPI budgets and recruitment; and often organised local focus groups to gather PPI views.

Panels were preferred by some because they were more focused on a specific question and were hoped to consult with a wider population:

I think that the PPI, you get from a reference panel is potentially more focused on specific important questions . . . for example a Trial Steering Committee which might only meet twice a year or, and if the layperson can’t attend.

ResearchNetwork02

Panels were felt to be a very effective way of using PPI time and skills, and easy access to existing groups made PPI feasible within the time line of a project. However, although convenient, recruiting solely through local PPI networks with trained and prepared members of the public was seen as not always appropriate for effective PPI. Researchers within some of our case studies expressed concern about the difficulty of recruiting more than just the ‘usual suspects’:

one of the problems is ensuring that you actually are getting real and representative user involvement and not just the people who happen to be around who always get signed up to these user involvement things . . . actually it’s quite easy I think to get the usual suspects.

CS23Res04-01

There was also evidence that relying on an external group had led to a ‘one-off’ model of PPI in which researchers had requested help for just one task and there was an absence of PPI throughout the rest of the research process. This limited the potential impact of PPI and was characterised by a more ambivalent view towards the value or impact of PPI. In a few case studies, the CI appeared unconvinced of the value of PPI:

but where’s the evidence that it [PPI] does work, maybe there’s scientific management for a programme that should just be done by scientists, internal, external and managers and maybe there are other elements that really need a PPI input, I don’t think that’s terribly well worked out at the moment.

CS01Res02-01

Case study 01 (Box 8) provides an example of this model.

BOX 8 - Case study 01: dementia

People undergoing routine brain imaging: comparison Description of patient and public involvement

Panel: an established local research network panel with a paid co-ordinator, who has worked with CI before. Panel asked for views on access to patient notes.

Normalisation Process Theory

Coherence: PPI co-ordinator feels PPI is about recruitment, researcher feels it is about better research and CI feels it is good for PIS and ethics committees but not necessarily design of studies. Both CI and researcher see dissemination to participants as PPI. A journal letter on patient involvement is focused on providing feedback to participants.

Participation: low legitimisation/enrolment. CI reservations that PPI can delay research, good for ethics but not for research design.

Collective action: CI prefers to deal with panel rather than having one or two potentially idiosyncratic views on a committee. Newer studies now have more contact with research network and PPI co-ordinator. Little support for PPI from researchers; this is provided through the research network.

Reflexive monitoring: CI’s view ‘maybe PPI does work, maybe it doesn’t, we are not really sure’. No systemic appraisal at research network, although a spreadsheet is maintained of PPI reps’ activity but not impact. PPI reps are becoming more involved, but it is not clear if researchers are using more PPI in further studies.

Radar plots

Patient and public involvement impact and outcomes

Study design: research design is acceptable to patients.

Recruitment: increase recruitment to studies (PPI co-ordinator’s priority to increase recruitment to studies).

Training

The evidence suggested that training around PPI for researchers was likely to be an enabling context, but there were mixed and even polarised views on training for lay representatives. Nevertheless, even in case studies with embedded PPI, researchers had not received formal training on PPI. Rather, they had learned through experience within an environment supportive of PPI.

Understanding PPI as different from study participation was a prerequisite for the development of appropriate structures, purposes and roles for PPI. Most researchers could articulate the difference; however, it was significant that many junior researchers could not. This lack of understanding was attributed by some to a lack of formal training:

there was no formal training, I just, you know, from what my colleagues and supervisors have just briefly mentioned but nothing formal on it and I think it would be an important part early on for trainees interested in research.

CS14Res02-01

Of those lay respondents in the case studies who had received training, the majority had been provided by CLAHRCs, public involvement groups or research networks. However, the need for training for lay representatives did not emerge so strongly from the data as for early-career researchers. Indeed, various arguments were put forward against training, such as potentially compromising the lay perspective, or skills already existing because of extensive PPI experience or transferable skills from work life:

I don’t think there was any necessity for training at all. There’s certainly obviously one or two people have got good committee experience who are sitting round the table, on the lay side, so no, I don’t think there was any necessity at all.

CS13PPI05-01

In stark contrast a minority of lay representatives suggested it was impossible to contribute meaningfully without some understanding of research:

I don’t think that you can just so easily shove inexperienced patients in to a PPI role without them being, having some training and having some support because there’s no question that, you know, the whole research agenda is incredibly complicated and they talk in a language which is, you know, if you’re not a researcher you sometimes have no idea what they are talking about when they’re talking about face validity and constructs and confidence intervals and so on and so on.

CS14PPI02-01

However, the above quote suggests that this lay representative felt it was essential for the PPI representative to be able to cope with the research world, rather than the research community needing to make the effort to include the lay world in an accessible way. Making the effort also included working towards recruiting lay representatives who reflected the diversity of a study population.

Strategies for recruiting lay representatives

Being able to understand the research through the viewpoint of a target population was seen as key:

I was sat on a table with speech and language therapist, physiotherapists, paediatricians, and a handful of parents, all as equals, you know? . . . and I was just thinking ‘What does that mean to me as a parent’, thinking of my child’s capabilities and all the other children that I knew.

CS21PPI01-01

However, finding lay representatives who they felt could present the viewpoint of the research study’s target population was often challenging for research teams. This was more frequently cited for BME groups. We attempted to include case studies that were likely to have a target population from the BME community. One case study specifically focused on the experiences of a particular ethnic group living with arthritis (CS18). One of the study sites was located in an area with a high percentage of people from this ethnic background, and the lay representative was recruited through that site:

I was going to the clinic, attending the clinic and the nurse actually asked me if it’s okay because I speak different, three or four different languages.

CS18PPI02-01

This opportunistic recruitment meant that CS18 benefited from the input of a lay representative who demonstrably understood the enablers of recruitment in far more depth than the research team:

what we had was an audio CD because a lot of people do not read the script, [specific language], so some of our interviewees did not speak English so we got the information sheet translated and put onto CD and she [PPI01] listened to that and gave feedback on that because there are different dialects of [specific language], apparently some are deemed posher than others and there’s all sorts of things that I was not aware of initially.

CS18Res01-01

While there was trust in this lay representative’s expertise demonstrated by the peak in the collective action segment of the radar plot (bold outline) (Box 9), the construct as a whole was affected by one major challenge: that the CI (who was effectively the PPI lead within this study) and the PPI representative were located at a significant distance from each other.

BOX 9 - Case study 18: arthritis

Treatment outcomes for ethnic group Description of patient and public involvement

Patient and public involvement representatives normally recruited to each study via an established PPI group based at the university. Only one took part in CS18. Expert by experience (has condition). This PPI representative was recruited through second university user group.

Normalisation Process Theory

Coherence: PPI representative slightly hesitant about role and impact. Researcher clear that PPI helped and provided reassurance. Another, more experienced, PPI withdrew through ill health. Both feel it important to give patient view and experience.

Participation: arthritis – outcome measures about pain were introduced because a patient spoke up.

Collective action: paid PPI co-ordinator. Training is provided once a year and group meet together. Sometimes difficult to commit, because of illness (flares).

Reflexive monitoring: PPI representative found it worthwhile for herself. CI found PPI input very valuable. In the future would involve two representatives. If the CI moved to another institution they would definitely set up a similar organisational PPI approach.

Radar plots

Patient and public involvement impact and outcomes

Data analysis: ensured it was culturally sensitive.

Dissemination: published paper with PPI representative in acknowledgements.

Cultural reassurance for CI (from other researcher and PPI representative).

Outcomes for PPI: hearing and reading about other people’s situations helped PPI representative personally.

Enabling contexts

Funder requirements: NIHR (C1). Study design: qualitative (C3). Research host (C7): CI located in an organisation with embedded ‘established system’; each research project is automatically assigned two patient representatives. However, this context was significantly counteracted by the research site being at a distance.

We also included four case studies focused on type 2 diabetes mellitus with the expectation that these would include a relatively high percentage of research participants from BME groups susceptible to this condition. One of these case studies (CS03) (Box 10) had used strategies (advertising and interviewing applicants) to recruit a diverse range of lay representatives:

BOX 10 - Case study 03: diabetes mellitus

Design of a self-management website: intervention design followed by randomised controlled trial Description of patient and public involvement

Experts by experience, different levels:

1. Focus groups involving diabetic and health professionals.

2. Eight to ten people with diabetes mellitus were selected through interview. The PPI recruited mostly through Diabetes UK magazine. Newly established group or panel meets once a month for a year.

3. Two patient representatives on steering committee (one PPI representative volunteers for local diabetes support group and Diabetes UK network, is involved in another charity and was involved in a project at the research centre before; second PPI representative not available for interview).

Normalisation Process Theory

Coherence: high. PPI representatives and researchers use the same language (everyday life, real-world knowledge), pooling experiences, diversity of views. Value for PPI individuals: confidence, mental stimulation. They value the importance of the research and website.

Participation: key person (one of the researchers) approachable, supportive, interested; trust is strong, all voices heard, researcher says ‘you have to take a back seat’ and that PPI have ownership.

Collective action: PPI representatives are a selected subgroup from a larger group. They know exactly what they have to do, they are given clear instructions and they are set appropriate tasks as homework, which is made clear at the beginning of each meeting. They are supported in terms of parking and venue. PPI embedded within department but not whole university. Some underestimation about what PPI representatives can do (by researchers).

Reflexive monitoring: PPI representatives know that their ideas have been used. Homework set each month and representatives given feedback on why their ideas have been incorporated into the website or why not. Comments clearly written up. Collectively think it is worthwhile, users shaping the intervention, broader measure will be if website gets used.

Radar plots

Patient and public involvement impact and outcomes

Recruitment: two PPI representatives on steering group involved in suggesting changes to documents.

Design of intervention: self-management tool.

Outcomes for PPI: PPI representatives feel valued, sense of worth, benefits for society.

we wanted a really representative mix of people with diabetes . . . we wanted to be quite diverse in sort of the sociodemographic, those variables so we got a really nice ethnicity mix and age mix and gender and so that’s what formed our regular groups . . . Because sort of in the population, diabetes is more prevalent in sort of the Afro-Caribbean, South East Asian population, . . . it was one of the things that sort of influenced our decision as well.

CS03Res03-01

Strategies used when recruiting individual PPI representatives reflected researchers’ attempts to involve people who could bring both specific and broader perspectives. Recruitment of individual lay representatives was from five main groups: general public; expert by experience; carer/parent; patient representative; study participant. Individuals could take on multiple roles and represent their views at different levels. However, relying on just one or two individuals also raised the issue of representativeness:

it’s impossible, you can’t expect one person to speak on behalf of everyone so I guess the question would be what would be useful, or truly influential patient involvement? Is it enough to have just one person or two people who will be very, very good and who will be enthusiastic, but the question is, how representative are their opinions into their input?

CS03Res02-01

Nevertheless, one model of using a single lay representative as a conduit to a broader constituency appeared to work well. The outreach model, as exemplified by CS02 (Box 11), involved one representative from a charity who then linked with 20 members of the support group in an iterative way to inform the study design and recruitment materials. An unforeseen impact from this PPI was to raise concerns over the safety of the study, resulting in extra study visits and a questionnaire (see Impact and outcomes of patient and public involvement).

BOX 11 - Case study 02: diabetes mellitus

To test whether or not older people benefit from lower doses of a particular drug Description of patient and public involvement

Steering committee: national representative. One PPI representative (senior position in charity) sits on trial steering committee. PPI representative then consults the (20) panel members, invites feedback, collates answers and then sends it back to research study team. Provided study with letter of support from charity. Involved at a number of stages during the research study. At second interview PPI was in a lull while trial up and running. PPI representative wanted to write something for charity newsletter. PPI representative leads and initiates involvement when appropriate.

Recruitment of PPI: PPI representative known to CI; charity recruits to panel.

Normalisation Process Theory

Coherence: high. CI and one researcher new to PPI, other researcher 2–3 years’ experience. Recognition that funding agencies now require PPI. Agreed PPI was about patient views, perspective, real life, everyday life. Overall, in agreement that PPI does bring value for research.

Participation: medium. PPI representative now gets approached by lots of people for PPI. One researcher had had negative experience of PPI, unsure about benefits and CI feels PPI might not be involved in research applications (PPI might not agree with what clinicians feel is an important research question). ‘Involvement might not be in the best interests of a partnership’ (CI).

Collective action: skills of PPI representative are put to good use, collating views, information more patient-friendly. Communication appears to be good. PPI representative is supported by her charity, not really by the research institution.

Reflexive monitoring: PPI representative believes PPI should be from the beginning and early involvement should be assessed by funding bodies. However, recognition that PPI does take more time. CI thinks PPI representatives not so useful at commenting on research design but great at commenting on how patients might feel to take part, adding questionnaire about safety and reassuring patients with an extra visit. PPI representative can see their ideas have been taken on board.

Radar plots

Patient and public involvement impact and outcomes

Design of study: protocol changed to include extra visit.

Safety of study: PPI suggested adding in a 3-month visit for patient reassurance. Patient concerns about safety so a safety questionnaire was added.

Recruitment: changed language of documents (more readable/PIS altered); designed a small card (with hospital telephone number, emergency number and information on); suggested avoiding the word ‘mortality’.

Some case studies also bought together project-specific panels. Case study 12 (Box 12) was particularly enabled by being located in a teaching hospital with relatively good resources for research (C4), which were sufficient for PPI honoraria.

BOX 12 - Case study 12: diabetes mellitus

Lifestyle intervention Description of patient and public involvement

Newly formed panel: experts by experience, participants on the trial, mentors on the trial. Less PPI at the early stages of the research, became involved with study documents. One lay representative had already worked with the CI, others were new.

Normalisation Process Theory

Coherence: researcher and PPI representative agree an outsider’s view is needed especially when testing a service which will be eventually used by pre-diabetics. Value of the psychological support for coming to terms with having diabetes mellitus (PPI). Good for ethics.

Participation: PPI representatives and researchers agree it should be part of their work. Wider than just the research project.

Collective action: PPI representatives do perform activities asked of them; relationships not a strong focus, but PPI representatives do not articulate this as a need, being very experienced working on committees (feel they do not need the support); skill set seems appropriate.

Reflexive monitoring: do not formally evaluate PPI, although meetings minuted; all feel PPI is worthwhile.

Radar plots

Patient and public involvement impact and outcomes

Ethics: enhanced approvals process as materials reviewed by PPI representatives before submission.

Recruitment: increase recruitment to study by clarifying paperwork. PPI looking at reasons for withdrawals from study.

Challenges in ensuring diversity

Researchers acknowledged there was much work to be done to ensure diversity in PPI. The fact that only a very few of our PPI respondents were not from a white British background exemplifies the common problem as described by one of the funders:

we have as much of a problem as anyone about, you know, diversity and all those issues are still issues which we need to explore further. I think often public, people who get involved with PPI are a self-selecting group and it’s really difficult to bring in groups wider and that’s a usual excuse and I hate it, and I hate it when researchers say it. But I, it, you know, for example if somebody couldn’t read and write how would we include them in this process? If someone wasn’t, we do manage to include people who are not computer literate but then these days how are they going to even find out about us if they don’t have a computer? How are they going to know who to ring?

Funder01

Difficulties in accessing ‘representative’ PPI were particularly marked for public health studies when the focus was on healthy or ‘at-risk’ populations with particular lifestyles. The difficulties for public health studies was also linked to the setting (e.g. broad communities) in which they took place. Those studies taking place in settings with ready access to patients obviously had greater potential for recruiting both participants and PPI representatives onto their studies:

the problem at the university is that those researchers that aren’t clinical or don’t have good links into the clinical setting, it becomes very difficult to identify service users to sit on a panel like that so we’re in quite a privileged position because we’re right next to the outpatients clinic and that makes our life a lot easier for maintaining that model.

CS18Res01-01

The dual role

Clinicians who were also researchers had the added advantage of having ready access to patients as potential PPI representatives. For example, one case study (CS14) (see Box 26) hosted by a district general hospital with few embedded PPI structures was able to involve patients through an active condition-specific support group facilitated by clinical department staff. However, the distinction between being involved and being a research participant was often blurred in this case study, with PPI being understood by many of the respondents as a dual role. This dual role was overt in another arthritis-related study (CS08) (Box 13), where the PPI respondent was also a study participant. While there are potential ethical conflicts here, this lay representative actively recruited participants through her own network and had a strong sense of purpose. This action-orientated approach to PPI resulted in a fuller quadrant in the radar plot denoting a positive appraisal of the PPI within CS08 (see bold outline in box figure).

BOX 13 - Case study 08: arthritis

Differences in joint functioning between those with and without osteoarthritis Description of patient and public involvement

Dual role: PPI representative already known to CI through being on funding committee but also has joint issues so is participating in the research as well as acting as an advisor.

Normalisation Process Theory

Coherence: there is a general understanding of what PPI means but also some confusion and acknowledgement that this is not a typical example of PPI. PPI representative has particularly strong views on the limitations of PPI whereas the CI appears to be increasingly converted to its possibilities.

Participation: the CI is fully enrolled, but the other two respondents have some doubts over whether or not the value of PPI outweighs the effort required. There is no key leader for PPI.

Collective action: the roles of involvement and participation are blurred. However, the relationship between CI and PPI representative is key and characterised by continuity and reciprocity. Both CI and PPI representative agree that there is the ‘right sort of person’ for PPI. There is mixed acknowledgement of reimbursement for time and travel, as some members of the research team see PPI as a voluntary role not requiring reimbursement for time.

Reflexive monitoring: CI can identify the impact of PPI in the study and to a lesser extent so can the PPI representative. However, the CI is concerned about the difficulties in evaluating PPI formally and talks of needing numbers for an evaluation. CI has clearly learned from this study and would have earlier and fuller PPI; all seem to think dual role is useful, particularly in terms of ongoing feedback during a study.

Radar plots

Patient and public involvement impact and outcomes

Design of study: changes in the way the study was run (subtle touches such as introducing staff, cups of tea).

Recruitment: actual recruitment by PPI representative to study, changes to PIS.

Enabling contexts

Study design: low-risk observational study so lay rep. comfortable recruiting others (C3). Research host (C7): study being conducted in labs with little interface with the public; CI relying on existing contacts for lay representation but this established relationship also enabled trust (C5).

Although respondents in CS08 were comfortable with this dual role, it was operationalised in an ad hoc manner that was unlikely to lead to any embedding of PPI. In contrast, this dual role was carefully constructed and managed in a longitudinal cohort study (CS13, see Box 16). A rolling PPI panel was drawn from study participants and the role was clearly defined and separated. The impact of this role included both increased public engagement with the study and improved participant retention. This model of PPI was enabled by topic area (public health with healthy participants) and the longitudinal design, which allowed relationships to be developed and sustained over time. This was a well-resourced study with dedicated PPI staff and good organisation, and was run from community hospitals at the heart of local populations. For the public undertaking this dual role, there was a sense of ownership of the study, with subsequent sustained engagement. Two tissue bank studies also had lay representatives who advised but also had donated samples. In CS17 (see Box 23), engagement was sustained, but for the other (CS16, see Box 20) the sense of ownership became a cause of conflict, which is presented in Enablers and barriers to trust.

If this dual role was to have a positive impact, then the purpose, structure and clear role boundaries needed to be defined.

Relying on one person to lead patient and public involvement

As described in Purpose, structure and role of patient and public involvement, having someone to lead and co-ordinate was important in embedding PPI. If there was not a designated co-ordinator, this role was often carried out in addition to a junior researcher role and was not part of the original job description. Different levels of commitment to this role were found and PPI representatives spoke of often not having one ‘designated person’. Continuity of personnel with responsibility for PPI was an issue. For example, one PPI representative reported seeing someone different at each meeting (CS05) (Box 14) and another said the enthusiastic support ‘gradually dwindled’ (CS14). If the whole team was not engaged with PPI, the loss of the PPI champion had a detrimental impact.

BOX 14 - Case study 05: arthritis

Reduced dosage of medication for people with arthritis, clinical trial of an investigational medicinal product Description of patient and public involvement

Steering committee: one expert by experience, recruited through national arthritis body, who was a patient of the CI. Was asked to change consultants. Nursing background and interested in research. Understands some of the medical terminology.

Stages: PPI representative on steering committee throughout the project. Steering committee meets every 6 months.

Normalisation Process Theory

Coherence: PPI representative is ‘patient voice’. Mixed views from the research team about PPI processes within the study. PPI representative believes PPI can be bridge between clinical decision and actual opinion. CI thinks PPI is ‘neutral’ in terms of recruitment.

Participation: PPI seen by some as just another change in the research landscape. Qualitative researcher within department (not involved with this research) very keen on PPI. PPI representative keen to take part.

Collective action: steering committee every 6 months and few e-mails in between. Not much work for PPI representative to do in between times. Lack of continuity with change of study managers (hard for PPI to maintain a relationship), PPI representative accepts she will not understand everything, she communicates the patient perspective. Researcher agrees that PPI representative might not be able to be involved in clinical practice. CI says PPI support needs to be tackled centrally, not at individual study level.

Appraisal: no feedback to PPI representative. PPI representative has some doubt about how much impact PPI has. PPI representative made comments on protocol, concern regarding radiography and reducing medication. PPI rep. feels views taken on board, but less evidence that research team evaluate these as changes.

Radar plots

Patient and public involvement impact and outcomes

Protocol: PPI suggested changes to protocol; concern regarding radiography and impact of reducing medication.

Patient and public involvement as a collective action

Case studies demonstrating agreement of all individuals in the team that PPI should be part of their work were particularly strong within the IDD topic area, where there is an acknowledged history and strong investment in PPI. The CI from CS11 (Box 15) clearly articulated this:

BOX 15 - Case study 11: IDD

Group intervention to reduce anxiety of (age-range) people with specific condition Description of patient and public involvement

Two PPI representatives are new to PPI in research (one voluntary worker, one carer) and one more experienced (expert patient). Preference for PPI recruitment from established condition-specific networks (built on previous relationships and personal recommendations) rather than local (generalised) PPI networks.

Normalisation Process Theory

Coherence: CI (clinician) has much more of an overview of PPI in the project (and its genesis) and deeper understanding of politics/history of PPI in this area than other (less experienced) researchers interviewed. PPI representatives have a good understanding of role/purpose of PPI in this study.

Cognitive participation: strong investment in PPI in this research topic, and on this project. CI drives participation. PPI members see participation as (1) part of their job, (2) their only job, (3) a voluntary role with potential benefits for those affected, including family members.

Collective action: PPI members feel valued and identify importance of being assertive. One PPI representative feels that research team could use that representative’s skills more. Interpersonal aspects of PPI: a few examples of issues which have arisen and been managed well are given, which stem from health condition and professional boundary issues. The word ‘mindful’ is used a couple of times in respect of this. Effective chairing of meetings very important.

Reflexive monitoring: nothing systematic in place, but some clear contributions outlined in minutes and interviews and lots of important small changes made, in line with PPI recommendations.

Radar plots

Patient and public involvement impact and outcomes

Intervention: refining intervention and testing acceptability of intervention.

Recruitment: better recruitment/retention of participants to trial.

Outcome measures: more meaningful.

PPI outcomes: taking part improves curriculum vitae to get more work of this nature, including consultant/trainer work.

in order to understand and develop theory effectively, you need to really have a conversation with service users and probably somebody who has the problem with the condition, and that is how psychological theory develops. You know, it is that, ‘Are you doing something to people or are you doing it with people?’ and doing it to people, you know, there’s such a power imbalance, and given the backdrop and the history associated with people with disabilities internationally, that’s not politically palatable to me personally and to, you know, society, I suppose.

CS11Res01

Although the NIHR funding (C1) was used to pay out-of-pocket expenses, this case study was hosted by a medical school, with the CI reporting limited engagement with PPI. However, it is the topic area (C3) that is the strongest enabling feature, suggesting a culture where ‘nothing about me without me’ shaped research practice. This CI also framed the challenge to explain technicalities more clearly to lay audiences positively, as good practice for communicating ideas to the public:

saying things like, ‘Oh, this is a stepped wedge design,’ and ‘We’re randomising,’ . . . you can use that language, but people aren’t going to understand it. So, you’re going to spend all your time explaining what you mean, so you might as well just use simpler language to start with, and perhaps really as a researcher or scientist, that’s a good thing because it helps improve your communication skills with lay people and the general public.

CS11Res01-01

This whole team engagement was not unique to IDD as a topic area. For example, in CS13 (public health) (Box 16) although a project-specific PPI panel was facilitated by a lead person, all senior researchers took it in turns to attend panel meetings. The embedding of PPI in this study was enabled through access to resources from grant monies (C4) and a longitudinal design (C3) that enabled relationships to be sustained (C5).

BOX 16 - Case study 13: public health

Relationships between dietary intake and cancer incidence: prospective cohort study Description of patient and public involvement

‘Consultation’ panel hybrid model, built upon earlier, less formalised (mass consultation, participant feedback) PPI panel drawn from large cohort study (now mostly retirement age). Panel meet quarterly with an annual general meeting with a rolling membership of 15.

Patient and public involvement in all research stages apart from data collection and analysis.

Normalisation Process Theory

Coherence: PPI approach (roles and purpose) of consultation type panel clearly understood across team with strong leadership; other historical PPI mechanisms have been utilised in cohort study, for example to identify study priorities.

Cognitive participation: run by dedicated and respected member of team. Strong buy-in communicated from outset/CIs: verified from researchers and PPI panel members interviewed. Lots of thought put into planning: how to set up the panel, following best-practice principles.

Collective action: All very satisfied with the way the PPI is run and managed. Shared decision-making regarding panel name, frequency of meetings, level of commitment, when/where meeting and level of input required. Light touch, structured but engaging, elements of fun. Panel only more recently being asked to help set the agenda of meetings, but do feel comfortable to challenge any points put to them for consideration.

Reflexive monitoring: panel are given clear, itemised feedback on why all suggestions are/are not taken on board. All queries are reported back on. They can clearly see their impact in before/after versions of documents. They enjoy taking part and feel highly valued. Annual general meeting reviews PPI contribution and model.

Radar plots

Patient and public involvement impact and outcomes

Research priority setting: helping identify research questions.

Design of the study: testing acceptability (refining) of data collection tools of proposed phases.

Recruitment: redesigning website to be more user friendly.

Design of intervention: testing acceptability of future intervention studies.

Standardise outcome measures: patient-focused outcome measures.

Dissemination: presentation to an audience of researchers on PPI and being a panel member. PPI representatives involved in public engagement work. Newsletters.

Other: improved participant retention.

Coherence in understanding the contribution of patient and public involvement

As the bedrock for trust, lay representatives and researchers needed to have a shared understanding of the purpose of PPI. Novice researchers were more likely to be unable to fully distinguish PPI as separate from research participation. For example, a registrar in CS14 defined PPI as:

patients contributing to research and enrolling into research studies, whether they had portfolio status or not.

CS14Res02-01

While this impeded an understanding of what PPI could offer to the research process, other researchers and PPI representatives provided a variety of reasons for carrying out PPI including gaining insight into the lived experience of a particular health condition, and to provide reassurance for researchers that their research question resonates with the patient community. This was exemplified in CS07 (Box 17).

BOX 17 - Case study 07: diabetes mellitus

Consultation between the practice nurse and the person with diabetes mellitus: randomised controlled trial and qualitative Description of patient and public involvement

Experts by experience panel (≥ 10 patients who have lived with diabetes mellitus for over 5 years). Recruited from diabetes research network panel. Members of the CS07 panel are on their own Diabetes UK voluntary group (access to another level of PPI). There is also an online panel. Involved in different stages (involved at beginning and in recruitment and CI hopes they will be involved in dissemination).

Normalisation Process Theory

Coherence: CI says ‘patients are the experts’, ‘find it incomprehensible that you wouldn’t ask patients first’. Researcher had been involved with PPI in service redesign before – ‘it’s the way forward’. Involve people from the concept and design through to dissemination. ‘PPI runs throughout the study’. Ethics committee had been very positive. PPI representative believes PPI to ‘humanise research’, ‘white coat jargon – words of one syllable’.

Participation: researchers buy into PPI as does the PPI representative.

Collective action: meetings set up only when needed. Training through research network, not sure PPI representative wants any more technical training as providing lay perspective. PPI representatives distributed questionnaires to their own Diabetes UK groups, so got lots of feedback. PPI representative accepts limitations of specialist knowledge and terminology. CI feels PPI is embedded within her organisation. Taxi organised for PPI representative as has mobility problems.

Reflexive monitoring: involved in protocol (randomisation), PIS, design of questionnaire, video used when recruiting practices to the trial. PPI representative intends to be involved in dissemination (not analysis). Researchers listen, PPI representative feels contribution is valued. Researchers recognise importance of feedback (it is an agenda item). PPI representative says feedback very complimentary on the video. Dissemination so far: PPI poster designed for UK annual conference, newsletters.

Radar plots

Patient and public involvement impact and outcomes

Design of study: psychological element (scale has been shortened), whether to do partial or full RCT. Panel commented on proposal of this study at the beginning. Introduction, how to phrase the study.

Design of intervention: refined – changed the questionnaire six times, eight pages to three. Validated the questionnaire (panel sent out to their user groups and through diabetes database).

Recruitment: through video – PPI representatives took part.

Case study 07 was enabled through an established diabetes (C3) network group (C6) housed in the host organisation (C7). The study focus on processes of care (C3) also enabled coherence. PPI members met regularly and were involved in a number of projects which helped promote sustained engagement (C5). However, lay representatives were not reimbursed for their time because of bureaucratic difficulties; rather, they were taken out for a social meal.

Whereas the experience of established panels was seen by many researchers as a bonus, others spoke of the tension between wanting someone who was ‘outside’ the research environment and wanting someone who was also able to engage with research studies (possessing some research skills) and thus perceived as being able to perform the necessary PPI tasks. This notion of wanting the right person with a short distance between direct experience and its interpretation but who also can feel comfortable in a research environment is maybe a contradiction in itself. There was evidence of an ongoing debate about whether or not the experience of a condition (direct or indirect) was seen as less legitimate because of that person’s professional background, experience or training in research or health care:

we’re all patients . . . yeah, and some of us have PhDs and some of us don’t, so it’s quite difficult to kind of define what is a lay representative . . . they’re really just people who, actually, through their qualifications, they should be research assistants but don’t happen to have a job in research but are on a lay panel. And I kind of think ‘what . . . are you really a lay person? You’re just an unpaid researcher’, but I’m not entirely sure whether that’s right . . .

CS17Res01-01

Our findings suggested that defining PPI as a discrete and commonly understood component of research was difficult but the range of PPI representatives can be summarised through a fourfold typology based on an insider/outsider perspective (on one axis) and a range of research skills (on the other) (Figure 8). Research studies can be placed along these axes; however, within one research study there may be a mixture of individuals with varying skills and perspectives.

FIGURE 8.

Insider/outsider perspective.

While the insider/outsider debate was articulated in a number of case studies, those that had embedded PPI were likely to have sustained engagement with the same group of PPI representatives over a long time.

Building relationships over time

We found that the relationship between the researchers and PPI representatives was a key factor for successful PPI. Relationships took time to establish, which was particularly difficult at the beginning of research (or even before the proposal is written). One finding from the case studies was that the majority of PPI representatives had a previous working relationship with the research team. Of the 51 PPI representatives, 33 (64.7%) had some prior working relationship with either the CI or members of the research team (three were patients of the CI and also members of a service user group, so had worked together before), nine (17.6%) had a more distant acquaintance (but described themselves as being within the same loose network) and nine (17.6%) PPI representatives were completely new to the research team (recruited from outside existing structures). The obvious advantage of having a previous relationship meant that PPI could start as soon as possible within the research project, whereas more time was needed to establish relationships through informal small meetings if the lay representatives and researchers had not previously met.

Research teams spoke of initial meetings where everyone got to know each other and having these times of informality before starting on the ‘academic’ business. The benefits of PPI representatives having the opportunity to get to know each other also provided support later in the project. Social elements were important in encouraging and maintaining continual engagement in the research study and should not be underestimated by researchers. Some of the case studies also ran social events outside the research, which again helped to build and maintain relationships:

We do these family fun days as well, which we get together once a year, just to get together at the zoo or something. [PPI co-ordinator] works phenomenally hard to build relationships with the families that . . . So, it’s a partnership really.

CS21Res01-01

In more formal advisory or steering committee meetings, PPI representatives spoke of being specifically welcomed into the meeting (sometimes being met beforehand). Most importantly, during the meeting, successful PPI was being ‘genuinely’ listened to and their comments taken on board. The relationship between the researchers and the PPI representatives was key to fully embedding PPI within the research study. The researchers needed certain social skills, which included being approachable, good listening skills and group work skills:

initially facilitating and always presented a positive, welcoming, friendly, supportive space to you.

CS17PPI01-02

Patient and public involvement representatives reported that meetings needed to have a ‘flat structure’ where PPI representatives felt free and at ease to speak.

Relationships could change over time, either through loss of trust or as trust developed, as illustrated in CS19 (Box 18). Although only subtly different, the radar plots show a change in all quadrants but particularly in collective action (bold box), where M10 (trust in each other’s PPI work and expertise is maintained) is included as a mechanism. This reflects clarification of roles and the improving relationship and trust in the PPI work:

BOX 18 - Case study 19: public health

Comparison of obesity treatments: randomised controlled trial Description of patient and public involvement

Committees: two representatives (one a coapplicant) on trial steering committee (meet twice a year). Two PPI representatives on trial management group (meet once/twice a year).

PPI in all but data collection and analysis stages of research.

Recruitment of PPI representatives: two research-related professionals already known to CI who had had experience of the intervention.

Normalisation Process Theory

Coherence: mixed understandings and tensions about the purpose and role of PPI in general and specifically on this project. CI sees PPI as normalised and indistinct from other research partnerships.

Cognitive participation: CI drives the PPI. Qualitative researchers unsure how PPI will impact. PPI coapplicant bought in from the start. Agreement on commitment to PPI but mixed views on how it needs developing further in the study.

Collective action: PPI coapplicant very effective to begin with but has had to step down. Other PPI representative feels that team need to think about patient experience and is unsure whether or not the team is responding to the PPI representative’s message. PPI representative feels both she and the other PPI representatives could be more involved.

Reflexive monitoring: no systematic process for appraising PPI in place, but clear, if weak, examples given of contributions of PPI to the study after initial protocol development; some reflections on how PPI could be strengthened.

Radar plots

Patient and public involvement impact and outcomes

Marketing of study: awareness that study logo might be inappropriate/insensitive.

Design of study: timing of nutrition questionnaire; PPI representative felt this is too invasive.

Design of intervention: testing acceptability of intervention; asking too much of patients in each visit.

Recruitment: recruitment pack, information.

Dissemination: PPI representative invited to coauthor a paper.

I don’t know what they want me to prepare for, I don’t know what they want me to do, and I don’t know if the confusion is because of my professional role [. . .] and I don’t know if they’re waiting for me to take the lead, and I’m waiting for them to treat me as a patient. . . . I’m a little bit disappointed. It feels almost tokenistic.

CS19PPI02-01

Conflict between roles resolved . . . CS19PPI02 feels happier now that they are no longer post-surgery, less vulnerable. They had a long chat with a rheumatology person in department who had done a lot of PPI work who reassured CS19PPI02 that one PPI voice is better than none, more confidence to feel you don’t have to represent all patients, can just give your own experience. Different way of looking at the value of their contribution.

CS19PPI02, notes of tracking interview 6 months later

Trust as the foundation for patient and public involvement

A relationship framed by trust was the minimum requirement for PPI to have a positive impact. Although PPI was not embedded within CS14 (see Box 26), the relationship between lay representatives and the clinician was based on trust built up from a clinician–patient relationship with continuity over time:

there’s a trust there, I trust them that they wouldn’t ask me to do something that wouldn’t be, you know, it would be difficult or wouldn’t necessarily be appropriate or whatever.

CS14PPI01-01

Some researchers also trusted their lay representatives to undertake the work in a way that was workable for the researcher:

It’s always good, you know, on research it’s you need good people you can work with and that’s what I would be looking for, to set up some relationship of trust and mutual respect . . . Without thinking of them as a hindrance.

CS15Res03-01

Case study 15 (Box 19) exemplified how, even when resources were minimal for PPI, the very existence of mutual trust resulted in positive PPI impact. This study was housed in an organisation with no PPI infrastructure and with minimal resources for PPI despite the study being funded by the NIHR. The research setting [intensive care unit (ICU)] provided particular challenges: consent and recruitment difficulties in a high-stress environment; difficulty in recruiting lay representatives, as they were often traumatised by their time in ICU; and lack of resources, as there was no large charitable funder. The lay representative in CS15 had an impact on recruitment processes for the study and provided a valid argument to present to the ethics committee for this approach:

BOX 19 - Case study 15: dementia

Prevention of delirium in intensive care: clinical trial of an investigational medicinal product Description of patient and public involvement

One national representative. PPI role confined to trial steering group.

Activities include review of PIS before ethical approval, project management. Involvement in dissemination expected.

A topic area with particular challenges in recruiting lay representatives.

Normalisation Process Theory

Coherence: lay representative very experienced in different PPI roles (although first clinical trial of an investigational medicinal product) but all research team very new to PPI and some find it hard to differentiate from obtaining feedback from participants. Despite some difficulties in clearly defining PPI and describing their own role in PPI, all the research team articulated potential value of PPI including bringing patient perspective, advising on consent process, improving recruitment and dissemination and raising awareness of the research.

Participation: CI sees role to drive PPI forward as a requirement of funders, whereas the research nurse feels this is something that could be developed in her role. There is no resistance to the idea of PPI and there is evidence that the team are likely to continue supporting PPI.

Collective action: some acknowledgement by the team that regular communication is a requirement for PPI. Trust is a key feature but unlike many of the other case studies it is trust in the PPI representative that is more articulated. Research team trust the PPI representative. This will make dealing with PPI representative easier, and representative will feed back appropriately and not mind if they do not take up PPI suggestions. The PPI representative sees the relationship with the CI as key.

Reflexive monitoring: minimal – had not thought of systematically evaluating PPI impact. CI evaluates PPI input as useful to date, but no evidence of any reconfiguration.

Radar plots

Patient and public involvement impact and outcomes

Recruitment: too early to tell but is seen mainly in terms of improving recruitment because of refinement of PIS, in particular making the wording more sensitive to relatives who will be consenting.

we put a thing here saying in the consent ‘to continue information now you’ve regained consciousness, we would like your permission to continue in the research’, now [PPI01 has] written ‘I realise this might not be possible but in my own experience of delirium I was regarded as having capacity long before I was in my right mind or capable of rational decision making. I appreciate this may be beyond the scope of the study but if there were any way to change the emphasis for the patient remaining in the study unless they chose to opt out, I think that would be much better’.

CS15Res03-01

It can be clearly seen how M10 appears as a peak in the collective action section of the radar plot (bold square).

Enablers and barriers to trust

Sustained feedback to PPI representatives was a clear enabler of trust and continued engagement:

and they’re always thanking us for our participation. All the time. Always saying, you know, I get it in e-mail messages, we get it at the meetings and they always say how valuable our input is, you know, they make us feel worthwhile and that is good.

CS13PPI03-04

This feedback was recognised by many researchers as being important:

I think it’s important, what I’ve learnt is that if you are going to ask them things you need to feed back to them so that it’s terrible to actually ask their opinion and they never hear again, so I’m quite good now, it’s an agenda item that you know, I feed back what I’ve heard from different people, because they’re left wondering . . .

CS07Res01-01

The forum in which researchers and lay representatives came together was also an important enabling context for trust. The majority of our case studies had PPI representatives as members of steering committees (one or more lay representatives per study) which met every 3 or 6 months to advise the research team but did not necessarily have a ‘hands-on’ role with the research. However, different research teams made different uses of their steering committee members, with some members making comments only during the meeting and others who were asked to comment on documents throughout the study. As shown in CS04 (see Box 22), there was a varying sense of equality within the committee:

it wasn’t run by the Professor from the top down in sort of military fashion, it was a round table discussion where anybody could contribute . . . whilst it could have been intimidating actually it was very welcoming, and the person who chaired the meeting was very sensitive to that.

CS04PPI02-01

However, the formal term ‘steering committee’ put some PPI representatives off: ‘if they said someone to be on the Steering Group I mean I would never, I would have just thought well that wouldn’t be . . . that sounds too high powered for me’ (CS04PPI01-01).

Despite it often being seen as good practice, there was little evidence that this relational integration was being formalised through job descriptions or contracts for the PPI representatives. However, at the start of the study researchers often clarified expected commitments to attending meetings; other PPI input was often less formalised. Those case studies which had high levels of PPI activity could achieve this through flexibility and allowing PPI representatives to ‘dip in and out’ as their other commitments also varied:

parents will stick with the programme because they know that they can step in and out if needs be, and the team also think about how they can include people that might not be able to travel, so whether it’s joining in a meeting by phoning in to it and participating over the phone or sending responses or comments and feedback in an e-mail, so it’s made it available and it’s made accessible.

CS21PPI02-01

However, trust in CS16 (Box 20) was lost because of a lack of formal agreements about access and ownership of a database. This issue appeared midway through data collection and the impact can be quite clearly seen on the radar plots tracked over time. The impact can also be seen in the way the PPI was appraised (hatched box); because of this experience the charity would get legal documents drawn up before working with researchers again.

BOX 20 - Case study 16: diabetes mellitus

Genetic and cell culture: tissue bank database Description of patient and public involvement

National expert network: charity. Dual role (participants and representatives).

Patients meet once a year but charity has ongoing contact.

Recruitment of PPI: through charity.

Normalisation Process Theory

Coherence: CI has worked with patients with a particular rare condition (which is related to diabetes mellitus) since the 1980s. CI received money to fund a clinic (but not the research). With parents of children with the condition, PPI and charity reps, the CI developed a grant application which was awarded by the charity. CI very committed to PPI. PPI representatives believe in patient voice. All three PPI reps work for charity, very involved. PPI representatives feel clinicians/researchers do not fully understand the condition as a whole. Come together for conference once a year.

Participation: PPI seen as essential, as patients/parents are in unique position to understand experience of condition.

Collective action: CI believes PPI not appropriate for all areas and sometimes realistically there is no time to involve patients – depends on the study. Scientists having an open mind to be challenged and be able to explain things simply. CI thinks funders should make PPI compulsory but the sponsor also needs to be signed up to PPI. The charity owns the database and wants to access it but there are research governance issues.

Reflexive monitoring: PPI representative keeps an events log of conferences, events. As study progressed, ownership of database became contested. PPI work will be reconfigured in future studies to include a formal contract.

Radar plots

Patient and public involvement impact and outcomes

Study design: sensitive research tools, how to undertake procedures and examinations.

Recruitment: changing the wording of PIS. Language: employ people at the charity who speak Urdu or Punjabi so not just relying on interpreter from the hospital – so can have open discussion.

Dissemination: will be from the charity, which is hoped will be more effective coming from patients.

Virtual patient and public involvement

There has been a recent move towards the increasing use of technology and teleconferences being used as a means to include lay representatives who might otherwise not be able to attend meetings. However, this move towards ‘virtual’ contact as opposed to ‘face to face’ has implications for the establishment of relationships. One lay representative said that they always wanted an initial face-to-face meeting to make any following (virtual) meetings easier. As presented in Strategies for recruiting lay representatives, being unable to meet in person also caused problems in CS18, where the CI and lay representative were in different regions. The challenges of building and sustaining relationships through virtual contact were also raised by a funder. Maintaining arm’s length relationships was acknowledged to be difficult for the army of remote lay reviewers, and they found it difficult to sustain any more active involvement. Case study 23 (Box 21) provided an example of virtual PPI with young adults, and also demonstrated how this medium provided an outreach model (see Discussion).

BOX 21 - Case study 23: diabetes mellitus

Mobile phone applications for diabetes service users: mixed-methods intervention feasibility study Description of patient and public involvement

Four YPD as advisory group.

One local community volunteer (experienced in PPI) also a coapplicant.

Another PPI coapplicant was meant to be a co-researcher but dropped out. A third coapplicant was an expert by experience.

Involved in developing initial idea, bid writing, design of study, giving feedback on mobile phone applications (apps), contribution to e-mail discussions.

Recruitment of PPI: coapplicant PPI through previous research contact, and local voluntary representative.

Normalisation Process Theory

Coherence: divergent understanding within the team regarding the PPI model, as research is user led. Concerns that the YPD on PPI advisory panel may not fully represent the (hard to reach) service users. Strong belief in value of PPI for more effective, relevant research/service development.

Cognitive participation: PPI in study has assigned co-ordinator from HEI cluster who runs YPD advisory group with research assistant. Strong buy-in across research team – allied to study being user-led research, but YPD have to be prompted to do their PPI tasks. Continued PPI has been a challenge to maintain here because of other demands on time and geographical remoteness. Coapplicant also dropped out.

Collective action: ‘virtual PPI’; never had a full team meeting – lots of e-mail discussion. YPD advisory group run separately (at their request) but mechanisms for feeding their input back to researchers unclear. Payment for PPI well handled – some flexible use of the project’s PPI budget to support participation. PPI on steering group believes they have challenged the team.

Reflexive monitoring: strong interest in evaluating PPI within the HEI cluster, but very little in evidence on this project. No minutes of meetings; mechanisms by which PPI informs study clear at bid development phase, but not always so in running of the project. Mixed assessments of the value of impact of PPI YPD advisory group.

Radar plots

Patient and public involvement impact and outcomes

Research priority setting: original research idea builds on PPI coapplicant’s website.

Study design: research question and design confirmed as relevant but more clearly defined by diabetes service users ‘expert patients’ panel; split original proposal into two target populations and approaches.

Recruitment: research web pages more younger person-friendly and language less clinical in style. Young person with diabetes mellitus helped to recruit participants to test apps via social media.

Dissemination: researcher and a main participant/informant on Twitter coauthored an invited journal paper.

The benefit of this virtual form of PPI was obvious:

I’ve got a main informant on Twitter . . . asked her if she could work with me on developing a paper and she’s a young person . . . and so she basically threw some question out to Twitter to her one and a half thousand followers . . . and we sort of put some questions to them and we’d written a paper within 3 weeks.

CS23Res04-01

Case study 23 also had an advisory PPI group made up of four young people with the condition. At their request, this group was hosted separately from the research team meetings and deliberately had an ‘informal feel’. These separate meetings were envisaged to be a series of evening, face-to-face meetings with monies allocated to cover travel expenses. However, in reality, as a result of competing demands placed on the attention of this age group, the group did PPI ‘virtually’. Co-ordinating such involvement was a significant workload, which was difficult to predict and cost for. However, as this was adapted to over time, the tracking radar plot became fuller.

Reimbursement

Reimbursing lay representatives for their time is intuitively an enabler of trust and seen as normal good practice, yet payment for time was surprisingly rare in the case studies, although all were paid for out-of-pocket expenses. This was not only because of financial restraints but also reflected a continuing debate from a minority of respondents on how payment fitted with the notion of volunteerism. For example, in CS08 (see Box 13) PPI was felt to be more of a voluntary role, which payment would then ‘professionalise’:

No, I think it should be voluntary . . . because you’ll be paying for services for like, I guess, advice or public feedback, if you’re paying them then it’s, I don’t think, it’s more official, I guess . . .

CS08Res02-01

This was rejected by many lay representatives:

It is and part of the problem is that people just kind of make an assumption, oh they’re patients, we won’t need to pay them. And you know our time in our life is just as . . . important as a researcher’s or a doctor’s.

CS14PPI02-01

Among the five case studies reimbursing for time, the amounts ranged from £25 per meeting to £75. This was often enabled by the host organisation, such as a CLAHRC or research centre. However, even within well-funded host organisations such as HEIs, researchers related institutional barriers to paying lay representatives within institutional systems reliant upon inappropriate standardised status classifications (such as lay people having to be classed as visiting lecturers), electronic forms and difficulties dealing with cash refunds, as well as payments affecting lay volunteers’ benefit payments and tax arrangements. Hence, the host organisations that managed reimbursement and arrangements seamlessly made lay representatives feel a collegial partner with the organisation.

National charity representatives reported that they were being asked to be involved with an increasing number of research studies, which is not necessarily the main part of their work. One PPI representative called for there to be some recognition of cost to the organisation and the consequences:

there must be other charities like me in other therapeutic areas who are getting the same issues . . . So I would have thought that we should be looking at this at a national level to say if patients are going to be involved in research in a way that they have not been previously then they need support to do that and it should not, all of the cost of doing that, should not fall solely on the third sector. We should be getting some kind of NHS/government DH [Department of Health] support for this.

CS14PPI02-01

One paid charity representative said her workload was such that she did PPI work in her own time. Perhaps in acknowledgement of this, another charity representative in a PPI role was contracted (part time) by the research project – yet further blurring the lines between PPI and researcher. It could also be argued that paid employment also introduces different obligations and commitments to involvement, although it has the potential to clarify an otherwise open-ended commitment.

Patient and public involvement and the research cycle

Performing the necessary tasks for PPI appeared limited to certain stages of the research cycle. Some researchers recognised the importance of having PPI before studies were funded but acknowledged the difficulty of finding appropriate PPI representatives in a short time frame. Many researchers felt comfortable involving PPI representatives in the general review of study documents but not so certain about involvement in other parts of the research process. In most case studies there was significant PPI during study set-up, but this often decreased as the research progressed, for example when patient recruitment or data collection started. Researchers often reported that the pressure to recruit research participants (during data collection) resulted in less capacity to enable involvement of PPI representatives. This impacted most on relational aspects, with changes evident in this sector of the NPT radar plots.

The distinctive characteristic of case studies that enabled involvement at all stages of the research process was the presence of a dedicated individual facilitating PPI (e.g. CS21, see Box 6). A complementary or alternative approach was having a lay coapplicant who was fully involved from the inception of the design, and sometimes undertook an outreach role, or worked as an integral part of the research team. Case study 04 (Box 22) had one experienced lay member as coapplicant; this was enabled by the research topic being focused on a patient evaluation tool, and by having an established PPI group with experienced lay members.

BOX 22 - Case study 04: diabetes mellitus

Questionnaire development: evaluation of self-management programme, mixed methods Description of patient and public involvement

Experts by experience.

Number of levels: focus group, consensus event (10 people) (not consented so counted as public involvement); two lay representatives on steering committee (one of the lay representatives is also on the research team and is particularly experienced; has been on diabetes user group for 3–4 years and been involved in other research projects).

Stages: involved throughout, as one PPI representative is on research team.

Recruitment of PPI: diabetes user group and known to CI.

Normalisation Process Theory

Coherence: both researchers new to PPI, but both felt PPI essential to research: ‘seems ridiculous . . . without’. Consensus event considered to be PPI. Researchers and PPI representatives agree on purpose of PPI; real situations, bigger picture, quality of life, well-being and sense of worth.

Participation: CI is approachable, in contact a lot; ‘invaluable to have someone with the condition involved’. PPI representatives view ‘amazing piece of research’.

Collective action: CI felt PPI not embedded within institution. PPI rep. talks of flat structure (relationship), managing expectations, welcomed, included, round table discussion, not intimidating. Other PPI rep. had thought she would not be sort of person for steering committee, as sounds too high powered, but persuaded through personal contact. Travel, support organised.

Reflexive monitoring: comments given on questionnaire are recorded. Second PPI representative does not consider self to have contributed very much. The end result is that questionnaire is unrecognisable from original form (so can see impact of PPI input). Felt to increase validity of research, more balanced. PPI representative suggested changing area of country when researching the education courses (national coverage). Questions were ambiguous and section on pumps changed.

Radar plots

Patient and public involvement impact and outcomes

Protocol: input into proposal (keep the patient in mind).

Design of intervention: questionnaire, issues added not thought of by researcher.

Dissemination: two papers written, one submitted to journal with PPI representative’s name as coauthor.

Outcomes of PPI: rewarding for PPI reps.

Patient and public involvement in different research designs

Funders and some researchers espoused stereotypical assumptions about the limitations of PPI in certain research designs, such as basic science, which has traditionally been an area with little PPI. A PPI co-ordinator working in a medical setting and talking about CTIMP (clinical trial of an investigational medicinal product) research observed that, because some researchers believed patients have difficulty in understanding the science and bench side of research, they find it hard to engage and have meaningful PPI. Similarly, one funder suggested PPI was more compatible with applied science:

I think it comes down to the complexity of science and the type of research that you’re talking about, some studies: when it’s very kind of lab based and it’s very molecular, doesn’t really lend itself to being able to involve the public or patient, you know, at a very kind of technical level. But there are a lot of opportunities in medical research where it is viable and it is possible to do it.

Funder03

Other researchers also felt that involving PPI representatives in ‘technical’ discussions about certain research designs was tokenistic, as they were able to contribute very little:

I mean we’re doing a review of reviews . . . so it’s methodologically very challenging, it’s conceptually quite challenging, half the academics feel out of their depth, intellectually and conceptually. The PPI feel . . . that’s not to denigrate them or to put them down in any way at all, it’s not what PPI is about, . . . No, they come to the steering group and then they feel a bit left out and I think it’s a shame, I think it’s, you know I think it’s disrespectful in a way that not having, I mean I think, for that project it would be more respectful not to have PPI, does that make sense?

CS03Res04-01

There was also some evidence that researchers felt concerned that some PPI representatives who lacked scientific understanding could jeopardise a study at the design stage. However, another funder challenged assumptions about the ability of experienced PPI to review less applied, more basic science applications:

when we set up these committees, there was a fear or a worry that they would have trouble reviewing basic science applications and therefore they would be more drawn towards the clinical ones, but actually what happens is, it’s quite funny really, it’s almost the other way round. I think they can see the pipedream or the hope at the end, and almost they’re more critical about clinical studies, I think maybe because it’s closer to the patient benefit.

Funder01

Indeed CS17 (Box 23) successfully involved PPI in approving bench research proposals in genetic epidemiology through a panel model whereby the PPI representatives are presented with lay protocols written specifically for the panel.

BOX 23 - Case study 17: diabetes mellitus

Tissue bank database Description of patient and public involvement

Established panel for over 10 years; about 30 on panel and two attend each monthly meeting (usually invite four). Meeting and reviewing papers. PPI mixture of research-naive and ‘professional lay people’. PPI work more widely than just the panel – work with primary care research network. Some of the PPI representatives do have samples in the biobank (therefore participants as well).

Stages: involved at the beginning at proposal stage, sometimes PIS.

Recruitment of PPI: diabetes mellitus research network, county shows, shop staff, researchers go out to recruit.

Members decide which researchers should have access to database. Some members also from diabetes mellitus research network.

Normalisation Process Theory

Coherence: new way of working for CI (was a tick box previously). Panel running some time (PPI now formal), previously PPI more ad hoc. PPI representatives had not realised how powerful PPI could be before being involved (changing the way studies go, outcomes relevant to patients). Researcher and PPI representatives seem to be in agreement about the purpose of PPI and the value of it. PPI to help other people, intellectual stimulation.

Participation: PPI representatives have sense of ownership. PPI representatives write articles for newsletters and magazines encouraging people to take part in the research. CI committed to PPI and is the main contact. All feel it should be part of their work. The CI uses the database budget to fund PPI.

Collective action: workload is manageable. PPI representatives know what they have to do, including commenting on proposals, PIS and consent forms; ensuring more accessibility to lay people. Direct feedback in the meetings, for example intervention is too onerous for people, information from own experience and own health background. The panel votes on whether or not they think proposed research is a good use of the database and samples. PPI representatives feel welcomed, communication is good. CI says group has to be managed well. CI would like to offer the panel some training in the future (governance processes and ethics). PPI representatives feel supported.

Reflexive monitoring: PPI representatives feel valued. PPI representatives suggested a toolkit for when you start on a project (time commitment, travel expenses for different research networks). Feedback all the time – PPI representatives know what happens after the meeting – whether proposals are approved or not.

Radar plots

Patient and public involvement impact and outcomes

Research priority setting: PPI representatives decide on which researchers have access to database/biobank.

Ethics: PPI adds strength to ethics committee submissions.

Recruitment: PPI representatives involved in writing articles to involve patients. PPI representatives tell friends about a new study. Research more patient friendly (PIS).

Outcomes for PPI: PPI provides intellectual stimulation; socially, meet like-minded people.

Systematic appraisal of patient and public involvement

Of the two case studies with systematic appraisal, CS13 (see Box 16) had an annual evaluation, with a review of the PPI contribution and model at the panel annual general meeting. The researcher co-ordinating PPI ‘measures’ the extent of PPI against published best practice guides to assess if PPI can be of use in further phases of the research:

The NIHR, five or six points in the research cycle in which people should be involved, I’ve always bought into that notion, so we try and use that as a template for judging how we’re doing internally.

CS13Res05-01

Case study 21 provided the RAPPORT study with numerous documents which recorded PPI activities and impact but currently did not formally evaluate PPI. However, the case study site was actively involved in piloting an evaluative framework for PPI with clear intentions to change its PPI practices in future to include systematic appraisal.

However, researchers often did not have the time or resources to record PPI activities and reflect on PPI input, and some did not see it as something they should be doing. Respondents from CS06 (Box 24) saw PPI as an implicit part of the research process and hence not requiring discrete evaluation:

BOX 24 - Case study 06: public health

Physical activity in older people in primary care: randomised controlled trial Description of patient and public involvement

Hybrid model consisting of one-off events for less experienced PPI, and an advisory group including a more experienced PPI with more commitment/ongoing responsibility required. Member of the general public. Three development focus groups (approximately 60 people) held with participants from earlier pilot.

One PPI focus group (nine people) to look at project participant paperwork, before ethics committee approval.

One PPI member on advisory group from local community voluntary service.

Chief investigator also got feedback on dissemination materials from three older people: one trial participant and relatives of researcher.

Stages: in all stages, apart from data collection/analysis.

Recruitment of PPI: participants in pilot study and local (recommended) voluntary representative.

Normalisation Process Theory

Coherence: CI less experienced at using PPI, so takes steer from highly experienced coapplicant (has a deep working knowledge of PPI in this field and at HEI, where PPI is strongly embedded). Other research team members see PPI as closely linked to trial participation. PPI representative on advisory group sees PPI in wider context of community voluntary service participation in service/policy consultation.

Cognitive participation: all research team very keen, PPI members in focus groups strongly perceived to have been very keen too – there was a waiting list.

Collective action: coapplicant is highly attuned to power dynamics of PPI. Research team employ their different professional skills/relationships with PPI to good effect. Previous/ongoing relationship of research nurse with PPI (part of GP practice) highly significant. Providing food/refreshments significant in light of no payment. PPI representative on advisory group has a lot of trust in team and their professionalism. Pacing of implementation focus group was perhaps too intense according to CI. No training offered or requested.

Reflexive monitoring: clear contributions systematically recorded, with implications, but changes not necessarily fed back to those involved in focus groups. PPI representative on advisory group sees the endeavour of PPI as intrinsic to research.

Radar plots

Patient and public involvement impact and outcomes

Marketing the study: changed project logo and images used on PISs.

Design of study: participants on previous (pilot) study ‘came up with’ study design for this RCT.

Design of intervention: refined participant intervention – time taken to complete, better tailored to older ‘healthy volunteers’.

Recruitment: changed language used to identify participants, not ‘seniors’ or ‘elderly’ but age range. Clarity of invitation letters, PIS, trial handbook improved. Include images of older people from different ethnic groups and both single and couples, and specify exactly what the intervention group and control group did in the trial.

I wouldn’t say it has added value, I think it’s actually fundamental to, to the research inquiry . . . it’s not just added value, it’s intrinsic to the inquiry.

CS06PPI01-01

However, as the NPT radar plots illustrate (bold box), although there was no systematic evaluation of PPI, respondents did individually appraise PPI as worthwhile in the study.

Other researchers struggled to evaluate PPI because there was little guidance or knowledge on how to carry out this task and recognised: ‘it’s quite hard to evaluate I would have thought’ (CS08Res01-02). Despite the lack of systematic evaluation, some research projects did record PPI activities, providing a baseline to document PPI input.

For some researchers a lack of robust evidence on the value of PPI was a problem. In CS10 (Box 25) it was explicitly recognised that what was seen as ‘worthwhile’ by the general public might not be seen in the same way by researchers:

BOX 25 - Case study 10: public health

Trial of self-help materials for the prevention of smoking relapse: randomised controlled trial with parallel qualitative study Description of patient and public involvement

Experts by experience, using a ‘consultation’ approach.

Two PPI representatives on trial steering committee (one could not attend meetings and did not comment on materials).

One PPI representative (ex-smoker) gave one-off feedback on intervention booklets.

Local PPI patient involvement group provided (one-off) feedback on PIS, for further linked qualitative study. PPI co-ordinator attached to this patient involvement group, little input.

New PPI representatives 2013: one person to take on the trial steering committee role (replacement) and another interested in the qualitative study (both have academic backgrounds).

Stages: development and developing study materials, refining intervention and on committee.

Recruitment of PPI: local PPI group and CI recruited other ‘lay’ representative. Mixture of smokers and ex-smokers.

Normalisation Process Theory

Coherence: research team less clear on purpose of PPI in this project, or its potential further application. PPI contribution as mostly collaborating to redesign booklets and giving feedback on additional qualitative study bid. PPI representative recruited by CI was not aware of PPI as a broader ‘movement’.

Cognitive participation: CI less experienced in using PPI and not clear on validity of PPI, especially when recruited via PPI networks. Researchers: normal practice to them; see the PPI in this study as relatively straightforward, unproblematic. PPI representatives have mixed views on the value of PPI in this study.

Collective action: researchers understand the importance of PPI in tailoring process to suit volunteers, although they question costs associated with local PPI group model. Secretarial aspects of checking project materials for clarity suited PPI representative but less confident about commenting on research design (although has had training). Lay PPI representative happy to help, would have done it for free (convenient, interesting, local).

Reflexive monitoring: CI and lay PPI question the contribution PPI actually has to project. Very little supporting documentary evidence, not clear what specific changes PPI made to intervention booklets or PIS for qualitative study. No PPI in minutes from first two committee meetings (did not attend or send comments), in final committee no comments attributed to new PPI person. PPI rep. feels they made a minor contribution.

Radar plots

Patient and public involvement impact and outcomes

Design of study: PPI representative on trial steering committee (along with other local panel members) provided timely feedback on lay summary for additional qualitative bid. Two new PPI members have input into the interview topic guide.

Intervention: refine self-help booklets – better tailored to population targeted. Clarity of PIS improved.

especially not to researchers. To people outside the world of research, I can reason very happily [on the value of PPI]. But researchers want evidence and that’s the holy grail, and I haven’t seen it yet.

CS10Res04-01

The rather mixed view on the value of PPI within CS10 can be seen in the radar plot in the case study summary. Within the tracking radar plot, the impact of questioning the value of the PPI in this study is marked.

Enabling contexts for evaluating patient and public involvement

Funders’ requirements (C1) were a strong enabling context for evaluation. For example, the requirement to report on PPI when NIHR-funded projects were completed was a major incentive for researchers to review processes and benefits of PPI in the project. Indeed, some case studies agreed to be part of the RAPPORT study as an aid to meet this requirement. However, approaches to evaluating the outcomes and impact of PPI were shaped by the funding organisation’s stance towards PPI. The data from the programme management organisations strongly resonated with the methodological argument for PPI; hence, the organisations were actively seeking ways of evaluating the impact of PPI on research outcomes. Approaches cited, however systematic, seemed descriptive and biased towards ‘good news’: the collection of illustrative case studies and the specific identification of PPI in the final research report, as described below.

I ask that all the monitoring programme managers, if they spot something really interesting or good or unusual or positive, that they just let me know. And then I keep a spreadsheet of all of those for posterity, for, you know, when I start doing, we’re doing a lot of work on impact here . . . we’ll have more ability to really get in and look at good examples of PPI.

Funder04

There was evidence that this focus on evaluation, although not fully assessing and comparing impact, was resulting in some modification to the funder’s PPI work, for example developing advice to support lay representatives on trial steering committees.

For the funding organisations framing PPI more in the engagement paradigm, evaluation was focused on the impact of public engagement activities, and a web-based tool was used for researchers to record their public engagement activities and outcomes. For the condition-specific medical charities, again perhaps shaped by their moral stance on PPI, there was anecdotal evidence of where PPI has had an observable impact: ‘I think that they [grant applicants] have an increased awareness of the fact that they have to write a good lay summary’ (Funder01). However, there was no systematic appraisal of this.

For lay representatives, a positive appraisal of PPI was shaped not only by a positive experience (C8), but also by personal outcomes. Described further later, these outcomes included increased understanding of their health condition:

I read some of the interviews from other patients . . . it really helped me because personally . . . reading about how people are suffering from it and I’m not the only one, other people are also going through the same sort of problems, yeah. So some people do get better, people still live a happy life.

CS18PPI02-01

For others it was a sense of giving something back to the health-care system:

I then felt well if I can do anything, you know for the greater good, I was happy to do it and so I get out of it the feeling I’m making a contribution towards the research and the further development of treatment for patients with rheumatoid.

CS14PPI01-01

A number of researchers also felt PPI was worthwhile to them personally beyond impact just on the research process and outcomes. Researchers spoke of PPI bringing them closer to patients or service users, and reminding the researcher about the real aim of the research:

remind you a bit why you’re doing it because sitting in front of a load of numbers you sort of think, you know you’re doing it for those people. Because that’s the population you’re working with.

CS13Res01-01

A positive experience of PPI was often the start of the process to establish PPI as normal practice:

she’s really been the one that’s been my epiphany for understanding patient involvement more than anyone.

CS08Res01-01

Equally, a negative experience of PPI could hold up this process for some time:

I did an informal questionnaire amongst patients who we saw in the secondary care clinic, research patients, asking them to identify, you know, suitable research topics. However, I didn’t find it was very useful, patients mainly said that research should be concentrating on identifying a cure for diabetes, and that’s not completely helpful as that’s like a very long-term goal in research . . . then that sort of coloured my views of PPI for a little while afterwards thinking, hmm, I’m not sure really what patients want to get out of it or whether they have enough knowledge about the research process and personnel involved to be able to contribute effectively.

CS02Res02-01

Modifying future work

The majority of the case studies reflected on their PPI approaches only informally towards the end of the study, so there was minimal evidence of any changes within actual case studies. However, in a longitudinal cohort study (CS14) there was also evidence that being a case study within the RAPPORT study provided the opportunity for participants to reflect on PPI and plan things differently for the future:

you’re making me think [interviewer], maybe I should do a little leaflet to send out to show them all what we are doing with their information. So as I’ve been talking to you that’s something I will be doing differently and I’ll bring it up at the meeting that we should be sending them an update or feedback.

CS14Res03-03

This case study (Box 26) was supported by minimal PPI infrastructure but the impact of reflection can be seen in the tracking radar plots (bold box).

BOX 26 - Case study 14: arthritis

Prognostic factors in early arthritis: cohort observational study Description of patient and public involvement

Hybrid: national representative, local service user group and expert by experience. However, these components did not come together and hence there was no cross-fertilisation. The national representative has advised nationally (multicentre) but the other lay representatives are just asked to be involved when needed, for example in the training of research nurses.

Recruitment of PPI: the national representative was formally recruited but the service user reps and expert patient were invited as CI’s patients. Service users also undertook dual role as study participants.

Age of study: started well before PPI in research movement, so rather ad hoc.

Normalisation Process Theory

Coherence: Less evidence of a shared understanding of the purpose and definition of PPI.

Participation: different levels of buy-in and some hesitation over recruiting PPI representatives who do not just talk to a personal agenda.

Collective action: relationship key and is underpinned by a long-term clinician–patient relationship. The national representative is finding the workload of being involved in a number of studies challenging. The lack of reimbursement of their time is a problem and has consequences for the national charity. Limited PPI infrastructure means ad hoc approaches. Blurring of PPI in service delivery and PPI in research.

Reflexive monitoring: more how it has impacted on research process rather than outcome. Lots of personal benefit for PPI representatives. Using a service user group was seen as having benefits but may require a different way of working in the future.

Radar plots

Patient and public involvement impact and outcomes

Design of study: PPI representative on trial steering committee (along with other local panel members) provided timely feedback on lay summary for additional qualitative bid. Two new PPI members have input into the interview topic guide.

Intervention: refine self-help booklets – better tailored to population targeted. Clarity of PIS improved.

For the majority of case studies, either modifications to PPI approaches would be operationalised in future studies, or the influence of that particular case study was affecting PPI approaches within other studies:

What is great now, lots and lots of studies are coming through . . . other departments who weren’t doing any PPI or patients, it is forcing them, I have access to a group of volunteers, pre-screened; you need to do PPI to get them! . . . we are getting sport scientists, oncologist, all sorts of people; the meetings are becoming more and more interesting . . . now the meeting [panel] has such a wide diversity of studies that are being discussed.

CS17Res01-03

Hence, while the CS17 (see Box 23) tracking radar plot showed minimal change in the reflexive monitoring segment, if expanded to the research environment in which it was hosted then a far more significant change could be noted.

Nevertheless, extensive future changes to PPI approaches could be inhibited by a lack of understanding of what is possible, as exemplified by CS09 (Box 27):

BOX 27 - Case study 09: IDD

Diagnostic testing during pregnancy: questionnaire Description of patient and public involvement

Two national representatives of two different support groups. Both PPI representatives sit on project advisory group; one of them carries out much of the qualitative analysis.

Normalisation Process Theory

Coherence: PPI representatives have seen an increase in PPI activity over the years. PPI rep. believes formal structures for PPI are a good thing. PPI believed to be important, valuable and worthwhile. Agreement among PPI representatives and researchers. PPI representatives feel listened to and not just because they have to have PPI.

Participation: can make medical ideas understandable for users; PPI is extra work but not a negative. Researchers will definitely use PPI in future (and one PPI representative is now a coapplicant on next grant).

Collective action: PPI representatives are confident to put views across, PPI representative doing qualitative analysis (skills), other PPI representative commenting on main themes (sees her lack of research skills as advantage), reassurance for researcher in a sensitive topic. One PPI representative and researcher know each other personally.

Reflexive monitoring: talk of track changes to documents but back copies not routinely collated. PPI representative has led on documents and views taken on board. Suggests a PPI mentor for less experienced PPI representatives. Research paper changed because of comments made by PPI representative. Researcher would like to work with individual women also but not sure how to go about it, unless through the charity.

Radar plots

Patient and public involvement impact and outcomes

Design of study: PPI representative on trial steering committee (along with other local panel members) provided timely feedback on lay summary for additional qualitative bid. Two new PPI members have input into the interview topic guide.

Intervention: refine self-help booklets – better tailored to population targeted. Clarity of PIS improved.

I don’t know how you go about involving patient and public in planning research, that maybe sounds a bit silly; in my mind I have a big question mark about how it works: before you recruit anyone to the study you need ethical approval, and to me asking patients potentially what they want research to entail, is in a way recruiting them to a mini study before the main study.

CS09Res02-01

While the radar plots indicate a comparatively embedded level of PPI, future potential impact of PPI is constrained by researchers’ beliefs. This reinforces findings presented earlier where potential PPI impact was inhibited by boundaries set by the research team.

Research priority and question setting

Five of the case studies reported that PPI representatives had been involved in research priority setting. For example, the panel in CS17 had input into deciding which research should be funded, although we received no documentary evidence to show the impact on decision-making. We were provided evidence of PPI impact in identifying research questions. For example, the research centre hosting CS21 has a large PPI group who are encouraged to put forward research questions, and this particular case study originated from one of the parents. The structure of the research centre and the priority that this organisation gives to PPI ensure that the ideas relevant to service users are carried out:

Because the very idea of the whole thing came from a parent, it’s been a very natural process to have families involved in it all of the time, and I think that’s the beauty of getting the idea from the family. You don’t have to then try and shoehorn PPI in, it’s already there. So, it’s just been from the word go.

CS21Res01-01

Linking to the wider community

Four case studies utilised PPI to provide links to the wider community. Case study 02 exemplified this approach, with the single lay representative on the steering committee consulting with wider range of people from the study target population. A newsletter to this group provides evidence of the impact of this approach (Box 28).

In CS23, this PPI bridging role to the wider population (in this case younger people) was done via social media and was an effective way of getting one-off input from an often less-heard group. This input was used as a basis for a practice-focused article and also informed the development of the study (Box 29).

Lay representatives were also used as a link to recruit study participants. Examples included writing articles for newsletters to encourage participants to take part, being filmed as part of the recruitment video, using social media or approaching people directly through their networks (support groups).

Such connections with other groups of members of the public or service users proved to be valuable in recruitment, and in harnessing a wider range of (less accessible) PPI representatives/experiences, reassuring researchers who were concerned about the PPI representatives not being a ‘typical’ lay person.

Marketing the study

Two of the studies had PPI input regarding study logos, and in the first study the logo was changed (Box 30).

However, in CS19 PPI advice was not taken on board, as the logo had been already commissioned and developed by the team (and PPI coapplicant), but it was later criticised as being potentially insensitive by steering group PPI representatives:

it was about their logo [. . .] I just hated it on sight, and their response to that was, ‘But we paid someone to do it and . . . maybe . . . [other] people wouldn’t feel like that.’

CS19PPI02

Design of study

Patient and public involvement representatives made comments on the protocol and design of the study in 13 of the case studies, particularly regarding lay summaries, data collection instruments and study information (Box 31). In other case studies PPI representatives were consulted over study design; for example, they were asked whether a RCT or partial randomised control design should be used, and PPI representatives’ suggestion of a RCT design (CS07) was accepted. In another case study the PPI representatives were asked if they approved of particular methods of accessing patients’ notes (CS01). Data collection tools such as questionnaires were ‘softened’ by PPI representatives in terms of language and sensitivity (CS09, CS21).

Safety of study

Involving patients’ groups in the governance of a study had evidence of potentially improving patient safety. In the original proposal of CS02, the researcher conceded that the research study was presented in a very clinical fashion and, because it was asking people to reduce the amount of a particular drug that they were taking, the safety element (from the patient’s point of view) had not been considered. PPI views (through a national charity panel) changed the application to include a safety questionnaire:

when I put together the original application, it didn’t really have much in terms of the safety element because I had taken it for granted that people are already on [x drug], reducing the dose wouldn’t make a big difference and I would have seen it in a clinical fashion, but as soon as we got the patients on board and had their comments coming through, one of the things that seemed to be coming through was a concern, or a potential concern from the patients that would it make them feel unwell or would it do anything to them? So therefore we then added an extra element asking patients about how they felt, safety questionnaires.

CS02Res01-01

The outcome of PPI here was reassurance for patients in terms of safety and also improved recruitment rates, and a 3-month visit (for reassurance) was added to the study design for participants.

Design of intervention

In those studies which contained an intervention as part of their study design, PPI representatives helped to design or make suggestions to existing interventions to make them more acceptable/feasible. In CS03 one PPI representative suggested there should be more content on pre-pregnancy and diabetes mellitus, and the documentation confirmed that more was added into the resource intervention (Box 32).

When the intervention was a questionnaire (CS04, CS07) the questionnaire was altered to include items not previously considered by the researcher:

I’m sort of embarrassed to admit it, but I went to that meeting thinking, ‘I know everything that should go in this questionnaire’, but I so didn’t, and loads of stuff came out in that meeting that I just never would have thought of.

CS04Res01-01

In another case study (CS07) PPI representatives were involved in the validation of 260 pre- and post-questionnaires (which were used as part of the intervention). The PPI representatives took the questionnaire back to their local user groups and asked them to complete the questionnaire as a pilot. However, there was also evidence that researchers perceived an unintended outcome of PPI. In CS23 the CI (who was also a clinician) felt that the PPI representatives had steered the project off course, and that the interventions developed were neither innovative nor adequately focused on the research question.

Research ethics submission

Two of the case studies felt that having PPI helped their ethics submission by saying that materials had been viewed and approved by PPI representatives:

they [research ethics committee] take note it, of course the committees have their own lay people and I mean the one thing about lay people is they’re individuals like all of us, it’s a bit like a peer review and you might get lay members of the panel who don’t really agree but they take note of it and I think it carries quite a bit of weight, doesn’t always carry the day though.

CS01Res02-01

Recruitment

One of the most frequently stated outcomes for PPI in our case studies was a perceived increase in recruitment rates to the research study (18 of the 22 case studies). Researchers commonly attributed recruitment success to the PPI input to materials (suggestions on layout, size, language, colour, length, readability, clarity and design; evidenced by tracked changes/before and after versions of documents) used for recruitment (PIS, invitation letters, consent forms). Suggested changes were made either directly by the PPI representatives themselves or by taking materials back to wider reference groups for further comments; for example parents showing the PIS to their children. PPI representatives made changes to language such as taking out the word ‘mortality’ or explaining technical words such as ‘lung function’. They introduced a summary section at the start of the PIS and produced information in a small, credit card-sized, format for people to carry in their wallets. Changes also included images of actual people wearing equipment or lying on scanners, and extra information about the length of procedure and the weight and size of any equipment was added (CS13 PIS).

One researcher felt able to quantify the increase in recruitment because of such PPI changes. A number of different versions of the PIS and invitation letter had been submitted to ethics committees:

we changed a few things but we did [a] stage [at a] time so we could see what had an impact, and that certainly did increase recruitment by, only you know, 1 or 2%, but 1 or 2% when you’re sending out thousands a month . . .

CS12Res02-01

Outcome measures

Research teams advocated the use of PPI for outcome measures that were relevant to patients; this was particularly relevant to one arthritis study:

it was appreciated that there were certain things that we were measuring that we thought were really important, these patients actually weren’t interested in at all and didn’t deem relevant, whereas other issues we didn’t think were . . . well, we thought they were important, obviously, otherwise we wouldn’t measure them, but we didn’t realise how important they were to patients.

CS14Res01-01

There was qualifying evidence here that the outcome measures had been changed or added to in the research design. Respondents also reported that this change of emphasis was fed into clinical guidelines, which may have an impact on patient experience.

Data analysis and collection

Two studies had PPI representatives involved directly in data collection. The PPI representative in CS18 was involved in data analysis; she was asked because of her cultural knowledge and to reassure the researchers that the analysis resonated with her experience of the ethnic ‘community’:

I want to check with the community that I’m not sort of completely going way off track here so I guess in some ways that was her main role, more than in this situation the experience of living with the illness really, it was actually her cultural knowledge that was most helpful.

CS18Res01-01

In one case study where this PPI role could have been applicable but was not utilised (CS21), this was a deliberate restriction because of ethical considerations; many of the study participants would have been known to the PPI representatives.

Patient and public involvement representatives also made a contribution to data analysis (Box 33).

The impact of this was clearly described by the researcher:

When I initially did a straight comparison women’s views versus health professional’s views, [PPI02] suggested I break down the women, into those that had experience with [xxx] or [xxxx], compared them to women who haven’t had any problem [xxxx] and we found quite big differences between the two groups.

CS09Res01-02

Dissemination

Two of the case studies (CS17 and CS21) mentioned that the PPI representatives had presented the study findings at conferences, and six examples of academic papers with PPI representatives as coauthors were submitted to the RAPPORT study. One of the PPI representatives, who was working as an academic researcher 3 days a week on the project, was first author of a paper (and her organisation was the charity mentioned in the paper). Other representatives were mentioned in the acknowledgements at the end of the paper. Some case studies in the RAPPORT study had not yet reached dissemination phases; however, our tracking interviews suggested that, in a number of cases where it had been planned that PPI representatives would be involved in dissemination, this had not happened.

Moral impact and outcomes

In addition to the outcomes which were identified in terms of the research project, the researchers and PPI representatives also identified personal outcomes associated with their PPI role. The personal benefits of PPI have been well documented, and those reported here echo previous studies. These included increased self-confidence and mental challenge. The PPI role helped people to keep mentally active in retirement or unemployment and provided the possibility of leading to other roles or responsibilities:

. . . enriches our research but also enriches their lives.

CS21Res01

. . . it just opens up a world of opportunities.

CS21PPI01

Interestingly, some PPI representatives spoke of the benefits of their involvement to the wider society: ‘maybe it’ll change the world a little bit’ (CS11PPI02-01). In CS03, the PPI representatives felt that they could pass back information to their wider network and again be ‘boundary spanners’ (which Williams161 describes as ‘individuals who have a dedicated job role or responsibility to work in collaborative environments’) between the academic world and the patient community:

I have met since my diagnosis other people who’ve had [condition] for many many years who haven’t necessarily had access to much information or they haven’t, or they have had access to information and services but it’s been a bit sporadic and I always wanted to kind of pass that information along or back to people that I’d met, so . . . I’ve kind of been a lay tutor.

CS03PPI04-01

Longer-term outcomes

The RAPPORT study was designed to include studies that concluded during our data collection period, with the expectation that outcomes of PPI could be summatively assessed. Among the eight that were completed, we found no evidence of any immediate outcomes of PPI for the research findings. However, out of these eight studies, five had demonstrated clear embedding of PPI and interestingly it was these studies that clearly identified the outcome as developing further grant applications, as more research was required. These five case studies were continuing to work with their lay representatives to generate further research questions. Out of the remaining three, one was in follow-up, we could find no evidence from the other two case studies of any particular outcome, and from a dissemination perspective there was nothing publicly accessible providing results or outcomes of the studies. It is clear that, as in all evaluation of research impact, longer-term outcomes influenced by PPI need an extensive time to detect.

Engagement, feedback or involvement?

While confusion between participation and involvement still persists among some junior and indeed some senior staff, the boundaries between ‘engagement’ and ‘involvement’ were sometimes even less firmly defined. In differentiating between engagement and involvement, we draw on definitions offered by INVOLVE167 and National Voices168 which view involvement as a deliberative activity characterised in various two-way processes such as collaboration or partnership. Our findings suggested that, in a minority of case studies, simply providing feedback on, for example, the study results, despite being one-way information giving, was seen by some participants as active involvement.

A recent study exploring PPI in commissioning services for long-term conditions57 suggested a model of involvement as the product of engagement plus experience. Peckham et al. 57 defined engagement as a commitment to respond or interact. In terms of PPI in research this could mean the public attending dissemination sessions, reading research summaries or going to engagement events. One of our respondents from a funding organisation very clearly defined PPI in terms of this kind of engagement, discussed in more depth later in this chapter. Our PPI respondents’ experiences were varied and included public service, experience of a health condition as patient or carer and being a research participant.

If PPI is limited to engagement, this precludes the full potential of involvement. The model of partnership synergy proposed by Lasker et al. 105 illustrates the full potential of involvement where the sum of the whole (PPI and researcher working together) is greater than the sum of its parts. Synergistic collaboration leads to thinking that is comprehensive and holistic, creative, practical and transforming.

In one RAPPORT case study (CS10), a PPI representative was identified as such because he had the appropriate experience of the public health issue being researched, but not of prior engagement with the research process. His involvement in this study was limited to a one-off event and the data suggested he was unlikely to actively seek to be involved in any future studies.

The study of Peckham et al. 57 also identified the impact of failing to maintain a two-way interactive process, which leads to disengagement. Some of our PPI respondents reported feeling disengaged from projects because of long periods of silence, and, if results of studies were not fed back, disenchanted with research overall.

The dual role

While some researchers appear not to understand the differences between involvement, engagement and participation in research,167 there is also developmental evidence of some explicit merging of research participant and PPI roles, in which patients undertake a dual role as participants and also as lay representatives. With the current focus on increasing patient participation in research trials169,170 and given that involvement and participation are listed alongside each other in the NIHR’s aims for patient and public awareness of research, the time may now be appropriate to examine the potential advantages and implications of this dual role further. In particular, the ethical implications and the robustness of knowledge gained in this way need to be explored further.

We found little evidence within the literature of the dual role, and discussions at our dissemination events also suggested that this was seen as very unusual, if not ‘bad’ practice. However, Goodman et al. 171 describe the benefits of involving young people in a study on transport and health as advisors, co-researchers and study participants. While they acknowledge it is more in the research team’s interests to recruit study participants, there were notable outcomes for the same young people in acting as advisors: increasing self-worth and understanding of research, and having enjoyable work experiences. While aware of potential ethical issues, the authors describe a reciprocal win–win situation for the young people, schools and research team. Where managed carefully, such experience for both research teams and lay representatives was also evidenced in the RAPPORT study.

Patient and public involvement: is it always needed?

While the majority of our survey respondents and case studies did report having some form of PPI, they varied considerably in how many research stages and which levels of PPI activity this involved, with a few case studies showing PPI as entailing only minimal, one-off activity. Not everyone believed that PPI was essential to each stage of a study or was persuaded that benefits far outweighed potential risks, particularly for studies nearer the laboratory end of the ‘bench to bedside continuum’. 172 Arguments for seeing PPI as less essential or more risky included there being a lack of evidence of PPI effectiveness in this type of research. However, our analysis suggested that underpinning this assertion were several researcher beliefs, including that lay representatives would not be comfortable with the more basic science and that they lack the knowledge to understand it. A small number of researchers appeared to minimise the general ability and skills of PPI representatives to contribute to research. Some respondents, including a funder, had changed their view on this, and after initial surprise reported now seeing their lay representatives as being able to understand and offer more than originally envisaged. 27

Narratives also included researchers’ stories of lay representatives being seen as overstepping boundaries and dabbling in science. A few researchers perceived a threat of the lay person taking over, an anxiety reported in other studies. 28 The supposed menace of experiential, subjective knowledge overturning scientific notions of neutrality, objectivity and distance173 has been described in terms of a clash of two worlds. 130 However, McKevitt et al. ,16 in reflecting on PPI in a stroke research study, argue that the moral status of PPI balances out any asymmetry between these two worlds. Some of our case studies at the basic science end of the research continuum did bear out the sustainability of such a viewpoint, with researchers making great efforts to ensure active involvement.

In terms of amount of activities, PPI was most likely to be limited in basic science studies, where lay knowledge was generally seen by researchers as less useful in this kind of research. 39 Callard et al. 172 argue that failing to have PPI in the early stages of translational research may result in failing to capitalise on the potential contributions lay representatives could make, including ensuring that the end product of bench research must be appropriately designed for the end-user, and that ethical issues arising between any academic and commercial interests are challenged. The basic science studies in the RAPPORT study that did have a firm PPI underpinning adopted a partnership approach,105 even though at times this partnership came under strain, particularly over issues of who had more say in decisions about the use of data. Conflict or tension, while posing difficulties to resolve, is not necessarily undesirable and, indeed, is evidence of engagement,105,174 so much so that where it was managed well, through, for example, clear guidelines, dialogue and consensus, it was seen to add to the strength of the case study in terms of how embedded PPI was.

Values shaping patient and public involvement approaches

Two main stances, methodological and moral, appeared to shape the ways in which our respondents viewed and operationalised PPI. A third stance (policy) described in Chapter 2, was the initial trigger for many of the researchers and the centrally funded funders to engage with PPI. Analysis of the data from these respondents suggested three pathways (see Figures 9 and 10).

A frequently found narrative, illustrated in Figure 9, was of researchers ‘having to do’ PPI because of revised governance requirements. 5 However, this often shifted to a subsequent ‘conversion’ to PPI when they appraised outcomes of PPI as having had a benefit on the research outcomes. Policy was then no longer required as the main trigger, so PPI activity became a self-sustaining virtuous cycle. A variation found in this pathway, also demonstrated in our analysis, identified not only a ‘conversion’ in terms of the methodological benefits of PPI, but also a move towards the moral value set whereby PPI became a process that one should do because it is ‘right’. Of course, some respondents were embedded in this moral paradigm and their pathways can be depicted in Figure 10. In particular, the funder representatives from condition-specific charities articulated PPI as a moral imperative and, perhaps paradoxically, had given little to no thought to if and how their PPI strategies should be evaluated as appropriate or successful. In contrast, some case studies from the IDD topic area clearly presented a strongly coherent moral stance on PPI. However, as well as seeing PPI as a given, the approach taken by researchers from these case studies was an open one, actively seeking to explore ways of capturing the impact of PPI on their research outcomes.

FIGURE 9.

‘Virtuous cycle’ of PPI triggered by policy.

FIGURE 10.

Pathways triggered by moral value set.

That values should have such a profound impact on shaping PPI processes is unsurprising and will be discussed further in other sections. A project running at the same time as the RAPPORT study explored values in more depth and suggested three value sets: normative (which we have termed ‘moral’), substantive (which we have termed ‘methodological’) and process. 25,175 Its process value set included concepts such as partnership and equality, respect, trust and honesty, which we have mostly subsumed under our ‘moral’, ‘methodological’ and ‘policy’ value sets. However, whichever labels are used to define value sets, it is clear that different value sets are seen to lead to diverse constructions of PPI,28,174 and can affect whether or not and how PPI is evaluated. 52

It was specifically notable that values could be seen to shape the way PPI was organised within funding bodies, with impact on how they evaluated PPI within their funded projects. The centrally funded programme management organisations placed a clear focus on developing ways of capturing the methodological impact of PPI, in contrast to the condition-specific charities, which followed a clear moral imperative to involve the people who put coins into the tin. The influence of values on funders was further highlighted by the approach of the large national funding body funded through endowments, for which the obligation was one of engagement rather than any moral imperative to involve. An earlier study on funders also suggested that charities and centrally funded organisations adopted different approaches and rationales for PPI. 176 It should also be noted that O’Donnell and Entwistle176 and van Bekkum and Hilton177 suggest that funders had many anxieties about PPI such as its ‘distorting’ the research agenda. This was not reflected in the RAPPORT findings on current views of funder organisations; the main concern from our data was the potential to overload lay representatives with reviews.

The metaphor of an ‘industrialised’ scale of research suggested by our data echoes the similar argument of Ward et al. 39 that the trend towards consumerism has shaped the commodification of research. Although we had no evidence of any of our lay respondents ‘shopping around’ for research projects to be involved in, it could also be seen that one of the condition-specific charities had framed the end-user of the research as being the consumer, and consequently including both patient and clinician as their ‘user representatives’. Kreindler27 also argues that the consumer paradigm is predicated on consumer satisfaction, so that methods for implementing PPI will centre on feedback. Our case studies offered examples of PPI approaches adopting such methods, such as focus groups.

Dynamic framework for patient and public involvement

The RAPPORT data supported a typology of PPI in research based on:

• who the PPI representatives were

• what the forums for the PPI representative–researcher interface were

• the approaches used.

This typology is not dissimilar to the conceptual framework suggested by Oliver et al. 132 However, the RAPPORT data suggested a far more dynamic framework (Figure 11) than that of Oliver et al. ,132 recognising subtle changes over time. Within this more dynamic framework, the PPI approaches, forums and PPI representatives may be seen to change during the research process, or several approaches and forums may run throughout the life of a study. In contrast to previous observations18 that PPI conceptualisations have not addressed the dynamic nature of PPI within research, our analysis, and the framework that it underpins, is able to take into account a dynamic concept of PPI within a research study.

FIGURE 11.

Dynamic framework of PPI.

Value sets greatly influence PPI processes within a study; and relatedly, in turn, the model of PPI in each study was seen to be influenced by the CI’s desired outcomes from the PPI input. 18 For example, one case study (CS01) recorded only a one-off narrowly defined PPI input, which reflected the CI’s reservations about any benefits of PPI.

Patient and public involvement within a study was also partially shaped by the lay recruitment method. Varying the methods of PPI recruitment was one means by which researchers tried to address the issue of PPI representativeness, an often-questioned issue. The quest to find a person (or a group) with characteristics as close as possible to a target study population173 was particularly challenging for research teams in public health studies, where the target population would not necessarily see themselves as having a health condition or need. Interestingly, those of our case studies which involved marginalised groups, such as those with learning disabilities and IDDs, did not have particular problems in recruiting PPI representatives thanks to the existing infrastructure for PPI developed in these areas. As argued by Barnes et al. ,178 representativeness in providing perspectives was achieved by merit of the lay representative’s personal experience, particular characteristics (e.g. being an older person) or links with other networks (e.g. representatives from local or national charities). However, what particularly vexed some researchers was the perceived repeated recruitment of the same group of lay people, rather than of those able to bring a fresh outsider perspective (discussed in Impact of involvement on patient and public representatives).

Our data also suggested that existing groups or panels were a pragmatic source of recruitment for many researchers, and had benefits such as PPI members having some basic knowledge about the research process, providing more than one voice and having support mechanisms in place. Many of these groups were also able to reimburse lay members for work in developing grant applications, a particularly challenging issue raised by many17 that could then be addressed. However, the PPI costs required in a grant application proved a surprise to a minority of researchers, suggesting that further work needs to be done in helping researchers understand the resource implications of supporting PPI. How often PPI representatives were involved in successive research stages in our case studies could be presented as a decreasing double bell shape, with activity focused on project set-up and to a lesser extent the dissemination phase.

The forums in which PPI representatives and researchers came together provided similarly contrasting data. The most common forum in our case studies was some form of study management committee. In many of these committees lay representatives felt well supported and that their voice was heard, often primarily enabled by relational work (discussed in Contextual enablers for patient and public involvement: relationships). However, in some committees, other PPI representatives reported feeling lost by language, procedures and lack of support, and unable to contribute effectively to these committees.

Scott179 suggests a number of rules and norms that govern such forums:

• regulative – the system of rules that shape and monitor actions

• normative – the rules for appropriate behaviour in that particular forum

• cultural-cognitive – the shared meanings that lead to unwritten agendas of ‘how we do things here’.

Regulative rules may be written and formalised, for example in research governance frameworks, but may not be understandable to lay representatives even if documents are provided. However, it is the other two sets of norms that can cause most problems, as they may be unwritten and therefore not visible to a PPI representative who is working with a research team for the first time, or is new to the work of study management committees. Barnes et al. 178 argue that these rules and norms have specific effects on such forums. First, proscribing groups with which studies engage may affect access to such committees. So, for example, certain lay representatives such as those with poor literacy or lacking access to a computer may not be invited to be part of the committee. Second, rules and norms can define agenda setting, including which items are included and who can take part in the discussion. Some of our data suggested that some lay representatives felt unable to contribute to any item on the agenda, and certainly had no say in adding any agenda item. Finally, the normative and cultural-cognitive rules and norms guide the way the deliberation proceeds and is conducted. So, for example, some lay representatives sitting on committees alongside eminent scientists followed unspoken rules that the scientists be allowed more talk-time than other members of the committee. Effective chairing could often ameliorate this, and several PPI respondents articulated the role of the chair in moving key PPI contributions beyond only tokenistic membership of a committee.

Models of patient and public involvement

While we found different forums, approaches and people involved in a dynamic framework of PPI, and traditional models of PPI with two or more representatives fully involved as part of a steering committee, our findings also suggested three specific models of PPI which resulted in different levels of PPI impact.

The one-off model of patient and public involvement

We found evidence of a model with one-off, limited PPI input into the study with boundaries set by the CI (Figure 12). While such involvement did achieve its purpose from the CI’s perspective, it also precluded any further potential positive impact of PPI.

FIGURE 12.

Model 1: the ‘one-off’ PPI.

As this model was indicative of a CI who had reservations about PPI, it was unsurprising that we had few case studies that exemplified this model, as we would expect that those agreeing to take part in the RAPPORT study would be more engaged with PPI. Therefore, it is likely that this model may be rather more common in the wider research community and so merits further exploration. It is also an interesting model, as our analysis suggests that an unintended outcome was commodification, whereby PPI was ‘bought’ by the CI for a specific task.

We can draw on Habermas’s theory of communicative action180,181 to illuminate this further. Habermas suggests that the modern capitalist society comprises a ‘lifeworld’ where public and private experiences and values are culturally and socially integrated, sustained and reproduced. Here, for example, we could see how people living with dementia exist in a ‘lifeworld’ where experiences can be shaped by stigmatising processes182 that are sustained and reproduced. One rationale for PPI in dementia research is therefore to be able to gain understandings of these experiences from the perspective of service user and carer, and how these experiences inform research priorities and design.

Alongside the ‘lifeworld’, Habermas argues, is the ‘system’ made up of bureaucracies, markets and policies;129 for example, this would include the bureaucratic and policy processes, and economic drivers of health research. A key mechanism within the ‘lifeworld’ is communicative action, a consensual process whereby people understand and share experiences. Within the RAPPORT study we can see communicative action within one particular case study with families of children with disabilities (CS21), where they shared experiences which were translated into research priorities. In contrast to the ‘lifeworld’, the ‘system’ is sustained by strategic action, which may be conscious or unconscious. Therefore, the requirement for PPI as a condition for research funding could be seen as a conscious strategic action. However, the tacit shaping of PPI through a process of commodification would, through the Habermas perspective, be seen as the colonisation of the ‘lifeworld’ by the bureaucratic and market interests of the scientific-technical system,183,184 which will then influence and rationalise the ‘lifeworld’. In the case study with more limited PPI, PPI was bought in by the CI as a commodity to demonstrate that those representing the target population would find the accessing of medical notes acceptable. Because there was no sustained engagement with the PPI representatives, we were unable to interview any of them, but our interviews with other PPI representatives suggested it would be unlikely that this one-off activity could lead to engagement with the research study, or therefore any connection with PPI representatives’ ‘lifeworld’.

The fully intertwined model of patient and public involvement

Seven of our case studies (CS03, CS04, CS07, CS09, CS13, CS17 and CS21) demonstrated a fully entwined model (Figure 13), in which PPI was integral and ongoing within the research study. This model demanded more resourcing, including a discrete co-ordination role, reimbursement of larger numbers of PPI representatives, ongoing and proactive communication and building and sustaining relationships between the research team, centre or host organisation and lay representatives. However, this model produced greater impact than the one-off model and, without exception, had the fullest NPT radar plots, indicating that PPI was embedded as normal practice.

FIGURE 13.

Model 2: fully intertwined PPI.

This model was particularly exemplified in two case studies (CS13 and CS21), which had the most embedded PPI as shown in the radar plots (see Chapter 6). The context for both was research being conducted in a well-resourced research centre, jointly or totally hosted by a university. However, they differed in research design, topic area (IDD and public health) and, hence, their tradition and history of PPI. They also differed in PPI approaches, with CS21 tending more to user-led research in that the research question had been identified by service users, but the actual research itself was conducted in partnership with researchers. In CS13, the approach was characterised by a collaborative approach, with the panel an integral part of the research process but as collaborators, not co-researchers. The key enablers for this model appeared to be the ‘work’ described in all four NPT constructs and, in particular, values, PPI co-ordination role and relationships. Both studies also had time, core funding and stability to develop a fully entwined model of involvement, where PPI was integral to the research process. Therefore, this model was not possible for all our case studies even when actively sought, because of limited resources.

The outreach model of patient and public involvement

A third model was identified, which we termed the outreach model. This still entailed entwinement but was looser and generally involved fewer PPI representatives: just one in some cases. The key feature of this model was that this small number of PPI representatives provided a conduit to and from a wider population from which they would bring back perspectives and views (Figure 14).

FIGURE 14.

Model 3: outreach PPI.

Recent literature has also promoted this model if the PPI representative is well connected to and has credibility with the target population. 185 Although this appeared to be an effective model which required fewer resources, the resource implications are from the research team’s perspective and we had evidence that this model increasingly relied on charities to perform the outreach function. This escalation of demand on charities without associated funding to support this work has significant implications for sustainability, and is discussed further later. Van Bekkum and Hilton’s study on funding bodies177 also suggests that this model is being adopted by some funders, seeking representatives who could act as a ‘middle-man’ between the worlds of research and the public. While CS02 demonstrated a collaborative approach, it also showed the potential for tokenism in this model if the individual is operating just to provide endorsement of the study by the target population.

Contextual enablers: organisation of patient and public involvement

We wished to test out in our case studies and survey whether or not the principles identified for successful PPI,137 or architecture for PPI in research,29 would be found to be prerequisites for effective and embedded PPI. The PIRICOM study architecture for PPI included the principles gained through consensus by Telford et al. ,133,137 grouped here under three headings: processes (PPI activities/research interface), resources and relationships (see Figure 15).

As reported in Chapter 6, having a clear purpose, role and structure for PPI was a salient action undertaken by case studies with embedded PPI. As illustrated in Figure 15, we found that adequate resources and the development of relationships over time were important enabling contexts, and these will be discussed in Contextual enablers for patient and public involvement: resources and Contextual enablers for patient and public involvement: relationships. However, what is not so explicit within the Brett et al. 29 architecture is the need for someone to lead, and in particular co-ordinate PPI.

FIGURE 15.

Architecture for PPI (adapted from Brett et al. 131).

The co-ordinator role

A major feature of the case studies where PPI was strongly embedded was the presence of a person with designated responsibility for co-ordinating and facilitating PPI within the research study. In some case studies this was undertaken by a paid co-ordinator with general responsibility for a research centre’s programme of work, for a UKCRN network or for a public involvement group serving a geographical area. In other case studies this role was undertaken by the CI or assigned to another researcher in the team. For the paid co-ordinators and assigned researchers, the role comprised various components (Figure 16). Some of these components were also evident in the work of the CIs and the funding bodies’ PPI leads, and all could be described as providing a bridge between two worlds: research and lay. This idea of bridging fits well with the social network theory-derived definition of ‘boundary spanner’161 (see Chapter 6). Further exploration of the specific features of this bridging role helps tease out the different approaches of these co-ordinators.

FIGURE 16.

Components of PPI co-ordination work.

Clavier et al. 186 propose three aspects of bridging: cognitive translation, strategic translation and logistic practices (Table 19).

TABLE 19 - Three aspects to the bridging role

Source: adapted from Clavier et al. (2012). 186

Aspect	Actions	RAPPORT findings
Cognitive translation	Bringing insights in from the public and translating them into research questions	Case studies 21 and 23 had forums specifically designed to do this, with members of the public facilitated in turning needs into research questions
Strategic translation	Circulating and translating information derived from research and public perspectives. Establishing committees and procedures. Clarification of expectations	Case study 17 PPI co-ordination was facilitated by the CI. Panels and structures plus effective lines of communication were a feature (see also CS13)
Logistic practices	The practical work and time required to enable both translations	Well-resourced case studies had discrete administrative support for PPI, or researchers would take the time and make the effort to carry out these tasks

One of our case studies (CS21) exemplified cognitive translation by bringing the public and service users into the research centre and helping them translate their perceived needs into research questions. Strategic translation was often undertaken by CIs who facilitated PPI within their study. They emphasised ensuring good communication and feedback, and setting up structures for PPI within their study. For both aspects of translation to work well, various logistical practices were required to ensure that PPI ran smoothly, usually in the administrative tasks such as payment of expenses. In some case studies the dedicated co-ordinator also carried out these tasks; in others this work had to be absorbed by the research team.

Our interviews evidenced respondents carrying out the PPI co-ordination role as displaying certain traits reflecting Williams’s boundary spanners’ skills requirements187 (Figure 17), focused on managing complexities through negotiation and building sustainable relationships. The nature, development and facilitation of such relationships will be discussed more in Contextual enablers for patient and public involvement: relationships, but our findings made clear that effective PPI co-ordination, whether within research teams or funding bodies, depended heavily on the ability to build and sustain relations.

FIGURE 17.

Skills for PPI co-ordination (adapted from Williams187).

However, to see PPI co-ordination simply as a bridge between two worlds disguises its complexity. As illustrated in Figure 11, within each case study, PPI was not static and boundaries between the research and lay world were dynamic, constantly shifting. For example, over time, PPI representatives may become less research naive and the nature of relationships and the boundary dividing the research and lay worlds may subtly shift. Equally, researchers’ assumptions regarding PPI may change with time and again affect the nature and permeability of the boundary. The evaluation of knowledge translation processes within CLAHRCs by Evans and Scarborough188 suggested that the bridging model described above may keep the two worlds apart. They argue that a ‘blurring’ approach is more effective in creating transformation. Rather than rely on a single PPI co-ordinator to perform the boundary-spanning role, the blurring approach requires both researchers and PPI representatives to adapt the skills they have to accommodate new ways of working. This theory may partly explain why in the case studies with minimal PPI co-ordination there could still be positive outcomes from PPI, and why in one case study with formal PPI co-ordination there was minimal PPI impact: the worlds of research and lay had remained separated by a boundary that only the co-ordinator could cross.

Overall, the co-ordination role facilitated effective PPI, including work to clarify PPI role descriptions and expectations.

Contextual enablers for patient and public involvement: resources

As seen in the findings, some of our case studies did include PPI during the grant development phase, but most exhibited an intense ‘PPI phase’, usually when developing ethics applications, and to a lesser extent when findings including the development of a lay summary were disseminated. Surprisingly, and in contrast to best practice,189 whereas all case studies reimbursed PPI representatives for their out-of-pocket expenses, only around a quarter of case studies reimbursed for time. As discussed earlier, there was evidence that this was becoming a very contentious issue, particularly for PPI representatives from relatively small charities. There were five such representatives. One was a representative from the charity partially funding the study so did not expect any remuneration. The second was being paid as a part-time researcher, and a third took an altruistic stance, giving time free in the interests of those she represented. However, two respondents who ran national organisations expressed doubts that it was sustainable for them to undertake this PPI work with no recompense, as their absence had implications for the work of the charity. They also called into question the idea that it was morally defensible to expect them to undertake an increasing workload of PPI requests without some significant financial reimbursement. There was evidence that both these charities were putting in place consultancy procedures and costs that would need to be met in future grant applications.

Interestingly, PPI respondents also rarely mentioned the need for training. Although some had received appropriate training, many lay representatives did not feel they required any particular training. However, there was some evidence that researchers (particularly those relatively new to research) did want more training and preparation for PPI.

The strongest theme to emerge from our data was the centrality of relationships to PPI. Whereas previous work has highlighted the importance of the researcher–lay relationship, the RAPPORT findings illuminate this further by teasing out the components that result in relationships being one of the most important enabling contexts in leading to positive PPI outcomes.

Contextual enablers for patient and public involvement: relationships

Truman and Raine190 suggested that ‘the creation of evidence in health research is shaped by the social relations of the research process as well as by the methodologies used’ (p. 215). This was very evident in the RAPPORT case studies, with several clearly recognising the importance of emotional as well as rational deliberation (cf. Martin 2011191). There were four components in enabling relationships: time, reciprocity, arenas and skills.

Skills for enabling relationships

Whatever arena within which relationships developed, there were discrete skills and traits needed by those facilitating relationships. Others have highlighted the challenge of encouraging and maintaining relationships in PPI,193 and the need for training to do this. 29,101

As discussed earlier, knowledge about boundary spanners helps illuminate the skills and traits that enable this work. Williams187 describes four aspects to building sustainable relationships (Figure 18), and these skills and traits were described within our data. Some of these aspects are amenable to development through training; others may be seen as personality traits. The RAPPORT data gave a sense of the PPI co-ordinator or person leading on PPI being that kind of person: approachable, sensitive, tolerant and open.

FIGURE 18.

Skills and traits for managing relationships in PPI (adapted from Williams187).

Impact of patient and public involvement

There is a drive towards public involvement and engagement in a number of sectors beyond health research. Impact of public or community engagement in these sectors may shed light on the PPI impact we may expect in health research.

Community engagement in housing through, for example, tenant management organisations has been cited as an approach leading to improved processes and outcomes in social housing. 199 A systematic review200 on the impact of community engagement on health and social outcomes found that community engagement helped with planning and management of housing services, enabled partnership working and better communication between stakeholders and provided a conduit to a broader community. Although there was evidence of impact on members of the public, such as development of skills, conclusive evidence of the impact of community engagement on service delivery could not be found. The authors suggest this is because of the difficulty in attributing improvements to PPI, as so many factors are involved.

There is also a broad body of work on public participation and involvement in ecological and conservation work. 201 Desired longer-term outcomes for PPI in this arena include sustained conservation202 and empowerment for the public involved. 203 However, as in social housing, longer-term impacts of PPI are rarely measured and noticeable long-term outcomes may occur only 10 years or more after the project has been set up. 201 Nevertheless, evaluation of PPI within social housing and conservation suggests that PPI enables positive outcomes in:

• processes including management and governance

• accessing knowledge that would otherwise be unavailable

• improved partnership working, communication and broader links with the community.

Impact and outcomes of patient and public involvement in the RAPPORT study

The most commonly reported outcome of PPI was in improving recruitment rates, particularly through refining participant information literature, and also through providing links with participant populations. However, there was no evidence on how this affected research outcomes. This effect of PPI on study recruitment has been reported widely elsewhere,70,78–86,88 but the range of case studies within the RAPPORT study allows the specific components to be teased out (Figure 19).

FIGURE 19.

Patient and public involvement processes and outcomes in study recruitment.

Five out of the 22 case studies had PPI representatives involved in setting research priorities, and again this has been previously demonstrated (see Chapter 2). Impact of PPI on this outcome was particularly facilitated by formal structures which ensured that lay representatives were part of the decision-making process and could influence the final decision.

Whereas the majority of outcomes from PPI for the research studies are reported previously, one notable exception was an unexpected outcome in one case study where both researchers and PPI respondent reported an impact on participant safety. Although this could be subsumed under PPI impact on the overall research design, it is particularly interesting that the lay representative, by being able to put herself in the shoes of the participants, could identify potential safety issues not seen by the research team. It is also significant that this outcome was unexpected by the research team, and carried implications for research teams who set strict boundaries on what outcomes PPI representatives may affect. It also raises some questions about current developments in building evaluative frameworks for PPI, which will be discussed in Evaluating patient and public involvement in research studies.

It should be noted that, while the RAPPORT studies did provide evidence of PPI impact on research design, the research stakeholder (including PPI) consensus study by Barber et al. 159 concluded that it was not possible to evaluate PPI impact in this area. The positive impact of the fully intertwined model on the research process was clearly observable, and ranged from identifying research questions to disseminating research findings; however, this was not exclusive to this model. Some of the case studies typifying a more traditional approach to PPI, such as two or more representatives involved in a project steering committee, also demonstrated a positive impact on research processes. Rather than a specific model being the enabler, it was the six salient actions as described in the CMO tables (see Chapter 6) that were key. However, it is significant that all the case studies demonstrating an intertwined model achieved the salient actions and exemplified fully embedded PPI.

Nevertheless, as with other sectors it was difficult to tease out the impact of PPI on the research results. This is unsurprising given the difficulties in distinguishing the impact of PPI from other factors, but, as recognised in the Higher Education Funding Council for England Research Excellence Framework, impact is often not immediate and can take 15 years to become visible. 204 We are therefore not able to determine whether or not the resources required for the fully intertwined model are merited in terms of long-term impact on research findings. However, our findings suggest that the fully intertwined model is likely to lead to increased relevance of research questions, and hence may have greater long-term outcomes in terms of research orientated towards patient priorities.

This issue also illuminates the ongoing debate on what constitutes PPI success. 205 Attaining seemingly small PPI impacts on research processes may mark important steps in achieving greater change within the complex system of health research. 205 However, there is consensus that the impact of involvement on PPI representatives themselves is easier to evaluate. 159

Impact of involvement on patient and public representatives

As reported widely elsewhere (see Chapter 2), involvement had a significant impact on many of our PPI respondents in terms of psychological outcomes and knowledge (see Chapter 6). It was also clear that, for some, involvement in a single study had been the start of a PPI trajectory that moved as the individual’s knowledge, confidence and interest grew. Some of the PPI co-ordinators had been patient representatives, and the PPI respondents we interviewed from funding programmes also described a journey from involvement at study level moving towards a more strategic level.

This PPI career trajectory is a hotly debated subject. Many of our researcher respondents expressed concerns about the same PPI representatives being repeatedly involved and the need to ‘refresh’, a euphemism generally denoting removing PPI incumbents and recruiting new ones. As raised by others,206 for the PPI representatives, the challenge was to remain authentically credible after having developed enough research knowledge to reduce some demands on the research team’s time. The ‘ideal’ PPI representative from many researchers’ perspectives would be at the stage indicated by the star in Figure 20.

FIGURE 20.

Credibility versus understanding research.

Making judgements about which lay representatives were credible and competent involved shaping processes and practice for PPI207 to exclude those moving beyond the star. So the argument ricochets between training and ‘professionalisation’ of PPI resulting in the loss of the lay perspective22,208 and also to counter the moral argument for PPI,208 and the need for some training209,210 to carry out certain tasks required in the PPI role. Although this was not articulated by any of our PPI respondents, in the ethnographic study conducted by Thompson et al. 46 they describe the balancing act carried out by the PPI respondents to have enough knowledge to be able to contribute within a trial steering committee, but also to remain credible.

It was clear in those RAPPORT case studies with strongly embedded PPI that developing knowledge and skills in research was not seen as detracting from credibility as an authentic PPI representative. Indeed, in many of these case studies the involvement trajectory was enabled and PPI representatives were encouraged to take on more strategic roles or act as co-researchers or coapplicants. This fits well with notions of equality, for it is unlikely that any researcher would in such cases be ‘refreshed’ because they had become too skilled.

Evaluating patient and public involvement in research studies

As indicated in the findings chapter, the weakest NPT construct in our analysis of case studies was appraisal, in which the PPI is evaluated and work is reconfigured in the light of this evaluation. There were few case studies which provided any evidence of systematic evaluation (see CMO Table 17) and these demonstrated very different approaches influenced by their research design. One RCT was able to demonstrate that rises in recruitment rates could be linked to participant information amended by PPI representatives. This was possible because of the duration of recruitment. However, in another case study, which was a systematic review, linking PPI to recruitment rates would not be relevant. In this study all PPI activities, input and outcomes were carefully recorded to provide a ‘storyline’ of impact.

The findings suggested that, in the case studies where PPI was strongly embedded, the research teams were already intuitively following evaluative processes suggested by the Public Involvement Impact Assessment Framework (PiiAF) project with values identification and impact theory. 175 Their value sets and understanding of PPI were coherent, and they were evaluating PPI impact appropriate for their study design. However, for many other case studies such systematic evaluation would be difficult because coherence levels were lower. In addition, many of these studies were working to short, intensive time scales, so would be unlikely to enter into the group thinking that the PiiAF project suggests lays the foundation for impact evaluation. However, outside the research study, it is likely that the value clarification exercises suggested by PiiAF would contribute much to the training and preparation for PPI requested by many of the less experienced researcher respondents. A point of note is that any evaluative framework must be flexible enough to capture any unforeseen outcomes; simply measuring whether or not expected outcomes had been met would not provide a full evaluation.

A key challenge for some of our researcher respondents (mainly clinical) was that their research approaches were clearly placed within the positivist paradigm, so they found it very difficult to articulate how they would evaluate PPI by preserving their focus on measurable and observable changes.

It should be noted here that some case studies were recruited to the RAPPORT study because they wanted an external evaluation of their PPI. Having regular tracking interviews revealed the complex temporal dynamics of PPI in some case studies; the challenge for these researchers is now to provide published evidence of the impact of PPI in as robust a form as in reporting their RCT findings. 21

Reflection, appraisal and evaluation

Normalisation Process Theory statements

1. Participants distinguish the intervention from current ways of working.

2. Participants collectively agree about the purpose of the intervention.

3. Participants individually understand what the intervention requires of them.

4. Participants construct potential value of the intervention for their work.

5. Key individuals drive the intervention forward.

6. Participants agree that the intervention should be part of their work.

7. Participants buy in to the intervention.

8. Participants continue to support the intervention.

9. Participants perform the tasks required by the intervention.

10. Participants maintain their trust in each other’s work and expertise through the intervention.

11. The work of the intervention is appropriately allocated to participants.

12. The intervention is adequately supported by its host organisation.

13. Participants access information about the effects of the intervention.

14. Participants collectively assess the intervention as worthwhile.

15. Participants individually assess the intervention as worthwhile.

16. Participants modify their work in response to their appraisal of the intervention.

Normalisation Process Theory statements

1. Participants distinguish the intervention from current ways of working.

2. Participants collectively agree about the purpose of the intervention.

3. Participants individually understand what the intervention requires of them.

4. Participants construct potential value of the intervention for their work.

5. Key individuals drive the intervention forward.

6. Participants agree that the intervention should be part of their work.

7. Participants buy in to the intervention.

8. Participants continue to support the intervention.

9. Participants perform the tasks required by the intervention.

10. Participants maintain their trust in each other’s work and expertise through the intervention.

11. The work of the intervention is appropriately allocated to participants.

12. The intervention is adequately supported by its host organisation.

13. Participants access information about the effects of the intervention.

14. Participants collectively assess the intervention as worthwhile.

15. Participants individually assess the intervention as worthwhile.

16. Participants modify their work in response to their appraisal of the intervention.

Normalisation Process Theory statements

1. Participants distinguish the intervention from current ways of working.

2. Participants collectively agree about the purpose of the intervention.

3. Participants individually understand what the intervention requires of them.

4. Participants construct potential value of the intervention for their work.

5. Key individuals drive the intervention forward.

6. Participants agree that the intervention should be part of their work.

7. Participants buy in to the intervention.

8. Participants continue to support the intervention.

9. Participants perform the tasks required by the intervention.

10. Participants maintain their trust in each other’s work and expertise through the intervention.

11. The work of the intervention is appropriately allocated to participants.

12. The intervention is adequately supported by its host organisation.

13. Participants access information about the effects of the intervention.

14. Participants collectively assess the intervention as worthwhile.

15. Participants individually assess the intervention as worthwhile.

16. Participants modify their work in response to their appraisal of the intervention .

London dissemination event

North East dissemination event

South West dissemination event

East of England dissemination event

Table 1: Top Priority: Ownership of PPI – it always happens under the radar and someone should take ownership – should this be Healthwatch?

Finance – lead up to grants (no resources), visible hubs (e.g. PPI groups to spread message), pathways for involvement and engagement.

Table 2: Top Priority: Making PPI standard across all research topic areas and healthcare areas – standardisation but at a local level.

Health research authority needs to make IRAS form more robust to make CI think about PPI throughout the research cycle.

Table 3: Top Priority: Educating researchers – researchers to listen to understand patient representatives – get them young when at university – get them to understand PPI right from the start. For more established researchers help them to understand barriers and enables of effective PPI.

Involving younger people more and understanding what incentivises young people.

Table 4: Top Priority: Overall marketing strategy needed, very difficult to get an understanding of who and what are involved, normalising PPI.

Who is mentoring lay representatives etc. succession planning etc.

Make sure we get the views of younger people and the older person.

Campaigns like the Australian one for sun protection to increase research participation.

Employment status of people in PPI – and their performance management.

Table 5: Top Priority: Feedback to 1. lay representatives and 2. general public re: possibilities of PPI. Start at school level and people must feel comfortable in challenging.

Involve lay people right from the beginning – need a balance of new and experienced lay people.

NIHR should have more of a monitoring role in PPI.

Publicity re: research at primary care level especially in GP surgeries.

Table 6: Education – start in schools e.g. health literacy and introduce partnership working.

Joint training, service users and researchers.

Training on innovative approaches for researchers.

Continue to build evidence base on PPI.

Co-ordination of different organisations involved in PPI, reduce confusion.

PPI has become an industry and is at risk of becoming introverted.

Increase diversity; Increase embedding of PPI.

Table 7: Evaluation of impact (e.g. Til Wykes) and qualitative research approaches.

Cost-effectiveness – public investment so need to be at least cost neutral.

Don’t lose good practice through transition to new structures.

One-stop shop to hear of local innovations and get out nationally.

Raising awareness – can’t do anything if people aren’t aware.

Table 8: Involvement should be at different levels so public, carers, service users, etc. should be able to be involved at what level they’re comfortable at.

Skill mix – new PPI reps and experienced PPI group meetings.

Grant allocation should be available and reflect the model of PPI; training for researchers.

Table 9: Top Priority: Annual audit (as per MRC [Medical Research Council]) of PPI, this will need costing. Data analysis @ NIHR.

Strategic level – people having PPI activities audited and include lay representatives.

More realistic costings of PPI.