An online supported self-management toolkit for relatives of people with psychosis or bipolar experiences: the IMPART multiple case study

Project Management Group

The initial Project Management Group (PMG) consisted of the grant holders who were responsible for developing the study ideas and delivering the project. This group included a relative who had spend many years supporting a family member with a diagnosis of schizophrenia (SF); two clinical academic psychologists (FL and SHJ) based in Lancaster; a clinical academic psychiatrist with a lead role nationally in EIP services (SJ) and a clinical academic GP who is director of an eHealth centre at UCL (EM); a trial statistician (CJS); a consultant clinical psychologist with a lead role in development and roll-out of EIP services (JS); the director of the McPin Foundation, which promotes service user involvement in research (VP); an internationally acclaimed expert in implementation theory and practice (JR-M); and a clinical lead for EIP teams in a local NHS trust (RS). The group met monthly via teleconference.

Additional members of the operational team based mainly in Lancaster evolved to include a lecturer in mental health with a special interest in implementation (and who joined the PMG) (NRF); two carer researchers; an IT developer (AW); five research associates over the lifetime of the project (VA, BG, EL, CM and PO); an individual who provided administrative support (BM); and two trainee clinical academics in psychiatry (GA-A) and clinical psychology (JB). They all met weekly in relation to specific tasks throughout the project and attended the PMG as required. Research associates based at the London site joined this operational meeting via teleconference.

Study Steering Group

A Study Steering Group (SSG) was appointed to oversee the project on behalf of the project sponsor (Lancaster University) and project funder (NIHR) and to ensure that the project was conducted to the rigorous standards set out in the Department of Health and Social Care’s Research Governance Framework for Health and Social Care70 and the Guidelines for Good Clinical Practice.

The group was chaired by a clinical academic and expert in the delivery of mental health interventions in the NHS (Professor David Kingdon) and included a service user or relative (this role was occupied by three people throughout the course of the study), a senior clinical academic with relevant methodological expertise in health service research, a senior NHS manager in EIP services and a representative of Lancaster University as sponsoring organisation. The SSG met before the start of the study, at the end of the pilot phase and then annually until the end of the study. All meetings took place via teleconference.

Phase 1: hypothesis generation

In phase 1, we outlined an implementation theory identifying the factors that we hypothesised would influence successful implementation of REACT and how they would lead to successful outcomes. Our study theory was informed by NPT, with specific hypotheses further informed by:

• a systematic review of relevant implementation studies of DHIs for people with psychosis or bipolar disorder and/or their relatives to identify factors affecting implementation81

• qualitative analysis of data relevant to implementation from the REACT feasibility trial,53 including interviews with EIP staff who have worked as REACT supporters and relatives who used REACT

• stakeholder workshops including staff and relatives at each participating trust

• synthesis of these data informed by our clinical and theoretical expertise in this area.

Phase 2: hypothesis testing

In phase 2, we used a case study design to test our hypotheses about which factors would influence the implementation of REACT. We developed an implementation plan to address these factors, and made it available in successively more developed forms across three waves.

Phase 2 was conducted across two geographical regions (North of England and South of England), with three NHS trusts participating in each. All six trusts were given the REACT toolkit and initial implementation plan [implementation plan version (IPv) 1] at the start of phase 2. The implementation plan included an online manual of detailed instructions outlining roles and responsibilities for key staff implementing REACT and face-to-face training sessions within each trust. Guidance for relatives on using REACT is embedded within the toolkit.

We collected detailed case study data in two of our participating trusts (one in each region) as wave 1. Key factors affecting implementation were identified and then discussed with stakeholders in the two trusts in wave 2. The research team worked collaboratively with these stakeholders to design IPv2 in these trusts. Further data were collected to test the impact of this and to identify additional factors affecting implementation. A third iteration (IPv3) was then developed with staff in the final two wave 3 trusts. Data from this wave, combined with ongoing longitudinal data from the four trusts in waves 1 and 2, informed the final version (IPv4).

Phase 2 required a mixed-methods approach, in which we collected quantitative data to describe the levels of implementation and outcomes within each trust, and qualitative data to identify the key factors that could explain these patterns of implementation. Consistent with the case series design, our data were first analysed within each trust before we attempted to analyse and explain similarities and differences between trusts.

The methods are described in detail in Chapter 4, and the design and objectives are summarised in Table 2.

Phase 3: finalising the implementation plan

The aim of phase 3 was to synthesise all of the data across all six trusts to identify the key factors affecting implementation and develop a national implementation plan for REACT IPv4 (objective 4), but also to draw out the generalisable learning for the delivery of DHIs into NHS services (objective 5). We used local ‘data analysis days’ to engage staff in the analysis of trust-level data; key staff involved in REACT roles across all trusts in integration of findings across trusts; and the whole project team including carer relatives in a final 2-day ‘explanatory synthesis event’ in which we produced the final explanatory synthesis outlined in Chapter 8. We also presented the iterative development of the implementation plans and drew on all of the study data to agree a final version of IPv4.

Normalisation process theory

Normalisation process theory was used to inform the structure of the framework into which our hypotheses were organised, and which then guided our data collection and analysis. The framework was used flexibly and, in phase 2, evolved to include additional constructs. However, in phase 1, we used the 16 core components, organised within the four core constructs of NPT as outlined on the NPT toolkit:89

1. coherence – the sense-making work people do individually and collectively when faced with operationalising a set of practices

2. cognitive participation – the relational work that people do to build and sustain a community of practice around a new technology or intervention

3. collective action – the operational work people do to enact a set of practices

4. reflexive monitoring – the appraisal work people do to assess and understand how the new intervention affects them and others.

We used the online toolkit to train the whole team to ensure that everyone had a good understanding of the theory. Although our team included internationally recognised experts in the field of implementation science (notably JR-M and EM), it was important that all team members, including carer researchers, and other methodological and clinical experts also had a thorough working knowledge of the theory so that everyone could contribute to the data analysis.

We then used this shared understanding to map the findings from our literature review, secondary analysis of REACT feasibility study data and SRG meetings into the framework. This helped to develop the generic mid-range NPT components into study-specific hypotheses for the IMPART study. The process for this is presented in Appendix 4, Tables 27–31.

As an example, the NPT component ‘relational integration’ in the construct ‘collective action’ hypothesises that an intervention is more likely to succeed if staff maintain their trust in each other’s work and expertise throughout the intervention. Data from all three studies in phase 1 contributed to making this a study-specific hypothesis in REACT:

REACT is more likely to be successfully implemented (more relatives are invited to use it) if REACT is offered as part of an integrated package of care for relatives, and not a stand-alone intervention; access is restricted to relatives already supported within the team; staff can see and are confident that the forum is well managed; staff can see and are confident that direct messages are being responded to; REACT supporters and IMPART leads are clear and confident about managing risk identified in REACT.

Systematic literature review

We conducted a systematic review of studies published between January 1995 and October 2017 that identified factors affecting implementation of DHIs for people with psychosis or bipolar disorder, or their relatives. 81 In this chapter, we focus on the key factors relevant to understanding the work undertaken by staff to implement DHIs, and the uptake and use by service users or relatives. The review identified 26 eligible papers describing DHIs for service users, but none aimed at supporting relatives. This further highlighted the need for the IMPART study.

The majority of factors for effective implementation of DHIs were focused on the characteristics of staff supporting the DHIs, or the design of the intervention. The first key finding was that users were more likely to engage with a DHI if it was facilitated by a staff member or peer supporter, and had been proposed by a staff member who could understand and clearly articulate its benefits. Enthusiastic staff can serve as champions for DHIs, reinforcing engagement among service users and providing practical guidance. They can also mitigate the experience of DHIs as generic and impersonal. The importance of human support in successful implementation was a strong theme across several studies, and is consistent with the findings of other systematic reviews. 90

Engagement among service users was facilitated by the DHIs’ accessibility (independently, in their own time and in their own home) and the ability to share information with friends and family. Some felt that the DHIs made communication with staff members easier on issues such as drug and alcohol use, which they would have found difficult to broach face to face, or relieved the boredom of time spent in inpatient facilities.

Key barriers to engagement for both staff and service users were concerns about cybersecurity, complexity and poor interoperability. Concerns about data privacy or security were associated with disengagement. DHIs that were difficult to use or time-consuming did not succeed, and DHIs that did not integrate with existing IT systems were found to be frustrating, either because they required additional logins or because important information was not being shared between platforms. DHIs that were considered either too complex for users or too simplistic were also less likely to be sustained by staff or service users.

Some service users expressed concerns about the potentially negative impact of a DHI on their mental health, with reports of increases in paranoia or other exacerbations of symptoms. Symptom severity, particularly episodes of acute ill health, and negative symptoms of schizophrenia were barriers to engagement. These factors highlighted the importance of user-centred design in all aspects, and including all users.

Finally, cost was a consistent factor in the review. Although digital interventions are promoted as offering potential long-term savings, insufficient funding in the short term was linked to inadequate staff time, training, space and necessary equipment, all of which negatively affected implementation.

These factors were mapped onto the relevant NPT framework constructs.

Secondary analysis of REACT feasibility study data

The aim of this secondary analysis was to identify factors affecting implementation of REACT (in its paper version) during the REACT feasibility trial (2008–12). 53 This was a stratified RCT in which 103 relatives were allocated to receive either treatment as usual (TAU) or TAU plus the paper-based REACT toolkit intervention, in a ratio of 1 : 1. Compared with TAU only, those receiving the intervention showed reduced distress and perceived increased support and ability to cope at 6-month follow-up.

Participants were relatives, partners or close friends of people experiencing psychosis who were supported by EIP services in three participating trusts. Each participant was offered an initial face-to-face session introducing the toolkit and agreeing arrangements for support, which was offered by e-mail or telephone (relatives’ preference) for up to 1 hour per week over 6 months, with a minimum of monthly contact.

Six REACT supporters working across the EIP teams were trained and offered monthly group supervision for the duration of the project. The first 14 relatives to complete the 6-month follow-up were invited to take part in a qualitative interview about their experiences of REACT. Twelve consented but one data file was corrupted and could not be analysed. Interviews were also conducted with the four supporters still involved in the study at the end.

Our secondary analysis of these data sought to identify specific factors affecting implementation of REACT. Despite the toolkit being mainly offered in paper form during the trial [an online Portable Document Format (PDF) was also available], we still felt that the data had important insights to offer into the process of implementing this self-management toolkit for relatives as part of an existing care pathway. See Appendix 2 for a detailed description of the methods, participants and findings. Appendix 2, Table 25, provides a summary of the main findings.

The key themes to emerge from our analysis were the importance of timing (when REACT is offered); the perceived benefits for the relative; that structure and delivery of the support; and the balancing of the REACT supporter role with other job demands.

Stakeholder reference groups

Stakeholder reference groups met in each participating trust during phase 1 of the IMPART study. The aims were to:

• establish good working relationships with key stakeholders in each trust

• finalise delivery of REACT and IPv1

• explore the views of EIP staff, service users and relatives on factors they thought likely to affect the implementation of REACT in their trust.

Each trust identified an IMPART lead to provide ‘on the ground’ insights into the workings of a particular site and to help researchers to access key data sources. Leads were asked to identify potential SRG members – ideally senior trust board members or EIP service leads, EIP team managers or clinicians, two EIP support workers, two EIP relatives and two EIP service users – and to help set up and co-chair the SRG meeting, held on NHS premises.

All SRG meetings were attended by at least two members of the IMPART research team: a research associate and the North of England or South of England site lead or PPI lead. See Appendix 3 for a detailed report of the methods and results forms including key themes arising from these meetings (see also Appendix 3, Figure 20).

The SRGs were well attended and provided invaluable predictions about factors that would be likely to affect implementation within each trust. However, staff were generally at a clinical and managerial level, with no involvement of senior trust representatives. In some trusts, the IMPART lead chose to hold separate staff and service user/relative meetings owing to fears that meetings would be used to air dissatisfaction with services.

There was a high level of consistency between trusts in terms of the factors considered likely to affect implementation of REACT. These factors included the promotion of its potential to meet the needs of relatives, saving staff time, helping trusts to meet their clinical targets, the flexibility of trusts to adapt REACT to their specific needs, staff confidence in the content and in their role in offering REACT, and the commitment of resources at an organisational level:

We need to make some decisions about the level of support we can give to it and whether or not we can afford to support the instant messaging and the forum, so I guess [we need] to have a clearer idea of what you think the time resource will be.

Staff member

Stakeholders felt that a key strength of REACT was its availability to relatives without requiring service user consent, as this targeted a currently unmet need.

Initial engagement from staff was anticipated to depend on engendering a sense of ownership of REACT within each team, which would be facilitated by ensuring that staff had easy access to the website and training, and by allocating key people to REACT roles. Ongoing use of the intervention by staff would be facilitated by the ability to monitor relatives’ levels of use, and to change and update the content in response to their feedback:

Will you be using any of the things that get spoken about in the forum to kind of tailor the toolkit? So if there are lots of discussions about sleep or diet will you then use that maybe to add to the toolkit for the future?

Staff member

Important barriers that stakeholders felt would need to be addressed before staff and relatives would engage were primarily linked to the REACT group (the online forum) and included confidentiality, security and privacy of data (mainly a concern of relatives); the challenge of managing disclosed risk of harm; and the website being used to air negative feedback (mainly a concern of staff).

The latter was particularly evident in one trust and was clearly linked to experience of an earlier online intervention in which some members of staff had been ‘trolled’. Most stakeholders felt that such risks could be managed by restricting access to the forum to invited relatives, and by having clear ground rules and regular moderation by a REACT supporter.

Despite the credibility of the intervention being clearly enhanced by multidisciplinary input to its development, including from service users/carers and health-care professionals, some concerns were expressed about lack of fit between the content of the toolkit and the ethos of the service. In one trust in particular, REACT was perceived as too medicalising by defining diagnostic terms such as schizophrenia. Staff in this trust wanted more content that focused on well-being and recovery than they felt was currently included:

It just seems to me that a lot of the discussion’s been around lifestyle stuff and families wanting to have more information about well-being in general – I just wonder if this is a very medical model and it’s coming from a very kind of, you know, medical perspective, and actually it needs to open up a little bit more to those kind of life.

Staff member

Challenges were also identified in using REACT to meet the diversity of relatives in terms of language, age, confidence with IT and reading skills. Generally, stakeholders felt that REACT was not suitable for those whose first language was not English, who were older, who were less IT confident or who had a lower reading ability:

[In] my family I’m the only one who speaks English for example, and how would they access this?

Service user

Relatives and staff in all trusts were clear that although they would value the information and peer support offered by REACT, particularly if offered early in their journey, it should be offered as part of an integrated package of care and should not replace face-to-face support:

It’s going to tick the ‘information for relatives and carers’ box, but I don’t want it to be instead of actually getting face-to-face contact and support with care co-ordinators and things like that.

Staff member

Key factors identified by the SRGs as likely to affect implementation of REACT were mapped onto the relevant NPT framework constructs.

Coherence

REACT is more likely to be successfully implemented (i.e. more relatives will be invited to use it) if:

• all staff have easy and independent access to the toolkit in their own time, with clear guidance on what it is, who it is for and what it offers

• staff are given an opportunity to discuss whether or not, and how, to use REACT within their service; decisions from this are visibly endorsed at senior management level

• staff can access training that clearly outlines their roles and responsibilities in delivering REACT

• staff are able to view the content of the REACT toolkit and can see clear benefits for staff and/or relatives, such as –

  • accessibility

  • relevance

  • credibility

  • reassuring

  • non-stigmatising

  • fits with EIP service ethos

  • clear user involvement

  • inclusive

  • safe to use.

Cognitive participation

REACT is more likely to be successfully implemented (i.e. more relatives will be invited to use it) if:

• there is a clear lead (‘champion’) who drives REACT forward within the trust

• staff make explicit links between REACT and existing key trust targets and priorities

• key roles are appropriately allocated

• training is provided for tasks that may require new skills (e.g. forum moderation)

• staff have a sense of ownership and take responsibility for promoting REACT within the service and offering REACT to relatives (and do not see this as the role of the researchers)

• delivery of REACT is included as a regular agenda item at relevant operational meetings within clinical teams.

Collective action

REACT is more likely to be successfully implemented (i.e. more relatives will be invited to use it) if:

• staff carry out their roles and responsibilities as outlined in the ‘how-to’ manual and have the resources to allow them to do this. Key tasks include –

  • IMPART lead creating REACT supporter and clinician accounts

  • all staff inviting relatives (requires access to computer and relatives’ details)

  • REACT supporters regularly moderating the forum (requires regular access to the online forum)

  • REACT supporters providing timely responses to direct messages

  • regularly updating blogs and local resources

• REACT is offered as part of an integrated package of care for relatives, and not a stand-alone intervention

• access is restricted to relatives already being supported within the team

• staff can see and are confident that the forum is being well managed

• staff can see and are confident that direct messages are being responded to

• REACT supporters and IMPART leads are clear and confident about managing risk identified on REACT

• IMPART lead and REACT supporter roles are allocated to people with the time, skills, organisational role and support to carry them out

• REACT is clearly visible within the relevant clinical care pathway trust policy documents

• REACT is customised with accurate trust details

• staff are allocated time for training, supervision and carrying out their tasks specifically related to REACT

• staff have easy access to computers or tablets and IT support to enable their online tasks related to REACT

• staff can work easily between REACT and existing electronic health-care record/IT systems.

Reflexive monitoring

REACT is more likely to be successfully implemented (i.e. more relatives will be invited to use it) if:

• staff can access regular data that show that relatives are using REACT

• feedback from relatives is reviewed and shared as part of an operational meeting and can inform ongoing work

• staff are able to gather direct feedback from relatives they have invited to use the website (if positive, this facilitates implementation)

• staff are able to request or enact improvements to REACT as they see fit.

Participants

Participants were all staff in EIP teams and relatives or close friends of someone with bipolar disorder and/or psychosis, referred by staff to use the REACT toolkit, and who then consented to take part in the study (subsequently referred to as research relatives). Relatives who did not wish to take part in the research were still able to access REACT and their data were used only in summarising levels of clinical activity (subsequently referred to as clinical relatives).

Staff were initially informed about REACT and the IMPART study through the IMPART leads. Staff and relatives were invited to SRGs and to other group workshops by the IMPART leads, with support from the research team. Individual staff were approached directly by the research team and invited to take part in individual interviews. A detailed description of sampling and procedure for individual and group interviews is given in Process evaluation.

Relatives were invited to use REACT via an invitation e-mail from a staff member, with instructions on how to set up an account. Relatives who visited the website were also invited by e-mail to take part in the collection of research data, and had the choice of completing questionnaires, taking part in an interview (see Chapter 7) or both. Relatives could use the REACT intervention without participating in the research, although all REACT users had to agree to terms and conditions that permitted collection and analysis of basic demographic and website use information. Relatives who agreed to participate in the research provided informed consent online before being asked to complete any research measures.

Personalisation of the intervention to each trust

This was intended to engender a sense of ownership and legitimisation. Each trust was able to customise some elements of the website content including adding a logo, times that the forum/direct messaging services were actively moderated and by whom, emergency contacts, and photos and biographies of the trust’s REACT supporters and IMPART leads.

Initial familiarisation with the intervention

This was aimed at building coherence and cognitive participation.

Establishing local champions

This promoted cognitive participation. An IMPART lead was identified in each trust to work with the research team to assist with the research process, and within the trust to implement REACT.

Each IMPART lead held a senior role in the EIP and had expressed interest in the study, but roles and selection varied between leads. For Moor and Woods Trusts, the IMPART leads were co-applicants of the project and senior clinicians in their trusts. In Ocean, the first IMPART lead had been involved in the development of REACT. In Seashore and Marsh, senior clinicians were recruited; in Lakes, a senior psychiatrist with an interest in research took on the role.

The IMPART leads were encouraged to customise their trust’s REACT website with support from the research IT team. They also helped set up a SRG of staff and service users or relatives in each trust as described in Chapter 3. These groups met periodically throughout the study, initially to support the research process, but quickly evolved to become an important data source. Finally, IMPART leads were often instrumental in allocating REACT supporter roles within the EIP service.

Establishing workflows

This promoted coherence, activation and reflexive monitoring. Staff needed accounts in order to visit the REACT toolkit, invite relatives and monitor whether or not their invitation had been accepted. An account was created by the research team for each IMPART lead who then created REACT supporter accounts for the people moderating the website and clinician accounts for staff who could refer relatives to the website. Clinicians then created accounts for relatives. This ensured that control over access to REACT remained with the trust staff. Table 4 shows each of the key roles related to REACT and the different levels of website access for each role. Figure 6 shows the pathway through the REACT website.

FIGURE 6.

Pathway through the REACT intervention. CTRC, clinical trials research centre.

Go live

REACT went live for all six trusts on 21 November 2016. Trusts were informed by an e-mail sent to each IMPART lead explaining that feedback on the pilot had been addressed and REACT was live in their trust. Instructions on how to deliver and use REACT were available via the online manual, and face-to-face training was available via the research team. This was followed up by telephone to arrange training. Figure 6 shows the pathway of how relatives were invited to use REACT.

Implementation outcomes

Clinicians’ activity was monitored throughout the study in each trust to assess their engagement with the REACT toolkit, as quantified by the number of accounts they created and the number of invitations they sent to relatives.

We collected basic demographic and web use information from all relatives on the REACT website (research and clinical relatives) to evaluate which users the intervention reached and which modules were most and least visited. The demographic questionnaire [available at www.journalslibrary.nihr.ac.uk/programmes/hsdr/140416/#/documentation (accessed February 2020)] completed at the time of creating an account collected participants’ age, gender, ethnicity, highest education, current employment status and internet access. Website use data were collected during the study period, including the total number of logins to the toolkit, the total number of visits to each module and to the forum, and the total time spent on the website.

Intervention outcomes

Those who agreed to complete the outcome measures (research relatives) provided informed consent online before completing the questionnaires outlined above. These measures were chosen because they have good psychometric properties and were acceptable and sensitive to change in the REACT feasibility trial. 53 Questionnaires were offered at baseline and again after 12 and 24 weeks to ensure that we captured short- and longer-term impacts. Participants received three automated reminder e-mails to complete the questionnaires at each follow-up point. Owing to limited resources, no further efforts were made to follow up non-responders. All data from relatives were collected online, anonymised and downloaded to a database held at the Liverpool CTRC.

Process evaluation

Factors affecting implementation outcomes were explored through individual staff interviews, SRG workshops, document analysis and meeting observations, and analysed together within the NPT framework, then were used to inform iterative versions of the implementation plan. Interviews with relatives explored their perspective of the implementation of REACT, but were analysed separately using an open thematic coding. Each step is described in the following sections.

Sample size

Our main purpose was to understand the process of evaluation, building a detailed picture in each trust by talking to relevant staff in EIP teams and in the wider organisation. We attempted to sample relatives, including those who had used REACT and those who had not, to interview across all participating trusts. The sample of relatives completing the quantitative measures was determined by the level of implementation.

Quantitative data

Descriptive statistics for variables measured on an interval scale were mean, standard deviation (SD), median, range and interquartile range (IQR); for categorical (including ordinal) variables, they were frequency (n) and percentage. Pearson’s correlation coefficient was used to quantify linear associations between interval variables, with Spearman’s rank correlation coefficient being used if the association was potentially non-linear. Independent sample t-tests (using the Satterthwaite approximation for the degrees of freedom where equality of variances could not be assumed) and Pearson’s chi-squared tests were used to explore differences between clinical and research participants. Paired t-tests were used to investigate the mean change from baseline to 12 weeks and to 24 weeks in the research participants’ self-reported scores; effect sizes were estimated by dividing the mean change by the baseline SD. Statistical significance required a two-sided p-value of < 0.05, and 95% confidence intervals (CIs) were presented. All data analyses were performed using IBM SPSS Statistics version 23 (IBM Corporation, Armonk, NY, USA).

Qualitative data: staff

Qualitative data sets collected from trust staff, meeting observations and document analyses were analysed using framework analysis85 and NVivo software. Data were analysed on two levels.

Qualitative data: relatives

Analysis of relatives’ interview data was led by Johanna Barraclough and followed the procedure proposed by Braun and Clarke. 99 This inductive thematic coding method allows themes to emerge from the data rather than relying on a pre-existing framework, but also offers a detailed description of the process that was easy for our carer researchers, who had no formal research training, to become involved in, ensuring multiple perspectives.

The first phase of analysis consisted of thorough immersion in the data by reading the entire data set of interviews several times and recording notes about ideas and meanings in a reflective journal. Ideas about interesting segments of the data were organised into initial codes, which represented areas of meaning. The entire data set of interviews was worked through methodically, with all potential codes marked in the margin of each transcript. A pictorial map of codes was created using sticky notes, enabling visual identification of patterns of meaning while retaining sight of instances of departure. To ensure that the process was valid and rigorous, two members of the research team with experience as carers of a close relative with psychosis or bipolar disorder independently coded six transcripts, each with the results discussed with the lead researcher (Johanna Barraclough). A high level of consistency provided reassurance that the process was appropriate, and discussions about the implications of the codes identified served as a useful foundation for building themes, with the lived experience perspective of the carers providing insight.

The codes were then compiled into a Microsoft Word (Microsoft Corporation, Redmond, WA, USA) document together with supporting quotations from interviews.

The next phase involved taking a broader perspective on the data and grouping the codes into potential themes. The visual sticky note map enabled codes to be grouped into clusters of overarching themes, and a structure of main themes and subthemes emerged, which was then translated into a Microsoft Word document together with supporting quotations. Seven members of the research team, including the two carer researchers and chief investigator, met to review the main researcher’s decisions regarding the potential themes. Discussions centred on whether or not the coded data supported each theme and the scope of each theme, with themes divided or amalgamated as needed. The descriptions given to the themes were discussed for accuracy. Discussions also explored how themes linked together and their illumination of factors affecting implementation of REACT from the participants’ perspective.

Following the meeting, the main researcher reviewed the whole data set once more to verify that the themes were robust, and the diagram was refined to illustrate the story conveyed by the data. This diagram was presented at the explanatory synthesis meeting (described in the following section) and refined further in the light of the process of integrating these data with data from other parts of the IMPART study.

Trust analysis days

These were held at each trust. IMPART leads invited staff and relatives who had been involved in the IMPART study to attend. The research team presented a preliminary summary of the factors affecting implementation of REACT at that trust, based on the data days analysis described in Level 2 analysis. Participants were encouraged to question, elaborate on and explore these findings, particularly highlighting any areas of misunderstanding or important missing factors. These events were a key step in ensuring that participants felt fully part of the research process, rather than just providers of data.

The IMPART study-wide data day

The IMPART leads and other key staff, including REACT champions and supporters, were invited to a 1-day event to support the interpretation of data gathered across all six trusts. The main aims of the event were to:

• present the draft final analysis, including overarching themes and key factors affecting implementation developed across the six trusts, and gather attendees’ feedback and reflections

• present all trusts with each iteration of the implementation plan and gather suggestions for IPv4

• ensure that attendees had all the information they needed about the immediate future of REACT, including how the findings of the IMPART study would be interpreted alongside those of the parallel REACT trial55

• maintain a collaborative working relationship and investigate how and if attendees wanted to continue contributing REACT in the year following the end of the IMPART study.

Explanatory synthesis event

All IMPART study team members, including members of the PMG, an IT developer, research associates and carer researchers, were invited to a final 2-day study event. The main aims were to:

• synthesise the analyses of all of the data collected throughout the study and develop a coherent story, including testing our study-specific hypotheses

• collaboratively develop an explanatory synthesis to understand the key factors that affected implementation of REACT across all six trusts

• finalise an implementation plan (version 4) that could be shared with other NHS trusts that were interested in using REACT

• reflect on the role of NPT in the IMPART study (i.e. successes and challenges) and implications for future development of the theory.

Staff activity and relatives’ uptake of REACT in each NHS trust

Figures 8–13 and Table 6 present the number of staff accounts created, invitations sent to relatives and accounts created by relatives in each of the six participating NHS trusts. Figure 8 also shows the timing of key research-related activities in each trust.

FIGURE 8.

Timeline of clinicians’ and relatives’ activity in wave 1 trusts: Woods Trust.

Wave 1 trusts

As seen in Figures 8 and 9, uptake, defined by the numbers of staff accounts created and numbers of relatives invited to use REACT, was relatively low, and similar in both trusts. Although more clinicians set up accounts in Woods than in Moor, subsequent activity was lower in Woods (fewer invitations were sent and fewer relatives signed up to REACT). There were no significant changes over the study period. Research team activity was minimal in wave 1 trusts, which received only IPv1.

FIGURE 9.

Timeline of clinicians’ and relatives’ activity in wave 1 trusts: Moor Trust.

Wave 2 trusts

Wave 2 trusts were given an enhanced implementation plan (IPv2; see Chapter 6). This appears to be associated with additional uptake and use in Seashore, with more relatives invited to register by staff members and more accounts created by relatives, but there was substantially less activity in Ocean throughout the study period, as shown in Figures 10 and 11.

FIGURE 10.

Timeline of clinicians’ and relatives’ activity in wave 2 trusts: Ocean Trust.

FIGURE 11.

Timeline of clinicians’ and relatives’ activity in wave 2 trusts: Seashore Trust.

Wave 3 trusts

The two trusts in wave 3 received the most intense implementation plan, the content of which is described in Chapter 6. As in wave 2, the trusts differed in levels of activity, with Marsh Trust showing more activity than Lakes Trust, as seen in Figures 12 and 13. IPv3 appears to be associated with an increase in the number of invitations sent and relatives’ accounts created, but not in the number of clinician accounts created. Marsh Trust was the only trust with more relatives’ than clinicians’ accounts. In contrast, the most clinician accounts were created in Lakes Trust, although the number of relatives’ accounts remained comparatively low.

FIGURE 12.

Timeline of clinicians’ and relatives’ activity in wave 3 trusts: Lakes Trust.

FIGURE 13.

Timeline of clinicians’ and relatives’ activity in wave 3 trusts: Marsh Trust.

Table 6 provides summary statistics for accounts created by staff, invitations sent in total (including repeat invitations), number of relatives invited and number of relatives’ accounts created. Table 6 also shows summary data for levels of support offered to relatives by each trust prior to using REACT, as reported in the National Audit of Early Intervention Service Providers conducted by the Royal College of Psychiatrists in 2016. 102

The highest number of clinicians signed up at Lakes Trust (n = 64; wave 3) and the lowest number signed up at Ocean Trust (n = 32; wave 2). Seashore Trust (wave 2) sent the most invitations in total (n = 112) and to individual relatives (n = 87), whereas Lakes Trust sent the fewest (despite having the largest number of clinician accounts). A minority of clinicians across all trusts sent invitations, with the fewest being in Ocean (n = 4) and the most in Seashore (n = 18). Marsh Trust had the most relatives’ accounts (n = 53; wave 3), and Woods Trust had the lowest (n = 7; wave 1). High uptake in Seashore may be linked to the lack of other forms of family intervention offered, based on national audit data. According to the audit, Lakes Trust offered family intervention to the highest proportion of relatives (62%), whereas Moor (wave 1) and Marsh offered family intervention to the lowest proportion (3%), much lower than the national sample average of 31%. As for carer-focused education and support, Ocean offered this to the highest proportion of relatives (87%) and Moor to the lowest (3%), again substantially below the sample average of the audit.

Data collection from relatives

Figure 14 illustrates the recruitment and attrition of relative participants. Clinicians across the six trusts sent a total of 355 invitations to relatives; 159 relatives’ accounts were set up across the six participating trusts. Sixty-two participants consented to complete outcome measures, of whom 56 (baseline), 21 (12 weeks) and 20 (24 weeks) completed at least some measures. Sixteen participants completed (at least partially) the measures at all three time points.

FIGURE 14.

Relatives’ demographic and intervention outcome data: recruitment and retention.

Participants’ characteristics

Table 7 summarises the sociodemographic characteristics of relatives. Comparisons were made between clinical relatives and research relatives, but there were no significant differences in any of the variables. The majority of participants in both groups did not work full time, were aged > 50 years, were female, were from a white ethnic background, used broadband or Wi-Fi to access the internet and were educated to Advanced level (A level) or above.

Table 8 provides information about the caring roles of research participants from the ‘About your caring role’ questionnaire completed at baseline (see Appendix 5 for 12- and 24-week follow-up; this information was not collected from clinical participants). The majority of participants had been in caring roles for > 1 year and had an average of 45 hours of predominantly face-to-face contact with the person they were supporting each week. Most participants reported that caring for their relatives had not affected their employment status or performance, because they either had not been in paid employment prior to their caring role or were still performing the same work. A small number of participants at each assessment point reported either stopping work or reducing their hours owing to their caring role. Participants reported caring for between one and four relatives, with the majority caring for one relative only. Demographic and clinical characteristics of the relatives cared for are also presented in Table 8 (see also Appendix 6). The most common relationships were mother (participant) and child (relative cared for). Individuals who were cared for were primarily aged < 25 years and living at home. The most commonly reported diagnosis was psychosis. However, there were relatively high rates of missing data or instances when participants did not know the diagnosis of their relative.

Web use of relatives

Table 9 summarises the 24-week web use of research and clinical participants, and the results of independent t-tests comparing their web use (see also Appendix 7). The mean number of logins for the group as a whole was 3.78 (SD 4.43), but there was a lot of variation, with a range of 0–31 logins (median 2 logins, IQR 1–8 logins). The mean total time spent on the website was 40.6 minutes (SD 54.54 minutes), with a range of 0–298 minutes (median 20.1 minutes, IQR 4.9–57.5 minutes). In general, the research participants used the website more than the clinical participants, and the difference was significant in the total number of logins, number of visits to modules 1–6 and module 11, number of visits to the forum, total time spent on the website and total number of module visits (composite measure of visits to the 12 different modules).

Figure 15 displays the average number of visits during the 24-week follow-up period for clinical and research participants to the different sections of the website. Module 1 (‘What is psychosis?’) and the forum were the most frequently visited sections for both clinical and research participants, and module 9 (‘Understanding mental health services’) was the least visited by both groups.

FIGURE 15.

Website use of clinical and research participants.

The impact of REACT on relatives self-reported measures

The descriptive statistics of self-reported measures at the three time points are summarised in Table 10. Results regarding the GHQ-28 screening scores (items scored as 0, 0, 1, 1) indicated that 34 participants (60%) scored above the clinical threshold of 4 out of 5 at baseline, compared with 11 (55%) at 12 weeks and 12 (60%) at 24 weeks. The average total GHQ-28 scores (items scored 0–3) were also above the suggested clinical threshold of 23 out of 24 at each follow-up point, confirming high levels of distress in the sample. Mean values of the GHQ-28 total scores, and subscales of social dysfunction, severe depression and anxiety and insomnia, the CWS well-being scale and eHealth literacy all improved with small effect sizes. Paired t-tests with 95% CIs compared baseline scores with those at the 12- and 24-week follow-ups; none of the changes from baseline was statistically significant (see Appendix 8).

Survey responses

Forty-eight staff members completed the costs and savings survey across five trusts (Table 11). Lake Trust had the highest response rate; no staff completed the survey in Woods. The majority of respondents were clinicians who had referred relatives to REACT, followed by REACT supporters. Given the variation in number and role of respondents, it is not valid to make comparisons between trusts or across different waves. However, a general picture across all trusts can be ascertained.

Costs

Savings

The final section of the survey explored whether or not staff members could identify areas where REACT contributed to savings of resources or other benefits to staff or relatives (Table 14). The majority of respondents did not feel that REACT reduced the time they spent supporting relatives. Those who did save time estimated this to be only 1–2 hours across the whole study period. However, > 80% of respondents reported that REACT helped their services to better support relatives by equipping staff members with a resource to share with relatives and helping them to reach both national policy and local service targets to support relatives. Respondents in Ocean also felt that the toolkit improved their confidence and offered a backup for their important conversations with relatives. They highlighted that REACT not saving time was a positive thing as it may indicate staff spending more time communicating with and supporting relatives as a result of having REACT and improved confidence. In Lakes, staff felt that REACT had improved communication with relatives owing to relatives having more understanding of psychosis/bipolar disorder. In Moor, staff reported that REACT was particularly helpful to relatives who do not wish to attend family interventions but were seeking a better understanding of mental health problems. Finally, in Seashore, a member of staff mentioned that REACT fostered thinking of alternative ways to support carers using digital/online resources.

Implementation plan version 1

Challenges for this first iteration were to deliver REACT in all six trusts, train staff by providing an online manual followed by face-to-face training, and appoint an IMPART lead as PI in each trust, as described in Chapter 4.

Implementation plan version 2

Implementation plan version 2 aimed to address some of the issues above. Factors identified in wave 1 fed into workshops in wave 2 trusts where IPv2 was designed and delivered. It attempted to increase staff engagement by making REACT more visible and user friendly. Some elements of IPv2 were further developed in IPv3: these are shown in italic text in Table 17.

REACT NHS booklets

These booklets, shown in Figure 16, were produced for staff to show to relatives to support their introduction to REACT. Designed to be an attractive description of REACT and its potential benefits for both staff and relatives, the booklets came with a business card with the REACT toolkit Uniform Resource Locator (URL) so that relatives would have the address to hand, and an information card to prompt staff to record the information (name and e-mail address) needed to later send an invitation from the REACT website.

FIGURE 16.

Booklets and merchandise for IPv2. Reproduced from Greenhalgh et al. 74 © Trisha Greenhalgh, Joseph Wherton, Chrysanthi Papoutsi, Jennifer Lynch, Gemma Hughes, Christine A'Court, Susan Hinder, Nick Fahy, Rob Procter, Sara Shaw. Originally published in the Journal of Medical Internet Research (www.jmir.org), 1 November 2017. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.

Reproduced from Greenhalgh et al. 74 © Trisha Greenhalgh, Joseph Wherton, Chrysanthi Papoutsi, Jennifer Lynch, Gemma Hughes, Christine A'Court, Susan Hinder, Nick Fahy, Rob Procter, Sara Shaw. Originally published in the Journal of Medical Internet Research (www.jmir.org), 1 November 2017. This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited.

The booklet included an invitation to use REACT with or without taking part in the collection of research data (data showed that this was a barrier for relatives – see Chapter 7). It was important that relatives could read a description of REACT without going online. The packs were welcomed, and fitted with how clinicians generally offered services and built relationships with relatives.

In IPv3, minor modifications were made in response to feedback, including using the term ‘friends and family’ on the cover to be more consistent with the terms services used.

Implementation plan version 3

Implementation plan version 3 was developed during wave 2 and aimed to address some of the key issues identified in wave 1, 2 and 3 trusts. Factors identified in waves 1 and 2 fed into workshops in wave 3 trusts where the implementation plan was designed and delivered.

Implementation plan version 3 primarily addressed concerns over staff not inviting relatives; staff training not being accessed, particularly by new staff members; the need for feedback about sent invitations; and the longevity of someone driving REACT in the trust after the IMPART study ended. All components of IPv2, including those shown in italic text in Table 17, were included in IPv3, which is described in Table 20.

Rationale and design

This section of the IMPART study had a qualitative design to allow relatives’ views and experiences of the implementation of REACT to be captured through a semistructured interview format. The key research question addressed was ‘What are the factors affecting relatives’ engagement with REACT?’.

These data were not analysed in waves, as the iterative nature of the implementation plan design was aimed at changing staff behaviour and getting more invitations sent to relatives, rather than changing the way in which relatives engaged with REACT.

The small number of relatives interviewed from each trust also prevented generalising to the experience of other relatives in that trust. However, we have highlighted instances when the study wave or elements of the implementation plan appear to have relevance, and explore this in the discussion.

The methods are described in detail in Chapter 4. Characteristics of the participants and the key themes are described in the following sections.

Participants

Twenty-three relatives participated in the study, all of whom had internet access at home. Table 23 shows the demography of the sample.

Relatives were recruited from five of the six participating trusts. We were unable to recruit any relatives from Lakes, and recruited the most relatives from Seashore. This pattern reflected the number of relatives agreeing to take part in the quantitative measures described in Chapter 5.

There was a high degree of variability in the use of REACT. The mean time spent on REACT was 63 minutes, substantially higher than the mean for all relatives (40 minutes, see Chapter 5). However, this group included relatives who did not use REACT at all, and others who spent nearly 5 hours on REACT. The amount of time spent by each participant on each module and on the forum is detailed in Report Supplementary Material 2.

As described in Chapter 4, emerging themes in the interview transcripts were visually identified by use of sticky notes. The final wall of notes is seen in Figure 17. The final thematic analysis of factors that influenced relatives’ engagement with REACT elicited four overarching themes and associated subthemes, summarised in Box 1.

FIGURE 17.

Sticky notes used to identify emerging themes of relatives’ engagement.

The relationship between these themes is presented pictorially by Figure 18. The relatives as participants are represented by the rainwater and their journey is shown by its path as it falls from the clouds to the hills and is captured by various buckets. The first theme, ‘Context’, is represented by the rain cloud, which holds the relatives and the complexity of their individual situations, illustrating the impact of their caring relationships on their emotions and on the practicalities of daily life.

FIGURE 18.

Relationship between key themes in understanding relatives’ experiences with REACT.

The flow of rainwater from the cloud represents the relatives’ journey to NHS EIP services, shown by the blue bucket. The brown bucket represents other forms of support that relatives may use, either in conjunction with EIP or separately, including support provided by charities and other sources of information.

The second theme, ‘introduction to REACT’, represented by the pipe, describes the way in which the participants are introduced to REACT by EIP staff, with the characteristics of this introduction determining the extent to which they initially engage with the intervention. The bends in the pipe represent the importance of an effective introduction in order to maintain the water flow through to the REACT ‘bucket’.

The third theme, ‘engagement with REACT’, refers to the participants remaining contained within REACT, with the factors that keep them interested grouped into subthemes. Theme 4, ‘disengagement from REACT’, is represented by rainwater escaping from the bucket and shows the aspects of REACT that participants felt could be improved in order to meet their needs.

The diagram shows links between the themes and subthemes as dotted lines. Although each participant’s journey may be linear (a relative of an EIP service user introduced to REACT, and then accessing REACT), the factors that affect their engagement with the intervention link back to their context. This reflects the fact that an in-depth understanding of the situation and experience of being a relative of someone experiencing psychosis or bipolar disorder is fundamental to designing the features of REACT and creating a protocol for the way in which it is introduced to relatives.

Impact of the caring role

Participants described their lack of understanding of what was happening to their relative in terms of their illness, and the emotional impact of this uncertainty. One participant conveyed how all of her time and energy were directed towards her relative, which left little space for understanding what was happening to the whole family. This created a feeling of a state of emergency, with no opportunity to gain perspective and analyse the problem. Despite the lack of emotional space to understand the situation, there was a strong desire to make sense of it and understand its origins:

It was very, very difficult to understand what was going on and where it had come from, and you were just so confused . . . you don’t have any space to process it or to understand it.

Mother, aged 56–60 years, Woods 08

The search for meaning and the cause of the difficulties was prevalent in participants’ accounts. In some cases, a sense of guilt emerged as participants questioned whether or not their actions had led to the difficulties being experienced by their relative, with one person questioning whether or not she had done something during pregnancy that had caused the mental health problems besetting her son.

Participants described their reaction to the onset of their relatives’ difficulties as a state of shock, accompanied by fear and uncertainty. This highlighted the trauma associated with the onset of psychosis or bipolar disorder. One participant described how she felt ‘lost’, evoking a sense of isolation and a lack of support and clarity about what was happening. In addition to physical demands of time and energy, the emotionally traumatising impact of shock suggested that the participant’s ability to seek support and process information at this time would be limited, despite this being a real need:

I think initially it was a tremendous shock and very frightening, and we were just lost.

Mother, aged 66–70 years, Seashore 06

The persistent trauma and shock were described as having a negative impact on carers’ own mental health and stress levels. This experience was not characterised by an isolated incident of stress, but continued over an extended period, resulting in high levels of distress. The intensity of the caring relationship was exacerbated by uncertainty about their relative’s behaviour. The need for constant vigilance made the caring role feel all-encompassing:

You just don’t know from you know hour to hour really how they’re going to behave.

Mother, aged 51–55 years, Woods 06

Because it’s so crazy for 2 years, trying to get support for him, and I was thinking . . . I’m going to end up having a mental breakdown.

Brother, aged 31–35 years, Seashore 11

Under these circumstances, the relatives’ own needs became supressed. This highlighted the need for information and support that were easy to access and digest, and could be used flexibly to fit in with the unpredictable nature of the caring role.

Family dynamics

Participants discussed difficulties with balancing the needs of the rest of the family because of the demands of the family member experiencing mental health problems. The challenge of allocating time to roles such as wife or husband, friend, employee or parent to other siblings linked with the previous theme stretched resources. Attending to the needs of other children was difficult, as was maintaining a commitment to work. Three participants felt that they were unable to work because of their caring responsibilities. Others said that they were able to remain in work only because of the flexibility shown by employers. One participant described how valued leisure activities were no longer possible, and acknowledged with resignation that this was part of being a carer. This showed how the needs of carers were often relegated in the face of other demands, sometimes with the carers not even recognising this themselves. Understanding this contextual factor is important in planning for how REACT is introduced and presented, as discussions about identifying and validating the needs of the carer may be required:

Yes, you’re not just the carer really, it’s balancing everybody else’s need in, ‘cos that’s the reality of families isn’t it?

Mother, aged 61–65 years, Marsh 03

I’ve given up work to be there 24/7 for my daughter who basically has had a couple of episodes which initially we thought was brought on by drug-induced psychosis.

Mother, aged 50–56 years, Seashore 12

I ended up being off work with him until April . . . There are big chunks of my life where I’ve had to take time out from what I do to care for [my son].

Mother, aged 46–50 years, Moor 09

Initial contact with health services

This subtheme covers contact between participants and clinical services, both physical and mental health, in the search for clarity at the onset of their relatives’ difficulties. Despite extensive attempts to find someone who could tell them what they should do to support their relative, participants were often very frustrated by the lack of support and information. They described time and effort spent travelling fruitlessly to different clinics. The sense of being excluded from services, and the effort and tenacity required on their part to even identify where help might be found, were palpable:

I’ve gone to one clinic after another and nobody can actually tell me how I get into the system.

Mother, aged 66–70 years, Seashore 13

Some participants contrasted the lack of clarity about what was happening to their relative with physical health, where definite diagnoses and treatment plans are made:

Initially I was very confused as to what was really wrong and I said this many times: if you’re dealing with measles or a broken leg then we’d all know what to do, but where mental health is concerned it’s anybody’s guess it seems.

Mother, aged 66–70 years, Seashore 13

All of this seemed to exacerbate carer distress. Participants spoke about feeling frustrated that their voice was not heard when interacting with services for their relative. In adult services, clinical staff were often reluctant to communicate with them owing to confidentiality. Participants felt that services discounted the value of their knowledge about their child and were excluded from discussions owing to the service user’s status as an adult. This further diminished the role of the participant as the carer. The contrast between how insignificant they felt their role to be considered by clinical staff and the reality of the amount of time and physical and emotional energy put into caring incited strong emotions of anger and sadness, but also reinforced feelings that their own needs as carers were not relevant. This was highlighted when one participant expressed her frustration at not being listened to by clinical staff:

I see him on a day-to-day basis; something’s not right, you know that, your mother instinct comes in.

Mother, aged 50–56 years, Seashore 12

I just think if the professionals actually sat down and interviewed us as well and maybe just not told us all of [son’s] confidential business but just listened to what I had to say.

Mother, aged 56–60 years, Seashore 04

Most frustration was focused on the initial process of trying to access support at the onset of their relatives’ difficulties, and carers primarily described interactions with NHS primary care services such as GPs, rather than the EIP service. Although there was variance in the length of time before appropriate services were accessed, positive experiences did not feature in participants’ descriptions of this initial period. Once participants were able to access EIP services, they described more positive experiences and provision of support.

Who?

The personal characteristics and relationship between the participant and EIP staff member as REACT introducer influenced the decision about whether or not to access REACT. A positive relationship with the introducer was characterised by trust and familiarity with the circumstances and needs of the carer, and made relatives more likely to sign up to REACT. The sense that the introducer has the participant’s best interests at heart was fundamental. Participants appreciated the care shown by the introducers and, in particular, that REACT focused on them as carers:

He said it was like . . . a toolkit, you know, you might find it useful, you know just have a look and see what you think, you know it’s being developed, so that’s what I did, I had a look ‘cos I liked him.

Mother, aged 56–60 years, Woods 08

I would go with anything that [care co-ordinator] had said and the person that worked with him previously, you know, they were very good and they just wanted to help.

Mother, aged 56–60 years, Seashore 04

Strong individual relationships emerged when participants felt that they were being listened to, taken seriously and not dismissed:

. . . But it was the Thursday of that week that the people from the EIP team first came, and I have to say they were absolutely brilliant. They spoke to me first, they were very concerned about how I was coping.

Mother, aged 61–65 years, Marsh 08

Early intervention team, amazing, really, really good in [town] – the one thing is that no one pooh-poohs you, they take it seriously.

Mother, aged 61–65 years, Marsh 10

Participants saw REACT as something that worked in conjunction with EIP, rather than being a replacement, with a strong existing relationship with EIP being important for this dynamic, and REACT was viewed as part of a package of care:

The toolkit . . . doesn’t replace the actual contact with the professionals, you know, that [contact] was invaluable, and still is actually. But it is useful in addition.

Mother, aged 46–50 years, Ocean 05

Participants were asked how they would feel about being introduced to REACT by another relative rather than a member of EIP staff, and most saw additional benefits to this. Other carers were viewed as offering potentially greater levels of understanding than clinical staff, which has implications in terms of involving carers to introduce REACT in future:

I think if you’re actually experiencing the problem yourself you really have a better . . . You really understand what’s useful and what’s not useful.

Mother, aged 46–50 years, Ocean 05

When?

Participants wanted REACT to be available to relatives as soon as possible, to address the need for clarity and support during the early phase of their relative’s difficulties. Several participants expressed frustration that REACT had been offered to them too late to be of use. The onset, with its shock and confusion, was the time when information about what was happening would have been much appreciated and could have reduced the trauma of the situation. Participants wanted to better understand what their relative was experiencing in order to provide more meaningful support to them, and felt that REACT could have had a role in providing this if it had been provided at the right time:

I do think the website itself is brilliant but it was probably just too late for us.

Sister, aged 16–20 years, Moor 04

You’re sort of there stumbling around in the dark really, so at that point initially if you’re given information or signposted to things, I think then you can start to then understand, OK, what they’re going through.

Mother, aged 56–60 years, Woods 08

However, some participants acknowledged that too much information early on might have been experienced as overwhelming. Despite this, they still wanted REACT to be available to them early, so that it was there when they felt ready to access it – they could make the decision about when the right time was:

I suppose for me it would have been valuable to give that information when I got a carers’ pack . . . Even if you don’t use it, basically you’ve got it, you know you can look at it.

Mother, aged 61–65 years, Marsh 03

How?

Participants needed to hear the benefits of REACT as part of the introduction. They wanted explicit detail of what was covered and specifically how this could be of value to them. Key to this was a clear message that REACT prioritised the needs of the relative. The opportunity to hear stories from other relatives about their experiences was a significant draw, especially when this was explicitly linked to reducing the sense of isolation that can accompany being a carer.

Although the initial introduction was seen as the most significant interaction in engaging relatives with REACT, gentle reminders from staff were also crucial to ongoing use. The discussions, explanations and ongoing reminders highlighted the importance of REACT being offered as part of an integrated service that included face-to-face contact:

. . . that it was just so important for me to look after myself because I was his primary carer, you know, that role was explained to me as well, and . . . I think it was after the second visit that [care co-ordinator] sent me the contact details or the user details for me to log on to the REACT website.

Mother, aged 61–65 years, Marsh 08

The question of whether or not REACT was introduced in the context of the IMPART research study is significant here, with opinion divided between participants regarding whether or not this motivated them to access REACT.

Some participants felt that engaging in a research project was positive, because it contributed to advancing knowledge that would benefit people in similar circumstances. Others felt that it would put them off using the toolkit as it represented one more demand to add to an already-heavy load:

I was just so grateful for the help that I did get and you know that I did genuinely want to, you know, contribute here in any way that I could . . . for other people.

Mother, aged 61–65 years, Marsh 08

I might have been irritated if somebody said, ‘oh yeah, it’s a research at that point’ ‘cos I was so stressed out . . . ‘Oh, do you think I really want to be worried about research when I’ve got somebody trying to kill themselves?’

Mother, aged 51–55 years, Marsh 09

Video clips

Participants were positive about REACT’s use of video clips, which illustrated situations faced by other relatives and how they dealt with them. Watching the clips reduced their feelings of isolation, as they identified with people in similar situations.

They also found the clips useful in suggesting strategies for dealing with difficult periods, and new, helpful ways of communicating with their relative, for example when experiencing scenarios such as hearing voices. (This subtheme links with context and family dynamics.) This enabled participants to change their perspective and improve their understanding of what their relative was going through, which then enabled them to consider different ways of thinking about the situation and change their behaviour:

I found the information on there absolutely brilliant, absolutely brilliant, because some of the people were even talking about the same experiences that I was having.

Mother, aged 61–65 years, Marsh 08

One of the video clips explained it beautifully: she said, ‘even though it may not seem real to us, it actually is a real thing for them, so as convincing as I’m saying to him no that didn’t happen . . . well, you shouldn’t say that to a person because it actually did happen, in their brain it happened’. So yeah . . . there was lots of good videos.

Mother, aged 36–40 years, Seashore 09

REACT usage data for the above participant (Seashore 09) supports her description: she visited the module containing the video clip she referred to three times during her total of eight logins to REACT, suggesting that this was the most engaging module for her.

Empowerment

Having more information about their relatives’ conditions increased relatives’ confidence and made them feel more empowered. When interacting with clinical services, the information they gathered from REACT meant that they felt more equipped to have meaningful conversations and were not so readily dismissed. One participant talked about how REACT reduced her feelings of panic about her relative’s difficulties, and led to a calmer feeling of acceptance:

I felt empowered honestly after having that little session online by myself.

Mother, aged 36–40 years, Seashore 09

We’re not being fobbed off with ‘everything’s going on OK’ [when] really it’s not.

Mother, aged 51–55 years, Moor 10

Once you start listening to other people’s stories or the literature, like I said, it makes you calmer about the situation, that’s what I’m trying to say . . . My attitude now, well not all the time but sometimes, is right we’ll ride the waves and see what happens whereas before I’d be like trying to stop the waves.

Mother, aged 51–55 years, Woods 06

Ease of navigation

Participants found REACT easy to navigate and understand. The modular structure allowed some participants to identify which section of REACT was most relevant to them, and others to work through REACT methodically and sequentially, as preferred. This echoes the earlier theme in which relatives also wanted choice about when they could access REACT:

Well I thought it was bright, and like I say the big boxes to click on to send you further into what you wanting, really easy, and it’s easy to navigate from one to the other, you know you can bounce around it, you don’t have to go step by step anywhere, so you can take bits whenever you want it, so yeah I found it quite user friendly and easy on the eye.

Mother, aged 46–50 years, Moor 09

This participant’s REACT usage data shows that she accessed 10 of REACT’s 12 modules over five separate logins, which supports her description of ease of navigation.

Technical problems

Some participants expressed frustration with problems they had with accessing REACT and with the initial account set-up process. Persistence and additional support from EIP staff were needed in some cases. For some, this meant that REACT was perceived as another task, another problem to overcome, rather than something helpful and supportive. Some participants felt as if they had done something wrong, and REACT added to their feelings of stress and burden rather than helping. One participant referred to REACT as ‘not liking’ them, which evoked an antagonistic relationship and a direct contrast with REACT’s aim of providing support. This emphasised the importance of resolving technical issues quickly, which was not always possible owing to limited resources.

Access issues also occurred because the REACT online trial ran in parallel to the IMPART study. Some IMPART participants used an internet search rather than through the link provided by their trust and accidently reached the REACT online trial website rather than the toolkit hosted by their trust. This problem would not occur in any future implementation of REACT:

I think my husband and I tried and initially there was problems getting onto the website, we were having issues.

Mother, aged 50–56 years, Seashore 12

So I do need to be able to get on it easily ‘cos that’s like another oh god not something else I’ve got to do.

Mother, aged 61–65 years, Marsh 03

Inactive forum

The forum was designed to allow relatives to connect with each other to share experiences, ask questions and exchange information. However, because each trust hosted a separate forum and the number of relatives using REACT in each trust was small, there was very little activity on the forums. This discouraged participants from posting, as they felt that this would make them conspicuous. This also gave the impression that it was not functioning as it should, thereby reducing confidence in the rest of the intervention. Participants wanted a well-populated and fully functioning forum to communicate with other relatives, reduce their sense of isolation and motivate them to revisit REACT to look at new content. Some participants checked the forum several times, and then followed this up by contacting EIP staff about whether or not the forum was active. Their persistence demonstrated the level of interest and need for this feature. REACT usage data support this point, as the forum was the most visited area of REACT, despite the lack of activity. One participant logged into the website eight times to check if anything was posted:

Only thing I was slightly disappointed in was the forum. I mean I don’t know how new the website is but there was just nothing on it, you know, I was hoping to be able to just go in there and read, you know . . . how other people had dealt with them.

Mother, aged 61–65 years, Marsh 08

Well, it didn’t seem to work. I did go on it a few times and initially didn’t get any response and then I mentioned it again to [care co-ordinator], I’m not sure how up and running it is or actually was.

Mother, aged 46–50 years, Ocean 05

Lack of specific information

Participants wanted more information about some topics that were not included in REACT to help in their supporting role. Specific medications and their side effects, comorbid health conditions and recovery were specifically highlighted; for example, participants were concerned about weight gain associated with some medications and wanted more information about this particular side effect, how to address it with their relative and, if necessary, introduce conversations about healthy diet and exercise:

Explain the medication, explain the risk of the medication, you know because . . . there is a lot of side effects, addiction to it. My brother put on about 4 stone in weight within a short period of time because of the medication – nobody told me about that, but how can a family member protect somebody . . . if they don’t know that information?

Brother, aged 31–35 years, Seashore 11

There’s not enough information about clozapine, aripiprazole, the mixtures thereof, getting those right, the side effects of that.

Father, aged 61–65 years, Seashore 10

Some participants said that REACT did not feel relevant to them as it did not provide case studies in which the service user had more than one condition. They had to adapt the information too much for their own situation. Autism, in particular, was a relatively common comorbid issue. Participants need to feel that REACT could be applied to the specifics of their situation and family, that it could be tailored to their needs and that it could account for the differences in families’ situations:

But what we didn’t know at the time, and didn’t know until March of 2017, that [daughter] is autistic as well.

Brother, aged 31–35 years, Seashore 11

Most participants were parents, and some felt that REACT lacked sufficient information about supporting their children to achieve an independent and fulfilling life. They were concerned about what would happen to their son or daughter if they were no longer able to support them, and this led to a desire to help their child to become more independent.

This emphasises the idea that participants feel that the information provided in REACT can be tailored to their individual situation and provide practical strategies for navigating problems. Participants expressed the need to help their relative complete day-to-day tasks independently and support them to build their own skills:

The recovery section was not particularly uplifting . . . and I know it’s a very slow process, even recovery, and just understanding more that how recovery is going to happen. How things move forwards. It just seemed quite a small section.

Mother, aged 46–50 years, Ocean 05

Independence to me means well managing the things you have to do everyday to be able to live independently.

Mother, aged 61–65 years, Marsh 03

Well I need them to be OK, I need them to get on with their lives.

Mother, aged 41–45 years, Seashore 05

Fit with organisational culture

Adopting a DHI into EIP teams required a huge shift in working culture. These teams were historically characterised by people talking to each other and recording activities using pen and paper. Direct human contact was understood to be the main agent of change. An online intervention was not something staff were naturally exposed to or generally seeking. Basic access to IT facilities and staff training was extremely limited, and experience of other digital health systems, including electronic health records, had been generally frustrating.

During our data collection, it was clear neither to us nor to staff who was responsible within trust organisations for the strategic direction of service development and who could facilitate the shift needed for staff to embrace DHIs such as REACT. Many were not sure who needed to give the ‘go ahead’ for staff to use REACT, and the distinction between clinical and operational managers seemed to add to the confusion. Some trusts (e.g. Marsh) had a ‘transformation manager’, but it was not clear how they would support the use of interventions such as REACT.

REACT came into trusts via R&D teams that were very keen to support research in their trust. Although the clinical teams agreed to this happening, often the decision was made by a key senior individual (who often then became the IMPART lead) without much evidence of team consultation. In one trust (Lakes), this person then left the team during the study period.

Several consequences stemmed from this lack of strategic engagement. First was that teams were sometimes being asked to adopt several new approaches in parallel, and change in many directions at once. For example, staff in Seashore were being asked to integrate a new DHI to support service user recovery at the same time as learning about REACT, which caused some confusion. Second, because the intervention had no clear clinical ‘home’ and was not adopted by any specific professional roles, there was no infrastructure or allocated resources to support ongoing sustainability or spread. REACT supporters were not given regular supervision to support their work (> 50% of REACT supporters responding to the costs and savings survey had received < 1 hour of supervision for their REACT role over the entire study period) and, consequently, there were no incentives for them to carry this out. Over time, REACT supporters disengaged from the intervention, leaving the forums often unmoderated. This disengagement exacerbated the fears of clinical staff regarding risk and trolling, which in turn reduced the likelihood of staff to refer relatives to REACT.

In all trusts, there appeared to be a lack of organisational capacity to strategically change practice: to reflect, adapt and reorganise. We noted that teams were constantly fighting to meet immediate service user priorities with low staffing and limited resources. Reported caseloads were higher than the EIP model was designed to accommodate. There was no evidence of any positive impact from the additional EIP funding or the government’s commitment to ‘parity of esteem’ for mental and physical health (although we did not collect comparative data for any physical care services).

Fit with the clinical pathway

In general, EIP teams offered support to relatives in the relative’s own home. Most staff did not have regular access to mobile technology on which they could show REACT to relatives in these face-to-face consultations. REACT was literally out of sight and, therefore, very often out of mind. Even when staff brought REACT to mind, social barriers impeded them from introducing it. Although most relatives had personal computers or tablets, staff did not feel that it was appropriate to ask the relative to log in to their own personal device to be shown the website, particularly as these were often in the bedroom or other private area of the house.

We tried to address this issue by providing (in IPv2) attractively designed REACT NHS booklets that explained what REACT offered and how to access it, and later (IPv3) printable PDF summaries of the REACT modules. These were generally welcomed by staff, and highlighted the complementary benefits of using digital technology alongside other forms of communication.

Staff who did refer relatives to REACT had no subsequent knowledge about whether or not the relative had accepted their invitation and used REACT and, if so, what their impressions of it were. This lack of feedback was demotivating, and led to disengagement among staff. As noted above, our attempt to address this via the newsfeed in IPv3 seemed to have the opposite effect as it highlighted the lack of activity.

Finally, at the time of the study there was no formal way for staff to record REACT-related activity in their electronic note systems, other than as a note in the service user’s electronic record. In all trusts, any staff time spent supporting relatives was recorded in the service user records, as carers do not have an independent record. Consequently, staff caseloads did not accurately reflect the number of people a staff member was working with. As the pressure on systems continues to grow, the cliché that ‘what gets measured gets done’ is likely to become increasingly true.

Measure uptake and use of REACT by NHS early intervention for psychosis teams and relatives

Over the study period of 18 months and across all six trusts, 281 staff accounts were created, 355 relatives’ invitations were sent, 310 relatives were invited and 159 relatives created an account. The most common characteristics of the relatives were that they were white British, a mother and aged > 50 years, and caring for their son/daughter with a diagnosis of psychosis who was living at home.

The mean time spent using the website was 40.6 minutes (SD 54.54 minutes) and the mean number of logins was 3.78 (SD 4.43), although there was considerable individual variation. We do not know how much use of REACT is required for effective change, but these levels of engagement compare relatively favourably with other online interventions, which often show lower than intended engagement118 and in some cases only a minority of participants visited the website more than once. 119

The most common pattern of staff behaviour was 1–2 hours being trained to use REACT (including presentation by the research team), < 2 hours familiarising themselves with the content and < 5 hours referring relatives over the entire study period. REACT appeared as an agenda item for a range of meetings, but was only a regular item at clinical team meetings for approximately half of the staff, and was commonly discussed for < 5 minutes. REACT supporters generally spent longer training and using REACT than other staff did, but received little or no supervision.

Identify critical factors affecting implementation of REACT

Chapter 8 presented a full explanatory synthesis of the critical factors affecting the implementation of REACT. In summary, staff in EIP teams were under great pressure, with high caseloads, several competing priorities and, in some trusts, high levels of staff absence. Staff found it difficult to prioritise time spent supporting carers.

Staff and relatives agreed that REACT could offer an appropriate way to increase access to information and support for carers, but only if it was delivered as part of a comprehensive care package that also included face-to-face support. Staff who used REACT felt that it had facilitated communication with relatives, saw it as a valuable information resource for staff and relatives, and an appropriate way to meet national and local clinical targets, but they did not generally feel that REACT had saved them time in supporting relatives, and may have even increased the amount of contact they had. Staff also reported frustrating practical and technical difficulties.

Relatives were generally very positive about the content of REACT. They valued the comprehensiveness of the modules, and particularly valued hearing about the experiences of other relatives and service users through the videos. However, many reported technical issues and were disappointed by the low level of activity on the forums, making them reluctant to post. This, in turn, made staff less motivated to refer more relatives, creating a vicious cycle working against sustained use.

REACT did not fit with staff members’ current ways of working, which were primarily paper based and community located. This incompatibility was exacerbated by a lack of up-to-date mobile technology. Staff felt that they needed more support and training to use DHIs, and had specific fears about being trolled online and managing risk. Fundamentally, they saw the human relationship as the main agent of change in mental health services, and felt that DHIs somewhat threatened this.

Finally, REACT was offered in the context of the IMPART research study and consequently staff saw the research team, not clinicians, as being responsible for driving it forward. They also recognised that REACT might not be available after the research period and lacked longer-term funding, making them question the value of changing their current working patterns to accommodate REACT.

There was some evidence that the iterative versions of the implementation plans developed during the study (see Chapter 6) led to increasingly more relatives being invited to use REACT. However, the relationship was far from straightforward, with wide variation between trusts.

Some of these factors are consistent with the findings of previous research in this area: specifically, staff concerns about the impact of DHIs on staff–patient (in this case relative) relationships, and the need for appropriate IT equipment, collaborative technology design and training and support have all been identified in other health contexts. 120 The negative impact on staff engagement of poor system design, IT failures and fears that the use of technology will dehumanise care have also been highlighted. 121 Staff fears about managing risk, being trolled online and the negative impact of the research context are unlikely to be particular to REACT, and need further exploration in other contexts.

Identify resources required (and cost implications) for successful implementation of REACT in early intervention for psychosis teams

Here, we outline the main resources required by our final implementation plan (IPv4). We have not allocated specific numerical costs, as these will vary depending on the organisational structure in any one trust (staff costs) and on the number of trusts adopting REACT (cost of delivery per trust).

Implementation plan version 4 (see Chapter 8) was designed to address each of the barriers to implementation and enhance identified facilitators, and hence suggested a fundamental change in the way in which REACT is offered.

Rather than each NHS trust hosting a separate version of REACT, restricted to relatives in their trust only, the intervention could be more easily delivered if it were available nationally from one centrally located and funded source, supported by dedicated trained REACT supporters, and open to all relatives in trusts that adopt the programme. This would overcome the challenges that staff found in dedicating time to the REACT supporter role and in accessing supervision. It would create a much larger population of participating relatives, generating a critical mass to stimulate an active forum. Finally, it would create an effective way to engage relatives as peer workers in the delivery of NHS services as they could be trained as REACT supporters.

Investigate the impact of REACT delivered by early intervention for psychosis teams on relatives’ self-reported outcomes

Relatives had high GHQ-28 scores at baseline, with approximately 60% scoring above the threshold for clinically significant distress. These levels of distress are consistent with those previously reported for relatives in EIP services. 53,122 Between baseline and the 12- and 24-week follow-ups, there was a general pattern of declining distress, social dysfunction, depression, anxiety and insomnia, and increasing carer well-being and eHealth literacy. However, these changes were small and none was statistically significant.

Develop a user-friendly REACT implementation plan and related resources to facilitate widespread use and dissemination

Implementation plan version 4 is described in full in Chapter 8. In addition to the necessary trust resources outlined against the objective Identify resources required (and cost implications) for successful implementation of REACT in early intervention for psychosis teams (staff time and training, merchandise, licence and IT), other key elements include updating the content and functionality of REACT; identifying a sustainable provider and long-term funding model at the national level; and building a mechanism to ensure that the ongoing audit of the reach, uptake and impact of REACT that is not onerous or perceived as research.

Use the findings from this study to further develop theories of implementation of digital interventions in real-world practice

Normalisation process theory73 was used to understand the work undertaken by staff in participating trusts to implement REACT. The NASSS framework74 was used to synthesise these data with the key factors affecting relatives’ engagement and the wider contextual factors. Rather than developing these theories further, we tested the utility of both in understanding the implementation of REACT.

Normalisation process theory proved very useful in informing data collection and analysis, particularly in relation to the work undertaken by the staff. It also worked well in informing the iterative development of IPv2 and IPv3. This was the majority of our data, and so this theory would have been sufficient to develop an effective implementation plan. However, we were keen to develop this plan further to ensure that it also addressed the complexity of the wider context and the engagement of relatives. We therefore used NASSS to synthesise all of the data from the IMPART study. This led to the development of a final implementation plan (IPv4) that differed significantly from our original implementation plan (IPv1).

The NASSS framework builds on many previous theories, and synthesising our data from staff and relatives in the key domains was relatively straightforward. Working across two different frameworks could have led to theoretical confusion, but in practice it worked very well. The theories proved highly complementary: NPT gave us a detailed focus on staff behaviour, and NASSS allowed us to then integrate the key themes identified by NPT with the data from relatives and the broader contextual factors, to present our explanatory synthesis. In combination, they helped us to meet our study objectives.

Study design

The research team had a dual role of delivering REACT while also collecting data to understand the process of implementation. This was confusing for trust clinical staff because it strongly embedded REACT within a research agenda, leading them to assume that REACT would be time limited and driven by the research team. This was a key factor in non-adoption. It also created a tension for the research team. On the one hand, our role was to observe and record, and to intervene only via the strategies outlined in our implementation plans. However, as originators, we also wanted REACT to be viewed positively by staff and, consequently, it was tempting at times to try to facilitate the implementation process. This highlights the need for independent implementation and independent evaluation of DHIs to overcome researcher bias.

Our study design included analysis of data from interviews, observations and document analysis. In the event, the number of data we were able to collect through observation was very limited. We worked with IMPART leads to try to identify meetings in which REACT was likely to be a key agenda item, and asked the meeting chairpersons if we could attend. There were very few meetings in which REACT appeared as a specific agenda item, and, where it was discussed as part of routine clinical practice, the staff did not think it appropriate that we attend the meeting owing to the confidential nature of material being discussed. Therefore, our analysis relied more heavily than we would have liked on interview data and document analysis, with less direct observation being possible.

In the ‘you said . . . we did’ cycles, staff and relatives made suggestions for how the obstacles to using or offering REACT could be overcome. The opportunity to influence the development of REACT was well received and the limits of what could be changed because of time and resources were understood. However, some changes that could not be made, such as offering REACT as a mobile app rather than a website, were repeatedly requested by two trusts (Lakes and Marsh), and this caused frustration for staff as these changes would have overcome the challenge of needing to remember a password.

For relatives, the confusion in our study design came from embedding our research within the clinical delivery of REACT. The invitation to complete questionnaires for the IMPART study initially came as part of the process of setting up a REACT account, leading some relatives to question if this was part (or even the whole) of the intervention. These processes were separated once we became aware of the issue.

Study theory

Normalisation process theory proved a useful theory to guide the collection and analysis of staff data, but was limited (at the time) in helping us to understand relatives’ engagement with REACT or in clearly specifying the wider contextual factors relevant to the implementation of REACT. We added additional codes to our coding framework to accommodate data specifically relevant to understanding the context (within EIP teams, and wider context) and the research process, which was identified as a particularly significant contextual factor. We analysed the relatives’ data using an open thematic approach.

Since the design of the IMPART study, NPT has evolved to more explicitly consider the importance of context. May et al. 123 in particular highlight the interaction between the plasticity of intervention components and the elasticity of contexts in determining success of implementation. Despite this development, we chose to use the NASSS framework to synthesise our findings.

Study process

We were very ambitious in the amount of work we set out to deliver, given the time scale and resources. Our lack of experience in designing and delivering IT projects meant that we underestimated the time and resources needed to deliver this, although this is not uncommon even in large-scale national digital projects. 124 Consequently, some staff were frustrated by IT failures, which, with a part-time IT developer, we could not address immediately. To meet our study objectives, we drew on the additional resources of a Master of Science student (GA-A) who led the systematic review of literature in phase 1, and a Doctorate in Clinical Psychology student (Johanna Barraclough) who led the analysis of interviews with relatives.

We underestimated the speed with which behaviour change can be achieved within the NHS, particularly when the change required is seen as part of a research agenda rather than a clinical agenda. We were unable to interview relatives as early as planned because this depended on REACT first being adopted by staff who could then invite relatives to use REACT. Consequently, these data had less impact on the development of IPv1–3 than planned. This also led to fewer relatives completing the outcome measures within the study period and, therefore, the pre–post analysis of outcome measures was likely to be underpowered to identify any significant changes.

The time frame of the study was limited, including only an 18-month window of observation following the initial deployment of the REACT intervention. Consequently, we were able to observe only the initial stages of adoption/non-adoption and early abandonment. Data collected over a longer time scale would be required to understand scale-up, spread and sustainability. This seems to be a gap in implementation science, with fewer studies funded to understand these stages of implementation. 125 (Note that a request for a funded extension to the IMPART study was rejected.)

Finally, the majority of participants in this study were white British, even in trusts that served populations with high ethnic diversity and, therefore, the generalisability of the findings is restricted to this population. As participants were identified by staff (rather than the research team), this bias may reflect a more general challenge across the services in engaging relatives from minority ethnic groups.

Despite these limitations, our findings have important implications for clinical service delivery, implementation theory and further research, which are outlined in the following sections.

REACT supporters

Six NHS supporters were initially trained to support REACT. All had an interest in mental health. One had been a service user and one was a parent of someone with mental health difficulties. Although feedback about the training was generally positive, one dropped out during training because he had left his post in the service, another dropped out following training because of changes in his role within the EIP team and one was unable to continue owing to ill health. Supporters who remained with the study worked flexibly to cover work across teams, and additional input was required from one of the research team, a trained nurse therapist (REACT supporter 1), to support three participants. Interviews were conducted with the four active supporters at the end of the study: two men and two women, with a mean age of 42 years. Three participants were working or had worked as support time and recovery workers in EIP; the other participant was a nurse, cognitive–behaviour therapist and a family intervention worker. As part of the original study, all participants consented to be interviewed, and for the data to be analysed to understand their experience of their role as REACT supporters.

REACT participants

Design and content of the toolkit

REACT supporters felt that the content of the toolkit was easy to follow and self-explanatory, as well as being bright, colourful, engaging and written in a friendly way. It was also described as being flexible and matching what families needed at that time, making it acceptable to a broad range of people. The REACT supporters reported that the case studies were helpful for participants too, as they were accessible, funny and they felt that relatives could identify with the specific challenges that the characters were facing, including duration of illness and stigma:

The toolkit is written so simplistically it’s so easy for them to follow.

REACT supporter 2

Toolkit’s very self-explanatory.

REACT supporter 3

It’s flexible so you can titrate the support according to need . . . looks official but it’s kind of informal and cool enough to not to put people off. It looks inviting, it’s easy to read. Erm, the content of the material, er, I think is good . . . is accessible, yes.

REACT supporter 1

Despite this, supporters recognised several limitations to REACT. They felt that REACT was helpful for people who were well organised, literate and used to reading or studying, whereas many of the people they visited had more chaotic lifestyles, were not as literate and did not use printed media, and so were likely to find REACT more overwhelming. The provision of static information that was difficult to update was also noted, implying that an online resource would be more appropriate in this context:

Erm, I think it probably needs fine tuning, as soon as you write something down it’s out of date, and I think that is one of the problems with um with printed material, with a printed resource like that is that, um, people are becoming increasingly used to getting information from a variety of sources, not just printed material. So, so I think in the [pause] it has this limitation in that respect.

REACT supporter 1

One supporter highlighted the fact that REACT was unlikely to meet the needs of all relatives, pointing out that:

We’ve got one size toolkit, but people come in different sizes. So it didn’t meet everybody’s needs.

REACT supporter 1

Relatives were also generally positive about what REACT offered:

Erm . . . at that time I was so desperate that I was just delighted to get anything, I had been all over the internet but you know yourself if you have ever done that it’s mostly either for professionals or for illness-specific type things and what have you. When I got REACT manual, I felt better because it showed me, it first of all it related to what was going on in my house.

Relative 024

Positive comments were made about the structure and layout of the toolkit, with relatives describing it as bright and colourful, which made it appealing to read. Relatives described it as ‘well laid out’, ‘very clear’ and ‘well presented’ with ‘nice colours’. Most relatives liked the modular aspect as they felt that they could access the parts they wanted by just looking at that particular section and they liked being able to take out one particular book instead of holding the whole toolkit:

It’s very nice to look at. I thought it was clear, very colourful. I loved the drawing. I liked the humour in it. Erm I like the fact you can take the books out, so you haven’t got to have all of it in your hand. You can find your way round it on your own.

Relative 046

Key aspects of the toolkit that relatives identified positively included creating a list of helpful telephone numbers for use in a crisis; contact with other services, which made them feel less isolated; self-reflection tasks; and case studies:

Examples in the books are good. The examples of the, you know, the case study things. Because they seemed real. And, er, yeah, I just did find it really useful.

Relative 046

One relative liked the real-life examples but felt that some of the solutions needed ‘a saint to react like that’; nevertheless, they found the different point of view interesting, even if they took only part of it away:

But I like the stories that they did. But having said that you know sometimes, the solutions what they used must be a saint that would think like that you know . . . Erm . . . but like one of them was saying it’s trying to ask you how you would react to it, and erm . . . I don’t think I will ever react like that erm . . . but it was interesting. I mean, they were like saints they way they were reacting to it, but it’s nice to somebody else’s point of view and if you just take a part of it away, I think that’s good.

Relative 040

Another suggestion was for more examples from more complex family situations, including managing the responses of other people involved in the family system:

Like a scenario where there is a family of five and there is a step-dad involved, who is not the natural father and he is having to, the poor wife is having to cope with him, as well as him and them, do you know what I mean, there is, there is . . . I have got to be, I have got to be in the middle of them all, got to keep them all happy.

Relative 032

Overall, the content of the toolkit seemed to cater for many of the needs of the relatives in terms of information they were seeking. However, individual relatives did make suggestions of things they would have liked even more information about, including occupational therapy with suggestions of activities, medication, and family therapy and working together as a family.

Timing of the REACT intervention

Although REACT was offered to relatives in EIP teams in the feasibility study, many of these relatives had been supporting someone with psychosis for many years. Both supporters and relatives felt that the toolkit came too late for these relatives, who have already had to try to find information on the internet to answer their questions (which supporters described as ‘pretty scary’) and figure out a way of dealing with things themselves.

One REACT supporter summarised the issue as:

Everybody I met loved it, there was not criticism for the toolkit, except for ‘I’ve already figured this information out for myself, but had this been a year ago . . . would have been a godsend’.

REACT supporter 3

They suggested that to overcome this problem, REACT needed to be included as an early step in the trust care pathway:

So I think what a good idea would have been is, which I know isn’t always practical in terms of the study, but if this was going to be rolled out as standard in the NHS, was as soon as the person is accepted by the EIP team, their next-of-kin carer was identified and approached and asked ‘Would you like one of these support kits to help you?’.

REACT supporter 3

The REACT supporters reported a lot of variation in how often and when they were called on to give support. In general, relatives who were in crisis or facing a lot of difficulties were more likely to use the support offered. A common pattern for those not in crisis was for the supporter to initially see the relative face to face then speak a week later; contact would then drop to fortnightly, then monthly:

With some of them, they didn’t use it initially. And it’s like with anything else, if you know it’s there, some people just didn’t need to use it . . . but if there was a crisis came up or there was someone who had a particularly, um, one of their children were in the services and they were having a particularly turbulent time with them, I’d hear more from them.

REACT supporter 4

Consistent with the views of the REACT supporters, the REACT relatives felt that the toolkit would have been more useful earlier on and might have changed the way they responded. They also felt that they would have been more likely to read it more in the early stages:

If I had had it there, then I wouldn’t have aged 10 years . . . would have been so much easier for all of us . . . taken the abject fear out of it . . . would have made a very big difference.

Relative 024

Difficult times were often a trigger for turning to the toolkit:

If she has been particularly bad sometimes I have looked just looked through it at whatever I have thought at that moment.

Relative 006

However, this was not universal. Some relatives read the toolkit when they received it, and would only occasionally go back to sections if they were prompted to do so by the supporter:

I didn’t need to keep going to it, because like I say once I have read it . . . seemed to stick with me but sometimes, I would talk to [supporter] as he used to ring me and I would say oh, something trivial was bothering me, he said well why don’t you look at section this, section that, and I shouldn’t have to be prompted but, he would and then I would start reading it again then you know . . . I was really yes, into it then.

Relative 032

Structure and delivery of support

Perceived benefits of the toolkit for relatives

Both REACT supporters and relatives were able to identify tangible benefits for the relatives of using REACT.

All of the REACT supporters recognised the advantages of offering information and support to relatives specifically designed for them, focus on their needs, and that empower them to navigate the system they are operating within:

. . . think relatives, and I’ve always said this, they need their own person that they can turn to. Not the care co-ordinator ‘cos you have conflicting, you know, the care co-ordinator’s there for the service user’s best interest. There is no one there and in fact I’ve seen a few times where parents or relatives just get cut out of things. And I just think at the end of the day they’re going to go, they are the main providers of the care, they’ve got to be involved.

REACT supporter 4

Knowing that this support was there, even for those who did not access it regularly, was seen as important:

But I think for a lot of parents, knowing that it was there and that they could ring ‘cos I’d always leave it open.

REACT supporter 4

The main tangible benefit was having access to update to date and accurate information that can be hard to access elsewhere:

It does really benefit if people understand what they are dealing with, it think that’s half the battle rather than if people just don’t understand anything about things, I think they are just fighting against the unknown really all the time.

REACT supporter 2

This was thought to lead to having the confidence to find out more and get more involved with services directly. For example, REACT includes a module outlining confidentiality and how to respond when as a relative you feel you cannot get the information they feel they need, or want to share information you feel staff need to know:

. . . realised that ‘yeah, OK, you can’t tell me the ins and outs because of confidentially but actually I have a voice too, and you probably would learn a lot if you listened to me for 10 minutes’.

REACT supporter 3

. . . saying to them ‘you can ask’ you know and just giving them the confidence through the support to do that.

REACT supporter 3

The indirect benefits of this are hard to determine in these data but there is some suggestion that REACT supporters perceived that engagement with the REACT toolkit might be affecting significant relationships within the family home:

I am not entirely sure if it was the REACT toolkit that changed his perspective, but he was much more . . . I don’t know if it was the support or the toolkit, but it became much more relaxed, much more relaxed as things went on. Uh, I think a little bit more confident that if she were becoming unwell that both she and him would be able to detect, so meaningful changes.

REACT supporter 1

REACT relatives felt that having support specifically aimed at them made them feel more valued, and knowing that there was someone to contact if they did have a problem made them feel more secure:

I need my space to, erm, I suppose to be valued as to what I am doing, do you know what I mean, being his main carer, I think that is all right to be valued as that, do you know what I mean, so I think he gave me that as well. Like, it’s tough this, you know you are doing alright.

Relative 065

Sometimes if nothing had have happened you just feel as if you are just chatting for chatting sake whereas knowing that you could just ring if something did happen was a lot better.

Relative 054

Relatives reported finding REACT reassuring, in that they would recognise strategies that they were already using and this made them feel that they were doing things right:

You can work out things as to what is what, but then it told you, so it confirmed what you had worked out for yourself, that you were actually on the right line.

Relative 008

It also highlighted to them that the challenges they faced were also faced by many other people who were also seeking the same kind of help:

. . . did make me realise that there are other people in the same boat as me, you know, the fact that there is a book actually there, been printed for people who have relatives.

Relative 006

Having contact details for various organisations was described as being helpful, as was having everything held together in one toolkit:

All the information is there in one lump.

Relative 008

Balancing REACT supporter role with general support worker role

The final theme relevant to implementation was discussed by only REACT supporters, and focused on the challenges of integrating their role as REACT supporter with their broader clinical role within the mental health team. REACT relatives talked about the general challenge of finding time to use REACT, but beyond this, did not go into detail that helped identify specific factors that could inform the development of implementation hypotheses for the IMPART study.

In general, supporting REACT seemed to fit well within the role that staff already held within their teams. Delivering the support was not seen as a problem, although attending regular supervision meetings that took place as part of the research project was more problematic, especially as these were held centrally, requiring supporters to travel for up to 1 hour each way.

One of the REACT supporters was also a trained cognitive–behavioural therapist and offered structured family intervention to some families within the trust. For him, the main challenge lay in trying to keep the focus on supporting relatives to use the toolkit, as opposed to offering a more active therapy:

It’s a bit more difficult from my perspective because I was providing sort of general support, rather than therapeutic support, which is what my job is, to be a therapist.

REACT supporter 1

There were also concerns about where the boundaries of the REACT supporter role lay, and when they were doing things to support the relative that might fall outside this, but would fall within a more generic support worker role. For example, one supporter, through talking to the relative, became aware that the person the relative was supporting was not receiving the level of care they should have been from the team. They reported this back to the team meeting, but this was later concerned not to be part of the role:

She’s not blaming me it’s just like, ‘My daughter’s CPN [community psychiatric nurse] hasn’t been here for weeks, um, what can I do about that?’ Or ‘Is that right? Should they be visiting more often?’ or ‘How can I get hold of them?’ or ‘maybe the other person has gone off sick and there’s not a new person here?’. So I think it might not have been in that particular case but in another case I ended up ringing the team involved and saying this is a concern that the relative has passed on, has there been a change of care co-ordinator? What’s happened to them? . . . So, you know, if they wanted a more practical active intervention. Or that might have been more appropriate to the difficulties that they were experiencing at the time.

REACT supporter 1

Finally, supporters expressed some discomfort when put in the position of listening to relatives giving negative feedback about the service, or about specific members of staff. There was uncertainty as to how best to manage this situation, recognising the need for relatives to have a chance to ‘sound off’ but having to choose what to feed back:

I still think that you need somebody that can back it up, so somebody there for the relatives. And I don’t think that that can be like a care co-ordinator, you know it can’t be the same because that’s conflicting. If it was somebody separate that they could just sound off to, you know and say ‘Well, actually I’m not happy about this and I’m not happy about that’ ‘cos I had a lot of that as well . . . I think they forget that I worked for the EIP service, but I was also delivering REACT? You know so I’d get ‘You know so-and-so’s no good and bla-di-blah’. Um, I had to manage that and be very careful.

REACT supporter 4

Summary of findings

Interviews with four REACT supporters and 11 REACT relatives who took part in the REACT feasibility trial53 were analysed to identify the key factors affecting their use of the REACT toolkit. Important factors included the design and content of the toolkit, in particular how relevant the content was to meeting the relatives’ needs, and how accessible and engaging the information and presentation were; the timing of when the toolkit was offered, and the flexibility and availability of the support to match when this was most needed by the relatives; the structure and delivery of the support, including the importance of an initial face-to-face introductory meeting followed by a choice of follow-up support by e-mail or telephone, depending on the relative’s preferences, to prompt relatives to use relevant parts of the toolkit, suggest additional resources where relevant, offer reassurance for appropriate coping strategies, and talk through strategies to manage difficult situations; perceived benefits for relatives, including the reassurance of knowing that they are not alone in facing these problems, that the information and support is available and easily accessible in one place, and the knowledge and confidence this provided to help them manage their situation and engage with mental health services; and the ability of the supporters to define the supporter role and differentiate this from other clinical roles they may have within the team, while also integrating the support with the wider clinical service offered to relatives.

Implications for IMPART study hypotheses

The aim of this analysis was to inform the generation of hypotheses for the IMPART study. First, it is important to understand differences in the design of the REACT intervention that is being used in the IMPART study. These changes have come about in response to extensive feedback from relatives in a series of focus groups54 and through the relatives’ advisory group that has helped to develop REACT.

The REACT toolkit used in the feasibility trial was a modular printed self-management toolkit consisting of 12 booklets, and was supported by a member of the EIP service clinical team. Support included an initial face-to-face session followed by telephone or e-mail contact.

Key changes to the REACT toolkit used in the IMPART study include:

• REACT is offered solely online, with no paper version.

• The content has been adapted so that it is suitable for relatives and close friends of people with psychosis and bipolar disorder.

• The content has been extended to include additional modules relevant to bipolar experiences, and more extensive video case studies of clinicians and actors retelling relatives’ stories.

• REACT is now supported within NHS EIP teams by clinical staff who are able to edit key aspects of the toolkit, including the resource directory where they can add additional resources they feel are relevant. REACT supporters support relatives online using asynchronous direct messages.

• REACT includes a peer support forum on which relatives can talk with each other. This is moderated by the REACT supporters.

Bearing in mind these modifications, we have considered the implications of these findings for the IMPART study hypotheses, and these are reported in Chapter 3.

Setting: stakeholder reference group focus groups

The phase 1 SRG focus groups were held during phase 1 of the study, between March 2016 and June 2016. At the time of the focus groups, NHS ethics and R&D approvals for the IMPART study had been given, and the research team had given a presentation on the study and the REACT toolkit to participating staff teams. These presentations included the rationale behind the IMPART study, an outline of the IMPART study design, a description of the toolkit, screenshots of key content and an outline of staff roles in delivering it. However, the REACT toolkit was in the process of being adapted for use in each site, and trusts did not have access to the website itself. PIs had been confirmed at each site. Each participating EIP service had been advised to allocate two roles (IMPART lead and REACT supporter) to members of their team in order to facilitate the research and implementation. The role of IMPART lead, as specified to teams, were to (1) provide an insight into the workings of a particular site, facilitate researcher access to key data sources, and meet regularly with the case study researcher, and (2) identify potential members of, assist with setting up, and co-chair the SRG. It was recommended that the REACT supporter role included (1) promoting REACT to other staff and to relatives, (2) moderating the REACT online peer support forum REACT group (check users are adhering to ground rules, stimulate discussion) and (3) providing information and signposting through the REACT direct messaging service.

The IMPART leads in each of the six IMPART study sites were approached and asked to set up a SRG meeting. Meetings were held on NHS premises at each of the six sites. IMPART leads were advised that the group should comprise individuals in their trust who were likely to be involved in the implementation of the REACT toolkit, and would ideally include senior trust board members or EIP service leads, EIP team managers/clinicians, two EIP support workers, two EIP relatives and two EIP service users. In some trusts, SRGs were set up almost independently by the IMPART lead. In others, researchers and the McPin Foundation offered varying levels of support.

The meetings followed a similar format across trusts, and comprised a presentation from the research team followed by a group discussion using a semistructured topic guide. The presentation comprised an introduction to the IMPART study [background, need for improved relatives’ support, the evidence–practice gap for online mental health interventions, study aims, design including timeline, quantitative and qualitative data collection, and expected study output (dissemination of findings and an implementation plan)], an introduction to the REACT toolkit (including descriptions and screenshots of key components), the role of the SRG and the aims for the first SRG meeting.

Participants

Participants in each SRG are shown in Table 26. SRGs in four trusts included both EIP staff and relatives and/or service users. In two trusts, relatives and service users were not invited to the initial SRG meeting, as staff felt that it was important to have an initial discussion with staff before consulting with relatives. In these trusts, a second SRG was held to include relative and service user SRG members. Senior management was noticeably absent from meetings. In most trusts, IMPART leads felt that it was unlikely that senior management staff would see their attendance as necessary. In others, senior management staff were approached but did not respond, or expressed an interest in attending but were unable to attend owing to other commitments.

Data collection

A semistructured topic guide was used to ensure that key topics were discussed in each SRG while remaining open enough to allow exploration of related views and experiences. Discussion was led jointly by the IMPART lead and researchers.

The topic guide was designed by the research team to elicit views on:

• current service provision for carers

• the REACT toolkit

• factors that might influence whether or not staff promote the REACT toolkit

• who will deliver the REACT toolkit

• factors that might influence whether or not relatives use the REACT toolkit

• what additional resources might be needed for successful implementation.

Stakeholder reference group members’ names and roles/job titles were also recorded.

Intervention factors

Organisational structures