An intervention to support adherence to inhaled medication in adults with cystic fibrosis: the ACtiF research programme including RCT

The clinical problem

Cystic fibrosis (CF) is the commonest inherited, life-limiting disease in white populations. In 2018, 222 new cases of CF were diagnosed, totalling 10,509 people with cystic fibrosis (PWCF) in the UK, with a median age at diagnosis of 2 months. 1 Although CF is a multisystem disorder, the upper and lower airways and digestive system are more commonly affected. 2,3 Gene mutations lead to clinical manifestation of the disease whereby secretions in the lungs become congealed and sticky. 4 In > 80% of cases, PWCF experience respiratory failure due to lung damage. 5 In the UK, the median predicted survival for a child born with CF is currently 47.3 years. 1

Cystic fibrosis management

Preventative medications both preserve lung function and reduce pulmonary exacerbations. 6–12 Typical treatments for CF target airway clearance, reduction in bacterial infection and inflammation. Patients self-manage their disease through a complex, time-consuming daily regimen of treatments that include inhaled therapies delivered via a nebuliser. 13,14 As with other long-term conditions, adherence levels to medication are low. Estimates by the World Health Organization suggest that 30–50% of prescribed medicines are not taken as intended. 15 In CF, observed objective adherence rates to nebulised therapies are 36% in adults16 and 67% in children,17 despite self-reported rates of adherence of 80%. 16 Treatments work only if they are taken; poor adherence predicts periods of exacerbation requiring intravenous antibiotics (IVAB). 18,19 Exacerbations carry both the burden of systemic side effects and significant mortality;20,21 frequent exacerbations are linked with accelerated decline in lung function and increased 3-year risk of lung transplant or death. 22

The cost of non-adherence

The cost of pulmonary exacerbations is high. The total UK spend in 2012 for CF was estimated to be £100 million, £30 million of which was spent on inhaled antibiotics and mucolytics (Paul McManus, Lead Pharmacist, South Yorkshire and Bassetlaw Area Team, NHS England, 2015, personal communication). In the UK, CF patients received 171,907 days of IVAB, with 93,455 days occurring in hospital, at an estimated cost of £27 million (Stephanie McNeil, Chief Statistician, UK Cystic Fibrosis Registry, 2016, personal communication). Medicines possession ratio (MPR) data suggests that PWCF with good adherence (MPR > 80%) have lower health-care costs than PWCF with poor adherence (MPR < 50%). Hospital admissions for IVAB are responsible for most of the excess costs. 23,24 At the heart of enabling preventative therapy is the task of making adherence visible and allowing this key metric to be available to both PWCF and their clinical teams to move from rescue to prevention.

Potential for cost savings

In the early stages of the programme grant reported here, Tappenden et al. 24 conducted an early, model-based cost–utility analysis in adults with CF with chronic Pseudomonas aeruginosa infection in a project funded separately from the programme grant. Therapies included nebulised or dry powder inhaled antibiotics prescribed by the NHS and Personal Social Services (PSS) over a lifetime horizon. The analysis suggested that an intervention such as ‘CFHealthHub’ had the potential to generate cost savings.

Interventionists

We adapted our initial plans to employ physiotherapists as interventionists because, early in the programme, CF clinicians identified that it was challenging to recruit physiotherapists because of workforce shortages. Intervention training was adapted to ensure that a range of health-care professionals could deliver the intervention competently through the use of tests and certification. Consequently, any member of the multidisciplinary team (MDT) could be trained to deliver the CFHealthHub intervention. Additional funding from NHS England allowed us to increase the amount of interventionist time from 0.8 to 1.0 whole time equivalent at randomised controlled trial (RCT) sites in England.

Primary outcome

The definition of the primary outcome – number of exacerbations – was changed from the standard Fuchs criteria (IVAB plus 4/10 symptoms)28 to the modified Fuchs criteria (IVAB plus 1/10 symptoms) to increase the sensitivity of the primary outcome, in line with other, recent CF research. 29

Number of centres and treatment costs

As a way of paying treatment costs, the NHS England patient activation quality improvement scheme authorised the CF Commissioning for Quality and Innovation (CQUIN). The CQUIN was offered to all 26 adult CF centres in the UK, adoption of which was a means of engaging with quality improvement by delivering the full-scale RCT in our programme. A total of 19 sites participated in the RCT, nine of which were funded by the CQUIN and 10 by a combination of local and central Department of Health and Social Care commissioning.

Fidelity

Fidelity checklists together with audio-recording sessions were used to assess treatment fidelity. CFHealthHub also records exactly what intervention components have been prescribed at each session, and the extent of participants’ independent engagement with the intervention. Interventionists were trained to audio-record all intervention delivery sessions of which a purposive sample were assessed based on the type and time point (phase) of the session administered. Data were analysed and rated for fidelity by two independent assessors for inter-rater reliability.

Changes to work packages

The WPs completed are shown in Figure 1. Changes to WPs included:

• Combining distinct WPs together – WP 2.1B was completed as part of integrating the findings of WP 2.1A with WP 1 to develop the CFHealthHub portal.

• Creating a smaller, distinct WP [WP 2.2B(1)] within WP 2.2B to allow us to recruit an initial set of participants to start refining the intervention.

Service user initiation

Dan Beever, a co-applicant on the ACtiF programme, is a PWCF and was the PPI panel chairperson throughout the programme. Dan Beever was involved in providing patient input to research design at the bid stage. The remaining panel consisted of five PWCF and two parents of PWCF. The initial PPI panel consisted of PWCF members from the Sheffield CF centre who were involved during the intervention development phase. To establish a more diverse panel, membership was extended to PWCF from across the country. A leaflet campaign was used to recruit patients, carers and other people with an interest in CF through sites involved in WP 2.1C. In addition, the opportunity was advertised on the People in Research website (URL: www.peopleinresearch.org/; accessed 12 May 2021), and contact made with some individuals known to Dan Beever. Roles and responsibilities of the group were discussed at the first meeting. During the course of the programme, the study manager provided regular updates on programme progress to allow co-learning30 and maintain engagement.

Development phase (work packages 1.1 and 2.2)

A number of frameworks and approaches that were used during the intervention development phase necessitated input from the PPI panel. Use of the agile software development approach required iterative cycles of development and feedback from both the research team and the PPI panel. This included giving feedback on the proposed input of the educational content, the CFHealthHub landing page and demo versions of the CFHealthHub website and giving advice on practical issues of sharing data on the platform. The PPI panel ensured that the CFHealthHub user guide was accessible and coherent. Frameworks to guide decision-making and prioritisation of work implementation in this process required PPI input to understand which tasks were the most important and relevant to PWCF.

Our PPI panel also contributed to discussions on the themes identified in the qualitative research conducted in WP 2.1 to provide context and additional insights. Feedback on the content and nature of PPI meetings was continuously sought from the panel. Alterations were made to agenda items to include agenda items that the panel wished to discuss, for example ongoing progress.

Feasibility and randomised controlled trial phases (work packages 3.1 and 3.2)

The PPI panel advised on research procedures, in preparation for both the feasibility study and the full-scale RCT. This included input on the best way to contact participants to arrange consent visits and collect prescription data as well as discussions on the most appropriate terminology and communicative approaches to use during intervention sessions and on the CFHealthHub digital platform. The panel advised and provided input on the essential trial documents and their administration with participants. One important element was testing the participant-administered questionnaires, with feedback on completion times and appropriateness of both questionnaire and text message wordings to participants.

Dissemination phase

The PPI panel advised the research team on dissemination strategies across the clinical community and participants. Co-learning facilitated sharing knowledge in a multidirectional way between the study management team and the PPI panel. The RCT findings were presented to the PPI panel members on the Trial Steering Committee (TSC), who provided input on the way in which these findings should be presented to PWCF. One of the PPI panel members with expertise in managing communications assisted with the development of a formal dissemination strategy for feedback of results to multiple stakeholders in multiple ways. The PPI panel agreed the dissemination plan.

Challenges and successes

We had regular contact with the PPI panel but input was limited by the inability of the group to attend face-to-face meetings because of infection control requirements. Visual conferencing facilities were a challenge to use in practice and this was not pursued owing to technical issues. Recruitment of the PPI panel was a challenge but we were able to expand recruitment from the initial Sheffield CF centre to several other CF centres through our leaflet campaign. Involvement was much more frequent in the earlier intervention development phase than during the evaluation phase. Two members of the PPI panel also became members of the TSC and, as such, were able to provide oversight to the programme as a whole.

One of the key successes of our PPI panel was their continued input and engagement, which was sustained throughout the duration of the programme. This is in part a result of having a patient co-applicant on the ACtiF grant who acted as the PPI panel chairperson. Dan Beever’s contributions were of particular value in the recruitment of the PPI panel, as well as in maintaining the interest and engagement of the panel throughout the duration of the programme. A short survey was distributed to members of the panel at the end of the RCT phase to seek their views on their involvement during the course of programme. Three completed responses were received, all reflecting positively on involvement, particularly in terms of contributions being valued by the research team.

Specific impacts

• When PPI panel members used the prototype website they found it difficult to locate appropriate patient videos, so suggested that these be tagged with labels for ease of use (change recommended 14 September 2016). This change was made to the intervention.

• We asked for advice from the panel around contacting potential RCT participants to arrange consent visits. The group recommended sending a text message to the potential participant the day before the planned telephone contact to prepare them, followed by a telephone call the next day, leaving a voicemail message if there was no answer (discussions held 23 June 2017 and 2 October 2017). This approach was used.

• A guide was produced for people using the intervention. The panel recommended a number of changes, for example referring to it as a ‘guide’ rather than as a ‘how to guide’ and being clear that the ‘problem-solving’ related to solving problems with adherence and not with using the intervention (changes recommended 13 November 2017). These changes were made to documentation.

Perceptions of cystic fibrosis and treatments

Acceptability of adherence graphs

People with CF in the sample generally found the charts interesting and easy to understand. They found the different ways of looking at adherence, overall, by week or by time of day, useful because they could see different patterns in their adherence:

I like the graph idea, if it gives you a visual aid. This is basically what you’ve done since your last visit.

P11, moderate adherence

They identified ways of improving the visual attractiveness of the charts. We also identified that there was a need for the research team to identify an approach to measuring adherence in the context of multiple and changing prescriptions.

Design

This was a pilot, open-label, parallel-group RCT with concurrent mixed-methods process evaluation. The pilot RCT has been reported in Hind et al. 41 The qualitative research from the process evaluation has been reported in Drabble et al. 42 and Hind et al. 43

Randomised controlled trial

Participants were PWCF at two CF centres. Inclusion criteria: aged ≥ 16 years and on the CF registry. Exclusion criteria: post lung transplant or on the active list, unable to consent, or using dry-powder inhalers.

Qualitative research

Sarah J Drabble, Samuel Keating and Alexander Scott interviewed intervention (n = 14) and control (n = 5) participants, interventionists (n = 3 on two occasions) and CF team members (n = 5) at the two CF sites. See Work packages 2.1A and 2.1B: a qualitative study – understanding the illness perceptions and treatment beliefs of people with cystic fibrosis for Sarah J Drabble’s credentials and experience. Participants were identified from RCT records and sampled purposively in a similar way to WP 2.1. A total of 49 patients agreed to be approached for interview. We were unable to contact 14 patients and did not actively invite seven, leaving a sample of 28, among whom 23 consented. People declined because they were too busy, had withdrawn from the intervention or did not show up for the interview. Patients were interviewed mainly at home and staff in their workplace or by telephone. The patient sample comprised nine male and five female patients, aged 17–69 years, across all deprivation levels and had fewer people with low levels of adherence than we wanted. The topic guide was used in the process evaluation of the full RCT is shown in Report Supplementary Material 8. All interviews were audio-recorded and field notes were taken and these were analysed using NVivo. Interviews lasted between 11 and 102 minutes (mean 56 minutes).

Interventions

Central randomisation using a computer-generated pseudo-random list and random permuted blocks of varying sizes (2, 4 and 6), stratified by site and number of IVAB days in the previous 12 months (≤ 14 days and > 14 days) on a 1 : 1 allocation to (1) the intervention arm, which involved linking a nebuliser with data recording and transfer capability to a software platform and strategies to support self-management with trained interventionists (n = 32), or (2) the control arm, which involved typically face-to-face meetings every 3 months with the CF team (n = 32).

Outcomes and processes

Trial feasibility was defined as recruitment of > 48 participants (75% of target) in 4 months (pilot primary outcome), valid exacerbation data available for > 85% of those randomised (future RCT primary outcome), change in percentage of medication adherence (secondary outcome in future RCT), use of CFHealthHub and positive perceptions of the intervention from qualitative interviews.

Objective

The objective of the RCT was to determine the effect of the CFHealthHub intervention on clinical and participant-reported outcomes.

Sample size

Sample size estimation was conducted using a between-group difference in mean exacerbations of 0.5 over the 12-month follow-up period, a standard deviation (SD) of 1.5, a design effect of 1.16 to allow for clustering, an alpha level of 5% and 90% power. After adjusting for 20% loss to follow-up, the recruitment target was 556 participants (278 per arm). The sample size was predicated on 2.0 exacerbations per year and reducing this by 0.5 to 1.5 per year. This is equivalent to an incidence rate ratio (IRR) of 0.75 (2.0 ÷ 1.5).

Participants

Potential participants were identified using the UK Cystic Fibrosis Registry. Eligible participants were aged ≥ 16 years and willing to take inhaled mucolytics and antibiotics via the eTrack nebuliser. Participants were ineligible if they were post lung transplant, on the lung transplant list, receiving palliative care, lacking capacity for informed consent or using dry-powder devices to take mucolytics or antibiotics.

Intervention and allocation

Intervention participants received the intervention described in Work package 2.2: development and refinement of the CFHealthHub intervention and Appendix 2. The intervention was delivered by full-time interventionists employed specifically for the research study to deliver both the intervention and the RCT (recruitment and some data collection). They were physiotherapists in 13 of the 19 centres and nurses, psychologists, a pharmacist and a dietitian in other centres. Some centres had two interventionists that shared the role, sometimes from different clinical disciplines. Control participants were given an eTrack controller and Qualcomm (San Diego, CA, USA) hub to enable accurate recording of inhalation data and calculation of adherence levels. They did not have access to CFHealthHub, that is, its adherence data, behaviour-change tools, educational content and visits from interventionists. Control arm participants received usual care.

Participants were allocated 1 : 1 to the intervention arm or control arm using a computer-generated pseudo-random list with random-permuted blocks of randomly varying sizes, via a central, web-based randomisation system. The allocation sequence was hosted by the Clinical Trials Research Unit at the University of Sheffield (Sheffield, UK), with the sequence created by a statistician (not otherwise involved with trial) and held on a secure server. The recruiting health-care professional logged into the server and entered basic demographic information, then the allocation was revealed. Stratification was by centre and number of past-year i.v. antibiotic days (≤ 14 days and > 14 days) – a predictor of current-year i.v. days. 21 The trial statistician remained blind to treatment allocation until database freeze. Participants and health-care professionals collecting primary outcome data were not blind to treatment allocation. The trial statistician remained blind to treatment allocation until database freeze; analyses were conducted unblinded.

An intention-to-treat approach was used, with all participants included in the arm to which they were randomised and exclusions being made only in the event of insufficient data for inclusion in the model for a given outcome. In addition, per-protocol and complier average causal effect (CACE) analyses were conducted, with protocol compliers defined as participants participating in both a first intervention visit and a review visit during which adherence graphs and/or charts were accessed.

Outcome measures

The primary analysis consisted of a between-group comparison of pulmonary exacerbation rates over the 12-month period from consent, with exacerbations defined as meeting at least one of the 12 Fuchs criteria and being treated by i.v. antibiotics. 47 The following sensitivity analyses were conducted to assess the robustness of the findings, applying the same model as for primary analysis: inclusion of all (including those not treated with i.v. antibiotics) exacerbations, multiple imputation for missing outcome data, best-case imputation, per-protocol analysis and CACE analysis (see Report Supplementary Material 1 and 2).

Key secondary outcomes included weekly medication adherence, forced expiratory volume in first second (per cent) (FEV₁%) predicted and body mass index (BMI). To calculate numerator-adjusted normative adherence, daily doses taken were recorded, capped at the number of doses prescribed if the participant took more than the prescribed dose, divided by the appropriate daily dose given the participant’s disease status and treatment regimen, and summarised as weekly means. Lung function and BMI were measured at baseline and 12-month follow-up visits. Health-related quality of life (HRQoL), beliefs and perceived behaviours were assessed by way of the following patient-reported measures:

• generic health status – EuroQol-5 Dimensions, five-level version (EQ-5D-5L)

• Patient Activation Measure-13 item (PAM-13)

• Confusion, Hubbub and Order Scale-6 item (CHAOS-6)

• perceptions of treatment adherence – Medication Adherence Data-3 item (MAD-3)

• Self-Report Behavioural Automaticity Index (SRBAI)

• Cystic Fibrosis Questionnaire-Revised (CFQ-R)

• Generalised Anxiety Disorder-7 (GAD-7)

• specific concerns and necessities – Capability Opportunity Motivation – Behaviour Beliefs About Medicines Questionnaire (COM-BMQ)

• Patient Health Questionnaire-8 item (PHQ-8).

Patient-reported outcomes were recorded at baseline and 12 months.

Participant safety was assessed by way of adverse and serious adverse event reporting. All randomised participants were included in safety summaries.

Statistical analysis

The statistical analysis plan is detailed in Report Supplementary Material 1. Analysis is summarised in this section.

Baseline and safety data were reported using summary statistics.

For the primary outcome (and associated sensitivity analyses), pulmonary exacerbation rates were compared using the IRR from a negative binomial model adjusted for stratification factors and including an offset for follow-up time.

Weekly numerator-adjusted normative adherence data were analysed using a longitudinal mixed model with random slopes and intercepts and adjustment for stratification factors and ‘baseline’ (weeks 1 and 2 post consent) adherence. The treatment effect was quantified using the adjusted between-group difference in mean normative adherence. Other secondary outcomes were analysed using 12-month follow-up data adjusted for baseline values and stratification factors. Treatment effects were determined by adjusted between-group differences in means.

For all models, treatment effects were reported with corresponding 95% confidence intervals (CIs). No adjustments were made for multiplicity. Adjustment for multiplicity was not specified in the statistical analysis plan, which was written, in accordance with the Clinical Trials Research Unit at the University of Sheffield (Sheffield, UK) standard operating procedures, before the data were analysed and was reviewed and approved by the independent members (which included two statisticians) on the TSC. There is no consensus on what procedure to adopt to allow for multiple comparisons. 48 Therefore, we followed Altman et al. ’s49 recommendation of reporting unadjusted p-values (to three decimal places/significant figures) and confidence limits, with a suitable note of caution with respect to interpretation. As Perneger concludes: ‘simply describing what tests of significance have been performed, and why, is generally the best way of dealing with multiple comparisons.’50

Analyses were conducted in R v3.6.1 (The R Foundation for Statistical Computing, Vienna, Austria) and SAS^® v9.4. (SAS Institute Inc., Cary, NC, USA).

We followed Consolidated Standards of Reporting Trials (CONSORT) reporting guidelines (see Report Supplementary Material 5).

Fidelity

The aim was to explore the fidelity of the intervention in practice. We used the Borrelli checklist54 as the framework to assess and monitor the fidelity of the intervention delivered in the RCT. Intervention sessions were assessed for fidelity at certification and for drift. CFHealthHub data on use of different parts of the website (click analytics) were included in this assessment. Details of the fidelity assessment methods, together with the fidelity results, are available in Appendix 3.

Usual-care survey

The aim was to understand usual care in each of the RCT sites, and how it changed over the time of the RCT, to assess how different the intervention was from usual care. We used an 11-item survey at baseline and at 12 months at each RCT site. Questionnaires were completed by the site interventionist and/or other members of the MDT. Questions included a mixture of items requiring five-point nominal scale and free-text responses. Medians, interquartile ranges and percentages by response were used to summarise categorical items. Free-text responses were summarised by identifying key themes. To examine change in usual care at sites over the course of the 12-month follow-up, change scores were calculated. All sites responded to the survey. Details of methods are reported in Appendix 4.

User acceptability survey

The aim was to measure the acceptability of different components of the intervention. We asked 11 questions about the perceived helpfulness of different components of the intervention in the 12-month follow-up questionnaire for those in the intervention arm. The questionnaire was either posted and handed to PWCF who had had the intervention for completion in the presence of the interventionist. A total of 257 out of 305 (84%) participants in the intervention arm responded. Details of methods are reported in Appendix 5.

Trial monitoring data

The aim was to monitor RCT progress in terms of numbers of people approached, reasons for not agreeing to participate in the RCT and numbers withdrawing from the intervention. This allowed us to consider reach and engagement.

Qualitative research

The aim was to explore perceptions of the intervention in practice. We sampled patients purposively using a similar approach to WP 2.3. A total of 84 patients agreed to be approached for interview. We were unable to contact 37 patients, and did not approach 12, leaving a sample of 35. A total of 32 patients consented and 22 were interviewed. Some patients declined and others said that they were too busy to participate; three were unwell on the day of the interview and one died. We approached and interviewed all 26 interventionists. We approached nine MDT members and did not get a response from four, so interviewed five.

We undertook face-to-face interviews with 22 intervention users in seven CF centres, 26 interventionists (some sites had more than one) and five members of the MDT who acted as principal investigators for the study at five RCT sites. Patients comprised 10 male and 12 female patients, aged 19–58 years, across all deprivation levels and all adherence levels.

The interviews were undertaken by Sarah J Drabble (see Work packages 2.1A and 2.1B: a qualitative study – understanding the illness perceptions and treatment beliefs of people with cystic fibrosis for her credentials) and Elizabeth Lumley, a female clinically trained qualitative researcher educated to master’s level with no experience of CF research. The relationship between researchers and participants, and the approach taken, was similar to those in WPs 2.1 and 2.3. The topic guide for PWCF included questions relating to acceptability of different aspects of the intervention and what aspects of the intervention, if any, helped them to increase their adherence. The topic guide for interventionists included questions on the delivery of the intervention, the trial processes and aspects of the context (see Report Supplementary Material 2 for both topic guides). Interviews were audio-recorded and field notes taken. Interviews lasted between 17 and 83 minutes (mean 42 minutes).

We used framework analysis,31 deductively coding to the TDF,32 mechanisms of action including Vassilev’s telehealth mechanisms of action,55 different components of the intervention and its delivery, and inductively to context. Three researchers (SJD, EL, AS) coded the data in NVivo. No participant checking occurred; the findings were discussed with the PPI panel.

Mediation analysis

Structural equation modelling was undertaken on the RCT data to identify the mechanisms by which the CFHealthHub intervention could have influenced medication adherence. Prior to analysis, a logic model (see Figure 3) was constructed to map the anticipated mechanistic pathway, along with potential effect moderators (including two-way interactions) from which a provisional directed acyclic graph (DAG) was created. The factors were further screened for inclusion prior to fitting the model by graphically assessing two-way associations and, for potential mediators, by calculating mean differences between the randomised arms. Factors with little apparent association (defined as an absolute correlation of < 0.1 or a mean difference < 0.1 SDs) were removed from the DAG prior to model fitting. Factors identified as potential mediator–outcome confounders were included in the model as fixed-effect covariates. Pearson’s correlation coefficients with their 95% CI (calculated using Fisher’s z transformation) were used as a guide to identify relationships between mediators. Model fit statistics comparative fit index (CFI) and root mean squared error of approximation (RMSEA) were used to select the final model. In addition, 95% bootstrap CIs were used to estimate the indirect effect of the chosen mediators as well as the direct and total effect of the intervention on medication adherence. Sensitivity analysis was carried out, removing intervention arm participants whose follow-up overlapped with the intervention system being unavailable owing to technical difficulties (i.e. from 20 March to 23 April 2019).

Participant flow

Participants were recruited from October 2017 to June 2018. Participants were followed up until trial completion in June 2019. Participant recruitment and disposition is shown in Figure 4.

FIGURE 4.

The CONSORT flow diagram. a, Exclusions due to missing covariates. Reproduced with permission from Wildman et al. 44 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

Reproduced with permission from Wildman 44 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure

Reasons for declining participation and premature discontinuation of intervention

Common reasons for declining to participate were unwillingness to change nebuliser (125/566) and that the trial would be too time-consuming (118/556). There were 54 premature discontinuations of adherence data collection (control arm, n = 29; intervention arm, n = 25) and 32 premature discontinuations of intervention delivery. Unhappiness with the device/nebuliser or a preference for a previous device was reported as a reason for discontinuation (see Report Supplementary Material 2).

Baseline characteristics

One participant withdrew prior to baseline data collection. Participant characteristics at baseline are shown in Table 3. There were no discernible between-group differences in baseline demographic characteristics. A difference was observed in ‘baseline’ numerator-adjusted normative adherence, which was measured in the first 2 weeks post consent. Participants in the intervention arm had slightly higher FEV₁% predicted and fewer i.v. therapy-days in the prior year. In accordance with CONSORT reporting guidelines, we did not carry out any significance tests of baseline differences. We carried out an analysis adjusted for covariates. We describe this analysis briefly in the statistical analysis section and in more detail in the statistical analysis plan. In summary, we adjusted for baseline stratification factors (site and previous years’ i.v. therapy-days) and baseline value of the outcome (where available) in all statistical models.

Primary outcome

The IRRs from the primary and sensitivity analyses comparing exacerbation rates between the intervention and control arms are presented in Figure 5. The IRR for the main primary analysis was 0.96 (95% CI 0.83 to 1.12; p = 0.638). The point estimate of the IRR is < 1, which favours the intervention arm. However, the 95% CI for the treatment effect included 1, which is consistent with no overall difference in exacerbation rates between the two randomised arms. The sample size was predicted on the assumption of 2.0 exacerbations per year prior to intervention, with a reduction of 0.5 exacerbations to 1.5 per year. This is equivalent to an IRR of 0.75 (2.0 ÷ 1.5). We observed 1.77 exacerbations in the control arm and 1.63 in the intervention arm. If we are looking for a 0.5 reduction in exacerbations from 1.8 to 1.3 then this gives an IRR of 0.72 (1.8 ÷ 1.3). Based on our sample size calculation, a clinically important IRR is between 0.65 and 0.75. Because the lower limit of the estimate (i.e. 0.83) is above this, our result is not statistically significant, and not clinically significant if we believe an important IRR is ≤ 0.75. Findings from sensitivity analyses were consistent with the primary analysis, with 95% CIs encapsulating the null IRR value of 1.

Further details about the exacerbation analysis can be found in Report Supplementary Material 2.

FIGURE 5.

Pulmonary exacerbation IRR from primary and sensitivity analyses. Reproduced with permission from Wildman et al. 44 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

Reproduced with permission from Wildman 44 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

Secondary outcomes

Adherence

The mean weekly numerator-adjusted normative adherence over the course of the RCT is shown in Figure 6. The adjusted between-group difference in mean weekly adherence was 9.5 (95% CI 8.6 to 10.4; p < 0.001) percentage points in favour of the intervention arm. Further details about adherence can be found in Report Supplementary Material 2.

FIGURE 6.

Mean weekly numerator-adjusted normative adherence. Reproduced with permission from Wildman et al. 44 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

Reproduced with permission from Wildman 44 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

Subgroup analyses

See later in Implications for randomised controlled trial subgroup and sensitivity analyses.

Extra analyses: longer-term outcomes

The above analysis is based on 12 months; however, some PWCF stayed in the RCT for up to 21 months. Analysis of the longer-term outcomes up to 21 months post consent back up the results of the primary analysis using 12-month data [no difference in exacerbations or forced expiratory volume in first second (FEV₁)]. There is a difference in adherence of a similar order to the 12-month analysis.

For exacerbations, there was a total of 1326 exacerbations (control, n = 693; intervention, n = 633) during that time. The observed exacerbation rate in the extended post-consent follow-up was 1.70 per year in the control arm and 1.58 per year in the intervention arm (compared with an exacerbation rate of 1.77 per year in the control arm and 1.63 in the intervention arm in the primary analysis). The primary analysis model included adjustments for the previous year’s i.v. therapy-days and site, which were stratifying factors in the randomisation schedule. The estimated treatment effect for this analysis, the IRR, was 0.97 (95% CI 0.84 to 1.12), which is < 1, favouring the intervention arm. However, the 95% CI for the treatment effect included 1, which is consistent with no overall difference in exacerbation rates between the two arms. The estimated treatment effect for the longer-term follow-up is very similar to the primary analysis, with 12-month post-consent follow-up of 0.96 (95% CI 0.83 to 1.12). For FEV₁, the estimated treatment effect was 0.6 (95% CI –0.2 to 1.4) percentage points and the time effect was –0.1 percentage points (95% CI –0.2 to –0.0 percentage points) decline in FEV₁ per month. The direction of effect favoured the intervention arm, but the 95% CI included zero, consistent with there being no difference between arms. There was a small trend for decreasing FEV₁% predicted over time and no significant interaction between randomised treatment arm and time. For adherence, the original mixed-effects adherence model was applied to the extended follow-up data. Increasing the follow-up time from 12 to 21 months increased the estimated treatment effect from 9.5 to 11.9 percentage points (95% CI 11.1 to 12.7 percentage points). The time coefficient was –0.2 percentage points (95% CI –0.2 to –0.1 percentage points), suggestive of a slight decreasing trend in adherence levels over time.

Fidelity (work package 2.3)

For both certification and drift, two persons independently assessed each intervention and the level of agreement between assessors was high. All interventionists were successfully certified as competent to deliver the intervention, including the first visit, review visit and phase review. A total of 110 assessments were assessed to explore drift in fidelity over the duration of the trial and a pass mark threshold of 80% was set for drift assessments. Among all paired assessments during the RCT, there was 97.2% agreement when comparing pass/fail decisions at the 80% threshold (207/213 assessments in agreement). That is, the RCT had good fidelity (overall fidelity by site, range 79–97%), with only one site not achieving over the mean threshold (> 80%) on drift assessments. See Appendix 3 for further details of the fidelity assessment methods and findings.

Usual-care survey

Although most CF centres reported using objective adherence data at baseline, this was described as ad hoc or infrequent at most sites, indicating that our intervention’s systematic approach to measurement was different from usual care. Change scores indicated that usual care within the sites was consistent from baseline to follow-up. There was variation in usual care between RCT sites. See Appendix 4 for further details.

User satisfaction

Among those intervention users completing the satisfaction survey, most rated the following intervention components as ‘very helpful’: the first intervention meeting (77.4%), adherence graphs/tables (68.5%), interventionist support to solve problems (60%) and telephone (58.7%) and face-to-face (67.2%) follow-up visits with the interventionist (Table 7). Videos of other PWCF were rated as less helpful. There was variation between RCT sites. See Appendix 5 for further details.

Trial monitoring

Among the participants approached for the RCT, 48% were recruited. A total of 566 declined recruitment, citing reasons such as not wanting to change the type of nebuliser they used and being too busy (see Report Supplementary Material 2).

The CFHealthHub digital platform was taken down for emergency technical work for 5 weeks (from 20 March to 23 April 2019). It was not available to participants in the RCT in that period. This affected the delivery of the intervention for a minority of PWCF in the intervention arm. This was taken into consideration in the mediation analysis and in a sensitivity analysis for the RCT.

Qualitative research

Process evaluations focus on mechanisms of action, implementation/delivery of the intervention and context. 45 The qualitative research focused on these issues. We report the findings in three parts: one focusing on a single mechanism of action (‘objective adherence data as proof’) (see Appendix 7), one focusing on the range of mechanisms of action operating in practice in the intervention, including some that were not identified in the feasibility study (see Appendix 8), and one focusing on variation in context and implementation between RCT sites/CF centres (see Appendix 9).

During data collection one of the qualitative researchers noted that PWCF and interventionists sometimes talked about the adherence data as ‘proof’. This mechanism of action was explored in detail by analysing codes related to mechanisms of action and identifying different aspects of this mechanism. The objective adherence data were described as offering proof to both self and others about adherence behaviour. PWCF perceived that this could offer benefits, including improving their relationships with their clinical team and their families, if objective adherence was higher than believed by these external parties (see Appendix 7 for further details about the role of proof in improving adherence).

During the feasibility study, we explored the range of mechanisms of action of the intervention. During the evaluation phase we continued to be interested in this. We had coded data to expected mechanisms of action and mechanisms of action associated with effective telehealth intervention and analysed these codes. There was evidence to support expected mechanisms of action around self-monitoring and self-regulation. Other mechanisms of action were also apparent, for example being monitored by others, which some interventionists believed affected control participants as well as intervention participants who mistakenly believed that clinicians could see their adherence data. The relationship between interventionists and PWCF in the intervention arm appeared to be an important mechanism, as found in the feasibility study. Open communication, home visits, continuity of relationship and time helped to build trust between interventionists and PWCF. This trust helped PWCF to talk openly and honestly about the challenges that they faced adhering to treatment. This meant that the interventionists understood more about the real-life challenges faced by PWCF and could help them to find ways to address those challenges. PWCF with high levels of baseline adherence reported gaining reassurance from the intervention. PWCF with very low levels of baseline adherence had challenging life situations that made improvement difficult. PWCF with low to moderate levels of adherence could improve adherence, with action plans to help establish treatment habits, especially if the time was right in their lives. PWCF found the components of the intervention acceptable, but some did not like the patient video clips and some struggled with setting formal action plans. (See Appendix 8 for further details about mechanisms of action).

There was considerable variation between the different RCT sites/CF centres in terms of the backgrounds of the interventionists delivering the intervention at each site, and the way in which the MDT engaged with the intervention and interventionist. That is, the context in which the intervention was delivered varied in the RCT, with the potential to affect implementation. See Appendix 9 for further details.

Mediation analysis

Awareness, habit, concerns, self-efficacy and effort were selected as mediators from the logic model (see Figure 3) because these were associated with both intervention and normative medication adherence. The standardised mean differences of the mediator between the intervention and control arms ranged from 0.6 to 0.2 and mediator–medication adherence correlation ranged from 0.1 to 0.5. Two-way interaction graphs showed that treatment effect had a stronger effect on awareness as baseline prescription increased, and treatment effect on effort was different for men and women. Hence, baseline prescription and gender were included as moderators. Age, gender and baseline measures of FEV₁, number of nebulisers prescribed, depression score, i.v. therapy-days (binary), medication adherence, awareness, motivation and chaos were included as fixed-effect covariates. All selected mediators except awareness had a statistically significant association between each other.

The final mediation analysis model included all the selected mediators, moderators and fixed-effect covariates, demonstrating a good model fit (CFI 1.00, RMSEA 0.00, 95% CI 0.00 to 0.01). The total effect of the intervention on mean normative medication adherence was 13.3 (95% CI 9.6 to 17.0). The direct effect on adherence (i.e. not explained via the selected mediators) was 6.1 (95% CI 2.7 to 9.5). The overall indirect effect of awareness was 5 (95% CI 3.2 to 6.8) and mediated 37.3% of the total effect, but interacted linearly with mean baseline prescription: awareness mediated 18% of the total effect for patients using a nebuliser on alternate days and mediated 58% of the effect for those using six per day. The indirect effect of habit was 1.3 (95% CI 0.4 to 2.1; 9.4% mediated), of self-efficacy was 0.2 (95% CI –0.3 to 0.7; 1.6% mediated) and of concerns was 0.2 (95% CI –0.3 to 0.7; 1.5% mediated). The indirect effect of ease of effort was 0.2 (95% CI –0.1 to 0.5; 1.4% mediated) but interacted with sex: effort had some mediation for women at 0.4 (95% CI –0.2 to 1.0; 2.8% mediated) but little mediation for men at –0.03 (95% CI –0.3 to 0.3; –0.2% mediated). Results from the sensitivity analysis showed a similar trend but with a slightly higher mediating effect as mean baseline prescription increased (14% to 63% for the baseline prescription range 0.5 to 6.0).

The results from the mediation analysis suggests that the intervention helped improve patient’s awareness of their medication usage. This increased awareness contributed to an increased medication adherence, with some evidence to suggest that this effect was more pronounced for patients who used several nebulisers at baseline. Another pathway in which the intervention could have affected medication adherence was by facilitating habit formation, resulting in decreased effort required to take medication, thereby reducing concerns and improving self-efficacy. The total mediated effect of all these mediators was 51%.

See Appendix 10 for further information.

Triangulation for process evaluation

The triangulation grid bringing together all the components of the process evaluation is reported in Appendix 11. This grid was used to draw key conclusions from the process evaluation. The synthesised conclusions were as follows:

• Implementation was very good. The intervention was delivered with good fidelity at all RCT sites with the exception of one. There was one period of 5 weeks towards the end of the RCT when the CFHealthHub platform was not available; this was considered in both the mediation analysis and the RCT sensitivity analysis.

• The intervention was acceptable. The majority of PWCF who completed the survey found it very helpful, particularly the graphs of adherence and the interventionists’ visits. Some caution is appropriate because not all PWCF completed the survey and PWCF may have wanted to please the interventionist, who might have been present at the time the questionnaire was completed.

• The expected mechanisms of action were evident (e.g. self-monitoring), and further mechanisms of action were identified for improvements in adherence to nebulisers. Changes in people’s calibration of their perceived adherence rates affected improvements in objectively measured adherence rates, showing the importance of the objective adherence data.

• The intervention was different from usual care.

• The context in which the intervention was delivered differed by RCT site owing to the differing strengths of the interventionists and the different levels of engagement of MDTs with the intervention.

Implications for randomised controlled trial subgroup and sensitivity analyses

Our proposed RCT subgroup analyses identified from the process evaluation were specified prior to analysis of the RCT data. Based on the process evaluation, we were keen to understand if RCT results differed by:

• good versus poor implementation at different sets of sites

• whether or not the CFHealthHub intervention was available (i.e. where it was not available for 5 weeks owing to technical difficulties)

• levels of baseline adherence.

In practice it was not possible to identify a set of RCT sites with poorer implementation because contextual issues varied greatly between sites. Therefore, we did not undertake this subgroup analysis. The issue about the lack of availability of the intervention for a few weeks was addressed in a sensitivity analysis and the mediation analysis (see Report Supplementary Material 2). The lack of availability of the intervention for 5 weeks did not affect the RCT results.

The baseline adherence subgroup analysis had been specified in our set of a priori RCT subgroup analyses but these planned subgroup analyses were undertaken on the primary outcome only. After our process evaluation we were interested in the relationship between baseline adherence and the key secondary outcomes of change in adherence rates and FEV₁.

Subgroup analyses are shown in Report Supplementary Material 2. There were no statistically significant subgroup differences in number of exacerbations (primary outcome). Post hoc additional subgroup analyses are also shown in Report Supplementary Material 2. The only statistically significant subgroup difference was related to improvement in adherence rates differing by baseline objective adherence (p < 0.001). PWCF with low and moderate levels of baseline adherence had the biggest improvement (18% and 15%, respectively) and high-level adherers had the least (3%) improvement and very low-level adherers had a 10% improvement. Adherence graphs by baseline adherence are displayed in Report Supplementary Material 2.

Within-trial economic evaluation alongside a clinical trial methods

The EEACT was undertaken using IPD from the RCT (Dr Martin Wildman, personal communication) Quality-adjusted life-year (QALY) gains for each patient were estimated as the ‘background QALYs’ accrued based on EQ-5D-5L assessments measured during routine clinic visits, less health losses associated with exacerbations (days spent receiving i.v. antibiotics). EQ-5D-5L measurements were mapped to the EuroQol-5 Dimensions, three-level version (EQ-5D-3L), using van Hout et al. 57

Background QALYs for each patient were calculated under four assumptions:

1. Mean utility was calculated using an area under the curve approach via the trapezium rule, assuming a constant rate of improvement/worsening in HRQoL between successive EQ-5D-5L measurements.

2. For patients with a final EQ-5D-5L visit beyond the primary outcome cut-off, the patient’s EQ-5D-5L score was estimated at day 365 based on assumption 1.

3. For patients with a final EQ-5D-5L visit before the primary outcome cut-off, the patient’s mean utility during the observed period was assumed to apply during the unobserved period (i.e. mean utility is ‘stretched’ over the whole primary outcome window).

4. Patients with missing baseline EQ-5D-5L measurements and those with fewer than two EQ-5D-5L assessments over the primary outcome window were treated as missing.

All QALY losses associated with i.v. days were estimated as the mean loss of utility between the first and last exacerbation-related EQ-5D-5L measurements (‘visit E3’, which took place approximately 4 weeks after the start of the exacerbation, minus ‘visit E1’, which took place at the start of the exacerbation) for the whole patient population multiplied by the number of i.v. therapy-days incurred by each individual patient. Total QALYs gained per patient were calculated by subtracting i.v.-related QALY losses from the background QALYs.

The EEACT includes the following resource costs: the CFHealthHub adherence intervention (applied to the intervention arm only); i.v. therapy-days in hospital; i.v. therapy-days at home; i.v drugs and consumables; visits from health-care practitioners; and nebuliser devices. Resource use estimates were derived from data collected using a standardised resource use form in the trial as well as expert opinion. The costs of nebulised drugs were not included in this analysis. Unit costs were taken from NHS Reference Costs 2017/18,58 the Personal Social Services Research Unit (PSSRU)59 and personal communication [Misbah Tahir, Clinical Pharmacist, Sheffield Teaching Hospitals NHS Foundation Trust, 2019; Carola Fuchs, PARI GmbH, 2019; Pauline Whelan, Farr Institute, and ACtiF Project Management Group (PMG), 2019]. The resource and cost inputs applied in the EEACT are presented in Appendix 13.

Missing data were handled using multiple imputation by chained equations over 10 iterations. 60 A complete-case analysis was undertaken as a secondary analysis. Total QALY gains and costs (with or without imputation) were included in a seemingly unrelated regression (SUR) model,61 which adjusts for baseline imbalances between the two groups in terms of baseline EQ-5D-5L and i.v. therapy-days in the previous 12 months (a stratification factor in the ACtiF RCT).

Within-trial economic evaluation alongside a clinical trial results

The results of the EEACT are summarised in Table 9. The base-case results for the EEACT suggest that the intervention generated 0.01 additional QALYs at an additional cost of £865.91 per patient; the corresponding incremental cost-effectiveness ratio (ICER) is £71,136 per QALY gained. The complete-case analysis suggests a higher ICER of £109,754 per QALY gained.

The results of this analysis should be interpreted with caution for two reasons. First, the short time horizon for the EEACT incorporates all of the upfront costs of the intervention but does not include the potential longer-term benefits that accrue as a result of these. Second, the allocation of the intervention costs across the trial group is unlikely to reflect the actual per-patient costs of delivering the intervention in practice, further inflating costs and associated ICERs.

Model structure and assumptions

The model-based analysis uses evidence from the RCT (Dr Martin Wildman, personal communication) and other external sources to estimate the cost-effectiveness of the intervention versus usual care over a lifetime horizon. The model structure was based on the model used to inform the National Institute for Health and Care Excellence (NICE) technology appraisal of dry powder antibiotics. 62 The model adopts a state transition approach, defined according to five health states: (1) FEV₁ ≥ 70% predicted, (2) FEV₁ 40–69% predicted, (3) FEV₁< 40% predicted, (4) post transplant and (5) death (Figure 7). During each annual model cycle, patients may remain in their current FEV₁% predicted state, transition to an improved or worsened FEV₁% predicted state or die. A small proportion of patients in the worst lung function state (FEV₁ < 40%) may undergo lung (or heart and lung) transplantation and do not subsequently receive further nebulised treatment. HRQoL is modelled according to FEV₁% predicted stratum and transplant history, with QALY losses applied according to the proportion of time spent receiving i.v. therapy. The model uses a 12-month cycle length over 70 cycles; a half-cycle correction was applied to account for the timing of events. Total QALYs are calculated as the sojourn time in each health state weighted by state-specific utility scores, less any i.v.-related QALY losses. The model includes the costs associated with (1) the adherence intervention (intervention arm only), (2) health state resource use conditional on FEV₁% predicted stratum (based on the trial resource use form used in the trial), (3) i.v. therapy-days at home or in hospital (including hospital inpatient stays, i.v. drugs and consumables), (4) nebuliser devices, (5) nebulised drugs and (6) lung or heart and lung transplant.

FIGURE 7.

State transition model structure.

The model employs the following key structural assumptions:

• At model entry, patients are assumed to be aged 30 years, based on the mean age of patients at entry into the RCT (Dr Martin Wildman, personal communication).

• Patients in any FEV₁% predicted stratum can progress/regress to any other FEV₁% predicted stratum during any model cycle.

• Patients who undergo transplantation cannot revert back to other alive health states; hence, the only remaining event for these patients is death.

• Mortality risk is conditional on model health state. General population and CF-specific mortality constraints63,64 are applied to ensure that the survival projection is plausible over the entire time horizon.

• Exacerbation risk is dependent on FEV₁% predicted stratum.

• HRQoL is dependent on lung function and whether or not the patient is receiving i.v. therapy for exacerbations.

• The impact of the adherence intervention is assumed to manifest through two mechanisms: (1) risk ratios (RRs) of switching FEV₁% predicted strata in a given year and (2) a relative rate ratio (RRR) for the number of i.v. therapy-days incurred (which is, in turn, assumed to be constant across all three FEV₁% predicted states):

• The model assumes that the treatment effects and costs related to the intervention accrue only in the years in which the intervention is given.

• In the base-case analysis, the intervention is assumed to be given for 10 years; this assumption is tested in sensitivity analyses.

• The intervention is not assumed to directly affect the probability of undergoing transplantation or death.

• The costs of antibiotics and ‘high-cost therapies’ are independent of adherence to those therapies. This assumption is tested extensively in sensitivity analyses.

Resource costs

The cost of the adherence intervention includes an initial cost associated with training and set-up (year 1 only), ongoing annual costs (including data transfer, monitoring, maintenance and hosting of the data platform), fidelity support and delivery of the intervention by interventionists. The model-based analysis assumes that 30 interventionists can support 5900 adult CF patients receiving nebulised therapies.

Resource use estimates and unit costs were taken from NHS Reference Costs 2017/18,58 the PSSRU,58 the British National Formulary (BNF),72 the ACtiF RCT (Dr Martin Wildman, personal communication) and personal communication (Misbah Tahir, Clinical Pharmacist, Sheffield Teaching Hospitals NHS Foundation Trust; Carola Fuchs, PARI GmbH; Pauline Whelan, Farr Institute, and the ACtiF PMG). Further details of the cost and resource use inputs are provided in Appendix 13.

Model evaluation methods

The model estimates the incremental cost-effectiveness of the intervention compared with usual care in terms of the incremental cost per QALY gained. Central estimates of cost-effectiveness were based on the expectation of the mean derived from the probabilistic version of the model, based on 2000 Monte Carlo samples. Parameter uncertainty was evaluated using both deterministic sensitivity analysis (DSA) and probabilistic sensitivity analysis (PSA). DSAs were used to identify key drivers of the cost-effectiveness. PSA was used to estimate the probability that the intervention is cost-effective across a range of willingness-to-pay (WTP) thresholds. Uncertainty was represented using cost-effectiveness planes and cost-effectiveness acceptability curves.

Validation

A number of activities were undertaken to ensure internal validity and credibility of the model; these are summarised in Appendix 16.

Model-based analysis: results

Table 11 presents the results of the base case analysis; the results of additional sensitivity analyses can be found in Appendix 17. Based on the probabilistic version of the model, the intervention is expected to generate 0.17 additional QALYs and cost savings of £1790 compared with usual care; therefore, the adherence intervention is expected to dominate usual care (i.e. it is expected to be more effective and less expensive). The results of the deterministic model are very similar. Assuming a WTP threshold of £20,000 per QALY gained, the probability that the intervention generates more net benefit than usual care is approximately 0.89.

The DSAs indicate that the conclusions of the economic analysis are sensitive to assumptions regarding:

• the duration over which health effects and costs of the adherence intervention apply

• the impact of the adherence intervention on the patient’s FEV₁% predicted stratum

• increases in adherence to nebulised treatments and associated effects on drug-prescribing levels.

Peer-reviewed journal articles published

Hoo ZH, Curley R, Campbell MJ, Walters SJ, Hind D, Wildman MJ. Accurate reporting of adherence to inhaled therapies in adults with cystic fibrosis: methods to calculate ‘normative adherence’. Patient Prefer Adherence 2016;10:887–900.

Tappenden P, Sadler S, Wildman M. An early health economic analysis of the potential cost effectiveness of an adherence intervention to improve outcomes for patients with cystic fibrosis. PharmacoEconomics 2017;35:647–59.

Hoo ZH, Coates E, Maguire C, Cantrill H, Shafi N, Nash EF, et al. Pragmatic criteria to define chronic pseudomonas aeruginosa infection among adults with cystic fibrosis. Eur J Clin Microbiol Infect Dis 2018;37:2219–22.

Arden MA, Drabble S, O’Cathain A, Hutchings M, Wildman M. Adherence to medication in adults with cystic fibrosis: an investigation using objective adherence data and the theoretical domains framework. Br J Health Psychol 2019;24:357–80.

Drabble SJ, O’Cathain A, Arden MA, Hutchings M, Beever D, Wildman M. When is forgetting not forgetting? A discursive analysis of differences in forgetting talk between adults with cystic fibrosis with different levels of adherence to nebulizer treatments. QualHealthRes 2019;29:2119–31.

Gao J, Arden M, Hoo ZH, Wildman M. Understanding patient activation and adherence to nebuliser treatment in adults with cystic fibrosis: responses to the UK version of PAM-13 and a think aloud study. BMC Health Serv Res 2019;19:420.

Hind D, Drabble SJ, Arden MA, Mandefield L, Waterhouse S, Maguire C, et al. Supporting medication adherence for adults with cystic fibrosis: a randomised feasibility study. BMC Pulm Med 2019;19:77.

Hoo ZH, Gardner B, Arden MA, Waterhouse S, Walters SJ, Campbell MJ, et al. Role of habit in treatment adherence among adults with cystic fibrosis. Thorax 2019;74:197–9.

Hoo ZH, Wildman MJ, Campbell MJ, Walters SJ, Gardner B. A pragmatic behavior-based habit index for adherence to nebulized treatments among adults with cystic fibrosis. Patient Prefer Adherence 2019;13:283–94.

Drabble SJ, O’Cathain A, Scott AJ, Arden MA, Keating S, Hutchings M, et al. Mechanisms of action of a web-based intervention with health professional support to increase adherence to nebulizer treatments in adults with cystic fibrosis: qualitative interview study. J Med Internet Res 2020;22:e16782.

Hind D, Drabble SJ, Arden MA, Mandefield L, Waterhouse S, Maguire C, et al. Feasibility study for supporting medication adherence for adults with cystic fibrosis: mixed-methods process evaluation. BMJ Open 2020;10:e039089.

Arden M, Hutchings M, Whelan P, Drabble SJ, Beever D, Bradley JM, et al. Development of an intervention to increase adherence to nebuliser treatment in adults with cystic fibrosis: CFHealthHub. Pilot and Feasibility Stud 2021;7:PMC7780635.

Wildman MJ, O’Cathain A, Maguire C, Arden MA, Hutchings M, Bradley J, et al. Self-management intervention to reduce pulmonary exacerbations by supporting treatment adherence in adults with cystic fibrosis: a randomised controlled trial [published online ahead of print September 23 2021]. Thorax 2021.

Conference abstracts

Arden MA, Drabble SJ, O’Cathain A, Hutchings M, Wildman M. ACtiF study: understanding adherence to nebuliser treatment in adults with cystic fibrosis using the theoretical domains framework. J Cyst Fibros 2016;15:S26.

Arden MA, Drabble SJ, O’Cathain A, Hutchings M, Wildman M. Applying the theoretical domains framework to adherence to nebuliser treatment in adults with cystic fibrosis. Eur Heal Psychol 2016;18:689.

Sadler S, Wildman M, Tappenden P. Cost-effectiveness of a complex adherence intervention to improve prevention in cystic fibrosis by empowering effective self-management: an exploratory health economic evaluation using UK Registry data. J Cyst Fibros 2016;15:S119–20.

Whelan P. Co-producing Digital Platforms to Support an Ageing Population, Employing Information and Communications Technologies in Homes and Cities for the Health and Well-Being of Older People, Sichuan University, Chengdu, China, 15–17 August 2016.

Whelan P. Using Digital Platforms to Improve the Lives of People with Chronic Conditions: A Connected Health Case Study of CFHealthHub, Mobile Apps & Behaviour Change for Connected Health Seminar, University of Manchester, Manchester, UK, 27 September 2016.

Wildman M, Buchan I, Ainsworth J, Whelan, P. CFHealthHub: Complex Intervention to Support Self-Management in People with Cystic Fibrosis. A Case Study for the Farr Institute ‘100 Ways of Using Data to Make Lives Better’. West De Moines, IA: Farr Institute. 2016.

Arden M, Drabble S, O’Cathain A, Hind D, Hutchings M, Whelan P, et al. CFHealthHub: The Development of a Digital Intervention to Provide Feedback on Objective Nebuliser Adherence Data for Adults with Cystic Fibrosis (CF). 3rd UCL Centre for Behaviour Change Digital Health Conference 2017: Harnessing Digital Technology for Behaviour Change, 22–23 February 2017, London, UK.

Arden MA, Hutchings M, Whelan P, Wildman MJ, CFHealthHub group. Understanding Nebuliser Adherence in Adults with Cystic Fibrosis: Comparing High, Medium and Low Adherers. 31st European Health Psychologist conference, 31 August to 2 September 2017, Padua, Italy.

Hind D, Maguire C, Cantrill H, O’Cathain A, Wildman M. CFHealthHub: complex intervention to support adherence to treatment in adults with cystic fibrosis: external pilot trial. J Cyst Fibros 2017;16:S40–1.

Hoo ZH, Curley RE, Campbell SMJ, Wildman MJ. Cluster analysis of objective nebuliser adherence data among adults with CF: change in adherence over time and association with clinical outcomes. J Cyst Fibros 2017;16:S60–1.

Hoo ZH, Waterhouse S, Nightingale JA, Dewar J, Allenby M, Haynes F, et al. CFHealthHub: using Leeds criteria and clinicians’ decision to determine the pseudomonas status among the 64 adults with cystic fibrosis in the two centre CFHealthHub (CFHealthHub) pilot study. J Cyst Fibros 2017;16:S105–6.

Horspool K, Whelan P, Wildman M. CFHealthHub: iterative co-production of a data feedback platform providing patient level adherence data to multi-disciplinary teams to optimise individual care and centre level adherence data to centre directors allowing benchmarking to enable centre level quality improvement cycles in a 3 centre improvement collaborative. J Cyst Fibros 2017;16:S155.

Kirkpatrick S, Arden M, Beever D, Bradley J, Cantrill H, Daniels T, et al. CFHealthHub: development and evaluation of videos incorporating peer description of successful self-management with inhaled therapies in adults with CF used to build self-efficacy to support self-care within the CFHealthHub complex intervention. J Cyst Fibros 2017;16:S155–6.

Whelan P. Data and mHealth Technologies. Digital Health Outlook, UK and Italy Experiences in a Global Context, 20 March 2017, Milan, Italy.

Whelan P, Hassan L, Fox S, Tempest E. Developing Mobile Applications for Health. Informatics for Health Congress, 24–26 April 2017, Manchester, UK.

Arden MA, Gao J, Hoo ZH, Wildman M. The Patient Activation Measure and Adherence in Adults with Cystic Fibrosis: A Think-Aloud Study. BPS Division of Health Psychology Annual Conference, 5–7 September 2018, Newcastle upon Tyne, UK.

Wildman M, Hutchings M, Arden M, O’Cathain A, Drabble S, Walters SJ, et al. Development and Evaluation of an Intervention to Support Adherence to Treatment in Adults with Cystic Fibrosis: A Randomised Controlled Trial and Parallel Process Evaluation Protocol. Sheffield: Sheffield Teaching Hospitals NHS Foundation Trust; 2018.

Arden M. Health and habits. Psychologist 2019;32:7.

Whelan P, Stockton-Powdrell C, van Velsen L, Tabak M, Wolf K. Interdisciplinary Perspectives on Connected Health Research: Facilitators and Barriers for Sustainable Platform Development. MedInfo: 17th World Congress of Medical and Health Informatics, 25–30 August 2019, Lyon, France.

Other publications

Beever DA, Wildman M. A personal reflection on being a co-applicant in the ACtiF study. Res Involv Engagem 2017;3:12.

Interventionist training, assessment and support

Interventionists received training in how to deliver the intervention in a variety of ways:

1. Training in use of equipment – interventionists received training in how to use the eTrack nebuliser and Qualcomm hub, how to pair the devices and how to register a new participant onto the CFHealthHub digital platform and eTrack system as part of their research procedures training. This was delivered face to face by the study manager and PARI GmbH and supported with a research procedures manual and ad hoc telephone support throughout the trial.

2. Training in delivery of CFHealthHub intervention – interventionists received training in how to use the CFHealthHub digital platform and how they deliver the CFHealthHub intervention. Training was delivered during a 2-day face-to-face training session followed by a schedule of online training to be completed over the equivalent of 4 days hosted on the VLE. Training consisted of presentations, with exercises in small groups or pairs, and supported use of CFHealthHub, role-play delivery of the intervention and discussion. A training version of the CFHealthHub platform was provided for use during training that included dummy data. Interventionists were paired to form buddies for support and additional role play during the online part of the training.

3. Competency assessment – interventionists undertook two competency assessments during the training period: (1) a theory test that assessed understanding of the content of the CFHealthHub digital platform content and data. This test was delivered through an online survey on the VLE and consisted of multiple choice and sort answer questions. The answers were marked according to a pre-determined marking schedule. Interventionists passed if they received a mark of ≥ 80%. Individual feedback was provided on the answers given and, where the first test was failed, additional tutorial support was provided and the test retaken until passed. (2) A practical test that assessed delivery of the first intervention visit of the CFHealthHub intervention. This was assessed through an audio-recorded role play. The part of the participant was played by a member of the study team and the interventionist role played their part. The intervention delivery was assessed using a competency assessment sheet that consisted of sections on preparation, delivery of intervention components and the quality of delivery. Two members of the training team looked at the completed worksheet for the session and listened to the accompanying audio-recording. They discussed the marks and agreed marks where there were any differences. Agreed marks for each section were averaged and the pass threshold was 90%. Interventionists received individual feedback on their performance and tutorial support where they had failed. The test was retaken until passed.

4. Competency to deliver a review visit and a phase review visit were assessed by listening to the first audio-recorded visit of that kind for each interventionist. Two members of the training team looked at the completed worksheet for the session and listened to the accompanying audio-recording. They discussed and agreed marks. Agreed marks for each section were averaged and the pass mark threshold was 90%. Interventionists received individual feedback on their performance and tutorial support where they had failed. The next audio-recorded visit of that kind was assessed where the assessment was failed.

5. Ongoing support – for interventionists this was delivered at a weekly teleconference, by e-mail and telephone support with the training team and through technical support by telephone and e-mail. The weekly teleconference provided a space where interventionists could discuss problems, successes and case studies (anonymised) to aid group learning. Individuals could also access members of the team individually and individual interventionists were targeted with support where they had failed their earlier competency assessment or where there were any problems identified.

Intervention schedule of delivery

The intervention schedule of delivery is described in Figure 13. The content of each kind of intervention session is described in this section. Within this schedule there were a number of different paths that were determined during delivery.

FIGURE 13.

Schedule of intervention delivery.

First intervention visit

This session always occurs face to face, although this can be in a hospital/clinic setting or at home. It lasts between 40 and 60 minutes. It is the first time that the participant accesses the CFHealthHub digital platform and sees their data. Interventionists must prepare for this session by entering the data from the COM-BMQ screening tool and checking that there is data coming through to CFHealthHub from the nebuliser.

The key things that happen in this session are:

• participant receives their log-in details and accesses CFHealthHub

• participant (optionally) downloads the CFHealthHub app onto their smartphone

• the following modules are covered –

  • My treatment

  • Self-monitoring

  • Confidence building

• the following modules are covered for those who want to increase their treatment adherence (sufficiently motivated) –

  • Goal-setting

  • Treatment plan.

Problem-solving intermediate review

The intermediate review is a short session that is designed to trouble-shoot ‘quick’ and easy-to-solve problems (e.g. an action plan that is not working). It is typically delivered by telephone and lasts 5–15 minutes. The review is less structured than other visits.

Ad hoc review

This follows the same structure as the intermediate review but is delivered where there is unplanned face-to-face contact with a participant (e.g. in a clinic).

Review visit

This session typically lasts 30–45 minutes and can be delivered face to face or by telephone. The session focuses on the data and what has happened in terms of adherence since the last visit. The precise focus will vary depending on the individual participant, for example a session with a participant who has met their goal would have a different focus to one with a participant who has not met their goal (or did not set one). The session will cover the following modules:

• My treatment

• Self-monitoring

• Confidence building

• Goal-setting and review

• Treatment plan.

Problem-solving phase review

The focus of this appointment is to facilitate reflection on progress since the intervention (or the current phase of delivery) began and to consider whether continued support is required or the participant wishes to manage their adherence independently. Ideally, this should be delivered face to face but can be delivered by telephone. It typically lasts 20–30 minutes and covers the following modules:

• My treatment

• Self-monitoring

• Confidence building

• Problem-solving.

Fidelity

See Work packages 3.2, 3.3 and 3.4: full-scale randomised controlled trial with concurrent process evaluation.

Question 3

Question 7

Summary

Adherence support was mostly considered very important and adherence was discussed with patients at least sometimes, across all sites. Some method of objectively measuring adherence to inhaled medications was identified in 75% of sites. Qualitative data highlighted that implementation of adherence support varied between RCT sites. Collection and utilisation of adherence data varied in frequency and formality; many sites reported using objective measures on an ad hoc or relatively infrequent basis. Change scores indicated that, on the whole, practice was consistent in usual care over the follow-up period.

Perceived changes in adherence

Most patients in this sample (20/22) believed that they had increased their adherence. Some patients made connections between changes in health and doing their nebuliser treatments, which reinforced the need for treatment. Small adjustments to routines allowed high-level adherers to maintain adherence. Participants perceived that moderate-level adherers increased adherence by establishing routines for additional treatments by making informal or formal action plans with the interventionist. For those with lower levels of adherence, there were more barriers to overcome, which interventionists sometimes struggled to address.

Expected mechanisms of action in different groups of baseline adherence

Self-monitoring was acceptable in this sample of PWCF and was the most used aspect of the intervention, usually accessed through the app. Self-monitoring seemed less important to high-level adherers who knew how they were doing and very low-level adherers who sometimes did not see the need to look at their adherence.

Tailored education and problem-solving were used successfully by some interventionists and PWCF to address individual issues for low- and moderate-level adherers such as unhelpful beliefs about treatments. The videos about how treatments worked were seen as particularly useful to patients and interventionists, with the exception of high-level adherers.

Tailored patient videos were useful to some patients provided that they found someone like them. However, some patients disliked seeing other PWCF.

Personalised goal-setting, goal review and rewards were acceptable and used in visits to set targets acceptable to patients. Lower- and moderate-level adherers often lowered their targets initially to hit their target. Some then increased their target and maintained it, although this could be sporadic to begin with.

Personalised action plans were sometimes problematic as patients did not understand their purpose and struggled to write them in the intervention format. High-level adherers did not need action plans; however, these plans were sometimes successful in forming habits for low- and moderate-level adherers through trial and error and tackling assumptions about when treatments could happen.

Additional mechanisms of action

The patient–interventionist relationship gave PWCF someone to talk to about adherence issues, someone who believed in them, and made them accountable to someone about adherence progress. Most PWCF and interventionists in the sample believed that visits needed to happen face to face and outside clinic visits to enable time to talk through issues with someone who cared.

Others (interventionists and MDT members) monitoring adherence was reassuring and motivating for most participants in the sample, particularly when the interventionist or MDT members praised PWCF.

The right time to receive the intervention was important for some PWCF who increased their adherence; for example, during a time of change, such as a new job, house or relationship, a recent hospital admission or a sense of getting older.