Financial incentives to improve adherence to antipsychotic maintenance medication in non-adherent patients: a cluster randomised controlled trial

Non-adherence and the effect on the individual

The range of negative outcomes for non-adherence with treatment can, in some cases, be severe. It has been suggested that non-compliance with treatment is a feature of at least one-quarter of suicides and homicides by people with mental health problems7 and has been found to be related to an increased potential for assault and dangerous behaviours, particularly during periods of psychosis. 1

Non-adherence and health-care costs

Aside from the effect of non-adherence on the individual, the health-care costs of patients with schizophrenia who do not adhere to medication are reportedly higher than the costs for those who do adhere. 8 It has been estimated that non-adherence accounts for 40% of rehospitalisation costs for patients with schizophrenia in the 2 years after their discharge from inpatient treatment. 9 Knapp and colleagues10 make the argument that given that between 25% and 80% of patients at some point in their treatment fail to take their medication correctly,11 total system-wide costs are likely to be substantial. These authors also found that patients who failed to adhere to their medication regimen were over one and a half times more likely than patients who did adhere to it to report use of inpatient services. Non-adherence can increase external service costs by a factor of three.

Interventions to reduce non-adherence in psychosis patients

Several strategies to help reduce non-adherence for psychotic patients have been developed in recent years. Newer atypical antipsychotic medications, which generally have different side effects, may make adherence easier to achieve and maintain; however, non-adherence can still be substantial, even with newer drugs. 12 Psychoeducation, which aims to inform patients about their condition, as well as family therapy have been found to be largely unsuccessful in promoting adherence. 13–16 In fact, for some patients, increasing their knowledge of their illness, medication and its side effects, may be disturbing and may reduce adherence. 17 Other interventions such as telephone prompting and compliance therapy have been found to have, at best, a very small effect. 18 Adherence therapy, a client-centred approach, which aims to improve adherence with medication by using a cognitive–behavioural approach, is specifically not recommended for people with schizophrenia, as it has been found to have no clear benefits for patients. 19,20

Currently, there is no consistent evidence for any intervention to significantly improve medication adherence in non-adherent patients with psychotic disorders. 3,18 It is against this background that financial incentives have been considered for increasing adherence to medication in non-adherent patients.

Financial incentive research

A great deal of research exists within general health studies which tend to support the effectiveness of financial incentives in shaping and improving behaviour. A systematic review found that 10 out of 11 randomised controlled studies21–31 using financial incentives described positive results for antituberculosis drugs, dental care, weight reduction, cocaine dependence and antihypertensive treatment. 32 The review found that a non-financial method of increasing compliance achieved a better result in only one study. 21

Carey and Carey33 tested the hypothesis that attendance at a day treatment programme could be increased by offering small incentives to patients with mental illness and substance abuse and found a modest increase in attendance over time. In 2006, Post and colleagues34 examined the attendance of 50 low-income African Americans with depression at therapy appointments without incentives for 12 weeks, followed by tracking 12 weeks during which US$10 payments were given at regular appointments. They found patients had better adherence when payments were attached to appointments. Roll and colleagues35 investigated the feasibility of using monetary reinforcement to promote abstinence from cigarette smoking in adults with schizophrenia. Abstinence was significantly greater when receiving money than when not.

Tidey36 conducted a review of studies evaluating incentive-based treatments for promoting tobacco and other drug abstinence, treatment attendance, medication use and increased physical activity. They concluded that given the medical and psychosocial costs of tobacco and other drug use, treatment non-compliance and physical inactivity and the efficacy of incentive-based treatments for improving these behaviours, incentive-based treatments should be further developed and integrated into behavioural treatment programmes for people with serious mental illness.

While these studies all show the benefits of financial incentives within mental health research, none of these studies investigated financial incentives and adherence to medication.

Evidence for financial incentives in mental health care

In psychiatric care, there are few well-designed studies that investigate how financial incentives are used to improve medication adherence. Tentative evidence does exist suggesting that financial incentives are effective at increasing adherence. A systematic review by Burton and colleagues37 focusing on mental health treatments found that the use of incentives was effective in encouraging attendance at therapeutic sessions and adherence to substance abuse programmes. In half of the included studies, the improvement was maintained even after the incentive had finished. There are, however, serious methodological problems with these studies. Only four of them were randomised controlled trials. Only one was conducted outside the USA, which means there is practically no evidence on the effectiveness of incentives within the unique setting of the NHS community psychiatric care system.

Two small sample observational studies have provided anecdotal evidence for the effectiveness of financial incentives for adherence to antipsychotic medication. A small study by Claassen and colleagues38 explored the practice of administering direct financial incentives to improve adherence in assertive outreach teams (AOTs) in England. Financial incentives were offered to five assertive outreach patients in east London. One patient declined the offer. Of the four in the scheme, three patients were not hospitalised after the introduction of the financial incentives. All four patients on the scheme were able to retain their independent accommodation and had fewer problems with their neighbours and the police than before. Only one patient on the scheme asked for the incentive to be increased. This was declined and the patient remained on the scheme. A further observational study was conducted by Staring and colleagues39 in the Netherlands, in which five patients with schizophrenia under an assertive community treatment team were offered financial incentives over the course of 1 year. All five patients’ adherence increased to 100% and only one patient was readmitted to hospital over the course of the year. Two patients in the study asked to receive their medication before their due date, but no problems arose when these requests were denied. While these results suggest financial incentives are effective, such results are anecdotal and few generalised conclusions can be drawn because of the lack of trial data.

Research within behavioural economics have found that financial incentives can lead to changes, or ‘spillovers’ in other behaviours not intended to be the target of the incentives; such as ‘crowding in’ or ‘promoting spillovers’, for which an increase in behaviour target by the incentives also leads to increases in other behaviours, or ‘crowding out’, whereby the likelihood of a behaviour being performed is worse as a result of being offered the incentives, as any intrinsic motivation has been removed. 40,41

The ethics of financial incentives

While there is little evidence for the effectiveness of financial incentives, there also is the question whether or not they are ethical. This is a contentious issue and has been debated widely with financial incentives being named as the key to improving health outcomes by those for incentives, and by critics as a form of bribery and rewarding people for unhealthy behaviour. Critics of financial incentives have argued that they are a form of coercion, which remove freedom of choice and autonomy from the patient who is forced to act out of financial need rather than personal motivation. Shaw42 has for instance argued that financial incentives place health professionals in a coercive role and can lead to the loss of a patient’s personal dignity and privacy.

Burns43 explored this argument, stating that the current debate on financial incentives uses oversimplified generalisations of concepts such as ‘autonomy’ and ‘coercion’, arguing that negotiation between the health professional and patient is a constant reality within mental health care. Financial incentives, rather than being an unethical manipulation of vulnerable patients, in fact offer a new model of respectful and mutual exchange. Ashcroft44 has argued also that too hasty a dismissal of incentives overlooks their potential benefits and may rely too heavily on a naive conception of the individual and their freedom of choice to accept or decline the incentive.

Surveys of clinicians have shown a general distrust towards financial incentives. An online survey conducted separately in the USA and the UK found that financial incentives were judged less acceptable and to be less fair than medical interventions (such as a weekly pill or injection). 45 There was a similarity in negative attitudes in both the UK and the USA. Claassen46 conducted a similar study in which team managers of AOTs were asked about their opinion of financial incentives. A total of 53 out of 70 managers mentioned concerns and expressed a negative attitude towards giving money for adherence to long-term antipsychotic injectable (LAI) medication also known as depot injections. Claassen’s paper makes a number of recommendations including receiving informed consent from the patient, an operational policy for the use of incentives, and randomised controlled trials as well as qualitative studies to evaluate their impact.

A focus group study by Priebe and colleagues4 among different stakeholder groups in the UK identified a number of issues or concerns relating to the use of financial incentives in improving adherence to antipsychotic medication. These concerns fell under four main themes:

1. Wider concerns: the value of antipsychotic medication, whether or not other services may suffer financially if health-care budgets were spent on incentives, how the incentives would be spent by patients and whether or not there were government motives underlying the use of incentives.

2. Problems requiring policies: the practicalities involved in implementing the incentives and how they can be incorporated as part of a clinician’s toolkit.

3. Inherent dilemmas: whether or not offering financial incentives are coercive or whether or not they are fair on others not receiving incentives.

4. Challenges for evidence and experience: whether or not the incentives would perversely incentivise patients to become intentionally non-adherent to qualify for the incentives or whether or not the incentives would affect the therapeutic relationship.

Despite controversial discussions in most groups, all talked about the importance of establishing solid evidence on whether or not financial incentives are effective and emphasised the need for systematic research on the issue. Overall, the debate over the ethics of using financial incentives would be greatly enhanced by more knowledge on whether or not incentives in fact work and how they are experienced by patients and clinicians.

Inclusion criteria

The inclusion criteria for AOTs and CMHTs were that they care for patients with psychotic disorders and had patients who had problems adhering to LAI medication. The exclusion criteria were a lack of willingness to participate and an already existing practice of giving financial incentives to patients.

For the patients in the teams, the inclusion criteria were:

• being cared for by an AOT or a CMHT for at least 4 months

• aged between 18 and 65 years

• capacity to give informed consent to participate in the study and actual written informed consent

• established diagnosis of schizophrenia, schizoaffective psychosis or bipolar illness according to International Classification of Diseases, Tenth Edition51

• being prescribed LAI medication

• poor adherence to antipsychotic medication, that is, having received ≤ 75% of prescribed LAI medication, over the 4 months prior to screening

• failure of all other methods available to the team to ensure adherence to medication.

Exclusion criteria

Exclusion criteria were:

• an intellectual disability

• poor command of English so that clinical communication and discussion of agreements is impaired.

Control group

Patients in teams allocated to the control arm received treatment as usual with no financial or other incentive for taking their medication. The type, frequency and dosage of the medication and other interventions were not affected by participation in this study.

Intervention group

Patients in the teams that were allocated to the intervention were offered a financial incentive for each LAI medication they received for a 12-month period, which was received either at the CMHT or in the patient’s home. Patients received £15 per injection. The total sum that a patient could receive could not exceed £60 for 1 month, as the maximum number of injections is four per month. The administering clinician gave the money in cash directly after the injection and patients signed a receipt on receiving the incentive.

There are several reasons why the sum of £15 was chosen as the incentive payment:

1. A fixed sum per injection simplifies the practice and makes it transparent for all clinicians and patients involved.

2. The sum of £15 is in line with the successful pilot study run in east London. 46

3. The sum is below the £20 per week limit, which would interfere with patients’ disability benefits. Most patients eligible for the study receive Disability Living Allowance, Income Support with Disability Premium or Incapacity Benefit. In all of these cases, patients are not entitled to have a separate income of > £20 without having their benefits reduced. That includes therapeutic earnings and income made through research participation.

4. £15 per injection is intended to be an incentive, helping motivate otherwise ambivalent patients. It is important to limit the total sum to a maximum of £60 per month so that patients do not become financially dependent on the additional income. The money was intended to provide an incentive but not lead to financial dependence on the scheme.

Primary outcome

The primary outcome was the percentage of prescribed LAI medication taken in the community within the 12-month trial period. In calculating the primary outcome, periods of hospitalisation were discounted. Their inclusion would likely inflate adherence under the assumption that patients cannot avoid taking their LAI medication while in hospital. Adherence during imprisonment is less certain, as patients may have the right to refuse their medication. Periods of hospitalisation or imprisonment were therefore excluded, except if the period out of the community was less than the duration of one LAI medication treatment cycle, assuming that patients absent for relatively short periods had a high chance of receiving their prescribed LAI medication in the community.

Secondary outcomes

The secondary outcomes were as follows:

1. The percentage of patients achieving at least 95% adherence, calculated as a dichotomous outcome from the percentage of prescribed LAI medication actually received by the patient during the 12-month study period, again based on electronic and paper records of patients’ receipt of LAI medication.

2. Time ‘slippage’ of taking LAI medication, which is defined as the percentage of the time interval following the day of the prescribed LAI medication that has elapsed before the LAI medication is taken.

3. Patients’ clinical global improvement at the end of the 12-month study period. This was rated by asking the consultant psychiatrist or other clinician overseeing the care of each patient to complete the clinical improvement component of the Clinical Global Impression (CGI) scale. 55 This single-item observer-rated scale asks clinicians to rate the extent of their patient’s improvement with a score ranging from 1 (very much improved) to 7 (very much worse). The score was then dichotomised to give a binary outcome of ‘improved’ or ‘no change, or worse’. CGI scale global improvement scores have been shown to correlate highly with change in depression scores during the course of antidepressant treatment56,57 and discriminate treatment responders from non-responders,58 indicating good convergent validity.

4. Patients’ subjective quality of life, which was assessed at the end of the 12-month study period using the DIALOG scale. 59 The scale consists of 11 items asking patients to rate their satisfaction with eight life domains and three treatment aspects, one of which is medication, on a scale ranging from 1 (lowest satisfaction) to 7 (highest satisfaction). The subjective quality-of-life subscale has been shown to have good psychometric properties60 including high internal consistency (Cronbach’s alpha = 0.71), a meaningful factor structure and good convergent validity with the Manchester Short Assessment of Quality of Life61 (r = 0.95) and divergent validity with the Positive and Negative Symptoms Scale62 (r = –0.37). The treatment satisfaction subscale has been shown to have acceptable internal consistency (Cronbach’s alpha = 0.57) and to have good convergent validity with the Client Satisfaction Questionnaire63 and divergent validity with the Positive and Negative Symptoms Scale. 62

5. Satisfaction with medication.

6. The number of involuntary and voluntary hospital admissions.

7. Adverse events:

   1. attempted and completed suicides

   2. incidences of physical violence

   3. police arrests.

8. Use and costs of inpatient care, outpatient and community mental health care and other health services during the 12-month treatment period, including the cost of the intervention, and in the subsequent 6-month post-intervention period.

Patient interviews

To explore the immediate and long-term impact of receiving the incentives, interviews were conducted at two time points, that is, at the end of the intervention and then 24 months after the end of the intervention. To be interviewed, patients had to receive financial incentives at least once, have a good command of the English language and have the capacity to consent to interview. Patients were not interviewed if they had not received any incentives throughout the intervention, spent the majority of time of the intervention period out of the community (e.g. hospitalised or imprisoned) or did not have the capacity to consent to interview. As the aim of the interviews at the end of the intervention was to explore patients’ views and experiences with the intervention, only those patients who completed the intervention within the previous 4 months were interviewed. No time requirement was used for the follow-up interviews.

An interview schedule consisting of open-ended questions was used at both time points. Questions posed at the end of the intervention included patients’ opinions about receiving financial incentives; whether or not receiving financial incentives affected their relationship with the clinician; whether or not the incentives changed the frequency of patients’ LAI medication appointments; how patients used the money; how they experienced the incentives having stopped; and how that affected them. Questions at the 24-month follow-up interviews included exploring the long-term impact of having received financial incentives. These included long-term changes in attitudes to treatment, relationship with family and friends and attitudes to receiving financial incentives.

At both time points, probing questions were employed for further development or clarification of a topic or a point. All participants were interviewed individually and the interviews were transcribed verbatim by an independent professional; the transcripts were checked for accuracy and any identifiable information was removed.

Clinician interviews

Semistructured interviews were carried out with the clinicians of patients in the intervention arm of the FIAT study. Interviews with clinicians were conducted at four time points to determine the immediate and long-term impact of offering the incentives, that is, 6 and 12 months into the intervention, and 6 months and 24 months after the end of the intervention. Clinicians were eligible for interview if they were involved in the patient’s care throughout the intervention period (e.g. team manager, consultant psychiatrist, care co-ordinator or nurse in LAI medication clinic). However, for interviews at 6 and 24 months’ follow-up, a number of clinicians or patients had transferred to other teams after the end of the intervention; therefore, it was not possible to interview all of the clinicians interviewed during the intervention period. To overcome this, other clinicians not originally interviewed but who were involved in the patient’s care during the intervention, or who had been involved since the intervention finished but knew the patient well enough during the trial itself to be able to answer questions, were interviewed.

An interview schedule with a mixture of yes/no questions and room for further unstructured responses was used at all time points. For interviews at 6 and 12 months into the intervention, questions included how clinicians thought patients spent the money, whether or not patients asked for more money or more frequent LAI medication appointments, the experiences of patients outside the trial (e.g. asked for incentives or become non-adherent to qualify) and the effect of the incentives on both the patient outcome and the interaction with the team. For interviews at the 6- and 24-month follow-ups, questions included whether or not clinicians continued using the incentives with patients within the trial or any new patients, whether or not patients within or outside the trial had asked for the incentives again or had become non-adherent to qualify, the long-term impact of offering/discontinuing the incentives on patient outcomes and interaction with the team, their opinions of the incentives before and after the intervention and the mechanisms underlying how the incentives worked. All responses recorded were written in shorthand.

Settings for data collection and interviews

Primary and secondary outcome data were collected from electronic and paper patient records and medication charts kept at the CMHTs and AOTs. Data on subjective quality of life and medication satisfaction were collected in person through interviews with patients either in their home or at their mental health team base. Qualitative interviews with patients were conducted either within their homes or with their mental health team base. Clinician interviews were conducted over the telephone, by e-mail or in person by a RA.

Issues arising from long-term antipsychotic injectable medication card data

The baseline adherence data were asynchronous with all other baseline data that were collected for the year before team randomisation. However, the LAI medication data were collected for 365 days back from the screening date. This means, when there was a gap between screening and randomisation (mean gap was 61 days, SD 45 days), the earliest LAI medication data collected during the screening period did not coincide with the rest of the baseline data. This was problematic for calculating adherence, as it takes into account periods out of the community greater than one LAI medication treatment cycle. If hospitalisations or imprisonments were unknown for the early part of the screening period, they could not be handled correctly. This issue was addressed by delaying the start of the baseline period to 365 days prior to the point of randomisation for patients with gaps. This was to ensure that the baseline period, although shorter than 1 year, would be contemporaneous with the other baseline data. Figure 2 shows an illustration of the problem with the gaps in LAI medication data and the solution.

FIGURE 2.

Illustration of the gap problem and solution. E, start of the period in which eligibility was assessed, that is 122 days (4 months) prior to the screening date; g, gap between the screening date and the date the team was randomised; I₀, intervention start date, which is fixed at 7 days after the randomisation date; I₁, intervention end date, which is fixed at 365 days after the intervention start date; R, team randomisation date; S₀, screening date, that is the patient’s adherence was initially assessed on this date; S_-1, screening date – 365 days.

Defining the primary outcome

The primary outcome was adherence to LAI medication defined as the percentage of prescribed LAI medication actually taken during a 12-month period. This was defined as the following:

The analysis population, which is available cases following intention-to-treat principles, for the primary analysis of the primary outcome was defined as all randomised patients, in the group to which they were randomised subject to having one continuous period of at least 4 months in the community in both baseline and intervention periods for which depot data were available. Patients who had died or moved away were included providing they met these criteria and had not withdrawn/been withdrawn from the study. Those who had withdrawn from the study or had been withdrawn were excluded.

Definitions:

1. In the community implies neither in hospital (either psychiatric or general hospital admission) nor in prison.

2. Periods out of the community that span less than one cycle are treated as in community.

The treatment cycle possibilities are defined as follows:

1. 1/52: weekly

2. 2/52: fortnightly

3. 3/52: every 3 weeks

4. 4/52: every 4 weeks.

The periods of analysis were:

1. Baseline: the period from 365 days prior to team randomisation date to screening date.

2. Intervention: the period from the intervention start date (i.e. period from team randomisation date + 7 days) to the end of intervention (i.e. intervention start date + 365 days).

3. Six-month follow-up: defined as 1 day after the end of intervention date + 183 days (covering the 6 months after the end of the intervention).

4. Twenty-four-month follow-up: this time period was defined as 24 months after the end of the intervention; however, this also included the 6-month follow-up period. Therefore, this analysis period only included 18 months after the 6-month follow-up, which was defined as 1 day after the end of the 6-month follow-up date + 548 days (covering month 7 to month 24 after the end of the intervention).

Primary outcome

The effect of the intervention on percentage adherence was estimated using a linear mixed-effects regression model with a random effect for mental health team, adjusting for baseline adherence, MINI score category (low vs. high) and average number of weeks between prescribed LAI medication during the baseline period. The main analysis used data from all individuals who had at least 4 months of continuous data during both the baseline and the intervention periods, on an intention-to-treat basis whereby those who did not receive any incentive as intended were also included in the analysis. A per-protocol analysis was also conducted in order to identify the effect of the incentive on adherence in those who received the intervention, using the same model specification as described in Defining the primary outcome. Finally, a sensitivity analysis without adjustment for baseline adherence was conducted, although still adjusting for MINI score category and average number of weeks between prescribed LAI medication during the baseline period, and with a random effect for clinical treatment team.

The effect of the intervention on the primary outcome at the 24-month follow-up was modelled using a simple linear regression model including only a fixed effect for treatment group. The ICC for adherence across mental health teams was negative (–0.05); therefore, including a random effect would have biased the standard error (SE) of the treatment effect upwards. Sensitivity analyses were conducted, with treatment group being the only fixed-effects variable.

Secondary outcomes

Models assessing the effect of the intervention on the secondary outcomes were adjusted for MINI score category and average number of weeks between prescribed LAI medication, and had a random effect for clinical treatment team.

The effect of the intervention on the binary secondary outcomes was modelled using mixed-effects logistic regression (i.e. achieving vs. not achieving at least 95% adherence and improved vs. unchanged or worse on the CGI scale). For each of these models the baseline value of the relevant outcomes was included as a covariate in addition to those described, with the exception of the analysis of clinical improvement, as a baseline assessment with the CGI scale was not conducted.

The effect of the intervention on the continuous secondary outcome satisfaction with treatment was estimated using a linear mixed-effects model, while subjective quality of life was estimated using a random-effects model fitted by generalised least squares, since the mixed model did not converge. In addition to the covariates described, both models adjusted for the baseline score on the outcomes. Secondary outcomes in which low frequencies were expected, that is, hospital admissions and adverse events, were summarised descriptively only.

Economic evaluation: cost-effectiveness analysis

The cost-effectiveness analyses were conducted from a NHS perspective. The study was powered to detect a 20% improvement in adherence; therefore, for the purposes of the economic analysis, the outcome measure was defined as the incremental cost to achieve a 20% increase in adherence to prescribed LAI medication taken over the 12-month intervention period. The incremental cost of achieving ‘good’ adherence (achieving at least 95% adherence to prescribed LAI medication) over the intervention period was also calculated. The modelling approach was also applied to the secondary outcomes of subjective quality of life and clinical improvement (treated as a binary variable as in the clinical analyses).

Service use and costs

Service use considered in the economic evaluation included inpatient, outpatient and community mental health services, general hospital and primary care services, prescribed oral medications and prescribed and received depot medications. The cost-effectiveness analyses examined costs and outcomes at the end of the 12-month intervention period. In addition, health-care resource use over the 6 months after the end of the intervention was collected for descriptive analysis only. All data on service use were collected by the study RAs via the case report form (CRF), from a combination of electronic and paper health records.

Unit costs

Established unit costs (for the 2010–11 year) from national representative and other published sources64–67 were used to estimate the costs of direct health care. Costs of a contact in any setting (office/service and home/community settings) were calculated, as opposed to the duration of contact with a mental health professional, as little duration data were available, drawing on unit costs taken from the NHS reference costs for England. 65 Oral and LAI medication costs were calculated using the prescription cost analyses. 68 The cost of the intervention itself was calculated as the total number of £15 incentive payments given over the study period, while contacts with nurses, including nurses giving LAI medication, were tracked as part of the data collection via the CRF. Feedback from researchers working with the participating teams indicated no other resources were consumed in providing the financial incentives. The number of incentive payments was variable, depending on the number of LAI medication appointments attended by patients. Unit costs are shown in Table 1.

Missing data

Cases treated as missing for the adherence outcome were treated the same way in the economic analysis. Cases were also counted as missing when all hospital or community service utilisation data were missing. The economic analyses drew on complete cases only.

A gap in the LAI medication data arose between randomisation and screening, necessitating imputation of LAI medication costs between the dates of screening and entry into the trial. So that the costs of the LAI medication would be in step with other costs, the period between the 12 months pre screening and the 12 months pre randomisation and the gap between the dates of the last LAI medication cycle and randomisation were calculated. The extra LAI medication cycles occurring in the gap were imputed by last observation carried forward. LAI medication unit costs were attached to the LAI medication units occurring over the pre-randomisation period and to the imputed LAI medication units in the gap, if the gap was as long as or longer than one LAI medication cycle. The total LAI medication cycle costs (including any imputed costs) were then adjusted by applying the proportion of prescribed LAI medication that had been taken over prior 12 months to screening.

Cost-effectiveness analyses

The financial incentive intervention should be considered cost-effective if it is more effective and less costly than the treatment as usual received by the control group or if it is more effective and more costly than treatment as usual and the purchaser is willing to pay the extra cost in order to gain the associated benefit. If the financial incentive strategy is more effective but also more costly, the additional cost per additional benefit produced [the incremental cost-effectiveness ratio (ICER)] would have to be less than the purchaser is willing to pay.

Incremental outcomes and costs were estimated through multilevel multivariate regression models, adjusting for the following covariates: intervention allocation, baseline measure of outcome (except in the case of the CGI scale, measured at end of intervention only), total costs in the pre-baseline year, high/low MINI score category and average LAI medication treatment cycle over the baseline period. The average number of weeks between prescribed LAI medication was controlled for, as some patients’ treatment cycles changed over the period. The modelling allowed costs and outcomes to be correlated both within and between clusters, with random effects for the participating teams as clusters.

Error terms for costs and outcome equations were assumed to be normally distributed. The coefficients on the allocation term in the costs and effects equations (giving the cost and outcome differences, respectively, between groups) were used to derive net monetary benefit values over a range of willingness to pay (£0–30,000) for the additional benefit associated with the intervention. The 95% confidence intervals (CIs) for the ICER were calculated from the model estimates using Fieller’s method. 69 Cost-effectiveness acceptability curves were constructed from the regression results, depicting the probability of the ICER being less than each willingness-to-pay value in the range.

Sensitivity analyses were also employed to examine the impact of altering key assumptions on the cost-effectiveness of achieving both adherence outcomes. The first analyses explored the impact of when patients ‘did not attend’ (DNA) an appointment, either through failing to turn up to an appointment or by not being in during a community visit (planned or unplanned). No data on the duration of contact for DNAs were available, and thus DNAs were excluded from the main analysis. However, it is possible that DNAs could consume a substantial amount of health professionals’ time, and to account for this, DNA costs were incorporated into total costs in a sensitivity analysis, assigning these visits the same unit costs as actual contacts with patients. Another sensitivity analysis explored the impact of the varying level of skill-mix of mental health professionals within teams. As a result of using national reference costs for contacts with professionals from CMHTs and AOTs, the costs may not reflect the variability in skill-mix within these teams or the actual duration of contact that patients had with particular mental health professionals. Therefore, to explore the impact of over- or underestimating these costs and their contribution to the ICER, unit costs of AOT and CMHT contacts were varied by 25%, 50% and 150%.

Qualitative analyses

Primary outcome result

The unadjusted difference in adherence, using data from all 131 patients with primary outcome data was 14% (85% in the intervention group vs. 71% in the control group). Adherence to medication was significantly higher in the intervention group than in the control group [adjusted difference in means (β) 11.5%, 95% CI 3.9% to 19.0%; p = 0.003], representing on average 11.5% greater adherence in the incentives group after adjustment for covariates. See Table 6 for the primary outcome results.

Table 7 shows only those patients with at least 4 months’ LAI medication data at baseline and follow-up. Table 8 shows the distribution of cluster size for clusters in which at least one patient had data at baseline and at the intervention period and Table 9 shows the summary statistics for LAI medication data cohort.

Six-month follow-up

Resource use over the 6-month follow-up (in cases where data were also available at baseline) is shown in Table 21.

There were few differences in resource use between groups in the 6 months after the end of the intervention. The mean number of visits from community psychiatric nurses in service settings were marginally higher in the intervention group (6.5 visits) than the control group (6.0 visits) (95% CI –1.6 to 2.6 visits), and in community settings (intervention 6.3 visits, control 6 visits, 95% CI –2.9 to 3.6 visits). The number of visits from mental health support workers in both community and service settings and the number of DNA appointments for community psychiatric nurses and psychiatrists were somewhat lower in the intervention group. While the number of inpatient admissions between groups did not differ significantly, the mean number of days spent in psychiatric hospital care was somewhat higher in the intervention group (3.7 days) than the control group (0.6 days) (95% CI –0.4 to 6.5 days).

Recruitment

Out of the potential 78 participants, 26 patients consented to be interviewed at the end of the intervention and 32 patients consented to be interviewed at the 24-month follow-up, with 11 patients interviewed at both time points. At the end of the intervention, reasons for not participating included ineligibility (n = 23), not being contacted as sample size had been reached (n = 14), deceased (n = 1), not being contactable (n = 7) or refused to take part (n = 7). Out of the 26 participants who consented to be interviewed, one refused to be interviewed with a recording device and was therefore not included in analysis.

Reasons for not participating in an interview at the 24-month follow-up included ineligibility (n = 8), the sample size had been reached (n = 12), refused to participate (n = 8), having deceased (n = 4), not being contactable (n = 10) and failing to attend an arranged interview (n = 4). Of the 32 patients interviewed, one patient was interviewed despite having received no incentives and was removed from the final analysis.

Patient characteristics

Patient characteristics for 45 patients whose interviews were analysed are reported in Table 22. Twenty-one interviews were carried out on the premises of mental health teams and four in patients’ homes at the end of the intervention, and 25 interviews took place on the premises of clinical teams and six were interviewed in patients’ own homes at the 24-month follow-up. On average, patients were interviewed 5 weeks (between 1 and 12 weeks) and 27 months (between 19 and 40 months) after the end of intervention and at the 24-month follow-up, respectively.

Results

The analysis identified five general themes: ‘structuring the day of the appointment’, ‘relationship with clinicians’, ‘extra expenditure’, ‘personal dilemmas’ and ‘impact once incentives had stopped’. The themes were similar at the two time points, and those patients who were interviewed twice expressed consistent views at both time points.

Clinician interviews at 6 months and 12 months of the intervention

Fifty-nine clinicians were interviewed with regards to the effect of the incentives on 73 of the 78 patients allocated to receive incentives during the FIAT trial. Twenty clinicians were interviewed about more than one patient – and for 19 patients, a different clinician was interviewed at each timepoint. The number of clinicians interviewed and the number of patients for whom interviews were obtained at each time point are presented in Table 23.

Positive effects on clinical management and relationships

Negative effects on clinical management and relationships

The clinicians of 19 patients reported negative effects of the incentives on the patient’s interaction with the team.

Positive Effect on Patient Outcomes

A positive effect was reported for 33 patients. This theme was further broken down into 3 sub-themes: Improved mental well-being; Improved social functioning; Insight.

Negative Effects on Patient Outcomes

The clinicians of 15 patients reported a negative effect of incentives on patient outcomes. These experiences all concerned negative effects of the payment itself, and were sub-divided into two sub-themes: Increased drug and alcohol use, and Other adverse effects of payment.

Link Between Themes

Clinicians endorsed different combinations of themes for each patient as depicted in Table 24. This shows that overall, for 77% of patients a positive effect on management and/or outcomes was reported, whilst for 33% a negative effect on management and/or outcomes was reported. These figures overlap because for some patients both positive and negative effects were reported.

Clinician interviews at 6 months and 24 months after the end of the intervention

Fifty-seven clinicians were interviewed for 59 of the 78 patients, with interviews for 25 of the 78 patients conducted at both time points; however, in some cases the same clinician was not interviewed. The clinicians for 19 patients were not interviewed because of clinician unavailability or because the clinician and/or patient had transferred to different teams since the end of the intervention.

Long-term consequences of the incentives being discontinued

Clinician opinions of financial incentives

Strengths and limitations

One of the strengths of this study was that it involved a large number of mental health teams across the UK, from areas of differing social deprivation. Outcome data were available for the majority of patients, 131 out of 141 (93%), and statistically analysed for 123 patients (87%) at the end of the intervention. The estimated treatment effect was also robust to a variety of sensitivity analyses at the end of intervention, and at the 6- and 24-month follow-up. A substantial number of patients and clinicians were interviewed about their experiences both during the intervention and at the follow-up periods. Patient interviews were available for 45 of the 78 patients (58%) allocated to the intervention group; clinicians were interviewed for 73 of the 78 patients (94%) during the intervention period and for 59 of the 78 patients (76%) were interviewed during the follow-up periods.

Nevertheless, the study does have some weaknesses. There were instances of protocol violations, including of patients in the trial who did not have a diagnosis of schizophrenia or who had > 75% adherence at baseline and were still included in the trial. The study was not blinded. It was impossible to blind patients and clinicians to whether or not patients received financial incentives. It can be argued that the majority of outcomes were collected from electronic databases, and therefore were objective, and that researchers had no knowledge of patients’ allocation. However, systematic efforts could have been made to ensure that researchers were blinded. Particularly, blinding should have been ensured when collecting patients’ ratings of subjective quality of life and assessments of patients’ clinical improvement rated by their clinicians. Future research should seek to address this limitation in their design to keep the scientific rigour as sound as possible.

Furthermore, all teams who participated in the trial consented to the practice of offering financial incentives. Whether or not the incentives would be effective among teams who did not choose to participate in the study is unclear, for example if negative attitudes towards using incentives may have influenced any outcomes.

The study was powered to detect a statistically significant difference in adherence, the primary outcome. However, it was not necessarily powered to identify significant differences in secondary outcomes. While the intervention group still had a significantly better subjective quality of life, the advantage of the intervention group in global clinical improvement, despite being marked, did not reach statistical significance. The sample was clearly too small to test differences in rehospitalisation and other adverse events that are rather rare.

While study retention was high during the intervention period, there was attrition at both follow-up periods for a number of reasons. However, the reasons for unavailability were accounted for (e.g. discharge, refusing medication, hospitalisation and imprisonment) and thus the data are not ‘missing at random’. Furthermore, a number of clinicians were lost at follow-up periods for the clinician interviews and the large majority of interviews were conducted at 6 months only; therefore, the longer-term impact of the incentives may not be generalisable beyond the first 6 months.

The study was not fully powered to address costs, and the full extent of patient costs from a broader societal perspective is unknown. However, the data collected for the effectiveness analysis did not indicate differences in such societally relevant outcomes such as police arrests or taking up education/training courses. Data on service use were collected from patient records within NHS trusts and any service contact with NHS services outside of these trusts or with primary care services is unclear and may be dependent on local information-sharing policies and practice. Furthermore, costs did not reflect the duration of contact with mental health staff or the skill level (and thus cost) of each staff member.

The qualitative interviews and analyses also demonstrate weaknesses. Patients were selected from a convenience sample; therefore, there may be differences in the characteristics of those who consented to interview compared with those who declined. While the majority of experiences were positive, the sampling method may have excluded those who may have had more negative experiences. For example, those patients whose adherence and mental health had deteriorated once the incentives had stopped and may not have had the motivation or capacity to agree to an interview. Moreover, all researchers involved in the data collection and analysis were also part of the FIAT research team. This might have introduced a bias in the way the interviews were conducted or in the analysis of the material.

With regard to the clinician interviews, the way the questions were structured may have biased how the responses were interpreted. There was little opportunity for further exploration if a clinician answered ‘no’ to a question; therefore, if there was no change in a particular outcome (e.g. therapeutic relationship) it was difficult to understand why. The interviews themselves may, therefore, have been more receptive in finding changes in behaviour during the intervention and follow-up periods. There is also the possibility that the answers clinicians gave were merely speculative. For example, one clinician reported the same outcome for five patients during the intervention period, which may have led to some data being conflated. How clinicians believed patients spent their money was also speculative and any links that were made between the incentives and changes in patient management or health may be subjective and not reflect reality. Therefore, making causal attributions between the incentives and any changes must be taken with caution. The interview schedules at the 6- and 24-month follow-ups differed slightly, with some questions into the same aspect being worded differently at both time points and some questions being asked at one time point only. This has two implications: first, the differences in the phrasing of the question may be interpreted in dissimilar ways by clinicians and thus the responses given may have been different; and second, the answers given to some questions reflect one time point only and cannot be taken as a general answer for the long-term effects of the incentives stopping.

Concerning the value of antipsychotic medication, whether or not other services may suffer financially if health-care budgets were spent on incentives, how the incentives would be spent by patients and whether or not there were government motives underlying the use of incentives or ‘wider concerns’

The incentives were spent in a variety of ways and patients enjoyed using the incentives towards some form of rewarding expenditure but, for some, this included the occasional expenditure on drugs and/or alcohol. Furthermore, offering financial incentives has been shown to be no more expensive that routine care.

Concerning the practicalities involved in implementing the incentives and how they can be incorporated as part of a clinician’s toolkit or ‘problems requiring policies’

Despite the effectiveness and a defined structure of how to provide the incentives in routine clinical practice, financial incentives have not been implemented since the study ended and thus have not become part of the clinician’s toolkit.

Concerning whether or not offering financial incentives are coercive or whether or not they are fair on others not receiving incentives or ‘inherent dilemmas’

The findings from this study overall cannot answer these questions in their entirety, as this was not the original research question, although there is some evidence from the patient interviews that some felt the practice of financial incentives may be coercive or unfair on others.

Introductory remarks

1. Hello. I would like to start by thanking you for coming and taking part in this research on financial incentives. We really appreciate your involvement. Just to let you know, there are no right or wrong answers to any of these questions and you do not have to answer any questions that you do not wish to answer and you can leave the interview at any time you want.

Establishing rapport

1. First experiences with mental health: So, to start off I would just like to ask you a few questions about your first experiences with mental health services. When did you first come into contact with mental health services? What effect did this have on you at the time, for instance, did it affect your day to day life?

2. Family and peer support: Okay, so can I ask you what effect this had on you in terms of your family and friends or your children? Were they supportive at the time? Did you feel excluded or discriminated in your local community or did you get support at the time?

Health and identity

1. Health: Now, just moving on to some broader questions, I just want to talk to you about your health in general. How healthy do you feel at the moment? Both in terms of your physical and psychological health. Do you feel that you have much control over both your physical and psychological health? What things in your life do you think would improve your health in general? Do you think health is something that is easy to change in general or is it something that you can’t do much about? (Maybe some probing is needed here like “Could you explain that a bit further?”).

2. Identity: Okay, so again, I’d just like to ask you a really broad question. How would you define yourself? If you had to describe yourself to someone else what would be the most important things to mention? In other words, what do you see as the most important thing in defining who you are?

Depot and medication

1. Medication and health care experiences: Okay, so now I just want to talk to you about your health care experiences so far. I’m really interested to know about how you feel about the service that you receive. Do you have a good relationship with the team and your care co-ordinator? What are your experiences with taking medication? What are the psychological and physical effects of taking the medication, if any at all?

2. Experience with FIAT: Okay, that’s great. So now I’d like to talk to you about being part of the FIAT study. What were your initial thoughts when you were asked to take part in the study? Was it something you wanted to do? Why did you want to/or not want to? Did you talk to friends or family about it and what was their opinion in you taking part? Did getting the money when you were given the medication affect your relationship with your care co-ordinator, or not at all? If so, in what way did it affect the relationship? Did your experience with receiving £15 for your depot change over the year that you participated in the study? If so, how so?

3. Financial issues: During the trial, did you feel that you wanted or needed more money than the £15 that was given? In general, what did you use the money for? Did you feel at any point during the 12 months that you were becoming dependent or reliant on getting the money from your care co-ordinator? How have you coped since the 12 months have ended and you haven’t received any more money? Has this been something that has been easy or difficult to adapt to?

4. Post-intervention experiences: What happened in the aftermath of the study, when you didn’t receive £15 for getting your depot? Was this something that was difficult or easy to adapt to? Can you explain why? Are you still receiving your medication? Do you feel that you have benefited positively or negatively from being part of the study? Can you explain why? Has being part of the study affected how much control you feel that you have now over your health and health care, or not at all. (That last question is quite a leading question, so might be best left out)

5. Ethical issues: Did you feel in any way that you had to take part in this study or that you had to continue with the study when you did not want to? If so, can you explain a bit further for me? Speaking in general, do you think there are any ethical issues with conducting a study like this, where patients receive money when they get their depot medication, if any at all? Can you explain a bit further?

Wrapping-up questions

1. Quality of life: Okay, so we are now coming towards the end of the interview. I just have one or two more questions for you. In general, how would you describe your quality of life to me, as it is now, for example in terms of stress, work, family and taking your medication?

2. Future plans: What are your plans for the future? Has being part of the trial affected your plans for the future, if at all? Do you intend to keep taking your medication, or is it something that you plan to stop in the future?

Closing remarks

1. Well, I have come to the end of my questions. Is there anything that you have said that you would like to withdraw or is there anything that you would like to add further? Anything that might be important to you, for example, that I might have missed? Okay, well, once again, thank you for taking part in the study. If you have any questions or queries about taking part please talk to your care co-ordinator and I can be in touch. We really appreciate your time and participation.

End of interview.

Introductory remarks

Thank you: Hello. I would like to start by thanking you for coming and taking part in this research on financial incentives. We really appreciate your involvement.

Financial incentives interview: I would now like to ask you about your experience with financial incentives.

Recording and confidentiality: There are no right or wrong answers to any of these questions and you do not have to answer any questions that you do not wish to answer and you can leave the interview at any time you want.

Also, if you don’t mind, I will record the interview, but all you say here will remain fully confidential.

Experience with FIAT

Closing remarks

Well, I have come to the end of my questions. Is there anything that you have said that you would like to withdraw or is there anything that you would like to add further?

Anything that might be important to you, for example, that I might have missed? Thank you for taking part in the study. If you have any questions or queries about taking part please talk to your care co-ordinator and I can be in touch.

We really appreciate your time and participation.

End of interview.