Primary care management of cardiovascular risk for people with severe mental illnesses: the Primrose research programme including cluster RCT

Burden of cardiovascular disease in people with severe mental illnesses

People with severe mental illnesses (SMI), including schizophrenia, bipolar disorder and other psychoses, make up around 2% of the UK population. 1

Cardiovascular disease (CVD) is the most important physical problem in people with SMI,2 and those aged < 50 years are three times more likely to die from CVD than those without SMI, whereas those aged 50–75 years have a twofold risk. 3 People with SMI die from CVD up to 20 years earlier than the general population3–6 and recent studies have shown that the mortality gap for people with schizophrenia and bipolar disorder is widening. 7,8

The reasons for increased CVD are multifaceted, including increased levels of smoking, diabetes mellitus, obesity and dyslipidaemia in people with SMI compared with general practice controls. 9,10 Systematic reviews conclude that components of metabolic syndrome (being overweight, abnormal lipids, hypertension and abnormal glucose) are more common in people with SMI. 11 Research has also shown that people with SMI ate a higher-fat diet, ate less fibre and were less likely to participate in physical activity. 12,13 Antipsychotic medications, such as olanzapine and clozapine, have been linked to increased appetite and subsequent weight gain as well as abnormalities of lipid and glucose metabolism. 14 Another theory is that long-term stress of SMI exerts cardiovascular risk via the hypothalamic pituitary axis. 15

Current NHS provision of cardiovascular disease care in severe mental illness

The majority of people with SMI use primary care services and see a general practitioner (GP) more often than people without SMI. 6,16 Routine annual CVD risk screening for people with SMI is recommended in national guidelines17,18 and the responsibility for CVD risk prevention is placed within primary care, while those prescribed antipsychotics should be monitored more regularly. 19

The primary care quality outcomes framework pays GPs for providing an annual physical health review to people with SMI. Indicators for this review have changed over the past few years and glycated haemoglobin (HbA_1c) and cholesterol measurements were retired in 2014/15. The 2017/18 indicators consist of a recording of blood pressure, smoking status and alcohol consumption. 20

Studies have shown that if CVD screening is offered to people with SMI, then they are as likely to attend the screening as people without SMI. 21,22 However, in primary care, people with schizophrenia were significantly less likely to receive blood pressure or cholesterol screening than practice controls,23 and BMI and blood pressure recording rates were significantly lower in people with SMI than for those with diabetes mellitus or chronic kidney disease. 24 This study also found that exception reporting rates were higher for people with SMI.

Cardiovascular disease risk prediction tools for people with severe mental illnesses

Cardiovascular risk tools are widely used clinically to predict an individual’s risk of developing CVD, usually over a 5- or 10-year period. The risk scores are algorithms of conventional risk factors, such as smoking, blood pressure and lipids, and the predictive ability of the combined models is greater than that of each single risk factor. The resulting risk scores are also used to determine thresholds at which different risk reduction strategies, such as statins, should be employed.

It is noted in the 2016 National Institute for Health and Care Excellence (NICE) guidelines for CVD disease, risk and reduction that people with SMI constitute a high-risk group for CVD and existing risk prediction tools may underestimate their CVD risk, a picture similar to that observed in South Asian people and people aged < 40 years. 25 Ethnicity-adjusted risk scores have been developed as a result,26 but this has not yet been done for SMI. People with SMI were excluded from the original cohorts, such as the Framingham cohort, from which existing risk scores have been derived. The 10-year risk needs to be accurately determined for CVD scores for people with SMI in order to decide the thresholds at which to intervene.

Evidence for treatments to reduce cardiovascular risk in severe mental illness

Although the evidence shows that there are higher rates of CVD risk factors and higher mortality in people with SMI, we know far less about interventions to decrease this risk. There is a lack of high-quality evidence on CVD risk-reduction strategies in people with SMI.

Statins have been found to reduce severe dyslipidaemia in people with SMI in small studies focused on particularly high-risk populations. 27,28 The best trials of smoking cessation show small changes in smoking and quit rates,29 and studies have shown that medication and behavioural interventions are effective for weight loss. 30,31 A feasibility trial in secondary care to improve screening for CVD risk factors, testing a nurse-led service working across primary and secondary care, improved screening rates, but was too short in duration to reduce CVD risk. 32 Systematic reviews of interventions to increase uptake of lifestyle behaviours found some beneficial impact on CVD risk factors; however, the methodological quality of many of the included studies was low. 33–35

Most studies target single-risk factors and do not seek to address the full CVD risk profile of patients with SMI. Only one trial was found that tested a life goals collaborative care intervention involving management strategies for mental health symptoms and CVD risk factors. The findings were that the intervention improved the primary outcome of quality of life (physical health) compared with usual care. CVD risk factors were measured only as secondary outcomes with significantly lower levels of low-density lipoprotein (LDL) cholesterol in the intervention arm than usual treatment at follow-up, but no significant differences in blood pressure, BMI, waist circumference or other lipid parameters. 36

The Primrose programme overview

The aim of this National Institute for Health Research (NIHR) programme was to develop and test better methods to predict and reduce the risk of excess CVD in people with SMI across three work packages.

The research was carried out between May 2011 and July 2017. In years 1–2 (May 2011 to May 2013), we developed and validated a CVD risk score tool specifically for people with SMI. We then developed an intervention in primary care to lower levels of cholesterol and reduce CVD risk factors in people with SMI through an update of a systematic review of the literature, focus groups and workshops with clinical and lived experience advisors (years 1–3: June 2011 to December 2013). We also explored statin prescription rates and the effectiveness of statins on lowering levels of cholesterol and preventing CVD in people with SMI using primary care databases (years 2–5: October 2012 to December 2015). Finally, we tested the clinical effectiveness and cost-effectiveness of the new intervention in a cluster randomised controlled trial (RCT) and assessed the extent to which the intervention was delivered as intended (years 3–6: January 2014 to February 2017). The links between the three work packages are summarised in Figure 1.

FIGURE 1.

Overview of the Primrose programme.

Investigating patterns of statin prescribing among people with, and without, severe mental illnesses

The aim of this study was to explore the statin prescription rate in people with SMI compared with people without SMI in primary care. This work was published in the journal Schizophrenia Research in 2017 (with an impact factor of 3.958 at the time of writing). 101 The link to this paper can be found in Appendix 3, Work package 2. 101

The uptake of physical health checks in primary care among people with SMI has increased substantially over time, reflecting the introduction of policies and incentives, such as the Quality Outcomes Framework. 20 However, the impact on cardiovascular interventions, such as statin prescribing, was unknown. We used data from THIN to examine the rate of new statin prescriptions in people with, and without, SMI over a 10-year time period. Overall, rates of initiating a statin were at least doubled in people aged 30–59 years with SMI than in those without, even after adjusting for cardiovascular risk factors. Among people 60–74 years, rates were generally similar for people with and without SMI. However, among the oldest (aged ≥ 75 years) with schizophrenia (but not bipolar disorder), the rate of statin prescribing was around 20% lower (IRR 0.81, 95% CI 0.66 to 0.98) relative to those without SMI. These findings suggest that older individuals with schizophrenia are less likely to be initiated on a statin than people of a comparable age without SMI, and that this group may therefore benefit from additional measures to prevent CVD. The higher rate of statin prescribing among 30- to 59-year-olds with SMI (relative to people with SMI) demonstrates that statin prescribing is an important element of CVD prevention for this group and highlights the need for further evaluation.

Estimating the effectiveness of statin prescribing for people with severe mental illnesses

This study aimed to examine the effectiveness of statins for people with SMI through a systematic review of existing evidence and a series of staggered cohort studies designed to estimate the effectiveness of statins for the prevention of CVD and for modifying lipids in people with SMI. 102 This study was published in the journal BMJ Open in 2017. 102 The link to this paper can be found in Appendix 3, Work package 2. 102

Although statins form a core part of CVD primary prevention in the general population, this evidence base was not readily transferable to people with SMI. This is because it is not known whether or not patterns of medication adherence differ. Furthermore, some antipsychotic agents interact with sterol regulatory binding elements (which control lipid synthesis) and might therefore counteract the cholesterol-lowering action of statins. In addition, some of the largest statin trials have excluded participants with psychological conditions or excluded individuals perceived as less likely to be compliant with treatment.

Therefore, a systematic review was undertaken to examine the effectiveness of statins for primary prevention of CVD among people with SMI. This review did not identify any information on CVD events, mortality or long-term statin use in people with SMI. However, two studies provided evidence that statin therapy is associated with significant reductions in levels of total cholesterol (decreases of 11%28 and 35%27 for pravastatin and rosuvastatin, respectively) and LDL cholesterol (decreases of 20%28 and 49%27 for pravastatin and rosuvastatin, respectively) over 12 weeks in 60–100 individuals with SMI. These findings suggested that a large-scale study to evaluate the long-term impact of statin prescribing, particularly on CVD outcomes, was needed.

To explore the effectiveness of prescribing statins to people with SMI, we used UK primary care data from THIN to develop a series of cohort studies. These cohorts included 16,854 people aged 40–84 years who had a diagnosis of schizophrenia or bipolar disorder, and did not have pre-existing CVD. Cardiovascular outcomes of statin users and non-users were compared for (1) combined first myocardial infarction (MI) and stroke (primary outcome), (2) all-cause mortality and (3) change in total cholesterol concentration at 1 and 2 years after initiating a statin. We adjusted our results for a wide range of characteristics (such as blood pressure) that are associated with being prescribed a statin and the risk of developing CVD. In the main analysis we used multiple imputation to estimate the value of unobserved data and also conducted a complete-case analysis (for individuals with fully observed data), which produced very similar results to the analysis of the imputed data.

We did not identify statistically significant reductions in the rate of combined MI and stroke [incident rate ratio (IRR) 0.89, 95% CI 0.68 to 1.15] or all-cause mortality (IRR 0.89, 95% CI 0.78 to 1.02) associated with statin prescribing. However, it was found that statin prescribing was associated with statistically significant reductions (equivalent to a 20% decrease) in the level of total cholesterol 2 years after initiating a statin (IRR 1.2 mmol/l, 95% CI 1.1 to 1.3 mmol/l). This finding is similar to the reduction in cholesterol level observed in trial participants without SMI (IRR 1.1 mmol/l, 95% CI 0.8 to 1.4 mmol/l) and suggests that medication adherence in people with SMI is sufficient to support effective lipid modification. This translates to approximately a 25% decrease in mortality and 30% decrease in CVD events. 103

The study investigated a wide range of confounders for which data were captured in THIN; however, we cannot exclude the possibility of residual confounding due to factors (such as diet and exercise) that were unmeasured within our data set. Of note, our estimates of effect for statins were compatible with those from randomised trials examining non-SMI populations, suggesting that the likely impact of unmeasured confounding on our results may be small.

Piloting the Primrose trial intervention and training programme

Seven practice nurses/HCAs attended a pilot session of the training programme before the start of the trial. Some minor changes were suggested that were incorporated into the final training programme, including more opportunities for role play of Primrose appointments and more simplified explanations of the theoretical frameworks used to explain behaviour change.

The first intervention appointment was then piloted with two members of the LEAP. The practice nurse responsible for delivering the training met with each LEAP member and conducted the appointment using the study manual. Feedback from these sessions was incorporated into the final version of the manual and training programme. This included an emphasis on goals being patient led and a need to consider and understand aspects of the person’s life that may make behaviour change difficult.

The Primrose trial intervention and training components

The final intervention consisted of 8–12 appointments with a practice nurse/HCA over 6 months. Nurses/HCAs were trained to support patients to identify and monitor progress with goals on cardiovascular health, including taking medication, improving diet, increasing physical activity, stopping smoking or reducing drinking. They were encouraged to compile a local resource directory and refer patients on to existing support services if available in the local area (e.g. weight management or Stop Smoking Services), or provide support directly if services were unavailable or if the patient requested one-to-one support. They were also encouraged to actively follow-up and monitor attendance at services and progress made towards achieving health goals.

Nurses/HCAs were given a manual to take away with them, which included step-by-step appointment delivery flow charts, help sheets on managing different CVD risk factors and help sheets on strategies to help patients to stay motivated and engaged. A health plan was also given to patients to take away and use in between appointments. The health plan was used to record chosen goals, create an action plan and record progress towards achieving the goal.

The training programme was delivered to between four and eight practice nurses and HCAs at one of six training sessions by a practice nurse with expertise in mental health, a health psychologist, a lived experience trainer and the programme manager. The training comprised lectures and active learning through group discussion and role play of appointments. There were 2 days of learning, which were 2 weeks apart so that the nurses/HCAs could deliver an appointment between sessions and bring any difficulties they might have experienced or areas they felt less confident with to practise at session 2.

Training session 1 consisted of (1) discussing the link between CVD risk and SMI, (2) mental health awareness, (3) effective interventions for lowering CVD risk, (4) behaviour change techniques and (5) practical strategies to encourage motivation and engagement. Appendix 4, Table 16, contains a more detailed description of the content of training session 1. Training session 2 was led by the trainees and involved discussing the appointments that they had delivered and practising strategies and appointment delivery through role play and discussion.

In conclusion, an evidence-based manual and training programme was developed using clinician, patient and clinical academic advice from focus groups and workshops alongside reviews of the best available evidence in both policy and RCTs of CVD risk-lowering interventions. The resulting manual and training programme were piloted with health professionals and patients and found to be acceptable and deliverable. The intervention was then tested in a cluster RCT described in Work package 3.

Methods

We successfully delivered a cluster randomised trial within which 76 GP practices across England were recruited and then randomised to either the Primrose intervention (n = 38) or the TAU (n = 38) arm. This total number of practices was larger than specified in the original grant application as the numbers of participants (cluster size) recruited in each practice was smaller than expected (mean 4.3 participants).

Participants in the trial were aged 30–75 years with a GP record of SMI, in line with the definitions already outlined in this report. They had a raised lipid profile defined as a cholesterol level of > 5.0 mmol/l or a total-to-HDL cholesterol ratio of > 4 mmol/l; and they needed to have one other cardiovascular risk factor, including hypertension, smoking, obesity, raised HbA_1c levels or diabetes mellitus.

For the practices in the intervention arm, allocated nurses or HCAs were trained on two occasions to deliver the Primrose intervention that was developed in the earlier stages of the research programme (work package 2). Briefly, this involved arranging up to 12 appointments with each participant to target the most relevant CVD risk factor and using agreed evidence-based approaches, and established behavioural techniques to maximise the chance of successful risk reduction. In the treatment-as-usual arm, the nurses were not trained in the intervention but they were allowed to offer standard treatment for CVD risk factors in line with routine practice.

Patients and staff could not be masked to the intervention, but researchers were not informed of the allocation. The analysis plan was predetermined and random-effects linear regression (adjusting for baseline characteristics) was performed on the primary outcome of total cholesterol level at 12 months to account for clustering within general practice. Secondary outcomes included CVD risk scores and other cardiometabolic parameters including glucose, smoking, blood pressure and diabetes mellitus as well as BMI. We also included validated measures of well-being, diet and physical activity, patient satisfaction with services and adherence to both psychotropic and physical health medication.

Results

We recruited 327 participants with SMI: 155 participants in the 38 practices within the Primrose intervention arm and 172 participants within the TAU arm. Attrition in the study at 12 months was lower than predicted (12% as opposed to 20%) so that the total number of patients with follow-up data at 12 months (for the primary outcome of total cholesterol level) was greater than the sample size calculation requirements. In general, the patients had a high level of CVD risk factors, as would be expected by the inclusion criteria. For instance, half were current smokers and the mean BMI was above the threshold for obesity in both arms.

For the primary outcome of total cholesterol level there were no differences between arms at 12 months (5.4 mmol/l Primrose vs. 5.5mmol/l TAU; coefficient 0.03, 95% CI –0.22 to 0.29). This remained the case when additional analyses were performed, adjusting for baseline cholesterol levels or for characteristics that differed between arms at baseline. Total cholesterol levels did decrease over 12 months in both arms (mean decrease Primrose 0.22 mmol/l; mean decrease TAU 0.39 mmol/l).

There were also no differences between arms on the secondary outcomes listed in the methods section above, including the cardiometabolic parameters, patient satisfaction with services or adherence to medications. Statin prescriptions were low and did not increase at 12 months in either the Primrose or the TAU arm.

A total of 30 serious adverse events were reported for 25 people. There were fewer serious adverse events in the intervention arm (seven events for seven patients, including one death, three psychiatric admissions and three general admissions) than in the TAU arm (23 events for 18 patients including three deaths, 11 psychiatric admissions for nine people, seven general admissions for six people, one admission to a crisis house and one diagnosis of cancer).

Participant attendance rates at Primrose appointments were moderately good, with only 32 (21%) participants attending no appointments. A total of 72 (46%) participants attended more than six appointments over the 6-month intervention period and 36 (23%) participants attended between two and five appointments, with the remaining 15 participants (10%) attending one appointment.

Care in the TAU arm may have been better than standard general practice care as the GP practices in the study had identified people with raised CVD risk factors, for whom they may have then clinically intervened. Furthermore, participants and the practices were motivated to take part in the trial and to reduce CVD risk factors. This may have minimised the chance to show superiority for the more intensive Primrose intervention.

The choice of primary outcome measure for the trial was a challenge given that the intervention was designed to target multiple CVD risk factors. Nurses/HCAs were trained to discuss cholesterol levels and statin intervention and adherence in the first instance and then move on to other goals relevant to CVD risk reduction; however, this may have been in conflict with goal-setting being patient-led. If other behavioural goals were chosen instead of statins, then they may have had less impact on levels of cholesterol. However, differences were not demonstrated in any other CVD risk factors.

Conclusion

Nurses and HCAs were successfully trained in the Primrose intervention and delivered it to the majority of patients. However, participants in the intervention arm did not do better on the primary or secondary outcomes than participants in the TAU arm in UK primary care. This may reflect good care in both arms, as cholesterol levels did decrease over the study period and satisfaction with services on the validated client satisfaction questionnaire-8 was high in both groups.

This was a pragmatic trial in which participants exhibited a range of clinical characteristics and CVD risk factors, and it may be that this variability made it less likely that the primary outcome of total cholesterol level was targeted by the nurses/participants or that the most effective interventions, especially statins, were chosen. This possibility will be explored in further fidelity and health economics work.

Methods

The aim of the economic evaluation was to evaluate if the Primrose intervention was cost-effective compared with TAU, for a range of values of WTP for a QALY gained from a health-care cost perspective over the duration of the trial (12 months).

To calculate QALYs, EuroQol EQ-5D 5 level (EQ-5D-5L) data were collected at baseline, 6 months and 12 months and calculated as the area under the curve adjusting for baseline differences. Data were collected using patient-completed questionnaires asking about health promotion activities over the past 6 months at baseline, 6 months and 12 months. Health promotion activities included services for hazardous and harmful drinking, Stop Smoking Services, nicotine replacement therapy, diabetes mellitus and weight management services. Primary and secondary care resource use was collected from patient medical records for the duration of the trial. The cost of the Primrose intervention was calculated from data collected as part of the trial on the number and duration of Primrose appointments attended, missed appointments and who delivered the intervention (primary care nurse, HCA or GP). Information on the cost of training was also collected.

Results

The Primrose intervention arm showed a mean of 0.769 QALYs (95% CI 0.751 to 0.787) compared with a mean of 0.780 QALYs for TAU (95% CI 0.764 to 0.796). The difference in QALYs was –0.011 (95% CI –0.034 to 0.011).

The total health-care cost for the Primrose intervention group was £1286, with a total cost of £2182 for TAU (mean difference –895, 95% CI –£1631 to –£160; p = 0.012). These lower health costs were mostly a result of fewer mental health inpatient stays and costs (£157 in the Primrose intervention vs. £956 in TAU; –£799, 95% CI –£1480 to –£117; p = 0.018). The total mean 12-month health-care cost per patient for the Primrose intervention (including intervention costs but excluding those who did not attend and training) was £2580 (95% CI £1899 to £3261), with a total mean cost of £3404 (95% CI £2467 to £4340) for TAU. This gave a cost difference of –£824 (95% CI –£568 to £1079) in favour of Primrose. The total incremental cost-effectiveness ratio (–£824/–0.011) is £76,245.

There is some uncertainty as to whether or not using the EQ-5D-5L to calculate QALYs is a suitable methodology for health promotion interventions.

Conclusion

The potential effect of increased contact with a primary care health professional on mental health inpatient admissions warrants further investigation.

Methods

To enhance fidelity in the intervention arm, we developed a study manual in which the detail of each component of the intervention was described. Nurses/HCAs were trained on delivering the intervention through strict adherence to the details provided in the manual. This facilitated the standardised delivery of the intervention across 41 providers in all 38 participating GP practices in the intervention arm.

Nurses/HCAs in the intervention arm were trained in procedures for audio-recording all of their appointments with recruited Primrose patients. After gaining consent from the patients and nurses/HCAs in the intervention arm, nurses/HCAs were asked to record all of their Primrose intervention appointments.

A random 20% sample of audio-recordings was selected for the fidelity assessment as specified in the original grant application. Fidelity was assessed by two independent researchers using first appointment and subsequent appointment checklists adapted from a reliable fidelity assessment method developed for behavioural interventions. 108,109 A score of ‘2’ was assigned if a provider behaviour was achieved, a score of ‘1’ when a provider behaviour was achieved to some extent, a ‘0’ when the provider behaviour was judged appropriate to do but was not delivered, and ‘not applicable’ for provider behaviours judged not appropriate. This scoring system was applied to the appointment transcripts, deriving a percentage fidelity score for each intervention component, appointment and provider. We also derived an overall fidelity percentage score for all sampled appointments and all providers combined. Inter-rater reliability for coding between the two researchers was 86% with a Cohen’s kappa of 0.668 (95% CI 0.63 to 0.70).

Results

One or more appointment audio files were returned by 33 out of 41 (80.5%) providers for 90 out of 123 (73.1%) patients. Out of 831 attended appointments, 431 (53%) audio-recordings were returned. A random selection of 86 out of 431 (20%) audio-recordings covering 23 out of 33 (69.7%) providers and 52 out of 123 (42.3%) patients were transcribed verbatim.

A total of 67.7% of intervention manual-specified components were delivered across all appointments indicating moderate fidelity, with considerable variation among activities. Fidelity was higher in appointment 1 (72.5%) than in subsequent appointments (66.6%). Fidelity also varied between intervention components and was lowest for ‘forming habits’ (47.8%) and highest for ‘reviewing progress’ (90.2%).

Out of 14 first appointments, none of the patients identified a goal that addressed statin adherence or initiation. Eight (57.1%) patients wanted to address diet or physical activity, three (21.4%) patients set a goal around reducing smoking, and two (14.3%) patients chose to reduce their alcohol intake. One patient did not set a goal. Nurses had higher fidelity than HCAs, with 79.5% of intervention components delivered by nurses compared with 64.3% by HCAs. This difference was significant [t(20) = 2.32; p = 0.037].

A potential limitation of the study was whether or not the fidelity sample of intervention practice nurses/HCAs was representative of the trial sample. Nurses were over-represented in the fidelity sample (60.9% vs. 43.9% in the trial) as were providers with previous research experience (52.2% vs. 39% in trial); however, the fidelity sample was randomly generated by an independent statistician.

Conclusion

Observed fidelity to the Primrose intervention was moderate, with some intervention components being delivered more than others. These results are comparable to other fidelity assessments of cardiovascular prevention programmes. 110 Statins were not focused on in initial appointments, which concurs with the main RCT findings in which few statins were initiated. HCAs had a lower fidelity of intervention delivery than practice nurses, which may have implications for future clinical and research work in this field.

Work package 1: development and validation of a risk model for predicting cardiovascular disease events in people with severe mental illnesses

The work package 1 risk score work developed new models for predicting CVD in SMI. These were delivered on time, using all the methods specified in the original proposal. We developed bespoke new Primrose models for predicting CVD in SMI and these performed well in people with SMI. However, existing models from the general population also performed fairly well and we did not generate the objective evidence to suggest that existing CVD risk scores in general practice should be replaced immediately with the bespoke SMI specific models.

A limitation of the study was that the performance of the CVD risk score models among different ethnic groups was not assessed, but the availability of routine ethnicity data is limited and this could be a focus of future work if data quality improves.

Given that a range of guidelines have referenced our work, it is likely that many clinicians are aware that standard risk scores may underestimate risk in people with SMI.

Work package 1: additional economic modelling work over and above the original protocol

The health economics modelling work regarding the new Primrose risk scores has recently been peer reviewed and published. 40 It uses the more recently recommended threshold of 10% CVD 10-year risk to explore which risk models would be better for people with SMI if used to drive statin prescribing. The Primrose BMI models provided the best economic results, which may have been a result of its classification of more individuals at a high risk of CVD and eligible for statin therapy than other algorithms, although the UK QRISK models also performed well.

Given that there was a small difference between the two tools economically, the decision regarding which algorithm to use in routine clinical practice becomes one of implementation, advocacy and ease of use. One could argue in favour of using a general population-derived lipid model as these are already used in UK general practice and hence require no change. On the other hand, the SMI-specific BMI model, although potentially requiring additional training and implementation costs, could confer additional benefit by raising awareness of the need to improve CVD outcomes in people with SMI, and providing a model that requires no blood test to estimate risk, a limitation of other CVD risk algorithms as many people, with and without SMI, decline blood tests. 111 The ease of implementation and delivery of the SMI-specific BMI model means it could be used in any setting, including mental health care and non-clinical settings without blood results. This is particularly important as many people with SMI do not attend primary care and monitoring of CVD risk factors remains low in other settings. 19,111–114 The SMI-specific BMI model provides an opportunity to target more people with SMI, to increase identification of those at a high risk of CVD and decrease the physical, social and financial burden associated with CVD.

We have made the Primrose model available on the internet so that it can be used by interested stakeholders. 40

Although the Primrose risk score results were delivered (and published) on time, the initial validation results were not strikingly different from current CVD screening practice for us to include the new algorithms in the Primrose intervention work or to use them as inclusion criteria for the trial as originally planned. 37 This was partly a timing issue as the risk score work occurred in parallel to the development work packages 2.1–2.3, which involved bringing together this intensive development work to finalise the training and content of the Primrose intervention.

Work package 2: development of a practice nurse-/health-care assistant-led intervention for lowering levels of cholesterol and reducing cardiovascular disease in people with severe mental illnesses

Three major pieces of research were performed to help design the nurse-led intervention for the main trial, as detailed in our original research protocol.

We also worked closely with our patient and public involvement (PPI) LEAP panel throughout this part of the programme and won a national prize for our PPI work, from the NIHR Mental Health Research Network (MHRN).

Work package 3: evaluation of a practice nurse-/health-care assistant-led intervention for lowering levels of cholesterol and reducing cardiovascular disease risk in people with severe mental illnesses – a cluster randomised controlled trial

Ethics approval and trial registration were successfully achieved on time to start the trial in 2013. The trial protocol was peer reviewed and published in 2016,106 before any analysis occurred.

The trial commenced in the north London locality and it soon became clear that recruitment procedures for the trial involved large amounts of work to screen for people with SMI in primary care, to check eligibility in terms of cardiovascular risk factors and to invite them to take part. Some potential participants did not have all the required information on CVD risk factors, so they needed to be invited to their GP practice to check eligibility. This was a rate limiting step to recruitment.

We realised that the numbers likely to take part per practice were somewhat smaller than expected, and required a lot of time and effort in the absence of the MRC GPRF and PCRN infrastructure. We therefore decided to increase the number of practices from 40 to a final total of 76, which increased the delivery time of the trial. However, we successfully recruited and retained enough people with SMI and CVD risk factors to achieve the final number of participants with primary outcome data at the 12-month follow-up (total n = 289), meeting the requirements of our published sample size calculation.

This was the first fully powered trial of a primary care-based intervention for lowering CVD risk in people with SMI. The successful delivery demonstrates that there is an appetite for this field in UK primary care and also that the CRNs were able to support this type of work. The large number of GP practices recruited and the geographical spread across both rural and urban areas in England is also a strength in terms of external validity.

The trial finished follow-up on time within our revised timeline (6 months ahead of the end of the overall programme funding as planned), allowing time for data cleaning and analysis of the results.

The training and intervention were successfully delivered across the general practices, and seemed acceptable to providers and participants with almost half attending six or more appointments in the Primrose intervention arm. This contradicts the negative attitudes that are sometimes voiced about uptake of this work in SMI.

The main results showed no differences in the primary outcome of total cholesterol levels at the 12-month follow-up. Furthermore, the trial did not show differences in any of the main secondary outcomes related to cardiometabolic risk factors, such as blood pressure, glucose, weight or smoking. Diet and exercise levels were similar in both arms and levels of satisfaction with services were high in both arms.

The richness and volume of the objective medical records data collected for the economic evaluation of the cluster randomised control trial was a strength of the study, but we did not have the time to conduct all of the analyses we were interested in. Additional work will include an analysis of 12-month data using multiple imputation to account for missing data and an analysis of the relationship between health inputs (health promotion activities) and health outputs (QALYs, reduction in unplanned health-care resource use).

In the assessment of fidelity using the audiotaped appointments, there was evidence that many of the behavioural techniques had been utilised within the appointments. This was true across providers, although fidelity seemed even higher for nurses than for HCAs. There was some evidence that when statin adherence or prescription should have been reviewed and targeted, the participant and provider did not choose statins as their primary focus.

The initial intervention had been envisaged as a nurse-led intervention, but in practice many GPs were unable to provide a nurse with time available to be trained in the Primrose intervention, so a health-care assistant (HCA) was identified instead. These practitioners do deliver CVD screening work to other populations, therefore, it was agreed that they would be trained in Primrose, partly to reflect real life and partly to allow the trial to be delivered practically and on time.

There is evidence in the primary care mental health literature that interventions that include supervision from a physician are more likely to reduce disease risk factors in people with depression,119 but this approach has not been tested in people with SMI. Future intervention studies may wish to test whether or not more intensive supervision and support for staff results in improved outcomes for patients; however, there may be cost implications of additional support.

There were some design issues that are a challenge to the external validity of the trial findings. This included generalisability, as both the GPs and the participants were people interested in physical health in SMI but perhaps not representative of the overall UK population and primary care landscape. A second issue was the cluster design and the fact that practices (and participants) allocated to the TAU arm were aware that there were Primrose participants with identified CVD risk factors who would not receive the Primrose intervention. Therefore, these participants may have received superior, or at least different, care to those undergoing standard care in usual UK general practice, by virtue of the screening process for the trial. In other words, TAU may not have been a ‘fair’ comparison with the Primrose intervention. The finding that levels of total cholesterol were reduced in both the intervention and the TAU arms despite a lack of focus on statins would support this.

It is interesting that mean total cholesterol levels decreased in both arms of the trial over the 12 months. The natural course of cholesterol level is to increase with age, so perhaps this decrease in cholesterol level reflects improvements in CVD risk for people with SMI in both arms and this may be one explanation as to why there were no differences between arms at 12 months.

The choice of an outcome measure for the trial was always going to be a challenge in a study targeting multiple CVD risk factors. The level of cholesterol was eventually chosen because the first aim of the intervention (and the original protocol) was to optimise statin prescribing in the first instance. However, if other behavioural goals were chosen instead of statins, these goals may have been less likely to have an impact on the primary outcome. Nonetheless, we did not show any differences in other CVD risk factors, nor in overall measures of well-being. However, the problem of identifying outcomes in trial settings such as this has been highlighted in other studies. 120 It is also acknowledged that in the longer term, cholesterol level reduction might not necessarily reduce CVD rates.

Future research related to work package 1: the risk score development work, and work package 2.3 – the primary care database work

Our bespoke SMI Primrose risk scores have been validated and can be accessed online (www.ucl.ac.uk/primrose-risk-score/). They have been assessed in terms of economic benefit of using them to guide statin prescribing and would provide a NMB.

This is a particular benefit when people with SMI do not have a blood test result for lipids or decline to have one taken. This may apply to 50% of the UK SMI population. 21

The Primrose risk scores could be validated externally in a separate data set and updated and also compared against newer risk scores, including the new QRISK3 score that was published in 2017. 121 The QRISK3 score should be explicitly evaluated for its benefits for people with SMI, especially as QRISK3 uses a definition of SMI that is inconsistent with the general meaning in UK primary care (i.e. it also includes depression). The inclusion of depression may be valid, but for now we lack evidence as to whether or not this new risk score is superior to the Primrose score for people with schizophrenia, bipolar disorder or other psychoses.

Regarding the statins database research in Primrose,102 it is important that further studies monitor the national prescribing of statins to people with SMI, especially older age groups who are undertreated in current UK practice.

Future study designs may be able to assess the impact of statins on the mortality gap and excess CVD in people with SMI, especially if databases in the UK or internationally achieve larger sample sizes with higher numbers of CVD events, as is currently planned in primary care research.

Future research related to work package 3: the Primrose cluster randomised controlled trial

The Primrose intervention was acceptable and deliverable in real-life practice, with patients attending appointments and behavioural goals being set. However, it was not superior in terms of a difference in levels of total cholesterol between the two arms of the cluster trial.

Future studies might need to move beyond standard trial designs if we are to understand what works for people with SMI in terms of reducing CVD risk. Studies might need to use routinely collected clinical data to assess outcomes for all patients in real-life settings, comparing areas where different interventions are delivered to different people. These naturalistic evaluations would overcome the restrictions imposed by trial inclusion criteria, and the unrepresentativeness of people (and practices) who agree to participate in trials. This could involve more applied methods where routine data are collected within health-care systems to evaluate gaps in CVD risk screening and interventions. These studies should explore the delivery of statins, smoking cessation and weight reduction techniques including pharmacological approaches, such as metformin and behavioural approaches.

Further analysis of the fidelity work could utilise the full data set and explore the content and conversational aspects of the appointments in greater detail to help shed light on the lack of difference in results between intervention and control groups. Future research work should also examine whether or not a conversation regarding statins is happening between professionals and patients, and explore the reasons why statins might not be initiated for those who would benefit.

Further analysis of the Primrose trial data set from work package 3 will be able to assess the content of care within both the intervention arm and the TAU arm and explore what did and did not work, and any predictors of response. Examples will include demographics, provider characteristics, participant characteristics, such as diagnosis, the availability of formal or informal support to the participant and further analysis on service use and prescription data collected for the cost-effectiveness analysis of the trial. The potential effect of increased contact with a primary care health professional on hospital admissions also warrants further investigation.

Additional studies emerging from the Primrose programme

A number of additional studies have emerged from the Primrose programme of work and are currently in application, set-up or analysis phase. This includes:

• An analysis of data from a qualitative study that used semistructured interviews to explore the experiences of those who delivered and those who took part in the Primrose intervention.

• As part of the data collection for the cluster RCT, participants consented to provide a saliva sample for deoxyribonucleic acid (DNA) extraction and analysis purposes to determine whether or not any variation in DNA exists, relevant to both SMI diagnosis and physical health outcomes. Participants were given the opportunity to opt out of providing the sample without affecting their entry into the trial. DNA extraction is currently under way with data analysis plans currently being finalised.

• The programme manager is carrying out secondary analyses on data collected through the trial and fidelity assessments for the purpose of fulfilling a doctorate. The research aims to explore the links between existing social support and adherence to CVD risk reducing treatments and behaviours for people with SMI.

• A further, independent study funded by the NIHR School for Public Health Research is being commenced to explore inequalities in the provision of smoking cessation interventions for people with SMI, and the effectiveness of interventions in the THIN database 2017–18. This work follows on from the work in work package 2.3, but for smoking rather than statins.

• Applications are currently being planned for studies to carry out naturalistic evaluations of the different risk scores in real-life settings as well as local interventions, which have been deployed in different regions of the UK to integrate physical and mental health care in people with SMI. An example is the Integrated Practice Unit for psychosis in Camden and Islington Foundation NHS Trust, which brings together health services and other providers to deliver a holistic approach to mental and physical health needs.

• A further study is being planned, to explore diet and exercise variables and interventions related to these, with a NIHR School for Primary Care Research PhD studentship.

• Primary care database studies continue to be used to explore CVD outcomes, risk factors and interventions in people with SMI, as new (and better-quality) primary care data become available.

Work package 1: development and validation of a risk model for predicting cardiovascular disease events in people with severe mental illnesses

We have demonstrated that general population risk scores can underestimate CVD risk in people with SMI. 37 We have also shown that a model that does not require lipid blood tests is most beneficial for deciding when to prescribe statins and prevent CVD in people with SMI. 40

We are now making this tool available for clinicians and will need to update it with contemporary data as it becomes available. We will explore integrating these scores into clinical software packages.

Our findings regarding risk scores in SMI have been highlighted in national guidelines, including the British Association of Psychopharmacology guidelines on weight management in people prescribed antipsychotics. 39

The underperformance of general population risk scores for people with SMI is also highlighted in the adapted ‘Lester’ tool, an algorithm to help clinicians to manage CVD risk in people with SMI. 38 This tool is endorsed by most Royal Colleges and medical/nursing organisations, as well as NICE. 17 The underperformance of general population risk scores in SMI is also recognised in the general 2014 NICE guidelines on lipid modification. 25

In 2017, a new ‘QRISK3’ CVD score was published and it has addressed some of these limitations. 121 This new score included SMI and second-generation antipsychotics. The existing QRISK algorithms do allow calculation of CVD risk without lipid levels, but this is by assigning an average value. The Primrose models use multiple imputation in the BMI model and, therefore, should be more accurate when lipids are missing.

There are no data validating the new QRISK3 tool specifically in SMI, and there are some uncertainties about the new tool as the definition of SMI in the QRISK3 study has needed clarification; it transpires that the authors include depression in their definition of SMI, which is unusual for this field and may limit the utility of QRISK3 for people with schizophrenia, bipolar disorder and other psychoses. The results need further validation and clarification. 122 In contrast, the new Primrose models have been validated and economically evaluated as superior for people with SMI.

Work packages 2.3.1: investigating patterns of statin prescribing, and 2.3.2 – estimating the effectiveness of statin prescribing for people with severe mental illnesses

Some people with SMI who are older may be less likely to receive statins, so this needs to be addressed in clinical practice, especially as this group has the highest absolute number of CVD events. It is important that people with SMI are offered statins, as we have provided robust evidence that they are effective in a large, representative UK sample of people with SMI.

Work package 3: evaluation of a practice nurse-/health-care assistant-led intervention for lowering levels of cholesterol and reducing cardiovascular disease risk in people with severe mental illnesses in primary care – a cluster randomised controlled trial

We demonstrated that a primary care intervention for people with SMI was deliverable across general practices in England. The training was successfully organised and when the nurses and HCAs organised appointments, there was good attendance by people with SMI, with almost a half of patients attending six or more appointments. The providers did deliver the behavioural interventions to a moderate degree of fidelity. This occurred despite some negative attitudes towards the work when the providers initially attended the training for Primrose.

There was some evidence that statins were not chosen as the first goal, even when they should have been initiated or reviewed, but this is preliminary evidence.

The intervention in its current form was not found to be superior for the chosen primary outcome of total cholesterol level, nor the secondary outcomes. However, the success of the trial delivery demonstrates the feasibility of organising care for people with SMI in the primary care setting, at least for a group of people with SMI, with good levels of satisfaction in both arms. It is also of interest that an intervention in primary care with regular appointments was associated with fewer serious adverse events and decreased costs in terms of mental health admissions.

There were some methodological considerations regarding the trial, especially whether or not TAU was superior to usual UK general practice care, and future research designs may need to be more naturalistic, using less constraining methods, which are more inclusive of all people and practices, and which can capture real-life service provision.

In terms of future interventions, our fidelity work demonstrated that behavioural techniques can be delivered during appointments by HCAs/nurses with a manual and with two brief days of training. Patient care may be enhanced through mechanisms to ensure that evidence-based CVD risk reduction strategies are being offered and explained to people with SMI, including reviewing risk scores and explaining the role of statins.

Intervention development group

A smaller, core LEAP intervention subgroup consisting of eight service users and carers was formed from the larger panel with the remit of translating the findings from the development work packages (focus groups, policy and systematic review) into the design of the intervention. The subgroup met four times over 1 year and developed 11 key recommendations, six of which were incorporated in to the intervention manual and training programme. 105 These recommendations were (1) one step at a time goals for behaviour change, (2) involvement of carers and mental health workers, (3) training of nurses to address attitudes towards mental illness and stigma, (4) appointment reminders to service users, (5) information to take away including the next appointment time and (6) involving service users in the training of nurses. For recommendations that were not incorporated into the intervention, the study team discussed the reasons with the subgroup members and answered any questions arising from these decisions.

Background

People with SMI are at increased risk of developing CVD and die at a younger age than the general population. This study evaluated and synthesised evidence from published systematic reviews and individual RCTs on the effectiveness of pharmacological and behavioural interventions for reducing modifiable CVD risk factors in people with SMI.

Methods

We searched The Cochrane Library for existing systematic reviews. We then searched the Cochrane Schizophrenia and Cochrane Depression, Anxiety and Neurosis Group Trial Registers between 1966 and 2014 for additional RCTs not included in the identified reviews. We searched for interventions to manage levels of cholesterol, diabetes mellitus, hypertension, weight, smoking and alcohol consumption.

Findings

Fifteen systematic reviews and 28 additional RCTs were included in the review, from 11,028 references. The synthesised data demonstrated good evidence of effective pharmacological and behavioural interventions for weight management and some evidence that combined pharmacological and behavioural interventions might be more effective than either alone. There was good evidence that pharmacological interventions were effective for smoking cessation, or reducing smoking, and some evidence for behavioural or combined interventions. Three studies reported effective interventions to reduce alcohol misuse. No studies were found on effective interventions targeting levels of cholesterol, diabetes mellitus or hypertension in SMI.

Conclusion

There is evidence that CVD risk attributable to weight and smoking can be managed effectively in SMI, using pharmacological and behavioural approaches. Future research should determine the effectiveness of interventions to manage cholesterol levels, diabetes mellitus, hypertension and alcohol misuse in this population.

Methods

We searched for published systematic reviews and additional RCTs that were not included in existing systematic reviews.

Inclusion/exclusion criteria

Studies were included if they met the following criteria: (1) they used a randomised controlled design, (2) ≥ 50% of participants were adults with a diagnosis of schizophrenia, bipolar disorder or other non-organic psychotic disorders including schizoaffective disorder, (3) they evaluated interventions aimed at managing levels of cholesterol, diabetes mellitus, hypertension, weight, smoking and/or alcohol consumption, (4) title and abstract were written in English (papers were only excluded on the basis of being written in a non-English language where translation of the full text was not possible) and (5) were published between 1966 and 2014.

Search strategy

We conducted the search strategy in five stages in the following order: (1) we searched the Cochrane group publications lists and Cochrane Library database for systematic reviews, (2) an information specialist from the Cochrane Schizophrenia Group searched the Cochrane Schizophrenia Group Trials Register for RCTs with schizophrenia and psychosis populations, (3) we searched The Cochrane Library database for additional RCTs using an adapted search strategy to include populations with bipolar disorder provided by the Cochrane Depression, Anxiety and Neurosis Group, (4) an expert reference group searched for non-peer-reviewed literature, (5) experts in the field were asked to identify any relevant publications not captured by the electronic search strategies and (6) principal investigators identified on trials registers were contacted for relevant publications.

The following search terms were used:

Schizophrenia* OR severe mental illness* OR bipolar or mania* OR manic* OR hypomani* OR psychos* OR psychotic OR postpsychotic OR “post psychotic” OR “rapid cycling” OR schizoaffective OR bipolar OR mania* OR manic* OR hypomani*

AND

*physical* OR *cardio* OR *metabolic* OR *weight* OR *Tobacc* OR *Smok* OR *medical* OR *alcohol* OR *nutrition* OR *diet* OR *health* OR *diabete* OR *blood pressure* OR *hypertension* OR *cholesterol*OR *statin*

Screening

Titles and abstracts of reports of RCTs identified in the electronic searches were screened by four researchers (LA, AB, MH and VT; see Acknowledgements). References not meeting the inclusion criteria and duplicates (including individual RCTs in included systematic reviews) were removed. To assess reliability of screening, reviewers independently screened 10% of each other’s allotted references resulting in < 5% level of disagreement. Disagreements were resolved by discussion.

Full-text reports were obtained and screened by the same researchers using the criteria described above. Data were extracted from reports meeting the inclusion criteria.

Data extraction

Data within the RCTs were extracted using a modified template from Methods for the Development of NICE Public Health Guidance125 to record methodological (publication, design, etc.) and substantive characteristics (participant, setting, etc.) of included studies.

Quality assessment

The quality of evidence in included reports was assessed using The Cochrane Collaboration’s tool for assessing risk of bias. 126

Data synthesis

Reference lists of published systematic reviews were cross-checked and reviews including the greatest number of relevant references were summarised. Papers from less comprehensive reviews were also included if they were not covered by the more comprehensive reviews. A meta-analysis of findings from individual RCTs was not possible owing to the heterogeneity of reporting, so a narrative synthesis of findings was created. Findings from RCTs with fewer than 10 participants, which may not be sufficiently powered to detect change, are summarised separately.

We performed a narrative synthesis of the evidence, first by selecting the most comprehensive systematic review and summarising this and all the additional trial results.

Findings

Fifteen systematic reviews and 28 additional individual RCTs of interventions met the inclusion criteria [Table 1 and Figure 2 for Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram of individual RCTs].

FIGURE 2.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow chart of additional individual RCTs. CCDAN, Cochrane Depression, Anxiety and Neurosis Group; CSzG, Cochrane Schizophrenia Group.

No trials or reviews of interventions directly targeting levels of cholesterol, diabetes mellitus or hypertension in SMI were identified. Systematic reviews and individual RCTs meeting the inclusion criteria were therefore grouped into three categories of interventions: (1) weight, (2) smoking and (3) alcohol consumption.

In this section, we summarise the main findings of the systematic reviews, grouping the interventions according to their main target CVD risk factor and whether they were behavioural, pharmacological or both.

Risk of bias

Risk of bias was assessed in individual RCTs identified through the electronic searches (n = 28) and in an additional five RCTs that were taken from systematic reviews that were discarded from this review because the remaining studies had all been included in one of the 15 more comprehensive reviews, making a total of 33 assessments.

Three RCTs were rated as being at a high risk of bias. Of these, two were open-label studies of pharmacological interventions54,55 and one had high attrition, which differed across treatment groups. 56

Seven RCTs were rated as being at a low risk of bias. 57–63

The remaining 23 RCTs were rated as being at an unclear risk of bias. Although all studies reported using randomisation, 20 of the 23 RCTs did not adequately describe methods of allocation concealment and 17 out of 23 did not describe how participants were randomised.

Pharmacological interventions targeting weight

Synthesising systematic review and individual randomised controlled trial findings

The most comprehensive systematic review98 reported the greatest effects for metformin on weight loss and blood lipids. Findings from more recently identified individual RCTs report varied effects for topiramate on weight management and limited evidence regarding topiramate and ramelteon on blood lipids.

Behavioural interventions targeting weight

Pharmacological interventions targeting smoking

Behavioural interventions targeting smoking

Combined pharmacological and behavioural interventions targeting smoking

Pharmacological interventions targeting alcohol

Behavioural interventions targeting alcohol

Additional individual randomised controlled trials of behavioural interventions targeting alcohol

No further RCTs were identified.

Combined pharmacological and behavioural interventions targeting alcohol

No systematic reviews or RCTs were identified.

Summary of main findings

We performed a comprehensive review of international evidence regarding interventions to reduce cardiovascular risk in people with SMI. We included existing systematic reviews as well as more recent RCTs. We found evidence for effective pharmacological and behavioural interventions to decrease weight and smoking, but limited evidence on effective interventions to manage alcohol misuse. We did not find RCT evidence regarding interventions directly targeting cholesterol levels, hypertension, diabetes mellitus or CVD multiple risk factors.

Regarding pharmacological interventions for managing weight, metformin appeared to be most effective over an average 13-week period, with a modest weight loss of nearly 3 kg on average. 98 Evidence from the 10 more recent individual trials focused mainly on topiramate, which was also shown to be similarly effective in managing weight,57,67,70 although adverse effects may limit its tolerability. 17 Incidentally, it also prevented an increase in the level of cholesterol. 70

For behavioural interventions for managing weight, those aimed at individuals (rather than groups), with both dietary and physical activity components were most effective in reducing BMI. 96 No trials were identified that assessed the effectiveness of physical activity alone. The most effective interventions to reduce BMI were those combining dietary, physical activity and behavioural support. 63,83,85 although adverse effects may limit its tolerability. 17 There was some evidence of effectiveness of behavioural interventions on metabolic outcomes with one of three RCTs reporting a significant effect on plasma glucose levels. 62

The one trial of a combined pharmacological and behavioural (diet and physical activity) intervention was significantly more effective in reducing BMI, weight and fasting glucose than the pharmacological or behavioural components of the intervention alone. 63

Clinical implications

CVD risk could be reduced in people with SMI by delivering combined pharmacological and behavioural interventions for both smoking and weight reduction. The exact content of interventions could be tailored to the preference of the individual as a variety of elements have been shown to be effective, mostly for individuals but also in groups.

There is little specific evidence regarding the effectiveness of interventions to reduce other CVD risk factors commonly found in SMI, including dyslipidaemia, diabetes mellitus and hypertension; however, there is evidence from the general population on what interventions might work to reduce and manage CVD risk. Algorithms are also available to guide treatment recommendations for reducing CVD risk in the general population in both Australia and the UK. 38

Our research findings are in line with recent international guidelines in both England17 and Scotland127 regarding the management of physical health in people with psychosis or bipolar disorder. The lack of evidence regarding managing lipids, diabetes mellitus and hypertension in SMI requires attention. We need to know whether this disadvantaged group of people are accessing and receiving the best interventions to prevent CVD, and then whether interventions, such as statins and antihypertensive drugs, are decreasing excessive rates of CVD. This will require international research to determine whether or not these effective interventions are actually being offered and accepted by people with SMI and ultimately to assess whether or not the stubborn mortality gaps for people with SMI can be diminished in years to come.

Review limitations

It is not straightforward to synthesise all the evidence regarding CVD risk management in SMI. The individual studies frequently have their own design problems and limitations, and evidence is extremely heterogeneous in terms of interventions, settings, target groups and outcome measures. Factors limiting interpretation of findings included small sample sizes, varying or undefined inclusion criteria, setting, short length of follow-up and completeness of intervention description.

Nine of the 23 RCTs not included in the systematic reviews of pharmacological or behavioural interventions to manage weight were tested on SMI patients with a minimum BMI or minimum weight gain since commencing antipsychotic medication (e.g. BMI of > 25 kg/m² was one of the inclusion criteria). 56–58,65–68,79,83 This limits the generalisability of findings to all SMI patients and highlights a reactive rather than proactive approach to weight management in SMI.

Most of the trials identified by this review were conducted in secondary care limiting their generalisability to other settings, such as primary care.

A further limitation is the length of time between intervention end and collection of follow-up data. As is the nature of pharmacological interventions, follow-up data were collected at the end of the intervention which was, on average, 13 weeks after baseline assessment (aside from one that collected data 4 weeks after intervention end). Of the behavioural and combined pharmacological and behavioural interventions the average length of time between intervention end and collection of follow-up data was 19 weeks. This is a relatively short amount of time in the lifespan of people with SMI, so conclusions regarding longer term effectiveness cannot be made.

There was a lack of detail used to describe behavioural interventions. This problem has been acknowledged in the wider literature on interventions to change behaviour. 128 Although guidance exists to promote detailed reporting to allow replication;129 for some interventions, replication would not be possible based on the descriptions in published reports we have reviewed.

Study aim

The aim of this study was to assess the fidelity of delivery (adherence to the manual and training programme) of the Primrose intervention. The study also aimed to identify whether or not there were differences in the extent to which key intervention components were delivered.

Objectives

1. To assess whether or not pre-specified intervention components of the Primrose intervention are delivered by nurses and HCAs.

2. To assess whether or not appropriate behavioural goals that targeted cholesterol level were set in the first appointment.

3. To determine whether or not particular intervention components are delivered more than others.

4. To assess whether or not there is a difference in fidelity of delivery between nurses and HCAs.

Method

A fidelity assessment of audio-recorded Primrose intervention appointments. The intervention was designed to lower levels of cholesterol and other CVD risk factors in people with SMI in primary care and its effectiveness tested in a cluster RCT (Primrose programme).

Sample

A random 20% sample of received audio files were selected for the fidelity assessment as specified in the original funding application.

Data collection

Patients and providers were asked to provide written consent to audio-record all Primrose intervention appointments. Providers were requested to spend 60 minutes delivering the first appointment and 15–20 minutes on subsequent appointments and were supplied with a digital audio-recorder and written instructions on operating and returning the audio files to the study team. Providers were asked to audio-record all appointments with participants who had given consent and to upload each file via a secure data protection system (IDHS Safe Haven, University College London, London, UK).

Measures

Scoring

The checklists were applied to the transcripts and a score was generated for each intervention component. The scoring methods of Hardeman et al. 109 were used. The total number of behaviours delivered within an appointment, were divided by the maximum score possible on the checklist.

A score of ‘2’ was given for provider behaviours that were performed; a ‘1’ for provider behaviours that were ‘performed to some extent’; a ‘0’ for provider behaviours that were judged appropriate to do but were not performed; and ‘not applicable’ for provider behaviours judged not appropriate to perform. The scoring criteria for strategies that were ‘done to some extent’ were developed iteratively by two researchers reviewing the transcripts and comparing their individually assigned codes.

Analysis

Audio files were transcribed verbatim and anonymised by a professional third-party transcription service. All transcripts were checked by members of the research team to ensure content accuracy and anonymity. Each transcript was then coded independently by two researchers and scores were compared to determine the reliability of the scoring system. Any disagreements or variations in coding were resolved through discussion between researchers until a final score was agreed on.

Researchers scored intervention components as ‘missing’ when there was insufficient information available in the transcript to be able to allocate a score of ‘0’, ‘1’, ‘2’ or ‘N/A’. All fields scored as ‘missing’ or ‘N/A’ were excluded from the denominator in the final analysis.

The fidelity score for each appointment was expressed as the total mean percentage of overall manual-specified intervention components that were delivered in the appointment divided by the total number of activities deemed applicable. In addition, to allow for comparisons between intervention components and to determine whether or not some activities were delivered more than others, the mean total percentage score for each individual intervention component was calculated across all appointments included in the fidelity assessment. A higher percentage score indicates greater fidelity of delivery. We adopted thresholds used in other intervention fidelity work,141,142 where 81–100% constituted high fidelity, 51–80% was moderate fidelity and ≤ 50% constituted low fidelity.

Results

Out of a total of 813 appointments attended, 431 audio files (53%) were received by the study team. One or more appointment audio files were returned by 33 providers (80.5%) for 90 patients (73.1%). A number of appointments were not audio-recorded although patients’ consent had been given; providers gave the following reasons: telephone appointment, audio-recorder batteries ran out, faulty audio-recorder, provider forgot to record and patient retracted consent to record the appointment.

For the purpose of this study, we randomly sampled 20% of first appointment audio files (n = 14) and 20% of all subsequent appointment audio files (n = 72) to assess whether or not the intervention was delivered to protocol. Table 8 presents the total sample and the fidelity sample for patients, providers and audio files.

Provider characteristics

A total of 41 providers (18 nurses, 22 HCAs and one GP) from 38 GP practices were randomised to receive training to deliver the Primrose intervention to all recruited patients at their GP practice. In three GP practices, only two members of staff were trained because the original staff member left the practice part way through the study. All providers gave their consent to be audio-recorded.

Twenty-three providers were included in the fidelity analysis, of whom 13 (57%) were nurses, 9 (39%) were HCAs and one was a GP (4%). The majority of providers were female (95.7%) and of white ethnic background (95.7%). The mean age of providers was 45.9 years (range 30–60 years) and the average length of experience working as a nurse, HCA or GP was 10 years (range 10 months to 27 years). Eleven providers (48%) had no previous experience of being involved in research. Table 9 presents the characteristics of the fidelity and trial samples.

Patient characteristics

A total of 155 patients were randomised to receive the intervention and 139 patients (89.7%) gave their consent to be audio-recorded. A total of 123 (79.4%) patients attended one or more intervention appointments and one or more appointments were included in the fidelity sample for 52 patients (42.3%). Table 10 presents the characteristics of the fidelity and overall intervention samples. The samples appear to be evenly matched with the exception of a lower number of patients in the ≥ 70-year age category in the fidelity sample compared with the overall intervention sample. The majority of patients in the fidelity sample were female (54%) and of white ethnic background (90%). The mean age of patients was 51 years in both groups with most patients falling into the 40- to 49-year age category (29%) and 46% of patients had a diagnosis of bipolar disorder.

Intervention appointment characteristics

The mean length of the first appointments in the fidelity sample was 28 minutes (range 19–42 minutes) and in subsequent appointments 14 minutes (range 4–32 minutes). More than half of the fidelity sample appointments (n = 49, 57%) were delivered by a practice nurse, 36 (41.9%) by HCAs and one (1.2%) appointment was delivered by a GP.

Fidelity

A total of 67.7% (median of 65.2%) of intervention manual-specified activities were delivered across all appointments, although there was considerable variation among activities. Mean fidelity to specific intervention components ranged from 47.8% for ‘forming habits’ to 90.2% for ‘reviewing progress’ across all appointments. Average inter-rater reliability for coding was 86% with a Cohen’s kappa of 0.668 (95% CI 0.63 to 0.70). Table 11 presents the percentage adherence to each specified intervention component in appointment 1, subsequent and all appointments.

Missing and not applicable intervention components

In total, 8.2% of all intervention components were not applicable and 4.8% were ‘missing’. Table 12 provides a breakdown of not applicable and missing scores by intervention component. Follow-up of supportive others had the highest amount of missing information in subsequent appointments, whereas ‘coping with setbacks’ was the intervention component that was the most likely to be not applicable.

Fidelity to setting a behavioural goal

‘Setting a behaviour goal’ was specified as an intervention component in appointment 1 and, therefore, only the CVD risk factors and agreed goals for the appointment 1 sample (n = 14) were explored. Table 13 presents each patient’s primary CVD risk factor, the risk factor that was targeted in appointment 1, the goal that was actually set, the goal that we would have expected to have been set and the awarded score.

Eight (57.1%) patients set a goal around behaviours that had an impact on lowering levels of cholesterol, while three (21.4%) patients set a goal around smoking cessation, two (14.3%) patients set a goal around reducing alcohol intake and one (7.1%) patient did not set a goal. No patients addressed statin adherence or initiation (Table 14).

Fidelity by appointment number

Higher adherence to intervention components was seen in appointment 1 (72.5%) than in subsequent appointments (66.6%) and adherence within individual appointments varied from 40% (appointment 12, n = 1) to 81.4% (appointment 8, n = 5) (Figure 3).

FIGURE 3.

Average fidelity score per appointment.

Fidelity by provider type

A mean of 79.5% of intervention manual specified activities were delivered during appointments by nurses (n = 13) in comparison to 64.3% in appointments delivered by HCAs (n = 9). The GP was excluded from this analysis. The difference in fidelity scores between nurses and HCAs was significant [t(20) = 2.32; p = 0.037].

Summary of findings

This study presents a thorough assessment of fidelity to a behaviour change intervention for lowering levels of cholesterol and other CVD risk factors delivered in a primary care trial (Primrose).

In response to the first objective of whether or not the intervention components of the Primrose intervention were delivered as specified, our results showed that fidelity was 67.7%. This score indicates moderate fidelity to the intervention. 141,142 This result is comparable to similar fidelity assessments of behavioural interventions including an assessment of a cardiovascular prevention programme110 but higher than the fidelity of delivery of a physical activity intervention (44%). 109 The fidelity observed in this study was slightly lower than for an intervention to increase attendance at a stop smoking service (71.3%);143 however, it was recognised by the authors that the sessions were delivered by two advisors who created a sense of shared responsibility, mitigated feelings of nervousness and encouraged higher adherence to the protocol in an environment where one assessor was being observed by the other.

The second objective sought to determine whether or not a relevant behavioural goal targeting cholesterol level was set at the first appointment. All 14 patients sampled for the first appointment assessment had a primary CVD risk factor of raised cholesterol levels and should have been supported by the provider to set a behavioural goal which would have had a direct impact on lowering levels of cholesterol. Although six of the patients had a risk score that exceeded the threshold for statin prescription (> 10%) and five of those patients were already prescribed a statin, none of these patients set a goal around increasing adherence to statins. Only eight (57.1%) patients set a goal related to behavioural modification (e.g. diet or physical activity) to lower levels of cholesterol, while three (21.4%) patients set a goal on smoking cessation, two (14.3%) patients on reducing alcohol intake and one (7.1%) patient did not set a goal.

As behavioural goals were intended to be patient-led with the support of the provider, the patient may have declined to work on reducing their levels of cholesterol and instead chose to set a goal that they felt was more relevant to their health profile and could produce more tangible results on their health, such as stopping smoking or reducing alcohol.

For the third objective, investigating whether or not particular intervention components were delivered more than others, the highest adherence was seen for ‘reviewing progress with goals’ (90.2%) and the lowest for ‘forming habits’ (47.8%). Providers may have been more comfortable reviewing patient progress as they may already be familiar with providing short-term support in usual practice, such as monitoring blood test results, weight change and blood pressure. Longer-term monitoring and support for behavioural change in primary care is perhaps less common because of short consultation times, limited resources and less continuity of care, which could explain why providers may have found it difficult to address habit formation. This finding may also suggest that if forming habits is considered a crucial element for behaviour change then low adherence to this activity may lead to decreased effectiveness of the intervention on health outcomes over time.

‘Goal-setting’ (80%) was discussed in depth at the start of the intervention training, whereas those BCTs with the lowest adherence scores (‘forming habits’ and ‘coping with setbacks’) were covered towards the end of the training in less detail. It is possible that providers remembered the earlier, more detailed aspects of the training over later topics when fatigue may have set in. A way of improving adherence to these techniques could be to revisit these BCTs at the start of the second training day or include specific role-play activities which explore their applicability in more detail.

The fourth research question assessed whether or not there was a difference in fidelity between provider types. Nurses were more adherent than HCAs, suggesting that the previous experience and skills of providers may need to be taken into consideration for future intervention training programmes. The Primrose trial originally aimed to train practice nurses as intervention providers but, owing to a shortage of nurses in some GP practices and the increasing involvement of HCAs in CVD prevention work, HCAs were trained to deliver the intervention to recruited patients if a nurse was unavailable. If the intervention was delivered solely by nurses, as originally intended, then adherence to the intervention may have been higher, clinical goals may have been more relevant and the intervention may have had a stronger impact on study outcomes.

First appointments were on average 28 minutes (range 19–42 minutes) in duration. This was less than half the time that providers were asked to spend with patients at the first appointment (60 minutes). This observation could be used to explain the moderate fidelity as there may not have been enough time to cover all of the specified components of the intervention.

Strengths

A strength of this study was the robust methodology. Transcripts were double coded and anonymised to remove potential bias. Inter-rater reliability of scoring between two researchers was 86% with a Cohen’s kappa of 0.668 (95% CI 0.63 to 0.70), indicating a moderate level of agreement between raters. 144 Our assessment of a random 20% of the sample is also seen as the recommended amount for fidelity studies. 145

An audio file return rate of 53% was achieved but the number of data provided was large. Health professionals with limited research experience and who had limited time for administrative tasks and a lack of technological support were responsible for operating the recorders and uploading the files rather than trained researchers. The complexity of co-ordinating the return of audio files across 41 providers in 38 GP practices must also be taken into consideration.

A further strength of the study was its external validity. Assessing the fidelity of delivery of an intervention in a real-world primary care setting with primary care nurses and HCAs, rather than in an artificial research setting with trained intervention specialists may increase the ability to apply the results of this study to training and intervention recommendations for health professionals working in primary care.

Limitations

A potential limitation of the study was the representativeness of the fidelity sample compared with the trial sample of intervention providers. There was an over-representation of nurses in the fidelity sample (60.9% compared with 43.9% in the trial) as well as an over-representation of providers who had previous experience of research (52.2% compared with 39% in trial). Although the sample was randomly generated by a statistician not involved in the fidelity analysis, the over-representation of nurses could be explained by a higher return of audio files and a greater number of overall appointments being delivered by nurses than being delivered by HCAs, meaning that nurses comprised a larger proportion of the sample in the fidelity study than in the trial. Similarly, those involved in previous research may also have been more likely to return audio files meaning they were more likely to appear in our fidelity sample. Future intervention training may need to be adapted to troubleshoot potential barriers for HCAs and those with limited research experience for providing the suggested number of intervention appointments to patients and returning the audio files.

Another limitation of the study was that, owing to a decreasing number of subsequent appointments from appointment nine onwards, the potential pool of audio files that the sample could be chosen from was small and for some appointments only one audio file was randomly selected for the fidelity assessment. Interpretation of the results of these subsequent appointments should therefore be treated with caution.

The Primrose intervention included up to 17 provider behaviours across seven different intervention components in the first appointment and 16 provider behaviours across eight different intervention components in subsequent appointments. Furthermore, patients presented with a range of CVD risk factors meaning that providers had to tailor the intervention to each individual. This was a complex intervention and previous research has shown that intervention complexity has a negative impact on fidelity. 146 This might contribute to the moderate score that was observed.

The structure and content of the intervention training could have been another contributing factor to explain the moderate fidelity score as the training may not have been detailed enough, or even too complex for providers who had little previous experience of delivering behaviour change interventions. 147 Providers were instructed to complete their first intervention appointments soon after having attended the intervention training sessions which could explain why overall adherence was higher in the first appointment (72.5%) than in subsequent appointments (66.6%).

Future work could take into consideration the wider content and conversational aspects of the intervention appointments, which were beyond the scope of this study.