Do higher primary care practice performance scores predict lower rates of emergency admissions for persons with serious mental illness? An analysis of secondary panel data

Notes

Permissions

Chapter 1 Introduction

Serious mental illness (SMI) encompasses a set of chronic enduring conditions such as schizophrenia, bipolar disorder and other psychoses. Although some people make a full recovery, most will develop a lifelong illness. 1 Schizophrenia is a psychotic disorder marked by severely impaired thinking, emotions and behaviours. People with schizophrenia are typically unable to filter sensory stimuli and may have altered perceptions of their environment including delusions and hallucinations. If untreated, people with schizophrenia may gradually withdraw from interactions with other people and lose their ability to take care of their personal needs. Schizophrenia is a disease that usually begins in early adulthood and the average age at onset is 18 years in men and 25 years in women. 2 Psychosis is a symptom or feature of SMI, typically characterised by radical changes in personality, impaired functioning, and a distorted or non-existent sense of objective reality exhibited by delusions and hallucinations. Bipolar disorder is a mood disorder that causes dramatic emotional changes and mood swings, whereby individuals experience alternating episodes of mania, or hypomania, and depression.

The prevalence of bipolar disorder is about 1–2% of the UK population, although bipolar spectrum disorder may affect as many as 8%. 3 The point prevalence of schizophrenia is around 0.7%4 and the lifetime prevalence around 1%. A systematic review of the incidence and prevalence of schizophrenia and other psychotic disorders in England found an overall (pooled) annual incidence for all psychotic disorders of 32 cases per 100,000 people,5 with much higher rates for psychotic disorders in young adults, in men, in black and minority ethnic groups and in more deprived neighbourhoods.

The total annual economic burden of schizophrenia (non-affective psychoses) in 2009 was estimated at £8.8B, of which service costs contributed 40%, informal care 13% and lost employment 47%, while for affective psychoses (bipolar disorder) the total cost to UK services and society per annum was estimated at £5.0B, with 80% coming from NHS costs, 3% from informal care costs and 16% costs of lost employment. 5 Therefore, SMI creates a high cost to society as well as to NHS services. Mental health is the single biggest programme budget expenditure area in the NHS out of 23 main programmes of care in England,6 bigger than cancer or cardiovascular disease, and schizophrenia and psychoses are a key driver of length of stay (LOS), bed-days and resource use in the NHS. 7

Life expectancy for people with schizophrenia and bipolar disorder is usually around 20 years less than for the general population,8–13 and people with a SMI die prematurely, the majority from preventable causes. People with a SMI are at higher risk of physical ill-health and thus hospitalisation. 14–19 Compared with the general population, people with a SMI have double the risk of diabetes, two to three times the risk of hypertension and three times the risk of dying from coronary heart disease,20,21 and experience a 10-fold increase in deaths from respiratory disease. 2,22 Owing to much higher smoking rates than the general population, smoking-related diseases, heart disease and premature death are more common in people with SMI. 23 People with SMI are at much higher risk of obesity because the atypical antipsychotic medications they take are associated with weight gain6 and their illness reduces their activities and impairs their ability to exercise. Poor compliance with medication is well recognised among people with these diagnoses and this may lead to relapse, poorer outcomes and admissions. Schizophrenia and bipolar disorder rank among the top 10 causes of disability in developed countries worldwide. 5

Despite its prevalence, considerable disease burden, poor outcomes and costs, there has been little empirical research on the processes of care for people with SMI. They are often disenfranchised and marginalised and experience stigma and thus do not receive the same priority as other chronic disease conditions. 2,24

In the English NHS, a number of different services provide care for people with a SMI. There has been a general trend away from long hospital stays in favour of shorter-term pharmacological stabilisation in hospital, followed by longer multidisciplinary follow-up in the community or primary care setting. In secondary care, patients may be seen by crisis resolution and home treatment (CRHT) services which provide intensive home-based care for individuals in crisis as an alternative to hospital treatment. 25 However, most people with a SMI are treated in primary care by their general practitioner (GP). People with a SMI consult their GPs more frequently26 and are in contact with primary care services for a longer cumulative time than people without mental health problems. 27,28 Recent evidence from the UK finds that around 31% of patients with a SMI are treated solely by their GP or other primary-care clinician and the estimated national rate is around 57% for schizophrenia and 38% for bipolar disorder. 29 Primary care is therefore central in the care of people with a SMI. The GP oversees care, prescribes medication and provides both mental and physical health services.

If we accept that high-quality provision of primary care may allow people to be cared for in a more comprehensive and proactive way, reducing the chance that they will be admitted to hospital as an emergency, we may find higher-quality primary care is associated with fewer unplanned admissions. In the UK, unplanned hospital admissions have risen steadily over the past 10 years and approximately 35% of all hospital admissions are unplanned, which places an increasing source of pressure on health system resources, costing the English NHS around £11B per annum. 30 Policy-makers are increasingly seeking ways to reduce unplanned hospital admissions, especially for people with long-term conditions.

A number of interventions have been proposed in primary care as a means to reduce unplanned hospital admissions. The evidence for an association between higher quality of primary care as an intervention and reduced rates of admission is however mixed. 31 Lower rates of admission for asthma were found in practices whose prescribing patterns suggested better preventative care. 32 Provision of diabetes clinics in primary care was significantly associated with reduced admission rates for diabetes, but the provision of asthma clinics was not associated with a similar reduction in admissions. 33 Conversely, a systematic review showed that high standards of diabetes care in primary care did not necessarily lead to reduced hospital admissions. 34

General practitioners are incentivised through the NHS Outcomes Framework to improve quality of care and outcomes for patients, with a stronger emphasis on mental health. 35 Quality indicators for mental health are routinely measured in English primary care as part of the Quality and Outcomes Framework (QOF) which was introduced in 2004. 36 The QOF is a voluntary incentive scheme for primary care practices which offers financial rewards for good-quality care such as meeting targets on clinical, organisational and patient experience indicators. In practice, nearly all GP practices participate. This may be related to the generous financial incentives attached to achievements on the QOF and the ease with which many practices have fulfilled the requirements. By encouraging the provision of better-quality care, the QOF has the potential to reduce unplanned, preventable hospital admissions, but evidence for this effect is mixed.

Table 1 shows a summary of the evidence on the association between the QOF and admissions for a range of disease conditions. No association has been found between admission rates and primary care quality indicators for coronary heart disease, asthma or chronic obstructive pulmonary disorder (COPD). 37–40 However, other studies have found a significant association between lower levels of achievement and higher emergency admissions for diabetes40,41 and a small effect for stroke. 42 Most of these studies are, however, based on cross-sectional data rather than longitudinal panel data. There has been no evidence to date on the relationship between primary care quality and admission rates for SMIs.

Effective primary care can have an important preventative role, and could therefore be associated with a reduction in emergency admissions. Conversely, better quality of care may result in more health problems being identified as part of regular screening activities and more frequent GP–patient contacts, thereby leading to more elective admissions for hospital care. If better-quality primary care can reduce costly emergency hospital admissions it may have knock-on effects for NHS expenditure and resource use. Better-quality primary care may also reduce LOS, if patients are effectively cared for outside hospital. LOS for patients with a SMI is typically much longer than for other patients and better management in primary care could shorten their LOS in hospital. 16

1. Our first research question therefore is whether or not better primary care practice performance on specific mental health QOF indicators is associated with:

   1. lower rates of emergency hospital admissions for SMIs for practice patients with a diagnosis of a SMI.

   2. lower rates of emergency admissions for SMIs for practice patients with a diagnosis of bipolar disorder.

   3. lower rates of emergency admissions for physical conditions for practice patients with a current or previous diagnosis of a SMI.

   4. higher rates of elective admissions for physical conditions in patients with a current or previous diagnosis of a SMI.

2. Our second research question relates to whether or not better-quality primary care as measured by specific mental health QOF indicators is associated with reduced resource use in terms of shorter LOS for people with a SMI following admission for a SMI.

3. Our third research question is whether or not better-quality primary care as measured by specific mental health QOF indicators is associated with reduced resource use in terms of lower secondary care expenditure for mental health services for people with SMIs.

Our null hypotheses are that there is no association between primary care quality and either admissions, LOS or costs. Our alternative hypotheses, as presented in Table 2, are that preventative care could lower emergency hospital admissions, reduce LOS and reduce mental health expenditure; and that regular screening could increase elective admissions.

The remainder of the report is structured as follows. Chapter 2 describes the QOF and the SMI domain within the QOF as well as how some cases are excluded from the calculation of the achievement rates in QOF (exception reporting). Chapter 3, Does better primary care reduce hospital admissions?, gives the data, empirical approach, results and sensitivity analysis for the first research question; Chapter 3, Does better primary care reduce inpatient length of stay?, provides the data, empirical approach, results and sensitivity analysis for the second research question and Chapter 3, Does better primary care reduce cost of care?, provides the data, empirical approach, results and sensitivity analysis for the third research question. Chapter 4 discusses the findings, while Chapter 5 includes discussion on the implications for research and practice (see Chapter 5, Implications for research, and Chapter 5, Implications for practice).

Chapter 2 Measures of quality of primary care for people with serious mental illness

Exception reporting in the Quality and Outcomes Framework

Practices can ‘exception report’ patients from specific indicators. 50 The GMS contract sets out valid exception reporting criteria, such as the patient is deemed unsuitable for treatment, is newly registered with the practice or newly diagnosed, or that the patient exercises informed dissent. This means that data on these individuals are removed from the achievement calculation for payment purposes. An analysis of the prevalence and reasons for exception reporting in 2008/9 found wide variation between practices and between indicators although relatively few patients were excluded for informed dissent. 51 Exception reporting boosted practice income by an average of £3834. One-quarter of this amount was explained by 2 of the 62 indicators studied, one of which was MH9, the annual review indicator. On average, practices exception reported 14% of eligible patients for this indicator, which is high relative to other indicators.

Reporting cases as ‘exceptions’ may reflect good-quality care – GPs are carefully reviewing cases to establish their eligibility for treatment – but could alternatively reflect ‘gaming’ by GPs, who can increase the number of points they earn by reducing the eligible population inappropriately,52 and so the legitimacy of exception reporting is ambiguous. As a conservative approach to assessing performance, we therefore include all patients in the denominator for each QOF indicator: those recorded as eligible, plus those who are potentially eligible but who were ‘exception reported’. Achievement is calculated as:

⁽¹⁾ achievement = A ( A + N A + E ) ,

where A denotes the number of patients for whom the indicator was achieved, NA denotes the number of patients for whom the indicator was not achieved and E denotes the number of patients exception reported. For simplicity, we refer to the sum of (A + NA + E) as the ‘register’ for this specific indicator. The exception report rate is derived as:

⁽²⁾ exception rate = E ( A + N A + E ) .

The QOF incentive regime rewards GPs on the basis of their achievement adjusted for exceptions, which is calculated as:

⁽³⁾ adjusted achievement = A ( A + N A ) .

We chose to analyse our data based on the achievement, not the adjusted achievement. This approach is justified by the fact that, given our data, we cannot distinguish whether an admitted patient did not receive the ‘QOF treatment’ or was exception reported (E). We conduct sensitivity analysis to ascertain whether or not our results are sensitive to this analytical choice.

A diagrammatic overview of how these performance indicators are constructed is provided in Figure 1.

FIGURE 1.

Overview of performance measures constructed for the regression analyses.

Chapter 3 Empirical analysis

Does better primary care reduce hospital admissions?

Overview

This section focuses on our first research question, namely whether or not better-quality primary care is associated with reduced unplanned hospital admissions for both mental and physical care. Good-quality primary care might be expected to reduce emergency hospital admissions for people with SMI. Conversely, better quality of care may result in more health problems being identified as part of regular screening activities and more frequent GP–patient contacts, thereby leading to more planned (elective) admissions for hospital care. Our null hypothesis is that better primary care, as measured by the SMI QOF indicators, has no effect on the number of hospital admissions, both elective and emergency, for both physical and mental health conditions. Our alternative hypothesis is that better primary care is associated with the number of hospital admissions, and we expect this association to be negative for emergency admissions, for both physical and mental health conditions, and positive for elective admissions for physical health problems.

We combined patient- and GP practice-level data for the study period April 2006–March 2011. The unit of analysis was the GP practice.

We considered three types of admissions and analysed these separately:

1. admissions for a mental health crisis – as indicated by a main diagnosis of a SMI [International Classification of Diseases, 10th revision (ICD-10) codes F20–F31]

2. admissions for non-mental health-related care (‘physical care’) – admitted as an emergency patient

3. admissions for non-mental health-related care (‘physical care’) – admitted as an elective patient.

Identification of patients was based on ICD-10 diagnosis codes. Physical care admissions included admissions for all primary diagnoses excluding R69 (‘Unknown and unspecified causes of morbidity’) and excluding all diagnosis codes in the range F00–F99 (‘Mental and behavioural disorders’).

Furthermore, because some QOF indicators specifically apply to people with bipolar disorders, we also considered:

1. admissions for a mental health crisis – for people with a diagnosis of bipolar disorder only (ICD-10 codes F30–F31).

All SMIs and bipolar admissions were deemed to be emergency admissions (irrespective of how they are coded by the hospital), which is in line with expert policy and clinical guidance provided by our steering group. In practice, there is inconsistency of coding by providers and it makes sense to combine admissions. Our interpretation is that considering all admissions as emergencies is a crude, but sensitive, metric of avoidable admissions.

In the analysis of bipolar admissions, we had data from 8042 practices; the remaining three analyses were based on data from 8223 practices.

Data

We merged QOF data from around 8500 GP practices in England with admissions data from Hospital Episode Statistics (HES) for the study period April 2006–March 2011 using a unique GP practice identifier. These data were linked to publicly available information on GP practice characteristics, characteristics of their patient population, such as disease prevalence, and to population characteristics such as deprivation and other potential confounders that are recorded at small-area level [i.e. lower super output areas (LSOAs)]. We also controlled for measures of access to care. All analyses were carried out at GP practice level and data were aggregated accordingly. We provide details about the individual data sets and the linkage process in the following subsections.

Hospital Episode Statistics

The HES database records all inpatient and outpatient activity in England that is funded under the NHS. The inpatient component of the HES data warehouse consists of approximately 18 million records per year, each of which provides detailed information about the patient’s demographic characteristics, medical condition, care pathway, as well as the GP practice with which the patient is registered. HES data are reported at the level of finished consultant episodes (FCEs) and a new FCE is created every time a patient is discharged from the care of one consultant to another consultant. To capture the entire care pathway and derive correct admission numbers, we converted FCEs to continuous inpatient spells (CIPSs) which cover the entire period from admission to final discharge. CIPSs also allow for transfers between providers of inpatient care.

We extracted information on all NHS-funded inpatient activity for people aged 18 years or over and diagnosed with a SMI. To identify people who have been diagnosed with a SMI, we searched all primary and secondary diagnosis fields for the relevant ICD-10 diagnosis codes. Although the QOF uses diagnosis to define patients eligible for practices’ SMI registers, ICD-10 is not used. Instead, GP practices record diagnoses using Read codes. We used a cross-mapping provided by the NHS Information Centre to translate Read codes to ICD-10 codes. We considered diagnosis codes that cover two large groups of SMI: schizophrenia, schizotypal and delusional disorders (F20–F25, F28–F29), and bipolar and mood affective disorders (F30–F31) (Table 4).

TABLE 4 - International Classification of Diseases, 10th revision codes used to identify patients with SMI

ICD-10 code	Description
F20	Schizophrenia
F21	Schizotypal disorder
F22	Persistent delusional disorders
F23	Acute and transient psychotic disorders
F24	Induced delusional disorder
F25	Schizoaffective disorders
F28	Other non-organic psychotic disorders
F29	Unspecified non-organic psychosis
F30	Manic episode
F31	Bipolar affective disorder

For some episodes of physical care, a diagnosis of SMI or bipolar affective disorder may not have been made or recorded in the inpatient records even though the patient suffers from this condition. This may occur if (1) the diagnosis is not deemed clinically relevant for the physical care provided, that is, the patient was treated for an unrelated medical problem and the psychiatric comorbidity did not interfere with the treatment; (2) the diagnosis is not important for reimbursement purposes; (3) the condition is not apparent at the time of assessment; (4) clinical coding is poor; or (5) to avoid any stigma associated with the diagnosis. As a result, people with SMIs (including bipolar disorder) who are admitted for physical care may not be identified as such when detection is based solely on diagnostic information contained in the current inpatient record.

To capture all relevant activity, we therefore linked patient records across time, based on the patient’s unique identifier, and identified all secondary care provided to this patient on or after a diagnosis of a SMI/bipolar has been made, whether or not the diagnosis is recorded in this specific inpatient record. This identification strategy can be justified on the grounds that SMI is an enduring illness that may increase or reduce in burden over time but rarely resolves. To ensure that activity at the beginning of our study period is identified correctly, we extended our search period retrospectively by 5 years to April/March 2011. Hence, if a patient was diagnosed with bipolar affective disorder (ICD-10: F31) in 2002 and received inpatient care for a physical condition in 2008, we will count the activity in 2008 even if no diagnosis of bipolar affective disorder was recorded then.

All admissions for patients who changed GP practices within a financial year were excluded from the data set because (1) it is not possible to assign the admission to a practice and (2) because it is unclear whether the patient changed practices before or after an admission. The GP practice identifier in HES refers to the practice to which the patient is discharged. We assumed that patients were already under the care of the same GP at admission if no change in practice association is evident from the data.

Quality and Outcomes Framework data set

We extracted data on practice quality performance from the QOF data set. For each GP practice, we obtained information on the number of patients on the SMI register, that is, the number of patients at risk of admission, and the practice’s achievement on four SMI-related QOF indicators discussed previously. We linked these data to the aggregate practice-level admissions data derived from HES through the unique practice-year identifier.

Practices were excluded from our sample if they did not report a SMI register or if the number of patients on this register was below 5. The latter exclusion criterion is justified by the potentially noisy measure of practice performance that would be derived if achievement were calculated on a very small number of patients. Owing to the typically small number of bipolar patients in each GP practice, we did not apply this exclusion criterion to the analysis of bipolar admissions, that is, all practices were retained that had at least one bipolar patient registered with them. Furthermore, we excluded practices for which the SMI or bipolar registers were inconsistent across indicators, for example when the denominator of MH6 was different from the denominator of MH9 even though both refer to the same set of patients with SMI in the practice.

Data sets used to generate other covariates

We controlled for a number of GP and practice characteristics from the GMS data set and attribution data set (ADS). These include the 2-year moving average practice list size as well as the average age of GPs, proportion of male GPs, whether or not the practice operates single-handed, and whether or not the practice is contracted under the Personal Medical Services (PMS) scheme. We dropped observations (practice-year) with a practice list size with fewer than 1000 patients because these are deemed unusually small and uncharacteristic of the way in which primary care is normally organised in the English NHS.

To control for local population characteristics, we linked data from the Neighbourhood Statistics Census (2001)53 and the Index of Multiple Deprivation (IMD; 2004) which is available at the LSOA level. LSOAs are defined geographic units that cover an average population of 1500 individuals. There are 32,482 LSOAs in England. GP practices typically care for people who reside in multiple LSOAs and the ADS provides a breakdown of the practice population by LSOA. Based on this information, we derived a weighted average of the local population characteristics of the practice and assigned this to the practice. From the ADS data set we obtained the average age and male proportions of each practice’s registered patients. We also derived a measure of deprivation based on the proportion of the population claiming incapacity benefit for mental health disorders – this variable is part of the IMD employment domain. Finally, we derived measures of ethnicity (percentage non-white) and rurality (percentage living in urban areas). ADS data are collected at the beginning of each financial year, but QOF data are collected at the end of the financial year. We therefore adjusted the estimates based on ADS by taking moving averages across 2 years of data.

We constructed several measures of access to care. First, we derived a measure of access to secondary care based on the distance between GP practice and the nearest (1) acute hospital and (2) mental health hospital. Distance was calculated on the basis of postcodes and grid co-ordinates. We also controlled for the availability of CRHT teams that provide alternative home care in an emergency and play a ‘gatekeeping’ role in admissions to hospital. 25,54,55 Data on CRHTs were collected as part of the Mental Health Services Mapping Data between 2000 and 2009 at the level of ‘local implementation teams’ (LITs) which partly cover the geography of local authority social services. 56 There is an almost one-to-one correspondence between LITs and the approximately 150 commissioning organisations in England at that time, primary care trusts (PCTs). Since we were only able to obtain service mapping data for two of our five study years (2008 and 2009), we instead used PC-level fixed effects to model differences in service provision by CRHTs. These 156 dummy variables capture all time-invariant differences between PCTs in terms of their resource capabilities. We tested the inclusion of LIT data for the 2 years that we had available and assuming the rest of the period’s data constant. This made little difference from the results using PCT-level fixed effects.

Finally, to reflect differences between practices and regions in terms of supply and access to care, we recorded the catchment population prevalence of NHS community psychiatric residential beds, the percentage of practice patients able to book an appointment within 48 hours (measured in the GP patient survey) and a measure of informal care provision (% of the catchment population providing informal care) based on census data; the last is intended to acknowledge that the level of informal care provided is often high for people with SMIs and may be considered a substitute for inpatient care. 57

Empirical approach

The aim of this empirical analysis is to relate the number of patients admitted to hospital from a GP practice to the practice’s quality performance, controlling for other factors that may drive admissions but are unrelated to the quality of care provided. The number of admissions per GP practice is a non-negative integer (i.e. count variable). We therefore estimated mixed-effects count models that acknowledge the data generating process and the nested structure of annual counts of admissions reported for each GP practice. 58,59 We estimated separate models for each admission type and allowed the two QOF indicators MH6 and MH9 (MH4 and MH5 for bipolar admissions) to enter separately (individually) or simultaneously (jointly).

Let adm_it be the number of hospital admissions from GP practice i = 1,. . ., I within the year t = 2006,. . ., 2010. The number of SMI (or bipolar) patients at risk of admission is denoted as risk_it and enters the model as an offset variable. We specified the Poisson regression model as follows:

⁽⁴⁾ adm i t = risk i t ∗ γ i ∗ exp ⁡ ( Q ′ i t δ + X ′ i t β + T ′ t λ ) ,

where Q_it is the measure of GP practice quality as measured by the QOF, X_it is a vector of covariates that capture differences in the practice patient population and the supply of and access to other mental health-care resources as well as an overall intercept term, and T_t is a vector of time indicator variables to control for general trends in admissions. We also introduced a GP practice-specific effect γ_i that captures unobserved, time-invariant differences between practices in terms of their admission propensity.

The GP practice effects γ_i are assumed to be randomly drawn from a gamma distribution with mean 1 and variance α and assumed to be uncorrelated with the other covariates. Alternatively, one can also assume a normal distribution for the random effects. However, the model with gamma distributed effects has a closed form solution and is therefore typically preferred. 59 We did not model the GP practice effects as fixed effects using indicator variables because (1) this would preclude the estimation of PCT fixed effects, and (2) because many of the dependent and independent variables of interest vary little over time, that is, there would be insufficient within-GP practice variation to estimate the model. In order to reduce any potential bias from unobserved practice-specific confounders and make the assumptions underlying the random-effects model more tenable, we included pre-sample baseline admission numbers per GP practice as an additional regressor. 60 These are taken as the average number of hospital admissions within the financial years 2003/4 and 2004/5.

As it is common to all Poisson models, the conditional variance, V(adm_it|Q_it,X_it,T_t,γ_i), is constrained to be equal to the conditional mean, E(adm_it|Q_it,X_it,T_t,γ_i). This property is known as equidispersion and is often found to be violated in empirical data. To allow for over- or underdispersed data, we derived bootstrapped standard errors for all parameter estimates using 200 replications and sampling with replacement.

We ran three sets of GP-level analyses. In the first analysis, our base case, the response variable is the number of admissions per practice per year. In the second GP-level analysis, we run a sensitivity analysis, in which our response variable is the number of people admitted at least once per practice per year. In the third GP-level analysis, we run a sensitivity analysis in which we include an unspecified main diagnosis code for SMI admissions. The sensitivity analyses are further described in Chapter 3, Sensitivity analyses.

Although the results from panel data models are regarded as more robust, we also estimated separate cross-sectional models for each of the years 2006–10 individually. We adapted the empirical specification presented above by dropping both the GP practice effects γ_i and the time indicator variables T_t, and report robust standard errors.

All coefficient estimates are presented as incidence rate ratios (IRRs). The IRR represents the estimated event rate under one scenario over the estimated event rate under a different scenario. For example, the IRR on a covariate indicating whether or not the practice is reimbursed under PMS measures the ratio of expected event rates under PMS over the expected event rate not under PMS. Values greater than one indicate that increases in the value of the covariate are associated with an increased number of admissions; this relationship is considered to be statistically significant (i.e. not a chance finding) if the lower confidence interval (CI) is also above 1. Similarly, the covariates with estimated IRRs smaller than one are expected to have a protective effect on admissions (with an upper CI below 1).

The presented IRRs reflect the percentage change in admissions for a unit change in the explanatory variable. This relationship is non-linear and IRRs need to be rescaled for changes that are smaller/larger than one unit using the following formula:

⁽⁵⁾ IRR x = IRR ^ ( 1 / x ) .

Here, x is the number of points in the new scale. Hence, if a unit change in a continuous variable is associated with 20% more admissions (i.e. IRR = 1.2), a 5% increase is associated with 0.92% more admissions (IRR10%=1.2(120)=1.0092). This transformation does not affect inferences, that is, the assessment of statistical significance. All models are estimated using the xtpoisson and Poisson commands in Stata 12.0 (release 12; StataCorp LP, College Station, TX, USA).

Results

Descriptive statistics

Our sample consists of 8223 GP practices that have reported treating patients with a SMI during the 5-year study period. The panel is unbalanced (mean t = 4.7) because for some years some practices either (1) do not report or participate in the QOF, (2) report having fewer than five patients with a SMI on their patient register, (3) report inconsistent QOF registers or (4) are yet to be established or have ceased to exist. The overall panel consists of 38,774 practice-year observations. Note that the number of practices and practice-year observations is somewhat lower for the bipolar sample because not all practices that treat people with a SMI (and produce a valid register) also treat people with bipolar disorder. The median number of people on the SMI register in a GP practice is 40 [interquartile range (IQR) 22–64] and the median number of people on the bipolar register is 6 (IQR 3–10).

Table 5 presents the total number of admissions, number of GP practices, number of GP practice-year observations, as well as the mean and median annual number of admissions per GP practice, all broken down by admission type. As expected from a count variable, the empirical distribution of the number of admissions is highly skewed. The median number of admissions per year and practice ranges from 6 (IQR 3–12) physical (emergency) admissions to 1 (IQR 1–2) bipolar admission. The mean number of admissions is generally higher. On average, the number of annual admissions per practice for SMIs was 3.52 admissions, and this ranged from 1.12 admissions [standard deviation (SD) 1.61 admissions] for bipolar admissions to 8.86 admissions (SD 9.24 admissions) for ‘physical’ emergency admissions. Figure 2 shows the development of the average number of admissions per GP practice over time.

TABLE 5 - General practitioner-level analyses: descriptive statistics for the samples

Admission type	Number of GP practices	Number of GP practice-years	Total number of admissions	Annual admissions per GP practice
				Mean	SD	50th percentile	25th percentile	75th percentile
SMI	8223	38,774	136,507	3.521	3.919	2	1	5
SMI with R69	8223	38,774	161,858	4.174	4.405	3	1	6
Physical elective	8223	38,774	128,382	3.311	4.628	2	1	5
Physical emergency	8223	38,774	343,486	8.859	9.244	6	2	12
Bipolar	8042	37,037	41,372	1.117	1.606	1	0	2

FIGURE 2.

Average number of admissions per GP practice.

There was a marked increase in the average number of admissions for physical care over time with the number of emergency admissions nearly doubling over the course of the 5-year period. This is in line with national trends which between 2001 and 2011 saw an increase in the number of emergency admissions per year for ambulatory care sensitive conditions of 40% and for all other conditions of 34%. 61 In contrast, the average number of admissions with a main diagnosis of SMI or bipolar disorder remained relatively stable over time.

Table 6 presents the achievement and exception rates for the four QOF SMI indicators. On average, GP practices report to have a comprehensive care plan in place for 76% of their patients with a SMI (MH6), and have reviewed 81% of their patients with a SMI during the last 15 months (MH9). However, 12% of patients were excluded from these SMI QOF indicators and practices did not achieve the indicators for the remaining 12% (MH6) and 7% (MH9) of eligible individuals. The achievement rates were higher (95% and 83%) (exception and failure rates were lower – 3% and 9%, respectively) for the QOF indicators MH4 and MH5, which are specific to bipolar patients. Figure 3 displays the empirical distribution of achievement scores for each of the four indicators across all practices in 2010/11. The distribution is approximately representative for the other 4 years. Table 7 presents descriptive statistics for all dependent and independent variables in our regression model.

TABLE 6 - General practitioner-level analyses: QOF achievement and exception rates

QOF indicator	n	Mean	SD	Median	25th percentile	75th percentile
SMI
MH6 achievement	38,774	0.76	0.17	0.79	0.66	0.88
MH6 exception rate	38,774	0.12	0.11	0.09	0.04	0.17
MH9 achievement	38,774	0.81	0.13	0.83	0.75	0.90
MH9 exception rate	38,774	0.12	0.11	0.10	0.04	0.18
Bipolar
MH4 achievement	37,190	0.95	0.12	1.00	0.93	1.00
MH4 exception rate	37,190	0.03	0.09	0.00	0.00	0.00
MH5 achievement	37,190	0.83	0.21	0.89	0.75	1.00
MH5 exception rate	37,190	0.09	0.15	0.00	0.00	0.14

FIGURE 3.

Distribution of QOF achievement rates across practices in 2010/11. (a) MH4 – serum creatinine and thyroid-stimulating hormone check < 15 months, 1 point; (b) MH5 – lithium within range, < 6 months, 2 points; (c) MH6 – comprehensive care plan documented, 6 points; and (d) MH9 – percentage reviewed < 15 months, 23 points.

TABLE 7 - General practitioner-level analyses: descriptive statistics for the variables

Max., maximum; min., minimum; ONS, Office for National Statistics.

Variable description	Source	n	Mean (or % where indicated)	SD	Median	Min.	Max.
Number of SMI admissions within a year	HES	38,774	3.52	3.92	2.00	0.00	55.00
Proportion of GP practices reimbursed under PMS	GMS	38,774	43%	–	–	–	–
Proportion of male GPs in GP practice	GMS	38,774	61%	–	–	–	–
Percentage of foreign GPs in GP practice	GMS	38,774	33%	–	–	–	–
Mean age of GPs in GP practice (years)	GMS	38,774	48.05	7.65	46.33	28.00	76.00
Practice list size	ADS	38,774	6707	4008	5908	1040	40,082
Patient population: average age (years)	ADS	38,774	38.91	4.15	39.33	21.56	56.43
Proportion male patients	ADS	38,774	50%	_	_	_	_
Proportion claiming incapacity benefit for mental health, practice catchment area	ONS	38,774	2%	_	_	_	_
Proportion providing informal care, practice catchment area	ONS	38,774	10%	_	_	_	_
NHS psychiatric residents per 1000 population, practice catchment area	ONS	38,774	0.19	1.12	0.00	0.00	63.57
Proportion of non-white ethnicity, practice catchment area	ONS	38,774	11%	_	_	_	_
Proportion living in urban setting, practice catchment area	ONS	38,774	82%	_	_	_	_
Proportion of practice patients able to access care within 48 hours	GP survey	38,774	84%	_	_	_	_
Distance (in miles) to closest acute hospital	HES	38,774	4.74	4.91	2.95	0.00	59.44
Distance (in miles) to closest mental health hospital	HES	38,774	10.55	8.27	8.21	0.00	74.05
Mean number of admissions between April 2004 and March 2006	HES	38,774	4.37	4.29	3.00	0.00	63.00

Main regression results

Table 8 presents the calculated IRRs and the 95% CIs for the estimated panel data models. Achievement scores are unadjusted, that is, all patients who have been flagged as unsuitable for this indicator are included in the denominator (see Chapter 2, Exception reporting in the Quality and Outcomes Framework). The first sets of results (columns 2–4) are derived from estimations in which both indicators were included simultaneously (joint modelling). The second and third sets of results (columns 5–7 and 8–10) show the results for models in which only one indicator was included. The first set of results (joint modelling) comprises the base case, as the statistical approach explicitly accounts for correlation between the two measures of QOF achievement. Because the focus of our study is on the association between QOF achievement and admissions, we refrain from reporting the effects of the various control variables that were included in all analyses. The interested reader is referred to Table 29 in Appendix 2, Further results. Full results and fit statistics are available on request from the authors.

TABLE 8 - General practitioner-level analysis 1: regression results (achievement includes exception-reported patients)

These models did not converge when including PCT fixed effects and we therefore dropped PCT fixed effects.

Indicator and admission type	Joint modelling		MH6/4 only		MH9/5 only
	IRR	95% CI	IRR	95% CI	IRR	95% CI
Admissions for SMI
MH6	1.020	0.944 to 1.102	1.113	1.039 to 1.192	–	–
MH9	1.210	1.104 to 1.327	–	–	1.226	1.129 to 1.330
Admissions for physical (elective) care
MH6	1.135	0.979 to 1.315	1.220	1.084 to 1.374	–	–
MH9	1.179	0.969 to 1.435	–	–	1.270	1.082 to 1.491
Admissions for physical (emergency) care
MH6	1.180	1.087 to 1.281	1.269	1.183 to 1.362	–	–
MH9	1.189	1.084 to 1.304	–	–	1.303	1.203 to 1.411
Admissions for bipolar disorder a
MH4	1.171	1.018 to 1.347	1.226	1.079 to 1.393	–	–
MH5	1.089	0.994 to 1.194	–	–	1.124	1.033 to 1.222

Several important findings emerge from these results. First, the association between QOF achievement and admission is generally positive (IRR > 1), implying that better QOF performance is associated with more admissions, not fewer. The IRR for MH9 (1.210) suggests that a change in QOF achievement of 10% is associated with an increase in the practice SMI admission rate of 1.9% (95% CI 1.0% to 2.9%). The strength of the effect varies across indicators and admission types, but may have important clinical and economic implications. Second, we find statistically significant associations between QOF achievement on MH9 and both SMI admissions and physical emergency admissions. Results are not significant for elective admissions, although these are still all positive. In contrast, the effect of MH6 on admissions is only statistically significant (i.e. the IRR is different from zero) for physical (emergency) admissions when modelled jointly. Of the two lithium indicators, results are statistically significant for MH4 when modelled jointly. Third, the estimated IRRs are generally larger and more often statistically significant when the association between admissions and only one QOF indicator is modelled instead of both QOF indicators. Both MH6 and MH9, as well as MH4 and MH5, tend to be highly correlated. Failing to account for this correlation may therefore lead to false conclusions. The cross-sectional models do not result in qualitatively different findings and are therefore not reported here (see Appendix 2, Tables 27 and 28).

Sensitivity analyses

We perform a range of additional analyses to test the sensitivity of our findings to different modelling assumptions or inclusion criteria.

Percentage of valid exception reporting

Practices vary both in the number and proportion of SMI patients they exception report (Table 6). Our base case makes the assumption that all patients who were exception reported could have been ‘treated’, that is they could have received an annual review and/or could have had a care plan developed. This assumption means that achievement scores are lower than they would be if exception-reported individuals are counted in the denominator) (see Chapter 2, Exception reporting in the Quality and Outcomes Framework). However, we know that some people are exception reported because they decline to attend the GP practice surgery (‘informed dissent’), and these exceptions are valid or ‘correct’ – individuals are free to choose whether or not they attend the practice for review and GPs should not be incentivised to use inappropriate pressure to encourage attendance. The absence of individual-level data on QOF achievement and exceptions means that we do not know what percentage of the exceptions is valid. To test the effect of exception reporting on our results, we therefore ran a series of regressions in which the percentage of exceptions deemed valid ranged from 0 (as in the base case) to 100, with the variable increasing by 10% in each regression.

In all analyses, as the percentage of exceptions considered to be ‘valid’ increases the magnitude of the effect on the number of admissions decreases. For example, in the analysis on SMI admissions that includes both indicators (‘estimated jointly’), the results that are significant in the base case (0% exceptions assumed valid) become statistically insignificant when all (100%) exceptions are considered to be ‘valid’ (Table 8 and Figure 4). Figures 5–7 show data for other admission types. This is also the case when indicators are modelled individually, aside from in one case: admissions for physical emergency care remain significant and positively associated with practice achievement on both MH6 (the care plan indicator) and MH9 (annual review indicator), regardless of how we treat exceptions.

FIGURE 4.

Change in IRR for the number of annual SMI admissions per practice when the percentage of ‘valid’ exceptions ranges from 0% to 100%. (a) MH6; and (b) MH9.

FIGURE 5.

Change in IRR for the number of annual physical (emergency) admissions per practice when the percentage of ‘valid’ exceptions ranges from 0% to 100%. (a) MH6; and (b) MH9.

FIGURE 6.

Change in IRR for the number of annual physical (elective) admissions per practice when the percentage of ‘valid’ exceptions ranges from 0% to 100%. (a) MH6; and (b) MH9.

FIGURE 7.

Change in IRR for the number of annual bipolar admissions per practice when the percentage of ‘valid’ exceptions ranges from 0% to 100%. (a) MH4; and (b) MH5.

This approach enables us to identify the approximate point at which the QOF ceases to have a statistically significant impact on admissions (i.e. the ‘switching point’). For the jointly estimated equations, the switching point range varies between 70% and 80% (e.g. for the effect of MH9 on SMI admissions) to between 90% and 100% [for the effect of MH6 on physical (emergency) admissions]. Table 9 summarises the switching points for each base case analysis. By combining these estimates with information on the SMI practice register size and exception rates, one can derive an approximate estimate of the maximum number of patients allowed to be ‘incorrectly’ exception reported for the result to hold. For example, given the median SMI practice register of 40 patients, the median MH9 exception rate of 0.12 (see Table 6), and a switching point of 80%, we calculate that approximately 0.96 patients per practice and year [= 40 × 0.12 × (1 – 0.8)] need to be ‘incorrectly’ exception reported for the positive effect of MH9 on the annual number of SMI admissions per GP practice to be statistically significant.

TABLE 9 - General practitioner-level analysis 1: approximate switching point – percentage valid exception reporting

NA, not applicable.

Indicator and admission type	Joint modelling	MH6/4 only	MH9/5 only
Admissions for SMI
MH6	NA – always insignificant	40–50%	–
MH9	70–80%	–	70–80%
Admissions for physical (elective) care
MH6	NA – always insignificant	80–90%	–
MH9	NA – always insignificant	–	70–80%
Admissions for physical (emergency) care
MH6	90–100%	NA – always significant	–
MH9	80–90%	–	NA – always significant
Admissions for bipolar disorder
MH4	70–80%	90–100%	–
MH5	NA – always insignificant	–	40–50%

Admitted at least once

Some patients with SMI are admitted repeatedly within a short time period, a behaviour known as the ‘revolving door’ phenomenon. 62 The observed admission pattern may be genuinely related to the quality of primary care, but may also be the result of the hospitals’ discharge management, the patients’ personal circumstances, or the availability of alternative care resources in the patient’s locality [e.g. access to CRHT teams or Community Mental Health Teams (CMHTs) or community nurses, availability of an informal carer, access to appropriate housing]. Owing to the high frequency of admissions, these individual patients have the potential to distort the admission rates observed at the GP practice level. In order to test whether or not our previous findings reflect this phenomenon, we conduct a sensitivity analysis on the number of patients being admitted at least once within a financial year. Hence, the dependent variable now reflects the number of patients who have required secondary care in a year, rather than the number of admissions in a year. Results are reported in Table 10 (assuming 0% exceptions are ‘valid’) and Table 11 (100% exceptions are ‘valid’).

TABLE 10 - General practitioner-level analysis 2: regression results (achievement includes exception-reported patients)

Indicator and admission type	Joint modelling		MH6/4 only		MH9/5 only
	IRR	95% CI	IRR	95% CI	IRR	95% CI
Admissions for a SMI
MH6	1.014	0.952 to 1.079	1.127	1.064 to 1.194	–	–
MH9	1.253	1.156 to 1.359	–	–	1.265	1.175 to 1.362
Admissions for physical (elective) care
MH6	1.060	0.981 to 1.146	1.181	1.100 to 1.269	–	–
MH9	1.260	1.163 to 1.366	–	–	1.309	1.214 to 1.411
Admissions for physical (emergency) care
MH6	1.091	1.030 to 1.155	1.190	1.131 to 1.252	–	–
MH9	1.212	1.139 to 1.290	–	–	1.280	1.211 to 1.352
Admissions for bipolar disorder
MH4	1.107	0.976 to 1.256	1.170	1.040 to 1.315	–	–
MH5	1.102	1.009 to 1.204	–	–	1.126	1.037 to 1.222

TABLE 11 - General practitioner-level analysis 2: regression results (achievement excludes exception-reported patients)

Indicator and admission type	Joint modelling		MH6/4 only		MH9/5 only
	IRR	95% CI	IRR	95% CI	IRR	95% CI
Admissions for SMI
MH6	0.980	0.921 to 1.043	0.984	0.926 to 1.047	–	–
MH9	1.015	0.908 to 1.135	–	–	1.001	0.899 to 1.115
Admissions for physical (elective) care
MH6	0.991	0.921 to 1.066	1.023	0.957 to 1.094	–	–
MH9	1.125	1.005 to 1.259	–	–	1.118	1.009 to 1.240
Admissions for physical (emergency) care
MH6	1.019	0.967 to 1.075	1.031	0.979 to 1.086	–	–
MH9	1.040	0.966 to 1.120	–	–	1.053	0.978 to 1.133
Admissions for bipolar disorder
MH4	1.110	0.946 to 1.303	1.136	0.969 to 1.332	–	–
MH5	1.043	0.946 to 1.150	–	–	1.060	0.962 to 1.167

Findings from the analysis of people admitted at least once a year were broadly similar to results from base case analyses (number of admissions per year). With one exception, the effects were significant and positive in all analyses in which indicators were modelled individually; in these cases, an assumption that up to 60% of exceptions were valid was sufficient for the relationship between QOF and admissions to be statistically significant. For the jointly estimated equations, the switching point for the percentage of valid exceptions lay between 20% and 30% for the effect of MH5 on bipolar admissions, but was between 90% to 100% for the effect of MH9 on physical (emergency) admissions. Table 12 summarises the approximate ‘switching point’ for these sensitivity analyses.

TABLE 12 - General practitioner-level analysis 2: switching point – percentage valid exception reporting

NA, not applicable.

Indicator and admission type	Joint modelling	MH6/4 only	MH9/5 only
Admissions for SMI
MH6	NA – always insignificant	60–70%	–
MH9	80–90%	–	80–90%
Admissions for physical (elective) care
MH6	NA – always insignificant	80–90%	–
MH9	NA – always significant	–	NA – always significant
Admissions for physical (emergency) care
MH6	60–70%	90–100%	–
MH9	90–100%	–	90–100%
Admissions for bipolar disorder
MH4	NA – always insignificant	60–70%	–
MH5	20–30%	–	60–70%

Patients with unspecific main diagnosis

Inspection of HES data at local area level revealed that SMI admissions in some regions were far below levels that would be expected given the demographic characteristics of the area. In some years, some mental health hospitals had coded over 90% of their cases with a primary diagnosis ICD-10 code R69 (‘Unknown and unspecified causes of morbidity’). 63 The use of R69 varied across providers and across years, but was neither unusual nor confined to a small number of providers (Figure 8).

FIGURE 8.

Use of R69 as primary diagnoses for psychiatric admissions. Percentage of provider admissions for SMI patients with primary diagnosis of R69. (a) 2006; (b) 2007; (c) 2008; (d) 2009; and (e) 2010.

We do not know how many of these ‘R69’ admissions were for individuals with SMI, but it is very unlikely that none of them were. Therefore, we undertook a further sensitivity analysis to test the relationship between QOF achievement scores and a revised measure of SMI admission that included records with a diagnostic code from our base case analyses (ICD-10 codes F20–F31), and records with a primary diagnosis of R69. The latter code was used if, and only if, the treatment specialty for the admission was one of the following: adult mental illness; forensic psychiatry; psychotherapy; or old age psychiatry (i.e. excluding learning disability, child and adolescent psychiatry and other types of service such as those for eating disorders). The HES field is ‘tretspef’ and defines the specialty in which the consultant was working during the period of care. We investigated the effect on both of our response variables (the number of admissions and the number of individuals admitted at least once) and varied the percentage of exceptions assumed to be valid, as previously (see Chapter 3, Percentage of valid exception reporting).

Findings are presented in Table 13 (assuming 0% of exceptions are ‘valid’) and Table 14 (assuming 100% of exceptions are ‘valid’). The effect of varying the percentage of valid exceptions did not alter the statistical significance of results from the base case analysis. Findings were also robust to the way that SMI admissions were defined, with the effect of a 10% change in QOF performance on practice SMI admission rates approximately 0.1 percentage point lower when R69 codes were included in the definition compared with the base case analysis that excluded R69 codes. In the jointly estimated model, the ‘switching point’ at which MH9 ceased to have a statistically significant impact on admissions lay between 80% and 90% (Table 15). Therefore, provided the percentage of invalid exceptions is no more than 10–20%, the positive association between practice achievement on the QOF review indicator, MH9, and the number of annual admissions is statistically significant.

TABLE 13 - General practitioner-level analysis 3: regression results (achievement includes exception-reported patients)

Indicator and admission type	Joint modelling		MH6/4 only		MH9/5 only
	IRR	95% CI	IRR	95% CI	IRR	95% CI
Frequency
MH6	1.019	0.949 to 1.095	1.116	1.049 to 1.188	–	–
MH9	1.220	1.118 to 1.332	–	–	1.235	1.144 to 1.334
Admitted at least once
MH6	1.019	0.963 to 1.080	1.128	1.071 to 1.188	–	–
MH9	1.242	1.151 to 1.341	–	–	1.258	1.175 to 1.347

TABLE 14 - General practitioner-level analysis 3: regression results (achievement excludes exception-reported patients)

Indicator and admission type	Joint modelling		MH6/4 only		MH9/5 only
	IRR	95% CI	IRR	95% CI	IRR	95% CI
Frequency
MH6	0.980	0.912 to 1.053	1.000	0.936 to 1.069	–	–
MH9	1.068	0.952 to 1.199	–	–	1.054	0.947 to 1.173
Admitted at least once
MH6	0.978	0.922 to 1.038	0.991	0.937 to 1.049	–	–
MH9	1.046	0.940 to 1.164	–	–	1.030	0.931 to 1.139

TABLE 15 - General practitioner-level analysis 3: switching point – percentage valid exception reporting

NA, not applicable.

Indicator and admission type	Joint modelling	MH6 only	MH9 only
Frequency
MH6	NA – always insignificant	50–60%	–
MH9	80–90%	–	80–90%
Admitted at least once
MH6	NA – always insignificant	60–70%	–
MH9	80–90%	–	80–90%

Does better primary care reduce inpatient length of stay?

Overview

This section focuses on our second research question, namely the effect of better-quality primary care on LOS in hospital. Better-quality primary care may also reduce LOS if patients are effectively cared for outside hospital. Our null hypothesis is of no association between QOF and LOS and our alternative hypothesis is that better management in primary care could shorten LOS in hospital for people with SMI.

We use the same data set as for the previous analyses on admissions, but instead of aggregating the admissions data to the practice level we keep the original patient-level structure. The analysis is limited to admissions with a non-zero LOS and practices with no SMI admissions are dropped from the data set. As before, QOF data are available only at practice level.

The data set includes 98,993 admissions for patients registered with 7912 practices.

Data

The analysis is based on the same data set as derived for the previous analysis (see Chapter 3, Data) but differs in several respects. First, we retain the original patient-level structure of the data and do not aggregate admissions to practice level. Second, we focus only on patients admitted with a primary diagnosis of a SMI, that is, admissions for physical care are excluded from the analysis. Third, we exclude patients who remained in hospital for more than 180 days (approximately 6 months) or were admitted after the 2 October 2010, that is, 180 days before the end of our study period. The latter exclusion criterion is applied to ensure that patients spent sufficient time outside the hospital to be able to be seen by their GP and interact with the primary care sector and thus potentially be subject to QOF activities. Fourth, we exclude patients with invalid discharge information, that is, where it is possible that the patient is still in hospital or it is unclear when the patient was discharged. Last, practices with no SMI admissions are no longer included in the data set.

The HES data warehouse contains detailed information on patient characteristics and characteristics of the hospital stay. We derive a range of variables that describe the medical and socioeconomic characteristics of the patient. These are age (categorised in 10-year age bands), gender, ethnicity (coded as white, Asian, black, mixed or other/unknown), proportion of people in a small area (LSOA) claiming mental health benefits (coded as quintiles), primary diagnosis (see Table 4), number of non-duplicate comorbidity codes recorded throughout the hospital stay, whether or not the patient had a carer, whether or not the patient was detained under the Mental Health Act 198349 during the inpatient stay, and whether or not the patient has a psychiatric history, that is, was previously admitted under the care of a psychiatric consultant. We also derive two variables that describe the hospital stay. These are discharge type (coded as discharge on clinical advice, self-discharge or died in hospital) and year of admission.

Analytical model

The aim of this empirical analysis is to relate the length of inpatient stay for people admitted to hospital with a main diagnosis of a SMI to the quality performance of their GP practices. In order to isolate the effect, the analysis also controls for other patient factors that may drive LOS but are unrelated to the quality of care provided in primary care, as well as the general efficiency of the hospital provider.

Length of stay is a continuous, non-negative variable that is non-normally distributed with large kurtosis and substantial skew (Figure 9). We used a mixed-effects linear regression model to analyse these data. In order to reduce the skewness and make the assumption of normality underlying the linear regression model more tenable, we transformed LOS using the logarithmic transformation; an approach frequently used in the analysis of cost and LOS data. 64 Some patients were admitted and discharged within the same day, that is, had a recorded LOS of no days. For these observations the logarithm of LOS would not be defined and the observation would thus be excluded from the analysis even though they still consumed hospital resources. Following this argument, we measured LOS as the number of days spent in hospital, rather than the number of nights, thereby avoiding the problem of undefined logarithms.

FIGURE 9.

Length-of-stay analysis: histogram of patient LOS.

Let LOS_ijk be the LOS for admission i = 1, . . ., I, where the patient is registered with practice j = 1,. . ., J and treated in hospital k = 1, . . . , K. We acknowledge the potential clustering of admissions within patients but do not specifically model it. However, most patients are admitted only once within each financial year. We also assume, at least in notation, that GP practices are clustered in hospitals, that is, submit patients exclusively to one hospital. However, we model hospital effects using dummy variables (‘hospital fixed effects’) so that this simplification in notation is not consequential. The empirical model is specified as follows:

⁽⁶⁾ LOS i j k = α + QOF ′ j k γ + X ′ i j k β + π k + μ j k + ε i j k ,

where X_ijk is a set of patient characteristics and QOF_jk measures the QOF achievement of practice j. The parameter α denotes the common intercept, whereas the parameters ε_ijk and μ_jk are random error terms at the admission and GP practice level and are assumed to be independently distributed as N∼(0,σc2) with c ∈ [i, j]. The parameters π_k are modelled as fixed effects and capture variation in LOS between providers owing to variation in efficiency and clinical practice. All models include year effects to account for common temporal trends.

The model is estimated via maximum likelihood using xtmixed in Stata 12.0. The reported standard errors are clustered at the GP practice level. Coefficient estimates can be interpreted as semi-elasticities, that is, a one-unit increase in a continuous variable is expected to increase/decrease LOS by a percentage equal to the estimated coefficient. Given the large number of observations in this analysis, the likelihood of finding a statistically significant but small and potentially clinically unimportant effect is large. We therefore caution the reader to consider the absolute size of the effect. Our discussions of the results are based at the highest level of significance (p < 0.001).

We ran several models, all of which tested the QOF indicators individually and jointly and all of which included year dummies. In the base case, we calculated achievement to include exception-reported cases, included a set of patient-level covariates and hospital fixed effects. We also ran a model without either patient-level covariates or hospital fixed effects, and a model with patient-level covariates and without hospital fixed effects. We ran the same set of models for QOF achievement measures calculated without exception-reported cases.

Results

Descriptive statistics

Table 16 presents the descriptive statistics for the LOS analysis. All variables are at an individual level, except for QOF achievement. The data set includes 98,993 individuals with a primary diagnosis of SMI and whose inpatient stay is 180 days (6 months) or less. The mean LOS was 42.3 days (SD 39.5 days) and the median LOS was 30 days (IQR 12–60 days). As shown from the descriptive statistics in Table 16, the mean age of these individuals is 45.22 years (SD 16.30 years) and 52% are male. Most are of white ethnicity (76%), with 2% of mixed ethnicity, 7% Asian and 9% black. Just less than one in five (17%) of these hospitalised individuals is formally detained under the Mental Health Act 1983. 49 On average, practices perform well on the QOF indicators for both the care plan indicator MH6 (mean 74%, SD 16%) and the annual review indicator MH9 (mean 80%, SD 12%). For both MH6 and MH9, around 12% of people with SMI are exception reported.

TABLE 16 - Length-of-stay analysis: descriptive statistics

Max., maximum; min., minimum.

Variable description	n	Mean (or % where indicated)	SD	Median	Min.	Max.
Length of inpatient stay – excluding admissions > 180 days (days)	98,993	42.29	39.50	30.00	1.00	180.00
Achievement on MH6 QOF indicator	98,993	0.74	0.16	0.77	0.00	1.00
Achievement on MH9 QOF indicator	98,993	0.80	0.12	0.82	0.00	1.00
Proportion with schizophrenia	98,993	41.3%	–	–	–	–
Proportion with schizotypal disorder	98,993	0.2%	–	–	–	–
Proportion with persistent delusional disorders	98,993	4.9%	–	–	–	–
Proportion with acute and transient psychotic disorders	98,993	7.8%	–	–	–	–
Proportion with induced delusional disorder	98,993	0.1%	–	–	–	–
Proportion with schizoaffective disorders	98,993	8.5%	–	–	–	–
Proportion with other non-organic psychotic disorders	98,993	0.4%	–	–	–	–
Proportion with unspecified non-organic psychosis	98,993	4.0%	–	–	–	–
Proportion with manic episode	98,993	3.3%	–	–	–	–
Proportion with bipolar affective disorder	98,993	29.4%	–	–	–	–
Patient age (years)	98,993	45.22	16.30	43.00	18.00	104.00
Proportion male patients	98,993	52.0%	–	–	–	–
Proportion detained patients	98,993	17.5%	–	–	–	–
Proportion white ethnicity	98,993	75.8%	–	–	–	–
Proportion mixed ethnicity	98,993	1.9%	–	–	–	–
Proportion Asian ethnicity	98,993	7.4%	–	–	–	–
Proportion black ethnicity	98,993	9.5%	–	–	–	–
Proportion other or undefined ethnicity	98,993	5.4%	–	–	–	–
Proportion patients who have a carer	98,993	6.3%	–	–	–	–
Discharge type: discharged on clinical advice	98,993	97.1%	–	–	–	–
Discharge type: self-discharged against clinical advice	98,993	2.4%	–	–	–	–
Discharge type: died in hospital	98,993	0.5%	–	–	–	–
Proportion claiming incapacity benefit for mental health, practice catchment area	98,993	2.26	1.62	1.87	0.00	20.41
Proportion patients with previous psychiatric admission	98,993	42.2%	–	–	–	–
Number of non-duplicate comorbidities recorded for the patient	98,993	0.98	3.10	0.00	0.00	288.00

Regression results

Table 17 presents the regression results of the LOS analysis. The reference categories in this analysis describe a patient with a primary diagnosis of schizophrenia, aged 24 years or younger, female, white, not detained, no carer, discharged on clinical advice, no previous psychiatric history, no comorbidities, and residing in an area with a low (first quintile) proportion of residents claiming mental health benefits. QOF achievement was calculated so that all exceptions were deemed invalid, that is, the number of exception reported patients was included in the denominator (see Chapter 2, Exception reporting in the Quality and Outcomes Framework).

TABLE 17 - Length-of-stay analysis: regression results

MH, mental health; SE, standard error.

Variable description	Joint modelling			MH6 only			MH9 only
	Beta	SE	p-value	Beta	SE	p-value	Beta	SE	p-value
Achievement on MH6 QOF indicator	–0.055	0.037	–	–0.034	0.029	–	–	–	–
Achievement on MH9 QOF indicator	0.042	0.044	–	–	–	–	0.003	0.034	–
Financial year 2007/8	–0.022	0.013	–	–0.024	0.013	–	–0.027	0.013	< 0.05
Financial year 2008/9	0.021	0.014	–	0.019	0.013	–	0.015	0.013	–
Financial year 2009/10	–0.019	0.015	–	–0.022	0.014	–	–0.028	0.013	< 0.05
Financial year 2010/11	–0.060	0.016	< 0.001	–0.063	0.015	< 0.001	–0.070	0.014	< 0.001
Schizotypal disorder	–0.062	0.064	–	–0.062	0.064	–	–0.062	0.064	–
Persistent delusional disorders	–0.109	0.017	< 0.001	–0.109	0.017	< 0.001	–0.109	0.017	< 0.001
Acute and transient psychotic disorders	–0.308	0.014	< 0.001	–0.308	0.014	< 0.001	–0.308	0.014	< 0.001
Induced delusional disorder	–0.229	0.123	–	–0.229	0.123	–	–0.229	0.123	–
Schizoaffective disorders	0.026	0.015	–	0.026	0.015	–	0.026	0.015	–
Other non-organic psychotic disorders	–0.314	0.055	< 0.001	–0.314	0.055	< 0.001	–0.314	0.055	< 0.001
Unspecified non-organic psychosis	–0.193	0.021	< 0.001	–0.193	0.021	< 0.001	–0.193	0.021	< 0.001
Manic episode	–0.167	0.018	< 0.001	–0.167	0.018	< 0.001	–0.167	0.018	< 0.001
Bipolar affective disorder	–0.132	0.011	< 0.001	–0.132	0.011	< 0.001	–0.132	0.011	< 0.001
Aged 25–34 years	0.074	0.027	< 0.01	0.074	0.027	< 0.01	0.074	0.027	< 0.01
Aged 35–44 years	0.035	0.026	–	0.035	0.026	–	0.035	0.026	–
Aged 45–54 years	0.101	0.027	< 0.001	0.101	0.027	< 0.001	0.101	0.027	< 0.001
Aged 55–64 years	0.261	0.027	< 0.001	0.261	0.027	< 0.001	0.261	0.027	< 0.001
Aged 65–74 years	0.512	0.033	< 0.001	0.512	0.033	< 0.001	0.512	0.033	< 0.001
Aged 75 years and over	0.773	0.031	< 0.001	0.773	0.031	< 0.001	0.773	0.031	< 0.001
Male gender	0.093	0.027	< 0.001	0.093	0.027	< 0.001	0.093	0.027	< 0.001
Patient was detained	0.465	0.012	< 0.001	0.465	0.012	< 0.001	0.465	0.012	< 0.001
Ethnicity: mixed	0.086	0.033	< 0.01	0.086	0.033	< 0.01	0.086	0.033	< 0.01
Ethnicity: Asian	0.089	0.018	< 0.001	0.089	0.018	< 0.001	0.089	0.018	< 0.001
Ethnicity: black	0.179	0.018	< 0.001	0.179	0.018	< 0.001	0.178	0.018	< 0.001
Ethnicity: other or undefined	0.035	0.020	–	0.035	0.020	–	0.035	0.020	–
Detained and ethnicity: mixed	–0.089	0.083	–	–0.089	0.083	–	–0.089	0.083	–
Detained and ethnicity: Asian	–0.066	0.031	< 0.05	–0.066	0.031	< 0.05	–0.065	0.031	< 0.05
Detained and ethnicity: black	–0.079	0.028	< 0.01	–0.079	0.028	< 0.01	–0.079	0.028	< 0.01
Detained and ethnicity: other	–0.040	0.036	–	–0.040	0.036	–	–0.040	0.036	–
Patient had a carer	0.074	0.023	< 0.01	0.074	0.023	< 0.01	0.074	0.023	< 0.01
Discharge type: self–discharged	–0.802	0.027	< 0.001	–0.802	0.027	< 0.001	–0.803	0.027	< 0.001
Discharge type: died in hospital	–0.048	0.053	–	–0.048	0.053	–	–0.048	0.053	–
Patient had previous psychiatric admission	–0.050	0.009	< 0.001	–0.050	0.009	< 0.001	–0.050	0.009	< 0.001
MH claimants – second quintile	–0.007	0.012	–	–0.007	0.013	–	–0.007	0.012	–
MH claimants – third quintile	0.006	0.013	–	0.006	0.013	–	0.006	0.013	–
MH claimants – forth quintile	0.011	0.014	–	0.011	0.014	–	0.011	0.014	–
MH claimants – fifth quintile	0.022	0.015	–	0.022	0.015	–	0.022	0.015	–
Number of comorbidities	0.051	0.004	< 0.001	0.051	0.004	< 0.001	0.051	0.004	< 0.001
Constant term	2.916	0.180	< 0.001	2.937	0.178	< 0.001	2.917	0.179	< 0.001
GP-level variance	0.211	0.012	< 0.001	0.211	0.012	< 0.001	0.211	0.012	< 0.001
Admission-level variance	1.033	0.004	< 0.001	1.033	0.004	< 0.001	1.033	0.004	< 0.001
Number of observations	98,993	–	–	98,993	–	–	98,993	–	–

The quality of primary care, as measured by the QOF scores, has no significant effect on LOS in any of the analyses. Longer LOS is associated with some primary diagnoses, older age and male gender. People with a primary diagnosis of persistent delusional disorder (F22), acute and transient psychotic disorder (F23), non-organic psychoses (F28 or F29), manic episode (F30) or bipolar affective disorder (F31) have significantly shorter stays than people with schizophrenia (F20). Formal detention under the Mental Health Act 198349 significantly increases LOS, and longer stay is associated with black and Asian ethnicity, compared with white ethnicity. People of one of these ethnic groups who were also detained are likely to stay even longer than those with just one of these characteristics, but the overall effect is smaller than the sum of the two individual effects. People with an informal carer were likely to have longer inpatient stays than those who did not have a carer, but people with a history of psychiatric admission tended to have a shorter LOS. The former may be an indicator of severity, whereas the latter may reflect clinical knowledge about the previous psychiatric history of the patient and a familiarity with the care requirements of the patient. Unsurprisingly, patients who decided to leave the hospital on their own responsibility, that is, self-discharged against clinical advice, had shorter stays than those who were discharged following clinical advice. Deprivation scores relating to the patient’s residential area did not explain LOS, but having a higher number of comorbidities did increase duration of stay in hospital.

Sensitivity analyses

Percentage of valid exception reporting

In previous analyses, we sought to identify the ‘switching point’ at which the relationship between QOF achievement and admissions became statistically insignificant. In the LOS analysis, we found that the way in which achievement was measured, that is, whether or not exception reported cases were included in the measures, had no influence on the statistical significance of the relationship between achievement and LOS. Results remained insignificant for whether we assumed 0% of exceptions were ‘valid’ or 100% of exceptions were assumed ‘valid’. Therefore, there is no ‘switching point’ to identify in this analysis. All covariates were approximately equal to those reported in Table 17 and are therefore not reported here.

Does better primary care reduce cost of care?

Overview

This section focuses on our third research question, namely whether or not better-quality primary care as measured by specific mental health QOF indicators is associated with reduced resource use in terms of lower secondary care expenditure for mental health services for people with SMI. If better-quality primary care can reduce costly emergency hospital admissions it may have knock-on effects for NHS expenditure and resource use. Our null hypothesis is one of no association between QOF achievement and secondary care expenditure while our alternative hypothesis proposes that preventative care could reduce mental health expenditure.

This set of analyses used the Mental Health Minimum Data Set (MHMDS) to investigate the relationship between QOF performance and the total per-patient cost of care. The MHMDS was available for 2 years: 2006/7 and 2007/8. The base case included 981,373 records on 711,820 individuals, clustered within 8064 practices. As there were no diagnostic codes in the database, we were unable to identify individuals with SMI. As far as possible we tried to match the LOS analysis in Chapter 3, Does better primary care reduce inpatient length of stay?

Data

The MHMDS was available for 2 years: 2006/7 and 2007/8. The data set contained details of community care, social care and hospital care received by people with mental illness in England, and so had a broader scope than HES which covers only hospital care.

The data comprised a series of records for each ‘spell’ of care received by an individual over the 2-year period. The MHMDS spell differs from that in HES: it covers a period of care that may or may not include a hospital admission. Unlike HES spells, MHMDS spells are not necessarily ‘closed’, or finished, before another spell is initiated – even if the spell is with the same provider. An individual patient in MHMDS may therefore have multiple spells running concurrently. A stylised example of a patient’s record in MHMDS is shown in Figure 10.

FIGURE 10.

Stylised example of a patient record in MHMDS.

Variables in the data set included demographic information (such as age, gender, ethnicity and marital status). Although two-thirds of the cases recorded marital status, the categories of ‘married’ and ‘separated’ were concatenated. The project steering group agreed that this undermined the value of the variable sufficiently to render it unusable. Variables included resource use data such as community care (contacts with social workers, community psychiatric nurses, physiotherapists, psychotherapists, psychiatrists and occupational therapists) and hospital care (outpatient visits, the number of days spent in inpatient care, in medium-secure units and in psychiatric intensive care units). Medium-secure hospitals/units are for people who are detained under mental health legislation and pose a serious danger to the public. 65 There were binary indicators for receipt of local authority services such as residential care, day care, home help and sheltered work. MHMDS also records use of electroconvulsive therapy (ECT) which is a treatment for severe depression. We use this variable to exclude patients from the data set, as the treatment is sometimes used for patients with bipolar disorder and we do not examine the QOF indicators for bipolar disorder (MH4/MH5). Aside from the variable for ECT, the MHMDS data sets contained no diagnostic and procedure codes. There was no variable indicating whether or not death had occurred, and there were no dates of admission and discharge to hospital.

The MHMDS also reported the code for the GP practice with which the individual was registered, the electoral ward of the patient and several PCT codes. There was no LSOA code or statistical ward code.

Some MHMDS data were too poorly populated to be used in the regressions:

• There were several variables relating to the outcome measure Health of the Nation Outcome Scale (HoNOS) that assesses the health and social functioning of people with a SMI. However, data were missing in over 90% of records.

• Fewer than 5000 (0.12%) records in the data set had a non-zero value for attendance at NHS day-care facilities: the majority attended for a single day, and the maximum use was 9 days (for two patients). These numbers appeared implausibly low and were too unreliable to inform the analyses. This judgement was supported by the NHS Reference costs for 2007/8, which recorded over 1.38 million regular adult attendances at NHS Trusts Mental Health Day Care Facilities. 66

• Similarly, MHMDS data on acute home-based care and stays in NHS residential homes (community bed-days) were insufficiently robust for inclusion in the regressions.

• The CPA is a needs assessment for people with severe mental disorders, and involves development of a care plan that is regularly reviewed. Therefore, we would expect almost everyone in the data set to have had at least one CPA review, but 40% of individuals had no record of having had a CPA assessment.

• The type of community team caring for each individual (e.g. substance misuse team, early intervention in psychosis team) is a potentially very useful indicator, but was missing from 82% of records.

Costing

The dependent variable was the total annual cost of care per patient. The MHMDS data sets did not include costs (prices), but there were data on resource use (activity) for hospital (inpatient and outpatient) care and community care provided by specialist mental health teams. There were no data on care provided by GP practices (visits, prescriptions) and only binary variables for most social care services. We applied the costing methodology used for the Resource Allocation for Mental health and Prescribing (RAMP) project. 67 Owing to the way the MHMDS data were structured in spells, we could not use completed spells as the unit of analysis as we did for HES. Instead, we estimated a total cost per year for each individual.

We used national NHS Reference costs66 to assign costs to mental health inpatient days, medium secure days, psychiatric intensive days and outpatient attendances. We used a single price year (2007/8) to find ‘real’ quality effects. For individuals of working age, inpatient costs were derived as weighted averages of the mental health adult inpatient costs for intensive care, acute care and rehabilitation. For patients aged 65 years and over, we assigned the mental health inpatient cost for the ‘elderly’. The unit costs of medium secure days and of psychiatric intensive days were taken from the NHS trusts’ Mental Health Secure Units sheet66 and separate adult and elderly outpatient visit unit costs were derived as weighted averages of first/follow-up attendances for face-to-face and non-face-to-face contacts.

For community and social care, we used the unit cost values from the RAMP report, adjusted for price inflation using the Personal Social Services Research Unit (PSSRU) Hospital and Community Health Services indices. 68

Owing to the absence of data on activity volume for local authority services, we could not attach costs to all of the resource use variables in the MHMDS and, as these were the only services recorded for a subset of the patients, this meant that our response variable (total annual cost) was missing for around 20% of patients in the MHMDS and these patients could, therefore, not be included in the cost analyses. As these patients had no record of hospital care, or care provided by members of the CMHT, they probably represent a group with less severe illness. It is, therefore, likely that our analysis focuses on individuals with more severe mental illness, but the lack of diagnostic information is a serious limitation of the analysis and introduces potential bias.

Data sets used to generate other covariates

The MHMDS included codes for the individual’s GP practice. We used practice codes to merge the MHMDS with data on QOF performance and GP practice variables. The QOF data were sourced from the Health and Social Care Information Centre (see Chapter 3, Quality and Outcomes Framework data set). We calculated practice achievement scores as the number of patients who achieved the indicator as a fraction of all patients who were reported eligible – including those who had been exception reported (see Chapter 2, Exception reporting in the Quality and Outcomes Framework).

For the MHMDS analysis, we included two of the QOF indicators, MH6 and MH9, as there was no diagnostic information in the MHMDS with which to identify people with bipolar disorder. As QOF performance data were available only at practice level, we did not know whether or when individual patients had received care under the QOF. We return to this limitation in our discussion section.

The MHMDS contained codes for the individual’s electoral ward, but the boundaries differ from those of the statistical wards used by the Office for National Statistics (ONS). The match between electoral wards in the MHMDS and the statistical wards used by the ONS was too poor to use ward-level characteristics.

Unlike HES, the MHMDS did not contain a small-area code (such as a LSOA code) at the patient level and so practice-level variables were used as controls instead. We incorporated measures of area characteristics at practice level using weighted average values based on the LSOAs in which registered practice patients reside (see Chapter 3, Data sets used to generate other covariates). These included deprivation, rurality, the number of NHS psychiatric residents per 1000 population and the prevalence of informal care.

Eligibility criteria used for the Mental Health Minimum Data Set analysis

As far as possible, we derived covariates that matched those of the LOS analysis and followed the same inclusion criteria (e.g. practice list size of at least 1000). The criteria used to identify records eligible for inclusion in the MHMDS analyses are:

• non-zero cost in year

• no ECT

• valid GP code

• valid ethnicity code

• aged 18 years and over

• learning disability was not the main treatment specialty

• LOS of no more than 180 days

• GP practice has at least 1000 registered patients

• GP practice has at least five eligible patients with a SMI

• GP practice has consistent records of numbers of eligible patients.

The base case analysis included 981,373 observations, which represents around 39% of the total records in the MHMDS. In the sensitivity analysis of people aged 18 to 64 years, the corresponding number was 685,154 (27% of the total).

Table 18 lists the reasons why records were excluded. The largest single reason for excluding a record was that there were no data on receipt of social care (almost 26% missing). Of cases with non-missing data, only 1% were recorded as having received social care. The other key reasons for excluding records were that we could not assign a cost, the record either did not meet the eligibility criteria or the MHMDS GP practice code did not match codes in the QOF data set.

TABLE 18 - Derivation of the regression samples

MH, mental health.

Reasons for exclusion: age < 18 years; annual LOS > 180 days; learning disability as main speciality of treatment; received ECT; invalid GP code; changed GP within year.

Matching: MHMDS practice code did not match codes in practice data set (QOF, GMS, GP survey, ONS variables).

Social care data: binary indicators for use of home help, social work, sheltered work, local authority day centre care, local authority residential care.

Reasons for record exclusion	Loss (n)	Total remaining (n)	Percentage loss (of total)
Total records in the MHMDS	–	2,537,324	–
Excluded recordsa	243,662	–	9.6
Unmatched recordsb	174,902	–	6.9
Excluded/unmatched records	292,683	2,244,641	11.5
Zero cost	482,594	1,762,047	19.0
Practice list size < 1000 or MH register < 5	10,974	1,751,073	0.4
Invalid ethnicity code	3352	1,747,721	0.1
No social care datac	652,690	1,095,031	25.7
PCT code missing	8644	1,086,387	0.3
Duplicate patient records	85,836	1,000,551	3.4
Missing at least one covariate	19,178	981,373	0.8
Patients aged over 65 years	296,219	685,154	11.7
Sample for base case analysis (% total)	–	981,373 (39%)	–
Sample for sensitivity analysis (% total)	–	685,154 (27%)	–

Analytical model

In all analyses, the (log of) total cost per patient per year was the dependent (response) variable. We took the natural logarithm because costs were positively skewed (i.e. characterised by a long right-hand tail). This helped to normalise the response variable (Figure 11).

FIGURE 11.

Mental Health Minimum Data Set analysis: histograms of total per-patient cost, before and after log transformation. (a) 2006/7; (b) 2007/8; (c) 2006/7; and (d) 2007/8.

The analytic model mirrored the approach adopted in the individual-level LOS analysis (see Chapter 3, Does better primary care reduce inpatient length of stay?). We merged the MHMDS data with QOF data and other data on practice characteristics from around 8000 GP practices in England. We restricted the sample of patients to those aged 18 years and over, used the LOS analysis categories for age and ethnicity, for area measures such as deprivation and rurality, and used the same criteria to define eligible GP practices in the LOS analysis. As there was no diagnostic information in the MHMDS, the two lithium QOF indicators (MH4 and MH5) could not be tested against the subgroup of people with bipolar disorder. We therefore tested only two QOF indicators, MH6 and MH9, in our analysis, individually and in combination (jointly) (see Table 19). We used a trim point of 180 days for combined LOS over the year (i.e. as an inpatient in any type of setting); the rationale for this was that a longer inpatient stay would reduce opportunities for primary care to affect cost. However, we could not fully replicate the methodology for the LOS analysis: the MHMDS did not include admission and discharge dates for hospital care, so individuals whose inpatient stay began < 180 days before the end of the study period could not be identified.

The MHMDS contains four variables indicating that an individual has been detained under the Mental Health Act 198349 and these were used to derive a dummy variable for detention.

We use xtmixed in Stata 12 (as used in the LOS analyses) to run mixed-effects continuous response models that took account of the clustering of patients within practices and practices within PCTs. Around 39% of individuals in the data set contributed data for both years, with 28% and 33% contributing 1 year’s data for 2006/7 and 2007/8, respectively. We included year indicators to allow for temporal trends, and controlled for individual- and practice-level factors. Table 19 lists the covariates used in the cost regression analyses.

TABLE 19 - Covariates used in the MHMDS analyses

QMAS, Quality Management and Analysis System.

Category	Variables	Sources
Individual-level characteristics: ‘need’ variables	Age group, gender, ethnicity, formal detention	MHMDS
Individual-level characteristics: ‘supply’ variables	Inpatient, day care, social work, residential care, home help, and sheltered work	MHMDS
GP-level characteristics	QOF achievement indicators:  MH6 (care plan)  MH9 (annual review)	QMAS data
	Percentage of practice patients able to access primary care within 48 hours	GP survey
	PMS practice (binary variable)	GMS data
	Mean age of GPs within a practice
	% male GPs within a practice
	% of non-UK qualified GPs
	Size of GP practice (small, medium, large)	ADS data set
	Mean age of practice patients
	% male practice patients
	Weighted average deprivation score of practice population (percentile categories), based on the number of people claiming incapacity benefit for mental health	ONS data
	Weighted average proportion of practice population providing informal care	ADS data
	Weighted average proportion of practice population living in urban areas	Census data
	Weighted average number of NHS psychiatric residents per 1000 practice population
Other	Year dummies	MHMDS

Around 30% of the individuals in the data are aged 65 years and over. There is no diagnostic information in the MHMDS, and at least some of these patients may have dementia rather than a psychotic disorder. We therefore conduct a sensitivity analysis that excludes people aged 65 years and over, as the prevalence of dementia is lower in people aged between 18 years and 64 years. 69

We ran four models in both the base case and sensitivity analyses, and tested the two QOF achievement indicators individually and jointly. A summary of the models is provided in Table 20.

TABLE 20 - Mental Health Minimum Data Set analysis: overview of regression models

✓, included in the model; ✗, not included in the model; A, the number of patients for which the indicator was achieved; E, the number of patients who were exception reported; NA, the number of patients for which the indicator was not achieved.

Variables included	Model 1	Model 2	Model 3	Model 4
Dependent variable	Log of annual per-patient cost
QOF achievement measure	A(A+NA+E)	A(A+NA+E)	A(A+NA)	A(A+NA)
Individual need variables	✓	✓	✓	✓
Individual supply variables	✗	✓	✗	✓
GP-level characteristics	✓	✓	✓	✓
Year dummy variable	✓	✓	✓	✓

The equation for specifying the models is:

⁽⁷⁾ log ⁡ _ cost i j k t = α + QOF ′ j k t γ + X _ d ′ i j k t β + X _ s ′ i j k t δ + v k + u j k + e i j k ,

where log_cost_ij is the log of annual cost of treatment for patient i = 1, . . . , I, registered with practice j = 1, . . . , J and based in PCT k = 1 ,. . . , K in year t = 1,2; X_d_ijk is a set of patient need characteristics, X_s_ijk is a set of individual supply characteristics and QOF_jkt measures the QOF achievement of practice j. The parameter α denotes the random intercept and the parameters e_ijk, u_jk and v_k are random effects at the patient, GP practice and PCT levels, respectively; these are assumed to be independent of one another and normally distributed with zero means and constant variances.

We ran multilevel mixed-effects linear regressions in Stata 12.0 (xtmixed) and fitted the models via maximum likelihood. For characteristics that entered as dummy variables, their proportionate influence was calculated as p = [exp(β) – 1]. 70 Given the large number of observations in this analysis, the likelihood of finding a statistically significant but small and potentially clinically unimportant effect is large. We therefore caution the reader to consider the absolute size of the effect.

Results

When the 2 years of MHMDS data were merged, the data set contained over 2.5 million records. After excluding ineligible records, the base case included 981,373 observations on 711,820 individuals in 8064 practices. The sensitivity analysis of individuals aged 18–64 years included 685,154 observations on 494,647 individuals in 8055 practices.

Descriptive statistics of the individual-level data for the two samples are provided in Tables 21 and 22. In the data set for the base-case (sensitivity) analysis, the mean annual per-patient cost was £3159 (SD £7278) (£3289, SD £7414), mean age was 52 years (41) and 44% (48%) of the patients were male. With regard to ethnicity, 83% (80%) were white, 4% (5%) were of mixed race, 4% (5%) Asian and 1% (1%) were black. Six per cent (7%) of patients are formally detained, 12% (12%) were hospitalised during the period and 6% (7%) had a social worker assigned to their care. Within any 1 year, < 1% of the patients used local authority day care, and in both analyses similar percentages applied for other local authority services such as home help and residential care. In both data sets, the use of sheltered work was rare (0.12% and 0.13%, respectively).

TABLE 21 - Mental Health Minimum Data Set analysis: descriptive statistics for the base case (all individuals)

Max., maximum; min., minimum.

Variable description	n	Mean (or % where indicated)	SD	Median	Min.	Max.
Total annual cost per patient	981,373	3159	7278	731	100	155,914
Natural log of annual cost	981,373	6.81	1.48	6.59	4.61	11.96
Financial year 2006/7	981,373	47.7%	–	–	–	–
Financial year 2007/8	981,373	52.3%	–	–	–	–
Patient age at start of spell/record period (years)	981,373	52.31	20.50	49.00	18.00	110.00
Aged 18–25 years	981,373	7.6%	–	–	–	–
Aged 25–34 years	981,373	15.0%	–	–	–	–
Aged 35–44 years	981,373	19.8%	–	–	–	–
Aged 45–54 years	981,373	15.7%	–	–	–	–
Aged 55–64 years	981,373	11.8%	–	–	–	–
Aged 65–74 years	981,373	9.4%	–	–	–	–
Aged 75 and over	981,373	20.8%	–	–	–	–
Proportion male patients	981,373	44.0%	–	–	–	–
Ethnicity, revised	981,373	1.50	1.21	1.00	1.00	5.00
Proportion white ethnicity	981,373	82.5%	–	–	–	–
Proportion mixed ethnicity	981,373	3.8%	–	–	–	–
Proportion Asian ethnicity	981,373	3.7%	–	–	–	–
Proportion black ethnicity	981,373	0.9%	–	–	–	–
Proportion other or undefined ethnicity	981,373	9.0%	–	–	–	–
Indicator for multiple spells in-year	981,373	10.4%	–	–	–	–
Indicator for learning disability speciality (any time)	981,373	0.0%	–	–	–	–
Indicator for patient being detained (year)	981,373	6.0%	–	–	–	–
Indicator for inpatient (year)	981,373	11.6%	–	–	–	–
Indicator for whether or not receiving local authority organised day centre care (year)	981,373	0.9%	–	–	–	–
Indicator for whether or not receiving local authority domiciliary care support (year)	981,373	0.7%	–	–	–	–
Indicator for whether or not receiving local authority organised residential care (year)	981,373	0.6%	–	–	–	–
Indicator for using sheltered work (year)	981,373	0.1%	–	–	–	–
Indicator for involvement of social worker (year)	981,373	6.2%	–	–	–	–

TABLE 22 - Mental Health Minimum Data Set analysis: descriptive statistics for the sensitivity analyses (working-aged individuals)

Max., maximum; min., minimum.

Variable description	n	Mean	SD	Median	Min.	Max.
Total annual cost per patient	685,154	3289	7414	815	100	155,914
Natural log of annual cost	685,154	6.88	1.49	6.70	4.61	11.96
Financial year 2007/8	685,154	48.2%	–	–	–	–
Financial year 2008/9	685,154	51.8%	–	–	–	–
Patient age at start of spell/record period (years)	685,154	40.93	12.18	41.00	18.00	64.00
Aged 18–25 years	685,154	10.8%	–	–	–	–
Aged 25–34 years	685,154	21.5%	–	–	–	–
Aged 35–44 years	685,154	28.3%	–	–	–	–
Aged 45–54 years	685,154	22.5%	–	–	–	–
Aged 55–64 years	685,154	16.9%	–	–	–	–
Gender: male	685,154	47.9%	–	–	–	–
Ethnicity, revised	685,154	1.56	1.25	1.00	1.00	5.00
Proportion white ethnicity	685,154	79.7%	–	–	–	–
Proportion mixed ethnicity	685,154	4.8%	–	–	–	–
Proportion Asian ethnicity	685,154	4.7%	–	–	–	–
Proportion black ethnicity	685,154	1.3%	–	–	–	–
Proportion other or undefined ethnicity	685,154	9.6%	–	–	–	–
Indicator for multiple spells in-year	685,154	10.4%	–	–	–	–
Indicator for if learning disability speciality (any time)	685,154	0.0%	–	–	–	–
Indicator for patient being detained	685,154	7.1%	–	–	–	–
Indicator for inpatient	685,154	12.1%	–	–	–	–
Indicator for whether or not receiving local authority organised day centre care	685,154	0.8%	–	–	–	–
Indicator for whether or not receiving local authority domiciliary care support	685,154	0.6%	–	–	–	–
Indicator for whether or not receiving local authority organised residential care	685,154	0.5%	–	–	–	–
Indicator for using sheltered work	685,154	0.1%	–	–	–	–
Indicator for involvement of social worker	685,154	6.6%	–	–	–	–

In total, 8064 practices contributed data for the base case. For each of the QOF indicators, practice achievement ranged from 0% to 100%. Average (mean) achievement was 69% for MH6 (SD 19%) and 81% (SD 14%) for MH9. When achievement was adjusted to exclude exception-reported individuals, the corresponding figures were 80% (SD 18%) for MH6 and 93% (SD 10%) for MH9. The exception reporting rates were 15% (range 0–88%) for MH6 and 13% (range 0–83%) for MH9. In the sensitivity analysis of individuals aged from 18 years to 64 years, these practice-level statistics were almost identical as almost all practices are represented (8055 practices).

The QOF indicators for annual review (MH9) and care planning (MH6) have no significant effect on total annual patient cost in any of the models, whether tested individually or jointly (Table 23). Results from the base case for model 1 are presented in Table 24 while the sensitivity regression analyses results can be found in Sensitivity analyses (see Table 26).

TABLE 23 - Mental Health Minimum Data Set analysis: regression results for the QOF indicators (all models, base case)

Note

For model definitions, see Table 20.

Model/indicator	Joint modelling		MH6 only		MH9 only
	Beta	95% CI	Beta	95% CI	Beta	95% CI
Model 1
MH6	0.016	–0.026 to 0.058	0.018	–0.015 to 0.051	–	–
MH9	0.005	–0.049 to 0.059	–	–	0.018	–0.025 to 0.060
Model 2
MH6	0.018	–0.019 to 0.056	0.017	–0.011 to 0.044	–	–
MH9	–0.003	–0.052 to 0.045	–	–	0.011	–0.025 to 0.047
Model 3
MH6	0.013	–0.026 to 0.053	0.008	–0.027 to 0.042	–	–
MH9	–0.019	–0.086 to 0.047	–	–	–0.008	–0.067 to 0.051
Model 4
MH6	0.010	–0.025 to 0.045	0.007	–0.023 to 0.037	–	–
MH9	–0.010	–0.067 to 0.047	–	–	–0.002	–0.051 to 0.047

TABLE 24 - Mental Health Minimum Data Set analysis: regression results from the base case (model 1), all individuals

SE, standard error.

–, p > 0.05

Variable description	Joint modelling			MH6 only			MH9 only
	Beta	SE	p-value	Beta	SE	p-value	Beta	SE	p-value
Achievement on MH6 QOF indicator	0.016	0.022	–	0.018	0.017	–	–	–	–
Achievement on MH9 QOF indicator	0.005	0.028	–	–	–	–	0.018	0.022	–
Gender: male	0.026	0.006	< 0.001	0.026	0.006	< 0.001	0.026	0.006	< 0.001
Aged 18–25 years	–0.101	0.012	< 0.001	–0.101	0.012	< 0.001	–0.101	0.012	< 0.001
Aged 35–44 years	0.032	0.007	< 0.001	0.032	0.007	< 0.001	0.032	0.007	< 0.001
Aged 45–54 years	0.042	0.009	< 0.001	0.042	0.009	< 0.001	0.042	0.009	< 0.001
Aged 55–64 years	0.020	0.012	–	0.020	0.012	–	0.020	0.012	–
Aged 65–74 years	0.043	0.019	< 0.05	0.043	0.019	< 0.05	0.043	0.019	< 0.05
Aged 75 years and over	–0.215	0.023	< 0.001	–0.215	0.023	< 0.001	–0.215	0.023	< 0.001
Ethnicity: mixed	0.222	0.019	< 0.001	0.222	0.019	< 0.001	0.222	0.019	< 0.001
Ethnicity: Asian	–0.022	0.016	–	–0.022	0.016	–	–0.022	0.016	–
Ethnicity: black	0.137	0.019	< 0.001	0.137	0.019	< 0.001	0.137	0.019	< 0.001
Ethnicity: other or undefined	–0.678	0.023	< 0.001	–0.678	0.023	< 0.001	–0.678	0.023	< 0.001
Indicator for learning disability speciality (any time)	0.275	0.216	–	0.275	0.216	–	0.275	0.216	–
Indicator for patient being detained (year)	1.943	0.051	< 0.001	1.943	0.051	< 0.001	1.943	0.051	< 0.001
Financial year 2007/8	0.032	0.019	–	0.032	0.019	–	0.033	0.019	–
Proportion of practice patients able to access care within 48 hours	–0.032	0.029	–	–0.032	0.029	–	–0.032	0.029	–
Indicator for whether or not GP practice is reimbursed under PMS	0.002	0.007	–	0.002	0.007	–	0.002	0.007	–
Proportion of male GPs in GP practice	0.027	0.012	< 0.05	0.027	0.012	< 0.05	0.027	0.012	< 0.05
Proportion of foreign GPs in GP practice	–0.007	0.010	–	–0.007	0.010	–	–0.006	0.010	–
Mean age of GPs in GP practice	0.000	0.001	–	0.000	0.001	–	0.000	0.001	–
MH claimants – second quintile	0.032	0.011	< 0.01	0.032	0.011	< 0.01	0.033	0.011	< 0.01
MH claimants – third quintile	0.058	0.012	< 0.001	0.058	0.012	< 0.001	0.059	0.012	< 0.001
MH claimants – fourth quintile	0.063	0.013	< 0.001	0.063	0.013	< 0.001	0.063	0.013	< 0.001
MH claimants – fifth quintile	0.061	0.018	< 0.001	0.061	0.018	< 0.001	0.061	0.018	< 0.001
Proportion providing informal care, practice catchment area	–1.995	0.570	< 0.001	–1.994	0.571	< 0.001	–1.989	0.570	< 0.001
NHS psychiatric residents per 1000 population, practice catchment area	0.003	0.003	–	0.003	0.003	–	0.003	0.003	–
Proportion living in urban setting, practice catchment area	0.046	0.019	< 0.05	0.046	0.019	< 0.05	0.046	0.019	< 0.05
GP practice list size (second tertile – medium)	0.006	0.007	–	0.006	0.007	–	0.006	0.007	–
GP practice list size (third tertile – large)	0.008	0.008	–	0.008	0.008	–	0.008	0.008	–
Patient population: average age	0.001	0.001	–	0.001	0.001	–	0.001	0.001	–
Patient population: proportion of male patients	0.234	0.170	–	0.234	0.170	–	0.233	0.170	–
Constant term	6.776	0.110	< 0.001	6.778	0.109	< 0.001	6.776	0.110	< 0.001
PCT-level variance	0.320	0.018	< 0.001	0.320	0.018	< 0.001	0.320	0.018	< 0.001
GP-level variance	0.153	0.006	< 0.001	0.153	0.006	< 0.001	0.153	0.006	< 0.001
Patient-level variance	1.337	0.008	< 0.001	1.337	0.008	< 0.001	1.337	0.008	< 0.001
Number of observations	981,373	–	–	981,373	–	–	981,373	–	–

The relationship between general practices’ performance on the QOF indicators and annual per-patient cost is not statistically significant at the 5% level in any of the models. Within each model, the relationship is insignificant when the indicators, MH9 and MH6, were modelled jointly and also when they were each tested individually. This relationship is robust to alternative calculations of the QOF performance measure (i.e. the inclusion/exclusion of data on exception-reported individuals), and to the inclusion/exclusion of patient supply variables.

Across the four models, patient-level factors most consistently associated with higher cost include middle age, being of black or Asian ethnicity, and being formally detained under the Mental Health Act 1983. 49 Male gender is associated with significantly higher cost, but this relationship is not statistically significant when supply-level factors such as use of inpatient care are accounted for. The relationship between age and cost is bell-shaped: relative to the reference group of people aged 25–34 years, costs are lower for people aged under 25 years, about 3–4% higher for those aged 35–55 years and similar for people aged over 55 years. Costs for people aged 75 years and over are around 20% lower than those of the reference group. Higher cost is also associated with greater deprivation scores for the practice population. Interestingly, the variable measuring the prevalence of informal care within the practice population residency is strongly and negatively related to annual per-patient cost.

In the two models that include covariates to control for supply-side factors (not shown), findings are broadly consistent. Having multiple spells within a year (see Figure 10) is associated with significantly lower annual cost, whereas higher cost is associated with receipt of inpatient care, and some local authority services such as day centre attendance, and social worker involvement.

Sensitivity analyses

As in the base case, the sensitivity analysis of individuals aged 18–64 years found no statistically significant relationship between general practices’ performance on the QOF indicators and annual per-patient cost in any of the four models (Table 25). For the other explanatory factors, findings were broadly similar to those of the base case (Table 26).

TABLE 25 - Mental Health Minimum Data Set analysis: regression results for the QOF indicators (all models, sensitivity analysis)

Note

For model definitions, see Table 20.

Model/indicator	Joint modelling		MH6 only		MH9 only
	Beta	95% CI	Beta	95% CI	Beta	95% CI
Model 1
MH6	0.024	–0.022 to 0.069	0.024	–0.011 to 0.059	–	–
MH9	0.000	–0.057 to 0.057	–	–	0.019	–0.024 to 0.062
Model 2
MH6	0.022	–0.018 to 0.063	0.023	–0.007 to 0.053	–	–
MH9	0.001	–0.051 to 0.054	–	–	0.019	–0.019 to 0.057
Model 3
MH6	0.021	–0.022 to 0.065	0.007	–0.029 to 0.044	–	–
MH9	–0.046	–0.117 to 0.025	–	–	–0.028	–0.089 to 0.033
Model 4
MH6	0.017	–0.022 to 0.055	0.008	–0.024 to 0.041	–	–
MH9	–0.029	–0.092 to 0.035	–	–	–0.014	–0.068 to 0.040

TABLE 26 - Mental Health Minimum Data Set analysis: regression results from the sensitivity analyses, working age individuals

SE, standard error.

Variable description	Joint modelling			MH6 only			MH9 only
	Beta	SE	p-value	Beta	SE	p-value	Beta	SE	p-value
Achievement on MH6 QOF indicator	0.024	0.023	–	0.024	0.018	–	–	–	–
Achievement on MH9 QOF indicator	0.000	0.029	–	–	–	–	0.019	0.022	–
Gender: male	0.034	0.008	< 0.001	0.034	0.008	< 0.001	0.034	0.008	< 0.001
Aged 18–25 years	–0.099	0.011	< 0.001	–0.099	0.011	< 0.001	–0.099	0.011	< 0.001
Aged 35–44 years	0.030	0.007	< 0.001	0.030	0.007	< 0.001	0.030	0.007	< 0.001
Aged 45–54 years	0.038	0.009	< 0.001	0.038	0.009	< 0.001	0.038	0.009	< 0.001
Aged 55–64 years	0.015	0.012	–	0.015	0.012	–	0.015	0.012	–
Ethnicity: mixed	0.248	0.021	< 0.001	0.248	0.021	< 0.001	0.248	0.021	< 0.001
Ethnicity: Asian	–0.004	0.018	–	–0.004	0.018	–	–0.004	0.018	–
Ethnicity: black	0.160	0.019	< 0.001	0.160	0.019	< 0.001	0.160	0.019	< 0.001
Ethnicity: other or undefined	–0.715	0.026	< 0.001	–0.715	0.026	< 0.001	–0.715	0.026	< 0.001
Indicator for learning disability speciality (any time)	0.233	0.202	–	0.233	0.202	–	0.233	0.202	–
Indicator for patient being detained (year)	1.909	0.051	< 0.001	1.909	0.051	< 0.001	1.909	0.051	< 0.001
Financial year 2007/8	0.050	0.020	< 0.05	0.050	0.020	< 0.05	0.052	0.020	< 0.01
Proportion of practice patients able to access care within 48 hours	–0.035	0.031	–	–0.035	0.031	–	–0.034	0.031	–
Indicator for whether GP practice is reimbursed under PMS	0.004	0.006	–	0.004	0.006	–	0.004	0.006	–
Proportion of male GPs in GP practice	0.009	0.013	–	0.009	0.013	–	0.009	0.013	–
Proportion of foreign GPs in GP practice	–0.014	0.012	–	–0.014	0.012	–	–0.013	0.012	–
Mean age of GPs in GP practice	0.000	0.001	–	0.000	0.001	–	0.000	0.001	–
MH claimants – second quintile	0.029	0.015	< 0.05	0.029	0.014	< 0.05	0.029	0.014	< 0.05
MH claimants – third quintile	0.071	0.015	< 0.001	0.071	0.015	< 0.001	0.071	0.015	< 0.001
MH claimants – fourth quintile	0.079	0.015	< 0.001	0.079	0.015	< 0.001	0.079	0.015	< 0.001
MH claimants – fifth quintile	0.076	0.020	< 0.001	0.076	0.020	< 0.001	0.076	0.020	< 0.001
Proportion providing informal care, practice catchment area	–1.771	0.610	< 0.01	–1.771	0.610	< 0.01	–1.761	0.610	< 0.01
NHS psychiatric residents per 1000 population, practice catchment area	0.001	0.003	–	0.001	0.003	–	0.001	0.003	–
Proportion living in urban setting, practice catchment area	0.056	0.018	< 0.01	0.056	0.018	< 0.01	0.056	0.018	< 0.01
GP practice list size (second tertile – medium)	0.012	0.008	–	0.012	0.008	–	0.012	0.008	–
GP practice list size (third tertile – large)	0.016	0.010	–	0.016	0.010	–	0.016	0.010	–
Patient population: average age	0.001	0.001	–	0.001	0.001	–	0.001	0.001	–
Patient population: proportion of male patients	0.320	0.212	–	0.320	0.212	–	0.319	0.212	–
Constant term	6.684	0.114	< 0.001	6.684	0.114	< 0.001	6.685	0.114	< 0.001
PCT-level variance	0.349	0.020	< 0.001	0.349	0.020	< 0.001	0.350	0.020	< 0.001
GP-level variance	0.173	0.006	< 0.001	0.173	0.006	< 0.001	0.173	0.006	< 0.001
Patient-level variance	1.329	0.008	< 0.001	1.329	0.008	< 0.001	1.329	0.008	< 0.001
Number of observations	685,154	–	–	685,154	–	–	685,154	–	–

Percentage of valid exception reporting

In previous analyses, we sought to identify the ‘switching point’ at which the relationship between QOF achievement and admissions became statistically insignificant. In the cost analysis, we found that the way in which achievement was measured, that is, whether or not exception reported cases were included in the measures, had no influence on the statistical significance of the relationship between achievement and cost. Therefore, there is no ‘switching point’ to identify in this analysis.

Chapter 4 Discussion

Our research investigated the relationship between quality of primary care for people with SMI, as incentivised by the UK QOF, and hospital admissions for physical and mental health conditions. Good-quality primary care management of patients with a SMI should reduce complications of SMI and comorbidities and should, therefore, be associated with lower unplanned (emergency) admission rates. Conversely, better quality of care may result in more health problems being identified as part of regular screening activities and more frequent GP–patient contacts, thereby leading to more planned (elective) admissions for hospital care. If better-quality primary care leads to reduced emergency admissions, it may also be associated with lower NHS expenditure. LOS for patients with a SMI is typically much longer than for other patients and better management in primary care could shorten their lengths of stay in hospital.

Our research questions were as follows:

1. Is better general practice performance on SMI QOF indicators associated with:

   1. lower rates of emergency hospital admissions for SMIs for practice patients with a diagnosis of a SMI?

   2. lower rates of emergency admissions for a SMI for practice patients with a diagnosis of bipolar disorder?

   3. lower rates of emergency admissions for physical conditions for practice patients with a current or previous diagnosis of a SMI?

   4. higher rates of elective admissions for physical conditions in patients with a current or previous diagnosis of a SMI?

2. Is better general practice performance on SMI QOF indicators associated, with shorter LOS for practice patients with a SMI following admission for a SMI?

3. Is better performance on SMI QOF indicators associated with lower secondary care expenditure for mental health services for practice patients with a SMI?

Our null hypotheses were that there is no association between primary care quality and either admissions, LOS or costs. Our alternative hypotheses were that preventative care could lower emergency hospital admissions, reduce LOS and reduce mental health expenditure, and that regular screening could increase elective admissions.

Our results showed, contrary to expectation, a positive and statistically significant association between QOF achievement (particularly for MH9 – annual health checks) and emergency admissions, for both mental and physical admissions. This implies that better quality of primary care is associated with more admissions, not fewer. The results showed a positive association between QOF achievement and elective admissions, as expected, although results were not statistically significant. The results for QOF achievement on MH4 (lithium therapy recorded) for patients with bipolar disorder also showed a statistically significant and positive association with admissions. Results showed no statistically significant effect of QOF achievement on either LOS or cost. All results were robust to sensitivity analyses.

There are a number of potential explanations for our findings and we present them in order of their perceived likelihood.

1. Higher quality of primary care, as measured by QOF, may not effectively prevent the need for secondary care. Current policy places an emphasis on ‘upstream’ prevention and ‘early intervention’ to avoid the need for more intensive and expensive specialist care. Our study findings do raise some doubt about whether or not higher-quality primary care, as measured by the SMI QOF indicators, is delivering on this policy aim. The QOF may be failing as an effective means of supporting early intervention and avoidance of crisis and hence emergency admissions. Indeed this chimes more broadly with questions about the effectiveness of current policy measures to prevent secondary care use through improved preventative management of certain clinical conditions given that these efforts have hitherto failed to reduce the demand for emergency care. 61

2. Better-quality primary care may be picking up unmet need for secondary care. Practices that attract people with a SMI who may be less likely to engage with services and those with relatively high levels of unmet need may uncover and treat this unmet need which may then entail admission.

3. We analysed QOF data at the level of practices and cannot be sure whether or not the specific individuals who were admitted had received QOF checks. It is possible that those who were admitted as emergencies had not received an annual QOF review. It is also possible that individuals admitted as emergencies did receive QOF reviews, but only after discharge from hospital, rather than prior to admission, which could also explain the positive association.

4. If people with a SMI know from their experiences with the health and social care system that some practices are more receptive to them, recognise their needs or are better organised to provide their care, then this may systematically influence their choice of practice. These practices may therefore achieve high scores for the QOF reviews but also may have a higher rate of emergency admissions for these individuals if they attract and treat people whose disease is complex or at the higher end of the severity spectrum.

5. The QOF indicators may not accurately measure the quality of primary care for a SMI. The QOF, like any other P4P scheme, may result in tunnel vision71 or a focus on areas of activities within the scheme which are incentivised, sometimes at the expense of other activities which are not incentivised. 72 Thus, high QOF attainment may not necessarily reflect high-quality care.

6. As hypothesised, higher rates of QOF checks were associated with higher rates of elective admissions for physical health problems for people with SMI, although these associations were not statistically significant. However, this trend is consistent with trial evidence of increased referrals following the introduction of regular health checks for people with long-term mental illness. 73

This study makes a unique contribution in that it is the first analysis to examine the relationship between primary care quality, as measured by SMI QOF indicators, and admission rates for patients with a SMI for both mental and physical health. It also makes a unique contribution in examining the resource implications of better-quality primary care for SMI patients. The strengths of the study are that it covers all practices in England and results are representative. It uses a consistent set of primary care quality indicators over the entire study period and employs robust panel data estimation as well as cross-sectional analysis. A comprehensive set of GP practice and patient population characteristics are included in the models. An array of sensitivity analyses was undertaken and results were found to be robust.

The limitations of the study include the fact that aggregate data were used to examine the association between QOF quality and admissions. This does not enable one to specify the nature of individual care pathways or determine causality. Although we attempted to model all known confounding factors, we were unable to capture time-variant aspects of the influence of CMHTs and could not account for disease severity (case mix). In addition, we cannot rule out the possibility that there are other, unknown, biases that impact our findings. Another limitation was the incomplete data available in the cost analysis. To help address this limitation, the resource use implications of primary care quality on LOS were also investigated, as the LOS data were drawn from a more comprehensive data set.

Chapter 5 Conclusions

Acknowledgements

We would like to thank the NHS National Clinical Classifications Service at NHS Connecting for Health for providing us with the cross-mapping results for Read codes version 2 and version 3 (which are a coded thesaurus of clinical terms used by GP practices to record patient diagnoses) to ICD-10 diagnosis codes used in HES admissions data.

We would like to thank the following steering group members for their invaluable contributions to this project: June Wainwright, Lauren Aylott, Suzanne McBain, Peter Bower, Paul Blenkiron, Liz England and David Daniel. We received sound and timely advice from our medical, policy and patient and public involvement (PPI) colleagues on a wide range of issues, not only at steering group meetings but also on many other occasions when we sought their expertise. They helped clarify our thinking on various issues and also provided valuable insights with the interpretations of our results. They read and commented on the interim reports produced for the National Institute for Health Research (NIHR) and also provided comments on this draft final report. Their input made a huge difference to the project. We are delighted that we have their support and continued commitment to our future research plans and we look forward to a long and fruitful collaboration with them.

We would like to thank Rachel Richardson for contributions to the project in producing the dissemination strategy and in writing the advertising materials in lay language to invite service users and carers (e.g. RETHINK) to a workshop we ran in March 2013 presenting interim results from the project.

We would also like to thank conference participants from the various conferences where we presented our work for valuable comments and feedback: Health Economists Study Group (HESG), Exeter, January 2013; Primary Care Mental Health Conference, University of Manchester, March 2013; Eleventh Workshop on Costs and Assessment in Psychiatry (ICMPE), Venice, March 2013; HESG, Warwick, June 2013; Society for Academic Primary Care 42nd Annual Conference, University of Nottingham, July 2013; Royal College of General Practitioners’ Annual Primary Care Conference, Harrogate, UK, October 2013; and the North American Primary Care Research Group Annual Scientific Meeting, Ottawa, November 2013.

References

1. Harrison G, Hopper K, Craig T, Laska E, Siegel C, Wanderling J, et al. Recovery from psychotic illness: a 15- and 25-year international follow-up study. Br J Psychiatry 2001;178:506-17. http://dx.doi.org/10.1192/bjp.178.6.506.
2. Maximising Resources in Adult Mental Health. London: Audit Commission; 2010.
3. Saunders KEA, Goodwin GM. The course of bipolar disorder. Adv Psychiatr Treat 2010;16:318-28. http://dx.doi.org/10.1192/apt.bp.107.004903.
4. Saha S, Chant D, Welham J, McGrath J. A systematic review of the prevalence of schizophrenia. PLOS Med 2005;2. http://dx.doi.org/10.1371/journal.pmed.0020141.
5. Osborn DPJ. Physical activity, dietary habits and coronary heart disease risk factor knowledge amongst people with severe mental illness: a cross sectional comparative study in primary care. Soc Psychiatry Psychiatr Epidemiol 2007;42:787-93. http://dx.doi.org/10.1007/s00127-007-0247-3.
6. Lethal Discrimination: Why People with Mental Illness are Dying Needlessly and What Needs to Change. London: Rethink Mental Illness; 2013.
7. Jaques H. NICE recommends dropping all QOF indicators on depression. BMJ Careers 2011. http://careers.bmj.com/careers/advice/view-article.html?id=20004103 (accessed 31 March 2014).
8. Miller B, Paschall C, Svendsen D. Mortality and medical comorbidity among patients with SMI. Psychiatr Serv 2006;57:1482-7. http://dx.doi.org/10.1176/appi.ps.57.10.1482.
9. Wahlbeck K, Westman J, Nordentoft M, Gissler M, Laursen T. Outcomes of Nordic mental health systems: life expectancy of patients with mental disorders. Br J Psychiatry 2011;199:453-8. http://dx.doi.org/10.1192/bjp.bp.110.085100.
10. Brown S, Inskip H, Barraclough B. Causes of the excess mortality of schizophrenia. Br J Psychiatry 2000;177:212-17. http://dx.doi.org/10.1192/bjp.177.3.212.
11. Osborn DPJ, Levy G, Nazareth I, Petersen I, Islam A, King MB. Relative risk of cardiovascular and cancer mortality in people with severe mental illness from the United Kingdom’s General Practice Research Database. Arch Gen Psychiatry 2007;64:242-9. http://dx.doi.org/10.1001/archpsyc.64.2.242.
12. Harris EC, Barraclough B. Excess mortality of mental disorder. Br J Psychiatry 1998;173:11-53. http://dx.doi.org/10.1192/bjp.173.1.11.
13. Marder SR, Essock SM, Miller AL, Buchanan RW, Casey DE, Davis JM, et al. Physical health monitoring of patients with schizophrenia. Am J Psychiatry 2004;161:1334-49. http://dx.doi.org/10.1176/appi.ajp.161.8.1334.
14. Murray CJL, Lopez A. The Global Burden of Disease: A Comprehensive Assessment of Mortality and Disability from Diseases, Injuries, and Risk Factors in 1990 and Projected to 2020. Boston: Harvard School of Public Health on behalf of the World Health Organization and the World Bank; 1996.
15. Bouza C, López-Cuadrado T, Amate JM. Hospital admissions due to physical disease in people with schizophrenia: a national population-based study. Gen Hosp Psychiatry 2010;32:156-63. http://dx.doi.org/10.1016/j.genhosppsych.2009.11.014.
16. Li Y, Glance LG, Cai X, Mukamel DB. Mental illness and hospitalization for ambulatory care sensitive medical conditions. Med Care 2008;46:1249-56. http://dx.doi.org/10.1097/MLR.0b013e31817e188c.
17. Osby U, Brandt L, Correia N, Ekbom A, Sparen P. Excess mortality in bipolar and unipolar disorder in Sweden. Arch Gen Psychiatry 2001;58:844-50. http://dx.doi.org/10.1001/archpsyc.58.9.844.
18. Brown S, Kim M, Mitchell C, Inskip H. Twenty-five year mortality of a community cohort with schizophrenia. Br J Psychiatry 2010;196:116-21. http://dx.doi.org/10.1192/bjp.bp.109.067512.
19. Connolly M, Kelly C. Lifestyle and physical health in schizophrenia. Adv Psychiatr Treat 2005;11:125-32. http://dx.doi.org/10.1192/apt.11.2.125.
20. Revisions to the GMS Contract 2006/07. Delivering Investment in General Practice. London: NHS Confederation (Employers) Company Ltd; 2006.
21. McElroy SL. Obesity in patients with severe mental illness: overview and management. J Clin Psychiatry 2009;70:12-21. http://dx.doi.org/10.4088/JCP.7075su1c.03.
22. Kendrick T. Cardiovascular and respiratory risk factors and symptoms among general practice patients with long-term mental illness. Br J Psychiatry 1996;169:733-9. http://dx.doi.org/10.1192/bjp.169.6.733.
23. McManus S, Meltzer H, Campion J. Cigarette Smoking and Mental Health in England: Data from the Adult Psychiatric Morbidity Survey 2010. www.natcen.ac.uk/media/21994/smoking-mental-health.pdf (accessed 4 April 2014).
24. Sernyak MJ, Leslie DL, Alarcon RD, Losonczy MF, Rosenheck R. Association of diabetes mellitus with use of atypical neuroleptics in the treatment of schizophrenia. Am J Psychiatry 2002;159:561-6. http://dx.doi.org/10.1176/appi.ajp.159.4.561.
25. Jacobs R, Barrenho E. The impact of crisis resolution and home treatment teams on psychiatric admission rates in England. Br J Psychiatry 2011;99:71-6. http://dx.doi.org/10.1192/bjp.bp.110.079830.
26. Nazareth I, King M, Haines A. Care of schizophrenia in general practice. BMJ 1993;307. http://dx.doi.org/10.1136/bmj.307.6909.910.
27. Kai J, Crosland A, Drinkwater C. Prevalence of enduring and disabling mental illness in the inner city. Br J Gen Pract 2000;50:992-4.
28. Lang F, Johnstone E, Murray G. Service provision for people with schizophrenia. Role of the general practitioner. Br J Psychiatry 1997;171:165-8. http://dx.doi.org/10.1192/bjp.171.2.165.
29. Reilly S, Planner C, Hann M, Reeves D, Nazareth I, Lester H. The role of primary care in service provision for people with severe mental illness in the United kingdom. PLOS ONE 2012;7. http://dx.doi.org/10.1371/journal.pone.0036468.
30. Purdy S, Paranjothy S, Huntley A, Thomas R, Mann M, Huws D, et al. Interventions to Reduce Unplanned Hospital Admission: A Series of Systematic Reviews. Final Report. Bristol: University of Bristol; 2012.
31. Purdy S. Avoiding Hospital Admissions: What Does the Research Evidence Say?. London: The King’s Fund; 2010.
32. Giuffrida A, Gravelle H, Roland M. Measuring quality of care with routine data: avoiding confusion between performance indicators and health outcomes. BMJ 1999;319:94-7. http://dx.doi.org/10.1136/bmj.319.7202.94.
33. Saxena S, George J, Barber J, Fitzpatrick J, Majeed A. Association of population and practice factors with potentially avoidable admission rates for chronic diseases in London: cross-sectional analysis. J R Soc Med 2006;99:81-8. http://dx.doi.org/10.1258/jrsm.99.2.81.
34. Griffin S, Kinmonth A. Systems for routine surveillance for people with diabetes mellitus. Cochrane Database Syst Rev 2006;4.
35. McEvoy JP, Meyer JM, Goff DC, Nasrallah HA, Davis SA, Sullivan L, et al. Prevalence of the metabolic syndrome in patients with schizophrenia: baseline results from the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) Schizophrenia Trial and comparison with national estimates from NHANES III. Schizophr Res 2005;80:19-32. http://dx.doi.org/10.1016/j.schres.2005.07.014.
36. About the Quality and Outcomes Framework (QOF). NICE; 2012.
37. Downing A, Rudge G, Cheng Y, Tu YK, Keen J, Gilthorpe MS. Do the UK government’s new Quality and Outcomes Framework (QOF) scores adequately measure primary care performance? A cross-sectional survey of routine healthcare data. BMC Health Serv Res 2007;7. http://dx.doi.org/10.1186/1472-6963-7-166.
38. Bottle A, Gnani S, Saxena S, Aylin P, Mainous AG, Majeed A. Association between quality of primary care and hospitalization for coronary heart disease in England: a national cross-sectional study. J Gen Intern Med 2008;23:135-41. http://dx.doi.org/10.1007/s11606-007-0390-2.
39. Bottle A, Millett C, Xie Y, Saxena S, Wachter RM, Majeed A. Quality of primary care and hospital admissions for diabetes mellitus in England. J Ambul Care Manage 2008;31:226-38. http://dx.doi.org/10.1097/01.JAC.0000324668.83530.6d.
40. Purdy S, Griffin T, Salisbury C, Sharp D. Emergency admissions for coronary heart disease: a cross-sectional study of general practice, population and hospital factors in England. Public Health 2011;125:46-54. http://dx.doi.org/10.1016/j.puhe.2010.07.006.
41. Dusheiko M, Doran T, Gravelle H, Fullwood C, Roland M. Does higher quality of diabetes management in family practice reduce unplanned hospital admissions?. Health Serv Res 2011;46:27-46. http://dx.doi.org/10.1111/j.1475-6773.2010.01184.x.
42. Soljak M, Calderon-Larranaga A, Sharma P, Cecil E, Bell D, Abi-Aad G, et al. Does higher quality primary health care reduce stroke admissions? A national cross-sectional study. Br J Gen Pract 2011;61:e801-7. http://dx.doi.org/10.3399/bjgp11X613142.
43. Doran T, Fullwood C, Reeves D, Gravelle H, Roland M. Exclusion of patients from pay-for-performance targets by English physicians. N Engl J Med 2008;359:274-84. http://dx.doi.org/10.1056/NEJMsa0800310.
44. Petersen LA, Woodard LD, Urech T, Daw C, Sookanan S. Does pay-for-performance improve the quality of health care?. Ann Intern Med 2006;145:265-72. http://dx.doi.org/10.7326/0003-4819-145-4-200608150-00006.
45. Roland M. Linking physician pay to quality of care: a major experiment in the UK. N Engl J Med 2004;35:1448-54. http://dx.doi.org/10.1056/NEJMhpr041294.
46. Mason A, Walker S, Claxton K, Cookson R, Fenwick E, Sculpher M. The GMS Quality and Outcomes Framework: Are the Quality and Outcomes Framework (QOF) Indicators a Cost-effective Use of NHS Resources? Quality and Outcomes Framework. Joint Executive Summary: Reports to the Department of Health from the University of East Anglia and the University of York. York: University of York; 2008.
47. Walker S, Mason AR, Claxton K, Cookson R, Fenwick E, Fleetcroft R, et al. Value for money and the Quality and Outcomes Framework in primary care in the UK NHS. Br J Gen Pract 2010;60:213-20. http://dx.doi.org/10.3399/bjgp10X501859.
48. Focus on QOF Payments. London: British Medical Association; 2013.
49. Mental Health Act 1983. London: The Stationery Office; 1983.
50. QOF Guidance. London: DH; 2004.
51. Doran T, Kontopantelis E, Fullwood C, Lester H, Valderas JM, Campbell S. Exempting dissenting patients from pay for performance schemes: retrospective analysis of exception reporting in the UK Quality and Outcomes Framework. BMJ 2012;344. http://dx.doi.org/10.1136/bmj.e2405.
52. Gravelle H, Sutton M, Ma A. Doctor behaviour under a pay for performance contract: treating, cheating and case finding?. Economic Journal 2010;120:F129-F56. http://dx.doi.org/10.1111/j.1468-0297.2009.02340.x.
53. Office for National Statistics . Neighbourhood Statistics n.d. www.neighbourhood.statistics.gov.uk/dissemination/ (accessed 23 September 2014).
54. Glover G, Arts G, Babu KS. Crisis resolution/home treatment teams and psychiatric admission rates in England. Br J Psychiatry 2006;189:441-5. http://dx.doi.org/10.1192/bjp.bp.105.020362.
55. Onyett S, Linde K, Glover G, Floyd S, Bradley S, Middleton H. Implementation of crisis resolution/home treatment teams in England: national survey 2005–2006. Psychiatric Bull 2008;32:374-7. http://dx.doi.org/10.1192/pb.bp.107.018366.
56. Glover G, Barnes D. Mental Health Service Provision for Working Age Adults in England 2003. Durham: Centre for Public Mental Health, University of Durham; 2005.
57. Holmes AM, Deb P. Factors influencing informal care-giving. J Ment Health Policy Econ 1998;1:77-8. http://dx.doi.org/10.1002/(SICI)1099-176X(199807)1:2&lt;77::AID-MHP10&gt;3.0.CO;2-5.
58. Cameron AC, Trivedi PK. Regression Analysis of Count Data. Cambridge: Cambridge University Press; 1998.
59. Hausman J, Hall BH, Griliches Z. Econometric models for count data with an application to the patents-R&amp;D relationship. Econometrica 1984;52:909-38. http://dx.doi.org/10.2307/1911191.
60. Blundell R, Griffith R, Windmeijer F. Individual effects and dynamics in count data models. J Econom 2002;108:113-31. http://dx.doi.org/10.1016/S0304-4076(01)00108-7.
61. Bardsley M, Blunt I, Davies S, Dixon J. Is secondary preventive care improving? Observational study of 10-year trends in emergency admissions for conditions amenable to ambulatory care. BMJ Open 2013;3. http://dx.doi.org/10.1136/bmjopen-2012-002007.
62. Johnstone P, Zolese G. Systematic review of the effectiveness of planned short hospital stays for mental health care. BMJ 1999;318:1387-90. http://dx.doi.org/10.1136/bmj.318.7195.1387.
63. International Statistical Classification of Diseases and Related Health Problems 10th Revision. Geneva: WHO; 2014.
64. Street A, Kobel C, Renaud T, Thuilliez J. on behalf of the EuroDRG group . How well do Diagnosis Related Groups explain variations in costs or LOS among patients and across hospitals? Methods for analysing routine patient data. Health Economics 2012;21:6-18. http://dx.doi.org/10.1002/hec.2837.
65. mentalhealthcare . Forensic Mental Health Services 2014. www.mentalhealthcare.org.uk/forensic_mental_health_services (accessed 31 March 2014).
66. National Schedule of Reference Costs 2007–08 for NHS Trusts. London: DH; 2009.
67. Sutton M, Whittaker W, Morris S, Glover G, Dusheiko M, Wildman J, et al. RARP35 Report of the Resource Allocation for Mental Health and Prescribing (RAMP) Project. Report to the Department of Health 2010. http://webarchive.nationalarchives.gov.uk/+/www.dh.gov.uk/en/Managingyourorganisation/Financeandplanning/Allocations/DH_4108515 (accessed 31 March 2014).
68. Curtis L. Unit Costs of Health and Social Care. Canterbury: Personal Social Services Research Unit, University of Kent; 2009.
69. Matthews F, Arthur A, Barnes LE, Bond J, Jagger C, Robinson L, et al. A two-decade comparison of prevalence of dementia in individuals aged 65 years and older from three geographical areas of England: results of the Cognitive Function and Ageing Study I and II. Lancet 2013;382:1405-12. http://dx.doi.org/10.1016/S0140-6736(13)61570-6.
70. Halvorsen R, Palmquist R. The interpretation of dummy variables in semilogarithmic equations. Am Econ Rev 1980;70:474-5.
71. Smith P. On the unintended consequences of publishing performance data in the public sector. Int J Public Admin 1995;18:277-310. http://dx.doi.org/10.1080/01900699508525011.
72. Eggleston K. Multitasking and mixed systems for provider payment. J Health Econ 2005;24:211-23. http://dx.doi.org/10.1016/j.jhealeco.2004.09.001.
73. Kendrick T, Burns T, Freeling P. Randomised controlled trial of teaching general practitioners to carry out structured assessments of their long term mentally ill patients. BMJ 1995;311:93-8. http://dx.doi.org/10.1136/bmj.311.6997.93.
74. Desai MM, Rosenheck RA. Unmet need for medical care among homeless adults with serious mental illness. Gen Hosp Psychiatry 2005;27:418-25. http://dx.doi.org/10.1016/j.genhosppsych.2005.06.003.
75. Community Mental Health Profiles 2013. Durham: Public Health England; 2013.
76. Wiersma D. Needs of people with severe mental illness. Acta Psychiatr Scand Suppl 2006;429:115-19. http://dx.doi.org/10.1111/j.1600-0447.2005.00728.x.
77. Buchanan M. England’s Mental Health Services ‘In Crisis’ 2013. www.bbc.co.uk/news/health-24546656 (accessed 31 March 2014).

Appendix 1 Patient and public involvement

Patient involvement was an important element in our research project from the outset. Ms June Wainwright (JW) who is a mental health service user and a carer and also a service user representative for the NIHR Mental Health Research Network (MHRN), was a co-applicant on our proposal and has therefore been a member of our project team throughout. June played a key role at the early stage in helping us formulate the research question and design and write the proposal. She also helped facilitate our overall PPI strategy for the project by advising us on the composition of our steering group, of which she is a member.

In addition to JW, our steering group includes two other PPI representatives [Ms Lauren Aylott (a mental health service user) and Ms Suzanne McBain (a carer)] both of whom have brought valuable perspectives to our project throughout. The steering group has met three times during the course of the project, in accordance with the schedule we set out in the proposal and our PPI members have contributed significantly to these discussions. At each meeting of the steering group, the principal investigator and a member of the project team outlined progress to date and the issues that have arisen for discussion in the group, in order to receive feedback and guidance on the next steps. The project team gave short presentations, which aimed to convey the main messages in an accessible way so that all members of the group were able to take part fully in the discussion.

The first meeting revolved around discussion of the QOF measures of quality of care that were available to the project team for analysis and the input of our PPI members was particularly valuable in terms of reflecting on the usefulness of measures in terms of what they mean in practice to patients. Their views on the nature of hospital admissions for people with a SMI were also important, that is, to what degree all admissions for this patient group can be seen as ‘emergency’ admissions, and also the potential impact of the CRHT teams. At this initial meeting we also discussed the other variables we were planning to include in our analysis and, again, our PPI members were able to give us an insight into issues related to home care and informal care, including possible sources of data. Planning for the workshop in which we disseminated our initial results also took place at the steering group meeting and we received advice from the PPI members on who we might invite to the workshop, including patient and carer mental health organisations and the PPI members were also invited to attend the workshop. JW also used her links with the MHRN to contact potential participants for the workshop.

The second meeting focused mainly on the initial results emerging from the analyses and specific points on which we sought and received advice from our PPI members, included the interpretation of the positive association between QOF and hospital admissions. Their views on this (as well as those of others on the group and project team) are reflected in this report in terms of possible explanations for the results. Another key question for our PPI members was around the annual review for patients with a SMI in terms of how invitations to attend are received and whether those being cared for mainly in the secondary care sector would view the annual review with the GP as an important or useful element of their care.

Our third, and final, steering group meeting reflected in detail on the final results and, again, the views of our PPI members on how to interpret the findings of a positive association between QOF and hospital admissions were invaluable. In particular, they helped shape our conclusions and our suggestions for further research to unpick and understand better our findings and the insights that the PPI members brought to this final interpretation of the results made a significant difference to our emphasis in this report. At this final meeting we also discussed the further involvement of two of our PPI members on a new proposal and subsequently they have attended meetings to discuss further work and have joined us as co-investigators on a proposal that has now been submitted to the NIHR. We are particularly pleased that we were able to engage the continued input from the PPI members in this way and this reflects the good relationships built up over the duration of the project.

In addition to the input at steering group meetings, we also were able to e-mail some of the PPI members to ask their views on specific questions as they arose in the research. We also asked for, and incorporated, their comments on the draft interim reports and the draft final report. Their involvement will continue beyond the production of the report because we will seek their input into the dissemination activities still to be undertaken, that is, the production of summaries targeted at decision-makers and service users and also anticipate their help in accessing channels through which the summaries can be distributed.

One of the challenges we anticipated for involvement of our PPI members was that they may not feel able to readily give their views at the steering group, given the range and background of other participants. We tried to address this by (1) ensuring that we did not just have one PPI member but three; (2) by thinking in advance of the meetings which were the key questions that would benefit most from their input, as opposed to other areas on which we were seeking mainly clinical or academic input; and (3) chairing our meetings in a way that was as relaxed and inclusive as possible.

Our payments to PPI members were made in accordance with the INVOLVE guidance. INVOLVE is a national advisory group, funded by the NIHR, to support active public involvement in NHS research.

Appendix 2 Further results

Glossary

Attribution data set

General practitioner practice data which include information on the age and gender and number of registered patients who are resident in each lower super output area (small-area level).

Black and minority ethnic groups

Patients in the black and minority ethic groups are at higher risk of compulsory admissions.

Care Programme Approach assessments

A needs assessment for people with serious mental disorders that involves the development of a care plan which is regularly reviewed.

Chronic obstructive pulmonary disorder

A collection of lung diseases, including chronic bronchitis and emphysema, that are often caused by smoking.

Community Mental Health Team

Community Mental Health Teams provide community-based services to people who are experiencing mental health problems. They are multiagency teams consisting of mental health professionals such as community mental health nurses, social workers, occupational therapists, psychiatrists and psychologists.

Continuous inpatient spell

The time between initial admission and final discharge. Patients are tracked over time, so a continuous inpatient spell can cover transfers between hospitals when this is part of a care pathway.

Crisis resolution and home treatment services

Developed to provide care for service users living in the community and experiencing a crisis requiring emergency admission to secondary care services. The services often have a gatekeeping function.

Electroconvulsive therapy

A treatment for severe depression.

Finished consultant episode

Period of care within a particular consultant specialty at a single hospital provider. This is the way in which Hospital Episode Statistics data are typically recorded.

General Medical Services contract

An alternative contract to Personal Medical Services contract status for general practitioner practices. The majority of practices have General Medical Services contracts.

General practitioner

Doctors who provide primary care services to patients who are registered with a general practice.

Health of the Nation Outcome Scale

A tool that assesses the health and social functioning of people with serious mental illness.

Health Services and Delivery Research programme

Part of the National Institute for Health Research funding regime (it subsumed the Service Delivery and Organisation programme).

Hospital Episode Statistics

Hospital Episode Statistics is an administrative data set that contains records of all patients in England who receive inpatient, outpatient or accident and emergency care. The inpatient data set contains around 18 million records annually. Hospital Episode Statistics patient (anonymised) identifiers allow tracking of patients over time and provide data on patients’ demographic and clinical characteristics, the area they live in and their registered general practitioner practice.

Incidence rate ratio

A way of expressing the coefficients in the regression model which is easier to interpret. This is typically the number of new cases of a condition in a defined (specified) group or population.

Index of Multiple Deprivation

Measure of deprivation which has subdomains such as education, crime, housing and health, available at lower super output area level.

International Classification of Diseases, 10th revision

Diagnostic categorisation system which is used in Hospital Episode Statistics to provide primary and secondary diagnosis codes.

Length of stay

Length of stay for inpatients, calculated from Hospital Episode Statistics data; proxy measure of resource use.

Lower super output area

Census-defined small-area level with an average population size of approximately 1500 individuals.

Mental health

Mental well-being, good mental functioning or having no particular problems in thinking, feelings or behaviour. The term ‘mental health problem’ or ‘mental disorder’ denotes the opposite.

Mental Health Minimum Data Set

Mandatory data collection for mental health provider hospitals (secondary care), patient-level data for people with severe and enduring mental health problems on hospitalisations, community and outpatient services.

Mental Health Research Network

Part of the National Institute for Health Research; helps research teams to recruit service user participants.

National Institute for Health Research

Funds health research through the Department of Health.

Office for National Statistics

Source of census data on population, demographics which are available at lower super output area level (small area).

Patient and public involvement

Getting patients or members of public involved in research; crucial underpinning of research activity.

Personal Medical Services contract

Forty per cent of practices have Personal Medical Services contracts. Personal Medical Services general practitioners earn about 10% more than General Medical Services general practitioners. Personal Medical Services practices take part in the Quality and Outcomes Framework, but, because they are already paid for some of the services counting towards the Quality and Outcomes Framework, they have points deducted from their Quality and Outcomes Framework score.

Primary care trust

Until March 2013 primary care trusts were responsible for commissioning primary, community and secondary health services from providers. They were responsible for spending around 80% of the total NHS budget. When primary care trusts were abolished in 2013 their work was taken over by clinical commissioning groups.

Quality and Outcomes Framework

A voluntary pay-for-performance incentive scheme for general practitioner practices, who earn points for achieving clinical targets for chronic conditions (including mental health problems). Points are also given depending on how well the practice is organised, the extra services offered and how patients view their experience.

Quality Management and Analysis System

A computer system used by the NHS for Quality and Outcomes Framework data.

Resource Allocation for Mental Health and Prescribing

A project commissioned by the English Department of Health to calculate resource allocation for mental health services.

Serious mental illness

Defined within the Quality and Outcomes Framework as those with schizophrenia, psychoses and bipolar disorder.

List of abbreviations

ADS

attribution data set

CI

confidence interval

CIPS

continuous inpatient spell

CMHT

Community Mental Health Team

COPD

chronic obstructive pulmonary disorder

CPA

Care Programme Approach

CRHT

crisis resolution and home treatment

ECT

electroconvulsive therapy

FCE

finished consultant episode

GMS

General Medical Services

GP

general practitioner

HES

Hospital Episode Statistics

HESG

Health Economists Study Group

HoNOS

Health of the Nation Outcome Scale

ICD-10

International Classification of Diseases, 10th revision

IMD

Index of Multiple Deprivation

IQR

interquartile range

IRR

incidence rate ratio

LIT

local implementation team

LOS

length of stay

LSOA

lower super output area

MHMDS

Mental Health Minimum Data Set

MHRN

Mental Health Research Network

NICE

National Institute for Health and Care Excellence

NIHR

National Institute for Health Research

ONS

Office for National Statistics

P4P

pay for performance

PCT

primary care trust

PMS

Personal Medical Services

PPI

patient and public involvement

QOF

Quality and Outcomes Framework

RAMP

Resource Allocation for Mental health and Prescribing project

SD

standard deviation

SMI

serious mental illness