Location of care for people with serious mental illness (LOCAPE): implications for service use and costs using a mixed-methods approach

The current need for comparisons between primary and secondary care

The NHS was asked to make unprecedented savings over 4 years up to the year 2014–15. Although there were a number of arguments about how best these savings could be made, care options that maintain quality but at a lower cost than comparators have been of particular interest. Mental health problems result in high costs. 18 Approximately 12% of the NHS budget goes on mental health care, with the bulk of spending on secondary mental health services. 19 Although there are treatments and therapies with established efficacy for people with SMIs, there is not a consensus as to the best location of care for patients. This is important because prices and costs differ markedly according to where care is provided, owing to differences in contractual arrangements, staff availability, infrastructure costs and overheads. Furthermore, similar to acute providers, care from specialist mental health providers will, in the future, be financed using payment by results and the tariff may be higher than for care provided elsewhere. This suggests that for some patients (e.g. those requiring relatively less specialist care than others) transfer from secondary care to primary care may be justified after their specific needs have been met by appropriate secondary care services. Other patients may initially come into contact with primary care services and it may again be considered appropriate to maintain their care in this location. Providing more care, where appropriate, in primary care settings may also help to address the high level of physical comorbidity in people with mental health problems. 20,21

The referral flows across primary and secondary care boundaries are determined by health service configuration factors such as the relative ease of access to specialist services and GPs. Perceived strengths of specialist services could include their function as a conduit to therapies not available in the practice and as a means by which to access social care services. Some general practices may feel very capable at dealing with SMI patients in distress, whereas others may feel less confident. Practices themselves may differ in their attitudes to SMI work. Some may consider it part of their role and take real pride in looking after ‘difficult’ patients. Others may feel that the complexity is too great. Providing data on the cost and benefits of transfers can inform investigations and assessments of the process of referring between these different agencies.

There are risks in transferring the lead role for co-ordinating and managing mental health care from secondary to primary services. For instance, if there is lack of rapid access to specialist mental health input, unplanned or emergency care may be more frequent. There is thus a need to identify the point at which patients may need care from a particular type of service and what the cost and clinical consequences of this may be. Determining the likelihood of unplanned care in specialist settings for patients managed in or transferred to primary care is crucial for those planning services, as is time to such care contacts. It is furthermore necessary to know what the costs of care would be for patients who are either not transferred or whose transfer is not as rapid as that of others. Information on these issues is currently lacking but may be derived from routinely available data, and may be of benefit to future assessments of patient care.

Specific services that have been set up in Lambeth may facilitate the transfer of ‘management’ to primary care and the maintenance of care in that location. As with any health-care interventions, these should be evaluated in terms of cost-effectiveness. There is a need to conduct evaluations efficiently, and we should make best use of existing data sets as alternatives to more expensive and time-consuming trials. These data will allow simulation models, which assess the costs and benefits of the different interventions, to be populated. There is also a need to assess care that is provided at a ‘whole-system’ level. The use of modelling is needed in such situations to ensure that results are generalisable. Models enable this by allowing specific characteristics/variables to be changed to reflect different circumstances in other areas.

This study therefore addresses a number of policy needs. It also takes account of the experiences and views of people with mental health problems regarding their care. The broad aim of the study is to examine the economic implications of different locations of management of care and the views of service users and staff regarding services set up as alternatives to secondary care. Specific objectives are as follows.

1. To identify people with SMIs whose care is (i) managed in primary care or (ii) managed in secondary care.

2. To identify people with SMIs who could be potentially transferred from secondary to primary care management.

3. To compare the demographic and clinical characteristics of these groups.

4. To measure and compare the use of services and to calculate service costs for these groups for the year prior to identification and the subsequent 34 months, adjusting for clinical and demographic differences in the groups.

5. To generate cost prediction models to enable resource consequences of patients with specific characteristics who are transferred from one form of care to another to be estimated.

6. To produce survival models to identify characteristics associated with time to transition from primary care to secondary care.

7. To investigate the experiences of patients receiving support from interventions designed to facilitate the transition from secondary to primary care.

8. To assess the economic impact of interventions to facilitate the transfer of care management.

The penultimate aim was revised in light of the characteristics of the recipients of the services. It had become apparent that many recipients had not, in fact, been recently discharged from secondary care services and some had conditions that would not be usually defined as SMIs.

In this report objectives 1–6 are explored in Chapter 2, objective 7 is explored in Chapter 3 and objective 8 is explored in Chapter 4. Each of Chapters 2, 3 and 4 contains a discussion. An overall brief discussion and conclusions are provided in Chapter 5.

Changes to protocol

Changes were discussed within the steering group and with NIHR monitors. The key changes were as follows: (1) the time frame for the quantitative analyses was originally January 2009 to October 2013. This was reduced to January 2010 to October 2013 owing to the availability of data; (2) we had hoped to include social care data in the final linked data set but the challenges in linking the health data precluded this and it was discovered that such social care data were historic and covered a small proportion of those in primary care registers; and (3) the intention of the qualitative component was to recruit participants with SMIs who had recently been discharged from secondary care services. However, many participants using the services had not actually been in touch with secondary care services for a long period of time and many did not have disorders usually defined as SMIs. These were still included as we considered it important to reflect the caseloads of the services.

A further change that is addressed in Chapter 5 relates to the input of service users to the research. We had hoped that interviews would be conducted by service users, but this was rarely possible for logistical reasons (timing of interviews, participants not appearing, etc.). However, service users were involved in the design of the topic guide and in the analysis of data. This was also a departure from the protocol, but a positive one.

Group definition 1: groups defined by discharge status

The mean age of patients discharged to primary care was similar to those remaining in secondary care (Table 1), and this was matched by a reasonably similar age distribution in each group as shown by the age bands. There was a slight difference in sex distribution, with relatively more men remaining in secondary care. In each group about one-third of patients were of white ethnicity and one-third were of black ethnicity, with no noticeable differences between the groups.

Nearly three-quarters of patients in secondary care had received a diagnosis of schizophrenia from specialist services, compared with only around half of primary care patients. However, about one-third of primary care patients had received no diagnosis from specialist services compared with < 8% of secondary care patients. There were clear differences between the groups in terms of history of violence, a forensic history, non-compliance with treatment and problems with physical health. In each case these were more common in the secondary care than in the primary care group.

Many of the clinical characteristics as recorded on LDN applied to relatively few patients. (It should be noted, however, that this was confined to characteristics recorded prior to the index date.) Hypertension, asthma, depression and diabetes mellitus were among the more common disorders noted by GPs. Receipt of thyroid-stimulating tests was also fairly frequent. However, the groups did not appreciably differ in relation to these.

Health of the Nation Outcome Scale (HoNOS) scores were available for only 2319 participants. This was an imbalance between the groups, with scores available for 94% of secondary care patients and 56% of those managed in primary care. We chose to use the scores that were recorded closest to the index date. The mean total score was 1.79 higher for the secondary care group, indicating greater severity. Subscores that showed the greatest differences were those relating to cognitive problems, hallucinations, relationships, daily living issues, occupation and other problems. These were all markedly higher for the secondary care group.

Group definition 2: groups defined by service use

The average age differed substantially between the three groups, with those managed in primary being, on average, 3.5 years older than those managed in secondary care and 8.4 years older than those who received neither primary nor secondary care (Table 2). Most patients were male, although roughly similar numbers of men and women were managed in primary care. Fewer than half the patients were of white ethnicity and, for those managed in secondary care, there were slightly more patients of black ethnicity.

Of those managed in primary care, half were diagnosed with schizophrenia but one-third had no diagnosis recorded. Three-quarters of patients managed in secondary care had a diagnosis of schizophrenia with affective disorders, comprising most remaining patients. Those patients who did not receive secondary care in the previous 6 months or primary care in the previous 12 months were most likely not to have a CRIS diagnosis. One-third had a diagnosis of schizophrenia. Interestingly, the LDN data show that relatively similar proportions in each group had a diagnosis of bipolar disorder. Rates of depression as recorded on the QOF were quite similar between groups, and for each group was less than 10%.

Physical health problems, as diagnosed by GPs, tended to be more prevalent in the primary care group and less prevalent in the neither primary nor secondary care group. The most common disorder in the primary care patients was hypertension (around one-quarter of the sample) and this was the same in the secondary care group (< 15%).

Health of the Nation Outcome Scale scores were available for 494 (38%) primary care patients, 1651 (93%) secondary care patients and 77 (20%) no care patients. Given the fact that a HoNOS score is unlikely to be given to a patient with no secondary care contacts, this distribution is unsurprising. Higher HoNOS scores indicating greater severity scores were noticeably higher for secondary care patients than for patients in the other two groups.

Group definition 1: groups defined by discharge status

During the year prior to the index date, around three-quarters of each group had received face-to-face contacts from their GP (Table 3). This increased somewhat in the subsequent years but there were few differences between the groups. Home visits were received by relatively few patients in either group, whereas the numbers receiving telephone contacts increased over time from around one-quarter prior to the index date to nearly half in the final year. Face-to-face nurse contacts were received by one-third to one-half of patients and again there were no clear differences between groups. Again, nurse contacts by telephone were relatively uncommon.

The number of patients with primary care consultations is shown in Table 4. Face-to-face GP consultations occurred, on average, about every 2 months, whereas face-to-face nurse contacts occurred around every 3 months. Differences between the two groups were limited. The costs of primary care were dominated by face-to-face GP contacts, accounting for between 74% and 84% of the total (Table 5). Contacts with nurses by telephone accounted for < 1% of the total.

Not surprisingly, use of secondary care services from the local NHS trust differed between the groups (Table 6). In the year prior to the index date there was a five-fold difference in the number of patients admitted to inpatient care, and in the subsequent years the difference remained substantial. For those who were admitted, the total length of stay across each year was fairly similar, with the exception of the first year where those who were subsequently discharged after the index date had more days in hospital (but were far less likely to be hospitalised in the first place). Although the number of days may seem high, this reflects multiple admissions and is also affected by extreme outliers. The cost of inpatient care mirrors the above findings, with far higher costs for the secondary care group. The proportion of patients with community contacts was also much higher among the secondary care patients, again with the largest difference during the first year. In each year, the number of community contacts for those who had them was substantially higher in the secondary care group. Finally, the table reveals that use and cost of psychotropic medication (as prescribed by the trust) was also much higher in the secondary care patients. Total mental health care costs in the year prior to the index date were 3.4 times higher among those in the discharged to primary care group. This ratio fell to 2.8 and 2.5 in 2011 and 2012, respectively, before rising again to 3.0 in 2013.

Between 14% and 20% of the patients had admissions for physical health reasons in each year (Table 7). This excludes the final period, where the figures are substantially lower than for other years, indicating that the data set was not fully updated. Inpatient days in the first years were somewhat higher for the secondary care group. During this year, the cost of inpatient care for physical health reasons was also higher for the secondary care group and thereafter the costs were more similar.

There was very little use of psychiatric inpatient care from other trusts, and differences between the groups were small. Around one-third of patients had outpatient contacts in each year (excluding 2013 for the above reasons) with around 4–5 contacts per user. Finally, one-quarter to one-third of patients visited A&E in each year. Use and cost of A&E was higher for the secondary care patients.

The mean total cost of the above services reveals that those in the discharged to primary care group had consistently lower costs than those in secondary care on the index date (Figure 1). The costs were lower in 2013 partly because this covered only 10 months. The mean (SD) costs for the combined follow-up period (2011–13) were £14,730 (£58,627) for the discharged group and £36,075 (£67,454) for the secondary care group.

FIGURE 1.

Mean costs per year by discharge group (2013/14 £s). PC, primary care; SC, secondary care.

The distribution of costs over time demonstrates that SLaM-provided secondary care accounted for most cost in each year (Figure 2). However, as a proportion it was (not surprisingly) somewhat greater among the secondary care patients.

FIGURE 2.

Distribution of service costs by discharge group. PC, primary care; SC, secondary care.

Group definition 2: groups defined by service use

More than 90% of the primary care group had received face-to-face GP contacts in the year prior to the index date (Table 8). This was substantially higher than for the secondary care group, and, by definition, none of the no care group had such contacts recorded for this period. For each of the 3 years following the index date, the proportion of the secondary care group with face-to-face GP contacts increased slightly, whereas the primary care group maintained similar levels of contact. What is striking, however, is that the patients in the no care group were increasingly likely to have GP contacts over the subsequent years. Telephone contacts with GPs were received by slightly more secondary care than primary care patients in 2010. The proportion of patients having such contacts increased in each group, particularly during the final year. For the no care group, telephone contacts did increase substantially but remained noticeably lower than in the other groups. Home visits by GPs were received by relatively few patients and proportions were always slightly higher for the secondary care group. Face-to-face contacts with primary care nurses were relatively common in the primary care and secondary care groups, with the rate being slightly higher in the former. By 2013 about half of each of these groups had such contacts. The no care group again saw increased use of this service, and by 2013 about one-third had these contacts. Telephone contacts with primary care nurses were far less frequently received by patients.

For those with primary care contacts, the numbers of these are reported in Table 9. It can be seen that the intensity of contacts did not differ markedly between the primary care and secondary care groups. Patients in the no care group who did have contacts in 2011–13 tended to have a lower rate of these than patients in the other two groups. Costs of care were again dominated by face-to-face GP consultations (Table 10). The costs of face-to-face GP contacts were very similar between the primary care and secondary care groups, and for each year both groups had higher costs than the no care group.

Around one-quarter of secondary care patients had psychiatric inpatient admissions to the local trust during the year prior to the index date (Table 11). This occurred for very few of the patients in the primary care or no care group (both of which by definition had zero use in the last 6 months of that year). Inpatient use fell slightly for the secondary care group in subsequent years, whereas the other two groups saw a slight increase. For those who were admitted, the number of days in hospital was always far higher in the secondary care group than in other two groups. Not surprisingly, the costs of inpatient care were by far the highest in the secondary care group.

Almost all secondary care patients had community contacts during the year prior to the index date. The proportion then fell over time to around one-quarter. The primary care group initially had a higher rate of community contacts than the no care group, but this difference disappeared over time. As with inpatient care, the cost of community contacts was much greater in the secondary care group than in the other two groups. Finally, the table shows that the use and cost of psychotropic medication was greatest in the secondary care group.

Use of inpatient care for physical health reasons was similar for the primary care and secondary care groups (Table 12). The no care group had a lower rate of use, except in the final year. For those who were admitted, the number of days in hospital was not consistently different between the groups. As in the earlier analyses, the use of psychiatric inpatient care in other trusts was uncommon. In each year excluding 2013, around one-third of primary care and secondary care patients had outpatient contacts, and the number and cost of these was similar. The no care group generally made less use of outpatient care. Finally, and similar to the earlier analyses, about one-quarter of primary care and one-third of secondary care patients visited A&E in each year (except 2013).

The mean total cost of the above services reveals that those in the primary care and neither primary nor secondary care groups have substantially lower costs than those in the secondary care group (Figure 3). The mean costs for the combined follow-up period (2011–13) were £7100 (SD £18,796) for the primary care group, £39,438 (SD £81,666) for the secondary care group, and £6870 (SD £13,655) for the neither primary nor secondary care group.

FIGURE 3.

Mean costs per year by service use group (2013/14 £s). IAPT, Improving Access to Psychological Therapies; PC, primary care; SC, secondary care.

The distribution of costs shows that secondary mental health care accounts for an increasing amount of cost in the groups defined on the index date as being in primary care or neither primary nor secondary care (Figure 4).

FIGURE 4.

Distribution of service costs by service use group. PC, primary care; SC, secondary care.

Analysis of primary care costs

Primary care costs were positively related to prior general medical outpatient costs, physical inpatient costs from other providers, A&E costs, age, use of antipsychotic medication, affective disorder presence, hypertension, asthma, depression and diabetes mellitus (Table 13). Lower costs were linked to prior psychiatric inpatient costs from SLaM, being male, being from a black and minority ethnic group, treatment non-compliance, having had blood pressure taken prior to the index date and having had a thyroid-stimulating test.

When groups were defined by service use, there was a negative impact of receiving neither primary nor secondary care on subsequent primary care costs (Table 14). Costs were positively associated with prior general medical outpatient costs, A&E costs, age, use of antipsychotic medication, physical health problems, presence of hypertension, affective disorders, asthma, depression and diabetes mellitus. Lower costs were associated with prior psychiatric inpatient costs, being male, being from a black and minority ethnic group, history of non-compliance with treatment, having had blood pressure taken and receipt of a thyroid-stimulating test.

Analysis of mental health care costs

Those discharged to primary care had mental health care costs that were 54% lower than for those remaining in secondary care (Table 15). Costs were positively associated with history of violence, physical health problems, treatment non-compliance, forensic care, antipsychotics, presence of schizophrenia and affective disorder hypertension. Costs were lower for those with higher previous primary care costs.

With the service use definition of groups, it was found that costs for those in secondary care were significantly higher than for the other two groups (Table 16). Costs of mental health care were positively associated with prior A&E costs, history of violence, treatment non-compliance, physical health problems, antipsychotic use, forensic services, schizophrenia, affective disorders and hypertension. Costs were negatively associated with prior psychiatric inpatient costs from other providers and the presence of diabetes mellitus.

Analysis of total care costs

Total health costs were significantly lower for those discharged to primary care (Table 17). Costs were positively associated with a history of violence, physical health problems, non-compliance, a forensic history, use of antipsychotics, presence of schizophrenia or affective disorders, and hypertension. Lower costs were associated with having had blood pressure taken.

Table 18 shows that patients receiving primary care or neither primary nor secondary care both had similarly lower costs than those patients in secondary care. As in previous models, we see a positive impact of violence, physical health problems, non-compliance, forensic care, a diagnosis of schizophrenia, and hypertension on costs. Costs are associated with age but the relationship is non-linear.

Propensity models with discharge to primary care as the treatment variable

In the first model the variables that were significantly associated with being discharged to primary care were age, being female, being from a black and minority ethnic group, not having a history of violence, not having physical health problems, not having used antipsychotics, not having schizophrenia or affective disorders and having hypertension (Table 19).

Propensity scores were grouped into bands and the numbers of people remaining in secondary care with different scores were examined. Of the 1397 people the distribution showed that only 5.8% had propensity scores of 0.7 or above. (In the absence of guidance on this, this was chosen as representing a reasonable likelihood of having characteristics that would indicate potential discharge to primary care.) If these were discharged to primary care then the cost differences for 2010 (computed as the difference in actual 2010 cost and the cost matched from a similar patient according to the propensity scores) would be very modest (the product of the numbers and the mean cost difference coming to £159,302). The ‘excess’ costs for primary care are also shown and these come to a total of £15,555. Therefore, if these patients in secondary care had been discharged to primary care, for 2010 their primary care and local secondary health-care costs would have been £174,857 less than they were on the basis of this model. Clearly, this latter qualification is crucial and a different set of predictors may have resulted in a different finding.

With location of care defined according to service use we see that even fewer (3.8%) secondary care patients have a propensity to be cared for in primary care over a threshold of 0.7 (Table 20). The total ‘excess’ care costs for these are just £80,642. Both of these models suggest that relatively few patients in secondary care have characteristics that strongly predict discharge to or use of predominantly primary care. This may suggest that there is a high level of appropriateness in terms of those receiving primary care.

Predictors of time to readmission to secondary care

There were 672 (48%) patients who had been discharged from secondary care who were readmitted during the follow-up period. The mean time to readmission was 385 days (SD 293 days). It is shown that re-entering secondary care is positively associated with a history of violence, treatment non-compliance, use of antipsychotics, physical health problems and a diagnosis of schizophrenia or affective disorders (Table 21). Secondary care was less likely for patients who had had blood pressure tests in primary care. The assumption of proportional hazards was met.

Predictors of time to reuse of secondary care

The number of primary care patients who reused secondary care was 408 (31%). The mean (SD) duration before reuse was 412 (291) days. Reuse of secondary was positively associated with history of violence, treatment non-compliance, physical health problems, diagnosis of schizophrenia or affective disorders and presence of depression (Table 22). Use was less likely for those who had received a thyroid-stimulating hormone test. Again, this model met the assumption of proportional hazards.

Group definitions and characteristics

There was not a straightforward indicator of location of care and clearly this would in any case be ‘fluid’ for many patients. We adopted two pragmatic definitions. First, we identified patients who had been discharged from a package of care prior to 1 January 2011 and were still outside the secondary care system on that date. Second, we defined care according to secondary care services used in the previous 6 months and primary care received in the previous 12 months. Both of these have their limitations, but primary care received in the previous 12 months does seem more valid, as discharge from secondary care does not indicate receipt of primary care. A clear dichotomy between primary and secondary care, even if apparent from service use data, may not be the reality and may not be appropriate. It may be that secondary care professionals are still aware of the care needs of those seemingly managed in primary care, and we have already seen that those managed in secondary care will usually receive primary care input as well. In addition, there may be mismanagement, whereby management location is uncertain and tasks are duplicated. These intricacies of management were not picked up in these analyses.

There were clear differences between the groups. Those in secondary care were more likely to have a diagnosis of schizophrenia than those in primary care, and the latter were more likely not to have received a formal diagnosis. It should be emphasised that the focus of the quantitative analyses was on patients with a SMI QOF record. This does not in itself mean that these patients will have such a diagnosis confirmed by secondary care services. There were few differences in demographic characteristics such as age, sex or ethnicity.

Data on severity of mental health problems were recorded by SLaM with the HoNOS. These scores were missing for many patients, but there were clear indications that severity was greater in the secondary care patients. History of violence, forensic care, physical health problems, use of antipsychotic medication and treatment non-compliance (all according to secondary care staff) were more prevalent among the secondary care group. We did not impute for missing values and this is a limitation.

Another limitation of the analyses was that we did not have access to mortality data. In fact, the sample consisted of these included in primary care recorded on 31 October 2013 and who had a diagnosis of SMI prior to 1 January 2011. Patients who had died after the latter date were not included. It is possible that death rates are different for those managed in the two care settings but this would not be picked up in our analyses. Similarly, the sample consisted of those who had not changed GP practice during the period of interest. As such, issues of mobility cannot readily be addressed. We did not have information on social mobility, and it may be that social isolation is an important factor in influencing service uptake.

Use and costs of services

Primary care use and costs did not differ substantially between the primary care and secondary care groups. Those not in receipt of primary care in 2010 did use progressively more in subsequent years. Mental health care costs were, however, substantially different. This was by definition in 2010, but carried on into future years. In Chapter 3 we focus on specialist services set up to act as alternatives to secondary care and to help facilitate the transfer between secondary and primary care. However, we do not have information from the linked data on the numbers using the specialist services.

Predictors of service costs

The models presented in this chapter attempt to identify the impact of location of care after controlling for background demographic, clinical and service variables. Primary care costs over the follow-up period were associated with being discharged to primary care, but only when previous primary care costs were included in the model along with the HoNOS scores. This indicates that discharge to primary care is associated with a greater change in costs rather than with costs in absolute terms and this seems logical. Higher primary care costs were also associated with the presence of disorders such as asthma and diabetes mellitus and, again, this is not unexpected. When the service use defined location variable was used, it was shown that those managed in primary care had higher primary care costs than those managed in secondary care but again this was only when previous primary care costs were included in the model. Men had lower primary care costs than women and this supports previous work showing that women are more likely to consult GPs then men.

Secondary mental health care costs were consistently associated with a previous history of violence, forensic care, physical health problems and treatment ‘non-compliance’. Being discharged to primary care was consistently associated with future mental health care costs that were about half of those for patients remaining in secondary care. Although this may not be surprising, it is of interest that this is after controlling for a wide range of background clinical and demographic characteristics. The models in which location was defined according to service use were very similar, with those in primary care at the index date having subsequently lower costs than those in secondary care. The models examining total costs are very similar to the mental health care cost models, which is to be expected.

Excess cost of care for patients who could be discharged

The use of propensity score methods enabled us to identify patients in the secondary care groups who were similar to those in primary care. This was done by focusing on those with propensity scores of 0.7 or higher. The expectation was that a large number of such patients would be identified and that as a result they would be in receipt of ‘unnecessary’ services with high costs. As it turned out this did not occur, with relatively few patients in this category. Of course, these findings are only as good as the models produced, but the indication is that substantial excess costs as a result of unnecessary secondary care are not apparent. These models focused on the costs in the year preceding the index date. Future analyses could use these models to predict future costs.

Time to reuse of secondary care

Finally, we produced models to identify characteristics that could explain the time it takes for those in primary care to re-enter the secondary care system. These were similar to the models used to predict costs. Key predictors were records of violence, forensic care, physical health problems and treatment non-compliance.

Recruitment of service user participants

The managers of the PASS, COT and the PS1 service were informed of the nature and requirements of the study, were given the opportunity to ask questions and were each initially asked to examine their service caseloads for 11 service users who met the following criteria:

1. having a history of severe mental health problems (primary diagnoses including schizophrenia, schizoaffective disorder, bipolar affective disorder and depression with psychotic features, excluding drug-induced psychosis)

2. having been discharged from secondary care within the previous 6 months

3. having the mental capacity to give consent to participation.

However, because the profile of people using the services had changed before data collection started, the first two criteria had to be changed. Although the services (particularly PASS and COT) were originally established to support people with SMI who had been discharged from secondary to primary care, many service users had been referred who had never been in the care of secondary services and who had a wider range of diagnoses. Under the revised inclusion criteria, although service users meeting the original first two criteria were prioritised, service users who had no previous involvement with secondary care, or who were diagnosed with a broader range of mental health problems, were also included in the sampling frame.

Service users who met the revised inclusion criteria were contacted by their caseworkers and informed about the study and asked if they would like to participate. If the service user was interested in participating they were asked for permission to forward their name and contact details to the research team. The senior researcher then made contact, informed the service user about the aims of the study and what participation involved, answered any preliminary questions and arranged a convenient date, time and venue for the interview if they agreed to participate. Before commencing the interview, each participant was provided with details of the study and was given the opportunity to ask any questions. Once satisfied, participants were asked to give their informed consent to participate (which also included their consent for the interview to be audio-recorded).

Only one PS1 service user was recruited before it was discovered that the service was no longer receiving funding and had ended. Because there was now no way to recruit former users of this service, it was decided to approach the managers of PS2 and PS3, who were also connected to the Hub, to ask if they would participate instead, and they both consented. These services are run by a charity that provides London-wide support for people with mental health problems, learning disabilities and autistic spectrum disorders and their carers. PS2 is a peer support service offering telephone and face-to-face support to Lambeth residents undergoing a mental health emergency, whereas PS3 is a peer-led service, which puts service users in contact with groups in the wider local community who provided activities that the service users were interested in.

In total, 46 service users were identified by the managers of PASS, COT, PS1, PS2 and PS3. However, 13 of these were unable to participate because of work, college or family commitments, or had moved away or were otherwise unable to be contacted despite repeated efforts, or who, while having been referred to a service, had not had any effective contact with it. Thirty-three participants were recruited, giving a response rate of 72%. The sample was formed into three groups as follows:

• 11 service users from PASS

• 11 service users from COT

• an amalgamated peer support services group, comprising one service user recruited from PS1, five from PS3 and four from PS2; one of the PS3 service users also had concurrent experience of using PS2.

Recruitment of focus group participants

After giving their informed consent, the participants were also given the opportunity to take part in a focus group to discuss their experiences of receiving help from their service. Ten of the participants were required: three each from PASS and COT, and two each from PS2 and PS3. The original intention was to use the focus group to inform the development of the economic model described in Chapter 4. However, this did not seem appropriate and so the focus group was used instead to supplement the information obtained from the interviews.

Recruitment of staff participants

When the managers of the participating Hub services were informed of the nature and purpose of the study and what it involved for the service user participants, they were also asked for their permission to include their staff in the study. When this was given they were asked to inform their staff about the study and to ask for three volunteers who would like to participate. Three staff members from PASS, COT and PS2 volunteered, as did three former PS1 workers. Because PS3 aimed to place their clients in mainstream activities with minimal involvement on their part, their staff were not included in the staff group. Each of the staff members who was interested in participating met the senior researcher who informed them in more detail about the reasons for the study and what participation entailed, and answered any questions before obtaining informed consent for interview and for the interview to be audio-recorded.

Ethical approval

Following discussion with the National Research Ethics Service and the SLaM/Institute of Psychiatry Research & Development Governance Manager, we were advised that this study was a service evaluation which did not require ethical approval. However, as described above, information about the study was provided to participants and their written consent was obtained.

Service user qualitative interviews

Once informed consent had been given, the service user participants were interviewed by the senior researcher or a service user interviewer, using an interview topic guide (see Appendices 2–4), which was based on that used in an earlier, unpublished small-scale pilot evaluation of the Hub’s services. This was further refined in consultation with the trained service user research interviewers who assisted with the earlier study and explored participants’ experience of referral to the service, what their expectations were, the nature of help received, their satisfaction with the support that they received and comparisons between the help that they received from PASS, COT, PS2 or PS3 and that received from secondary services or their GP. After the interview all service user participants were given a £15 gift voucher as a token of appreciation for their participation. Participants were approached for a second interview (mean time between interviews = 195 days; range 127–254 days) to explore any help that they had received after the first interview and whether or not their views on the service had changed in the intervening period.

Service user focus group

When the focus group participants had been recruited, they were asked to attend the focus group in March 2015. Besides discussing their experiences of their respective Hub services, they were also asked to complete the Client Satisfaction Questionnaire (CSQ),24 and the European Quality of Life-5 Dimensions 5-level (EQ-5D-5L) questionnaire. 25 The CSQ is an eight-item scale for the evaluation of service user satisfaction with health and social care services that they receive, whereas the EQ-5D-5L involves rating the severity of problems with mobility, self-care, usual work, study, housework, family or leisure activities, pain/discomfort and anxiety/depression, as well as an overall rating of health quality. Each participant completed the CSQ with respect to the service received from the Hub and to the help received from other services, and the EQ-5D-5L was completed in respect of participants’ health before referral to the Hub, and to how they were feeling on the day of the focus group.

Staff qualitative interviews

After obtaining their informed consent, the staff participants were also interviewed by the senior researcher. The interview was directed by a topic guide, which explored areas including comparing the current post with previous work in mental health and social care, improvements that could be made of the service, the process of work in the current role, the peer supporter role and relationships between peer support services and mental health and social care professionals. After the interview all staff participants were given a £15 gift voucher as a token of appreciation for their participation. Participants were approached for a second interview (mean time between interviews = 215 days; range 168–260 days) to explore if any changes in the service or their experiences after the first interview had led to any changes in their feelings about working for the service.

Analysis

The interview tapes were professionally transcribed verbatim. Thematic analysis was conducted using an inductive (data-driven rather than theoretical) approach and followed the stages described by Braun and Clarke. 26 These stages are: familiarisation with all the data, generating codes, identifying themes, reviewing themes, defining and naming themes and finally writing up themes into a report. The senior researcher (SW) was assisted by the research interview-trained service user interviewers in identifying, reviewing and defining and naming the themes.

Time 1: sociodemographic characteristics of service user participants

The sociodemographic characteristics of the sample are summarised by service in Table 23.

Clinical characteristics and service use

The clinical characteristics of the sample are summarised by service in Table 24 below, and the service use characteristics of the participants are summarised in Table 25.

Details given by participants concerning length of time in contact with service, typical frequency of contact with service and typical duration of contacts were often quite imprecise, such as ‘4 to 6 weeks’, ‘every 2 to 3 weeks’ or ‘15 to 45 minutes’, respectively. When this occurred, the mid-point of the range given was used to calculate the mean.

Characteristics of the focus group sample

On the day, one of the PASS participants did not attend because of work commitments. The social and demographic characteristics of the nine participants who attended the focus group are summarised in Table 26.

Characteristics of the staff sample

Of the 12 staff participants, four were male, four described their ethnic background as white British or white other, and the average age was 42.17 years (range 26–59 years). All but two of the PS1 workers had previous experience in mental health care (in either paid or voluntary work), with the average length of this occupational experience being just over 9 years. Average length of time in their current post was 21.17 months. This information is summarised for each service in Table 27.

Five of the six workers in the peer support services had previously received treatment for mental disorders [one for depression and comorbid substance misuse, one for bipolar affective disorder, one for hypomania and borderline personality disorder, one for anxiety and depression, and one for anorexia, depression, obsessive–compulsive disorder and personality disorder]. Mean length of time spent receiving mental health services was a little over 13 years and 6 months (range 8–20 years). Three of these workers had received inpatient psychiatric treatment at least once (one having been formally admitted), with the mean length of time since last admission being 13 years (range 10–18 years). The peer support worker who had not received treatment for mental health problems was caring for a relative suffering from schizophrenia.

Peer support participants only

• Differences between peer support and other services:

  • mutual experience makes for a different relationship

  • peer support more proactive

  • other services give less time-intensive support.

• Do you prefer peer support or help from mental health or social care professionals?

  • peer support and mental health/social care professionals have different roles and both are necessary

  • mutual experience makes peer support helpful and effective.

• Was anything particularly helpful about peer support?

  • mutuality/shared experience

  • peer supporter as role model.

• Any problems with peer support?

  • sometimes felt pressured into accepting visits from PS2. Perhaps because they were paid to do these?

All participants

• Initial expectations of support:

  • interventions addressing mental health symptoms

  • help with housing and/or benefits

  • help with addressing social isolation/establishing a daily routine and activities

  • general support.

• Support received:

  • signposting/facilitating help from other agencies

  • addressing social isolation/establishing a daily routine and activities

  • interventions to treat mental health symptoms

  • help dealing with bureaucracy

  • emotional support.

• Practical help

  • help around education/employment.

• Did help received match expectations?

  • help received did not address needs

  • suggested activities were not appropriate to needs or accessible

  • planned events were poorly attended

  • no interesting activities were suggested

  • expected more support: proper diagnosis/change of medication

  • some listeners not sharing about their experiences, thereby undercutting mutuality of peer support.

• Problems with help received?

  • some problems could not be helped through peer support

  • some suggestions were unhelpful or impracticable

  • planned events usually fell through.

• Could help have been done differently or better?

  • team did not recognise service user’s ability to manage their condition

  • team were slow to refer on to appropriate help

  • service was disorganised: too little structure

  • volunteer helpers needed more support

  • service was focused on unemployed service users, which restricted activities available to them

  • the service’s website did not work.

• Has the service helped, and how?

  • by changing medication

  • therapeutic interventions to relieve mental health symptoms

  • helping to address social isolation

  • help with employment

  • signposting to other sources of help.

• Who has made the best job of supporting you, the Hub service or other mental health services/GP?

  • Hub service, through helping to complete benefit claim and following it up

  • both have been useful in different ways

  • Hub service was more diligent and reliable.

• Most important thing helped with?

  • medication

  • psychological help: talking through problems, offloading, coping strategies

  • benefits and bills

  • daily structure and activity

  • social isolation

  • practical skills

  • signposting to other services.

• Anything about how you were helped that was appreciated?

  • empathy/feeling listened to

  • support/empowerment

  • egalitarian nature of relationship (PS2)

  • being helped with practical problems

  • being helped to be less isolated

  • validation of incremental progress

  • help in dealing with bureaucracy.

• Anything unhelpful?

  • insufficient time for intervention

  • contact too frequent

  • behaviour of volunteer staff at team base gave an unprofessional impression

  • when the listener stepped out of the peer role (PS2).

• Anything that you would like to change, or that could be done better?

  • longer sessions, and more sessions

  • direct access to a psychiatrist

  • service should communicate with service users better

  • access to a wider range of activities

  • less advice and more sharing from peer supporter.

• Anything that the service did not offer, but should?

  • continuing, occasional contact after discharge

  • in-house counsellor

  • out-of-hours service

  • transport for community visits.

• Is there anything that other services could learn from the Hub?

  • be more egalitarian and open, non-judgemental, non-pathologising and less clinical/role bound

  • listen and offer time

  • respond more quickly to crises

  • be aware of the Hub’s services.

• Why would you recommend the Hub to others in a similar situation (or not)?

  • because services users felt cared for and supported

  • because they addressed social isolation

  • because they gave a professional, depathologised, non-pressurised service

  • because they offered time and privacy

  • because of practical help

  • because of signposting

  • because of flexible, pro-active, individualised service

  • would recommend only to people not in full-time work (PS3)

  • would not recommend: could not see how this service could help (PS3)

  • mixed experience.

Most of the participants expected help to combat social isolation and lack of daily activities and structure, and most received help for these problems. Help was also given beyond what was expected. For example, more participants reported being generally listened to and supported than were expecting it. Several expected and received clinical input much as they had received from secondary services, as did others who did not expect such help. Similarly, most participants who expected help with housing and benefits received other forms of help as well.

But it was nice to actually have someone like [PASS worker] listen to me, because when people come up to you and they say, ‘How are you?’ As soon as you start telling them, they blank out; they don’t want to know [. . .] What a waste, why ask, if you’re not going want to know? And it was nice to sit here [. . .] and speak to [PASS worker] about the problem that I’ve got.

Participant 32, PASS service user

Other forms of support included signposting towards or facilitating help from other agencies, help in dealing with officialdom and bureaucracy, practical help with arranging household repairs and furnishings, and in the case of one of the peer support service users, giving her small amounts of money and taking her out for meals.

Everywhere I go to seek help – nothing. So [PS2 worker] said, ‘OK [. . .] I don’t know nothing [sic] about immigration but I’m going to try, bring your documents to me’, and he read the letters and he put a letter together and he sent it to Mrs May [Home Secretary] and one to [her] MP [. . .] No other supporting group in Lambeth have ever written to the Home Office on my behalf – but [PS2] did.

Participant 18, PS2 service user

The service users particularly appreciated feeling supported and cared for, feeling listened to and being allowed to proceed at their own pace. Concrete help with practical matters and in dealing with officialdom and bureaucracy were also appreciated, as was feeling empowered by their support worker to regain control of their lives.

I go to talk to a professional person about problems, not being judged but [. . .] it’s just that aspect of talking and being listened and also practical, writing letters and as well in a way financial, when I can probably get help to pay for the college.

Participant 6, COT service user

Support from PASS and COT was typically described as recovery-focused, depathologised and personalised. Service users felt listened to and felt that goals and action plans were negotiated, with no time pressure in a friendly and supportive atmosphere, which contrasted sharply with the support offered by GPs and secondary services.

They’ve obviously got a lot more time for you than the GP has, I mean understandably they’ve got like 10-minute slots or whatever, they can’t listen to you yammering on for an hour about all your various problems. It’s more interactive – clearly, because they do things with you and come with you to places and kind of support you in finding stuff.

Participant 2, COT service user

I’ve seen one somebody that was like [PASS worker] one time, it’s a couple of years ago [. . .] and that man was [. . .] real good. [. . .] But [apart] from that, I haven’t seen nobody [sic] else who really sit down and listen to my case like [PASS worker], really take a little time to really listen to me and understand.

Participant 30, PASS service user

[My COT worker] was chasing it up and this is why I’m saying I think [she] has done more than what [my CPN (community psychiatric nurse)] has done.

Participant 9, COT service user

The help from here is very practical and the amount of time that you’re gonna need is long enough to talk about your problems and to find a solution, it’s like 20 minutes, so it’s very basic stuff, saying how you feel and you get the treatment.

Participant 6, COT service user

There was little focus on the whole sort of psychiatry side. I mean obviously, you know, some things had to be monitored, you know. Any sort of thoughts of self-harm, medication, etc., and I fully understand that but beyond that [. . .] the whole mental health stuff was very much de-emphasised. It was all about, you know, what’s being called recovery. [. . .] Getting on with things and [. . .] that was very satisfying because there’s always a concern, rightly or wrongly [. . .] about getting too pulled into a mental health system.

Participant 21, PASS service user

The nature of the service provided – how the care was given, rather than what was actually done – and feeling supported were the most frequently cited reasons for recommending their service to others, although concrete help in addressing social isolation and practical problems and signposting to other services were also reported.

I would say, no matter how anxious or nervous, or depressed you are; they don’t bite. Once you get past that initial fear of walking in there, they’re very friendly, they’ll put you are your ease. They make suggestions, they don’t give you orders, they make offers of things that you might like to try and if you say you don’t want it, they don’t get all huffy about it or anything, they just look for something else.

Participant 2, COT service user

He built my hope up; he did give me faith. He followed me to appointments I had where sometimes I’d go on certain appointments. [. . .] He followed me to appointments at International House [International House London, London, UK] [. . .] for my homelessness and he went somewhere else with me and he made quite a few calls for me. He recommended me to [. . .] the man who helped me chase up the solicitor to get my back my passport and papers.

Participant 30, PASS service user

Among those who used the peer support services, mutuality and shared experience was frequently reported. This made the helping relationship less formal and more comfortable, and made it easier to discuss problems.

With peer support, you’re sort of meeting about and [. . .] it’s not formal or [. . .] like doctor/patient relationship or [. . .] nurse and patient relationship. [. . .] You’re more sort of likely to reveal [. . .] much more to someone who is in the same boat as yourself.

Participant 1, PS1 service user

These service users saw peer supporters’ experience of having had mental health problems themselves as giving them credibility, which gave their support and encouragement more weight.

Well because these people are people who seem to have gone through some of the things that I have, you know. [. . .] Even though they want to play life coach but they seem more human.

Participant 29, PS2 service user

The example that peer supporters show in their helping role, having experienced mental health problems themselves, also provides a valuable model of recovery. The need for support from mental health professionals was recognised, and the clinical and professional support provided by them was appreciated (again, particularly with regard to support in their various dealings with officialdom and bureaucracy, although one peer support service user also reported considerable help from her service in assisting with visa problems).

Two particularly valued aspects of the support offered by PS3 were its involvement with inpatient ward teams (whereby workers would contact patients who were nearing discharge), and supporting their service users soon after discharge in attending activities that they had stated an interest in.

I did appreciate them helping me, coming in the beginning and telling me all about it [. . .] I was in hospital then and I didn’t know.

Participant 28, PS3 service user

I really appreciate the socials with [PS3] [. . .] going out and [. . .] doing things. [. . .] It kind of gave me that boost that I can do this one all right, then [. . .] it kind of helps me to build up to then make my own plans and [. . .] call a friend and say [. . .] ‘Do you want to go here or [. . .] go watch a movie or something [. . .]. It [. . .] just helps me more be a bit involved, active I suppose. I appreciated that [. . .] It’s helped me get more out of going when I first came out [of hospital] and that I think is one of the things that can be [. . .] a bit worrying when you first come out. You know, like going out on your own and going places and stuff. [. . .] It can feel a bit daunting at first and stuff when you’ve been in hospital for the last 4 months.

Participant 28, PS3 service user

Many participants who used PASS and COT said that the 12-week intervention period was insufficient and that longer-term follow-up was needed (perhaps at a lower level of involvement). Other service improvements suggested by these participants included improved access to psychiatrists, ongoing support groups for service users and an out-of-hours service. The premises themselves were also seen as problematic, being hard to find and inadequately signposted, and the security system being rather forbidding (especially for service users who had experience of formal admission). The interview rooms in reception were also seen as not providing adequate privacy and one user remarked that horseplay among the volunteer reception staff gave an unprofessional impression. Another service user complained that letters from the service that contained several typographical errors also conveyed a lack of professionalism. Furthermore, she felt that the major need that she presented with (her husband’s need for psychiatric assessment and treatment in order to relieve pressure on her) was not promptly attended to, she felt that her ability to manage her condition was not recognised or supported by the team, and that she had received a totally inappropriate referral to a time-banking service. Some service users reported frustration that agencies to which they had been referred by their service had not yet seen them.

One peer support service user commented on an unhelpful tendency to give advice rather than provide a supportive listening role by some listeners, which violated the lack of distinction between the helper and the help that the service user particularly valued.

I mean, when you talk to them they give you advice and bulldoze you, yeah, but a bit of a balance, you know, would be nice [. . .] to hear about them [. . .] Rather than just stand back listening to what I say [. . .] To be more interactive peer support I think.

Participant 29, PS2 service user

A problem with activities that were organised by PS3 was that they were often poorly attended or fell through because of non-attendance. Service users also reported problems such as a lack of interesting activities proposed by PS3, suggested activities that were not appropriate to the level of involvement that the service user wanted (e.g. recommending a highly competitive badminton club to a service user who just wanted fun, recreational games) or activities that were inaccessible to the service user (a ceramics class whose costs for materials was far beyond the service user’s means). Their activities were also predominantly targeted at people who were not in work, and, although such people are in obvious need of daytime activities in order to add structure to their lives, service users who were in full-time work but who still had problems with social isolation were not catered for. Another problem arose when the manager left the service and, in the break before the new manager was appointed, few new service users were followed up. The first version of the service’s website also did not function well.

Time 2 service user findings

Service user participants were re-interviewed in order to assess whether or not there had been any changes to their mental and physical health since time 1, whether or not they had received support from their service for any other problems since time 1 and if they had anything to add to what they reported at time 1 or if their views about the support that they had received had changed in the intervening period.

In total, 27 of the original 33 participants were re-interviewed at time 2, giving a response rate of 82%. Three PASS service users were not seen; two of these declined interview and had experienced a worsening of their mental health in the intervening period, and the third was of no fixed abode at time 1 and could not be located at time 2. Two COT service users were not interviewed. One had re-entered full-time employment and work commitments prevented him from being re-interviewed, whereas the other did not respond to any attempt to contact him. One PS3 service user felt unable to be interviewed, having just been discharged from a mental health admission.

The average time between time 1 and time 2 interviews was 210 days (range 183–254 days) and the average duration of the time 2 interviews was 25 minutes and 6 seconds (range 6 minutes and 58 seconds to 1 hour, 21 minutes and 20 seconds).

At time 2, one of the PASS participants had his care transferred from secondary services to primary care and the remainder of the participants were still being cared for by the same service as at time 1. All but two of the PASS participants and all but two of the COT participants were no longer in contact with their respective services (one PASS participant had been transferred to COT and one COT participant made occasional telephone contact with the service). Two of the participants were still in regular contact with the service (in fact, one had been sought out by PS2 when they had heard that she had suffered a bereavement), and none of the PS3 participants was still in contact with the service. Three PASS participants, four COT participants and four PS3 participants had received input from at least one other non-statutory service.

Improvement and worsening of mental and physical health and other problems identified at time 1 are summarised in Table 28.

One PASS participant reported being formally admitted since time 1, but no COT, PS2 or PS3 participants reported any mental health admissions. One service user from PASS and one from COT reported experiencing a mental health crisis. Neither sought help with this, in the case of the PASS service user because she does not trust services to help her, and in the case of the COT service user because he felt that he had been neglected in the past when seeking help from his GP. Three of the PS2 service users reported experiencing mental health crises. Two sought help from their GPs, whereas the other sought no help because she did not feel that this had helped in the past, and she felt that the quality of listeners on the crisis telephone line was variable, so she preferred not to ring. Three of the PS3 service users also reported experiencing mental health crises, all of whom sought help from others (one from his GP, one from Gamblers Anonymous, and one from his father or a friend, as he felt let down by professionals in the past.

I think professionals, to be honest, have let me down. In the past, the track record is that they have let me down. They’ve let me down, so there hasn’t been the interest of phoning up, ‘Hi there, what can I do for you?’

Participant 26, PS3 service user

Overall, the service users tended not to have changed their feelings and opinions about the support that they had received from the services. If they were satisfied with the support they had received at time 1, they remained so at time 2, and vice-versa. An exception was a PASS service user who at time 1 had felt uncomfortable about coming to the team’s location because he felt intimidated by the security measures and had felt uneasy with some of the volunteer reception staff. With hindsight, he attributed his discomfort at the security measures and the reception staff’s behaviour to the hypervigilence and negative thinking that were characteristics of the anxiety from which he suffered.

I said to my girlfriend, when we first saw the psychologist practitioner, I thought they were gonna lock me up and then when I went here and I saw the bars and all, I thought, ‘This is a hospital innit’ and I thought,’Ooh no, I’m not gonna get out of here’. So I was thinking, ‘I better be careful what I say.’ [. . .] I was paranoid and everything. But now, with hindsight, I realise that it’s not like that. It’s more like to stop people breaking in and smashing the place up. So now, with hindsight, I think it’s a good place to come [. . .] I think it was me being negative and just being a bit, you know when you’re not feeling great but I think you try and, you see every little nitpicky detail in other people I think [. . .] When you’re positive, you look at it and think ‘I can’t be bothered’, but when you’re down, I think you notice every stupid little thing you know.

Participant 22, PASS service user

However, although service users’ views of the support that they had received had not changed between the interviews, there were several comments made that reinforced or expanded on what had been said at time 1. For example, one PS2 user expanded on the dilemma concerning the nature of peer support that she raised at time 1:

If they don’t say anything about their story, they’re acting just like professionals. [. . .] My interest in using a peer support anyway is to meet peers, in plural right. Then when it’s run so much like I don’t know Social Service, or like the professional services the Monday to Friday ones, then it’s, I am disappointed with that anyway. [. . .] I just feel like people’s needs should be addressed more you know. [. . .] I mean I would like to meet people who are dealing with mental health problems and where people share what they’ve been going through, so I didn’t really get any of this. Any of it, I feel I’m having to ask them and they’re not that comfortable with it so, then [. . .] that’s not very peer is it?

Participant 29, PS2 service user

In contrast, another service user really valued PS2’s support and was able to speak with the same listener on more than one occasion.

I felt like somebody cared for me and somebody was listening to me. Somebody heard what I was saying and offered some continuity.

Participant 19, PS2 service user

As at time 1, the very loose structure of PS3 came in for criticism at time 2.

I mean you know, the few things which I did try and arrange or do whatever, it all fell through, every single thing. So it was pointless.

Participant 15, PS3 service user

However, the value of a service like PS3 was clearly suggested by a service user who was not involved with PS3:

I mean in the sense of inclusion, or community? How do you build that? Also, it’s I think it’s quite good to have instead of things being so spare, sort of one-on-one, I think I should be, I guess at the COT day centre is, I don’t know, but if people could meet you know, there could be more events held anyway where people could meet each other.

Participant 29, PS2 service user

A suggested improvement was that PS3 should adopt an approach based on a more structured assessment of a service user’s needs and interests.

I think they need to have a real understanding of you and an evaluation of you and your situation before they meet you, so that the time isn’t wasted because they’re watching the clock and they’ve said, ‘We have 4 months to see you’, or ‘We have three, four visits to see you and then you’re done with us’. So they need to do a proper evaluation before they meet you so that the time they can work with you really counts. I get the impression that it’s all very hunky-dory but then nothing materialises out of that. That’s really sad for those like myself who, in mental health, they really need that support.

Participant 27, PS3 service user

Another service user offered mitigation for the service’s shortcomings, pointed to the service’s potential and suggested changes:

I think in conclusion you can say that they were very much in flux at the time when I was there and they were attempting to set up a couple of initiatives and so the one that I was most involved with and as far as I know it’s come to nothing. [. . .] The good intentions were certainly there but I think they were still trying to feel their way into an established service. [. . .] There were definitely sort of like employment initiatives and it had the potential of going somewhere, it was still very much in its infancy and I didn’t get the sense that certain tutors were involved in the actual running of things. I mean, that was basically the model that they wanted to go for but I didn’t get that there were enough people taking initiative.

Participant 26, PS3 service user

Community options team service users were particularly impressed with the relaxed, unhurried but attentive atmosphere of the team and with the diligence of their COT workers.

I still think they are great. They listen. The service is not something where you go and you just have 15 minutes, say your thing and you have to go or, because your time finished. You really feel like these people care about you.

Participant 6, COT service user

She did all the ringing about – everything, the appointments and all that and she just, as best I can remember, she was involved all the way through to make sure everything was working OK, you know kept in contact by phone, ringing up to make sure I was OK and everything. [. . .] She didn’t have to do any of these things for me. She didn’t have to; she could have just made a phone call, but she didn’t, she kept in touch with me. She went over and above what she had to. [. . .] She didn’t just treat me as just another client and ‘OK, you’re time’s up . . . right next one come in . . .’. She’s just so caring.

Participant 12, COT service user

I’d recommend them any day of the week and I if I was talking to you as a mental health patient myself, I would be saying to you, ‘Look, don’t be frightened about it, they’ll take good care of you, they won’t give you a hard time, they’ll listen to you’, which would be a big plus because you don’t feel like anybody listens to you when you’ve hit bottom. No, they were fantastic.

Participant 2, COT service user

Well, the atmosphere, you know, it’s less clinical here, whereas in the doctors it’s very clinical. You know, and then you always have that time thing of, you know, are they waiting for the next patient or . . . Here you might have that but not in a, it’s not apparent.

Participant 4, COT service user

However, one COT user reported that his worker did not know about eligibility for tax credits, which meant that he did not receive his full entitlement.

Hindsight also ameliorated the view of one PASS user who had been critical of the amount of time it took for the treatment to which his PASS worker had referred him to materialise.

Like she said to me you need CBT [cognitive–behavioural therapy] and then I’d say to her, ‘Well when’s it gonna start?’ and she’d go, ‘Right I’ll fire off some e-mails, I’ll make some calls.’ And I come the next couple of weeks and it would still be, ‘We’re waiting for an appointment’. But that’s not her fault, it’s just the system is swamped.

Unknown participant, PASS service user

As with COT, simply taking the time to listen to the service user was greatly appreciated.

It’s nice to sit in front of someone and they say, ‘How are you?’ and you tell them, and they’re listening. That makes me feel good inside.

Participant 32, PASS service user

For another PASS service user (who happened to be a mental health professional), the recovery-focused support that he received felt appropriate for his conception of his difficulties.

I would recommend it. [. . .] I think it’s particularly suitable for people who may probably not have had a significant history of mental illness, but have found themselves in difficult times and have had a reactive, depressive, or anxious disorder and I think that the level that the team is set, enables people who’ve had a reactive crisis to move on more quickly and I know from my own experience, the reluctance of people who don’t see themselves as having had a mental health history, being pulled into a system which involves diagnosis etc. [. . .] I’ve worked with that myself and so for myself, I think I’d say the same, it’s that I don’t have a significant mental health history and so the level it was set at, was far more comfortable to deal with.

Participant 21, PASS service user

For another PASS service user, being encouraged to do things for herself built her confidence, and help with form-filling and bureaucracy was particularly appreciated.

The PASS team kind of started me off of doing things by myself, you know, it’s given me that bit of confidence out there. Any other thing else, I would say if you’re really down, extremely down like you, you don’t know how to do your benefits and paperwork and things like that, that’s what I say, I would definitely recommend them because they was there supporting me in that field.

Participant 13, PASS service user

Focus group findings

Service satisfaction and health quality

The highest possible score on the CSQ is 32. The average satisfaction score for the service received from the Hub service was 18.7 (range 10–32), compared with 18.4 for other services (range 12–25).

The highest possible total score on the sum of the EQ-5D-5L subscales is 25, and the overall rating of health quality is out of 100. Average total subscale score and average overall rating of health quality before referral were 15.9 (range 9–23) and 36.3 (range 0–90), respectively, compared with 17.6 (range 9–23) and 53.8 (range 0–80) on the day of the focus group. (We did not use the utility weights for the EQ-5D-5L given that this was not primarily for use here in an economic evaluation.)

Time 1 staff findings

Mean duration of the time 1 staff interviews was 35 minutes and 15 seconds (range 20 minutes and 28 seconds to 48 minutes and 47 seconds). The PASS and COT staff participants were asked how their current role and their previous post differed (none of the peer support workers had previous paid or unpaid work experience in mental health).

The transcripts of the time 1 staff interviews were also reviewed by the senior researcher, who coded the participants’ responses to the questions. Once again, these codes were reviewed by the senior researcher and the service user interviewers to sort the codes into themes. The themes derived from the codes are presented and the questions to which they relate are summarised below.

• What was it about the post that appealed to you? What were your expectations?

  • change of role: less stress and frustration, more fulfilment and satisfaction, novelty

  • a chance to use previous experience

  • the role was interesting, better

  • a chance to earn an income, get work experience and opportunities (all peer supporters).

• Are you pleased with how it has turned out?

  • has given opportunities (for peer supporters)

  • role not as depressing/serious/intimidating as expected (peer support)

  • referral patterns have changed – not so much about facilitating transfer from secondary to primary care (PASS workers)

  • financial benefits not as good as expected (peer support)

  • turned out as expected, but wondering how to maintain enthusiasm

  • not sure if role helped recovery as expected.

• What challenges does the role present?

  • 12-week intervention period not sufficiently long (PASS and COT)

  • hard to talk to people contemplating suicide (peer support)

  • hard to talk to people who are not getting the help that they need from mental health services (peer support)

  • hard to help people who cannot communicate what their needs are (peer support).

• Have you ever worked with people that you felt unable to help? What was the problem?

  • service user can be too acutely ill, have longer-term needs, or other problems (e.g. substance misuse, debt), which can put their needs beyond the remit of the service

  • concerns with safety.

• What are the main differences in role between your current job and previous work (COT and PASS only)?

  • emphasis on short-term, planned work

  • less emphasis on crisis management

  • current clients are less acutely ill, more motivated and able to make choices

  • smaller/more manageable caseload

  • more role autonomy

  • easier communication with GPs

  • slower access to medical/psychiatric assessments.

• How is the work that you do in this post different from what you did previously (COT and PASS only)?

  • previous work more institutional

  • more proactive, with activities done in the community

  • less continuous relationship than in residential care

  • easier access to other resources (in the Hub).

• How do the challenges in this job differ to those in previous work (COT and PASS only)?

  • time-limited intervention means that disengagement can be difficult

  • different demands require different knowledge and skills.

• How are the clients that you currently see different from those you have worked with in different posts (COT and PASS only)?

  • less acutely ill

  • more insight, more collaborative, more capable of independent living

  • more convenient and less stigmatising for client to be seen at GP surgery

  • having more time means better assessments can be made, and therefore increased client satisfaction.

• How are your relationships with your current client different from previously (COT and PASS only)?

  • less fire fighting, less authoritarian/custodial/prescriptive, more facilitative/collaborative/supportive

  • different pace of work affects relationship

  • client more capable of responsibility, and has more options.

• Is there anything that you think secondary services could learn from the way your service operates (COT and PASS only)?

  • services need to be better informed about charity/voluntary sector resources

  • closer working with GPs

  • residential service users should be encouraged to become more involved in their care

  • services should work more holistically

  • some (perhaps limited) scope for time-limited and recovery-based work.

• Is there anything that could usefully be applied from secondary services to your service (COT and PASS only)?

  • access to consultant psychiatrists

  • better safety systems for working in the community

  • social care professionals need to be better informed about clinical issues

  • more rigorous case presentations.

• Are there ever times when you wish you were working in a different service? What is preferable about that (COT and PASS only)?

  • would prefer a more challenging client group.

• Do you ever work alongside service user-led services? What has your experience of this been like (COT and PASS only)?

  • mental health/social care professionals need to ensure that peer support services are included in service provision, and that their contribution is respected

  • service user-led services are good, offering an added dimension to the service

  • peer supporters are insightful, and engage well with service users

  • Concerns regarding

    • peer supporters’ safety

    • peer supporter’s experiences possibly leading to tensions and conflict with mental health/social care professionals

    • mental health problems of peer supporters creating problems

    • peer supporter involvement in referral meetings. There is a case for this, but confidentiality might raise problems.

• Is there anything that your service does not do well, or should be changed?

  • better communication between teams, and definition of team roles

  • more secure funding is needed

  • more staff support needed

  • arrangements for sessions should not be made at the last minute (peer support)

  • more preparation needed for peer supporters who are nearing the end of their engagement

  • attention needs to be paid to not moving from crisis support to a more support work role

  • problems in assimilating new workers into the service.

• Is there anything that this service does not offer, but should?

  • ongoing support groups for discharged service users

  • in-house resources such as a housing officer or IT resources for service users

  • improved monitoring and support to prevent relapse in peer supporters

  • direct referral route to A&E or community mental health team (CMHT)

  • longer peer support sessions to enable more interesting activities and travel

  • better safeguarding for peer supporter home visits

  • representation at discharge meetings (peer support).

• Is there a down-side to peer support work? (peer support only)

  • boundaries and closure – disengaging after 12 weeks of intervention (more time needed)

  • exposure to clients’ distress can be retraumatising, negatively reminding the peer supporter of being in a similar position

  • uncertainties regarding continued funding

  • sufficient and adequate supervision, training and support

  • feeling pressured into becoming the sole means of support, rather than giving supplementary support.

• What special insights do you think your experience as a service user enables you to offer? How does your experience affect how you offer support? (peer support only)

  • experience gives credibility

  • experience offers a role model – gives the message that they can be helped

  • experience gives specific knowledge that helps them give proper support, such as looking at strengths rather than weaknesses, knowing that the service user might be bewildered, and not know how to ask for help, and active listening which engages with the emotional content of what is said, communicating value of the service user.

• What differences are there between the relationships you have with those that you are trying to support and other service users? (peer support only)

  • peer support relationships had more boundaries – it’s a one-way relationship, and you are more accepting in the peer supporter role. Also you need to take care not to take on a supporter role in other relationships.

• Have you helped someone in ways that a mental health professional could not have done? What was different about what you did? (peer support only)

  • offering more practical, hands-on help

  • more mutuality in the relationship – more equal.

• What impact has your work had on your own recovery, and on life in general? (peer support only)

  • peer support work can be emotionally draining

  • peer support work has many positives: financial reward, a sense of achievement and altruism, given an educational goal, and increased confidence

  • positive effect on views of mental health services and professionals: more appreciative of help given, has increased respect for them, and has humanised them

  • peer support work has directly opened up work/career opportunities and direction, and has shown possibilities for expanding other work projects.

• Has anyone ever had this kind of role with you? What difference did it make? If not, what difference would it have made? (peer support only)

  • similar relationships have/would have given role models

  • would have given social contact and daily structure

  • would have provided a person who she could have felt safe confiding in.

• Do you get involved in meetings with mental health professionals. How is your input treated by them? (peer support only)

  • there has previously been a lack of recognition of the value of peer support work

  • mental health professionals seem to be more accepting of peer support now

  • some professionals are distant and hard to understand, but others are easy to get along with.

Some were mindful of the risk of peer support being co-opted in mainstream service provision, and thereby losing credibility with service users.

Both COT and PASS participants reported that their role emphasised short-term, planned and proactive work, as opposed to less focused, longer-term involvement, which was predominantly concerned with dealing with or averting crises.

When I became a Care Co-ordinator, I thought I would have the freedom and flexibility to work with people and you know and meet as many of their needs as I could, but the frustration is you can’t, because you are trying to keep people safe. So for me, this felt an opportunity to work with people who maybe we could work more creatively with and have more time to spend with and have more time to actually do that work with, that I’d gone, you know, into secondary care, became Care Co-ordinator to do.

Staff participant 1, PASS team member

Another contrast that was commented on was the diagnostic heterogeneity of their client group, which meant that the service was operating in a slightly different way from what was originally envisaged.

I am working with a lot of people with affective mood disorders I must say, rather than say psychosis, serious mental illness, at this stage and we seeing more, a lot of people which, I don’t really like using the label ‘personality’ but I’d say ‘personality traits’, where it’s maybe people that the GP’s don’t really know how to manage, people who have multiple attendances at the GP practice, take up a lot of their time, the GPs just really don’t know how to best support them.

Staff participant 1, PASS worker

Another difference for the PASS workers was working in GP surgeries, which carried with it a wide range of changes. Seeing a service user at their GP’s surgery was seen as less stigmatising than seeing them in a CMHT and more convenient to arrange for those service users who were in employment. This also made liaison with GPs easier.

At first I wasn’t happy going into the GP practices, I must say, I was almost avoidant, dreading it, because each GP practice works so differently and they’re so busy, you don’t even know where the GPs are sitting, you don’t know where the rooms are, you know, the two GP practices are massive, quite impersonal [. . .] no one kind of gives you an induction, no one gives you a tour of the building, you’re just given a room and ‘get on with it’, [. . .] and also, I mean, two of the rooms I work in, there’s no buzzer and no panic alarm or anything. Touch wood, nothing’s happened or likely to happen [. . .] so you feel a bit isolated when you first work in the GP practices, so what I would do is I would just bring it back [. . .] to my team and say, ‘Look I’m not happy about this’, or ‘I don’t feel comfortable with this’, or ‘I don’t feel safe with this’ [. . .] but I think gradually as we’ve known how the GPs work and we’ve got to know people, that’s felt a bit better as well.

Staff participant 1, PASS worker

Accommodating all the Hub’s services together in one building had both advantages and disadvantages. Although this meant that it was easy to access other Hub services, medical assessments took longer than in secondary services. Another problem was that the lack of a direct referral pathway from the Hub to secondary services meant that referrals to them took longer.

Both COT and PASS staff participants also commented on their smaller, more manageable, caseloads and an easier pace of work than before. The relative lack of time pressure in turn creates time to make better assessments of the service user’s needs and more time for working with the service user. In turn, this increases the client’s satisfaction with the service.

The clients on staff members’ current caseloads were seen as less acutely ill and more able to be motivated to actively address their problems, live independently and work collaboratively. They were also seen as having a wider range of options available to them and as being more able to actively choose options that they could pursue, and be more able to accept responsibility for their decisions.

It sort of felt like, you were sometimes almost sort of, almost like a mothering role I suppose, where you’re sort of saying, look you really, really need to do this and even if they didn’t exactly want to, you’d have to really encourage it, whereas here it’s, the support really is on the client’s terms.

Unknown staff participant, COT team member

Another contrast with previous posts was that activities were undertaken in the community and that workers had more autonomy than they would have in, for example, a residential setting.

The staff participants were overwhelmingly satisfied with their current role, although one said that they would prefer to work with a more challenging client group. The current role was seen as less stressful and frustrating, and more satisfying and exciting (as it is new and operating in a more integrated fashion than had been the case with previous roles).

Participants from all the services also commented that their current role offered opportunities to use and capitalise on previous experience while offering the opportunity to develop new skills, interests and career directions.

I think on the whole I must say that I really, I really enjoy working with the people that I’m working with [. . .] I look forward to coming to work every day, most days and [. . .] I do work with a variety, a diversity of people with different needs, different symptoms, different presentations, so that’s been good for me as in challenging me and extending my skills and myself, informing myself and learning more and finding out more things.

Staff participant 1, PASS team worker

Peer supporters particularly commented upon the opportunities that their work gave them in relation to earnings, new work opportunities and the opportunity to prepare to undertake paid employment in the future.

Basically it’s made me feel more confident, been given back my confidence.

Staff participant 9, PS2 worker

I want to study [. . .] I’m doing a Psychosocial Graduate Course and then I want to do a Master’s in Mental Health.

Staff participant 10, PS2 worker

The staff participants were asked if there was anything that their service did not do particularly well, or if there was anything that it did not do, but perhaps should. The major problem identified was the amount of input offered. The 12-week intervention period used by COT and PASS (and formerly by PS1) was considered insufficient, as service users (particularly those with complex social needs) often had longer-term needs that were hard to address in such a limited time frame. A suggestion for mitigating this problem was to provide some measure of continuing support by running support and social groups for former PASS and COT service users. Former PS1 peer supporters also felt that the 2-hour sessions that were used by this service were too short for activities outside the local area (they were not paid extra if they worked for longer than 2 hours in a session).

Both PASS and COT staff participants also felt that sometimes service users were referred too early by CMHTs. This problem was also mentioned by the peer supporters, who thought that the expectations of some service users and of those who refer them to peer support could be unrealistic, with people who were too acutely ill to gain the full benefit of the service sometimes being referred.

There was one person who was really tough and I was way out of my depth and I didn’t feel particularly supported on by the professionals, because when something goes wrong there, they all fall to their like, textbook – we do this, we do this, we do that and then it happened to be that my meeting with him was when he was having his mental health breakdown and I was really scared and I wasn’t equipped to deal with it.

Staff participant 8, PS1 worker

Another problem lay in confusion around the different roles of the PASS and COT teams (particularly in relation to which needs the different services addressed), and a COT staff participant also mentioned the need for their colleagues to have a better understanding of medication and of medication side-effects. Both PASS and COT staff participants observed that housing the two teams in the same office facilitated consultations and support between individuals in each of them and it was felt that better communication was necessary at a team level. Practical suggestions for improvements to the service included having a psychiatrist in the team (or at least having better access to a psychiatrist), having an in-house housing officer and providing IT facilities for service users.

Participants in all three services saw the insecurity of funding for their services as a problem and the need for better safety systems when working in the community.

The ethos of peer support is one of companionship, empathy and empowerment borne of shared understandings and experience, which often conflicts directly with a prevailing discourse in mental health service culture in which ideas concerning treatment and care, control and compliance and professional expertise are central. With mental health and social care professionals working alongside peer support services in the same Hub, there is the potential for conflicts to arise. The peer support staff participants reported that their work did not initially appear to be taken very seriously, but that their contribution had become much more valued over time (although sometimes this lack of recognition resurfaced). They also commented that some (although not all) mental health and social care professionals were also seen as distant and hard to understand. They were also concerned that peer support services would be co-opted into the mainstream and thereby lose credibility among service users, and that they could inappropriately change from their adjunctive role into the only service available for some users.

I think also [. . .] the problem with it actually can be is that when you’re supporting individuals who have been discharged [. . .] it becomes more stressful because you’re having to support somebody with complex needs and you’re not trained for that and the person automatically will become dependent on us because it’s their only support.

Staff participant 10, PS2 worker

Having said that, one peer supporter pointed out that PS2 was in fact the only port of call for people who were missed by other services. For their part, COT and PASS staff participants acknowledged that they needed to ensure that peer supporters felt included and that their input was respected at meetings, and that tensions could arise out of peer supporters’ own experiences of mental health services and in negotiating how to co-work with a service user. It was further acknowledged that some progress needed to be made before they could be considered to be fully working together.

Despite these conflicts, there was considerable mutual respect shown for each others’ work. Peer supporters reported that working alongside mental health and social care professionals increased their respect for them, and their experience of supporting other service users increased their appreciation of the help that they had received. Both COT and PASS staff participants were positive about the contribution made by peer supporters, particularly noting that the insight and ability to engage with service users that their shared, lived experience gave them added a valuable dimension to the Hub’s services. One suggestion was that peer supporters become more fully involved by being included in referral meetings, which would mean that their help could be used more quickly, although it was recognised that patient confidentiality might make this difficult. Another difficulty with this plan lies in payment for the peer supporters, because they were paid only for the time spent working directly with service users. The fact that time for meetings and supervision was unpaid was a source of some dissatisfaction and resentment.

Both PASS and COT staff participants highlighted concerns about the support offered to peer supporters and about their continuing mental health problems. The peer support staff acknowledged that their work could be emotionally draining and that the exposure to the distress of their clients could remind them of their own experiences, and they stressed that equal attention should be paid to how they are supported and to the benefits that they can provide.

There was a time some of them [peer supporters] felt they were not really included in the conversation [. . .] that had taken place in the meeting and I think people need to understand them more that they’re actually a crucial part of what is happening now and [. . .] very effective.

Staff participant 7, PASS team member

Time 2 staff qualitative findings

Participating staff were re-interviewed to explore whether or not any changes that may have occurred during the intervening period had affected their work and their feelings about it. Eleven of the 12 staff members who were interviewed at time 1 were re-interviewed (one of the participants having taken up a post elsewhere), giving a response rate of 92%. The average time between time 1 and time 2 interviews was 220 days (range 151–322 days), and the average duration of the time 2 interviews was 22 minutes and 11 seconds (range 7 minutes and 26 seconds to 36 minutes and 55 seconds).

The PASS staff mentioned the upcoming introduction of the community incentive scheme (CIS) as being a potentially great change, and noted the successive changes in the role of the service over time.

Originally, the remit was for us to help facilitate people being discharged from secondary care and then that changed, so we were more meant to be primary care-focused, working from GP practices and now obviously the CIS thing developed.

Staff participant 1, PASS team worker

The CIS is intended to support GP practices in working with people who have SMIs who have been discharged from SLaM services in Lambeth. This raised concerns that this would change the profile of service users who are referred to PASS, and that it might not be possible to offer equal quality of service to service users with and without a SMI. Furthermore, service users with SMIs may find it harder to understand the nature of the support that PASS offers, and might be more likely to have more serious relapses than the other service users, which would mean that closer links between PASS and secondary services would be needed.

I worked mostly, well for many years in secondary care, so I do miss working with those people who had quite very high dependency needs, so I quite look forward to working with those people. I just want to make sure that I’d be able to offer the same time that maybe if you call it two separate groups of people with low needs and high needs, that I would hope that I could still offer the same quality of service and input to both. So that you don’t end up almost like working with the most – well, I shouldn’t say needy, or unwell, but the ones who are most likely to have higher needs and then maybe not spend as long with the people who have less but, they still need those needs met in order to move forward.

Staff participant 1, PASS team member

Another change commented upon by the PASS participants was that caseloads were larger, leading to an increased pressure of work. This had led to some team members leaving the service and few of the original team remained. All of this contributed to a feeling of demoralisation. Along with this turnover of staff came three different team managers in a 7-month period, all of whom had different approaches to their task, which, along with a general tightening of procedures and working practices, led to more unsettlement. The lack of a staff room that was separate from the working area also contributed to this feeling of pressure.

I’ve seen some staff, maybe think it’s too much and we’ve had some leave because they think the pressure is too much. It’s just the environment because there is no staff room to go out and maybe, because you are working there, you have your break there – everything happens in there.

Staff participant 7, PASS team member

The team was also expanding in size, which increased pressure on resources (particularly on workstations), which, combined with problems in accessing GPs’ information systems to update service users’ notes (entries to notes often had to be faxed to the GPs from the PASS team base), added a further element of frustration. The expansion of the PASS team was also seen by one of the COT participants as putting the Hub in danger of becoming unbalanced, and of the COT team thereby becoming identified with SLaM. Although she felt that it was good that the two teams worked closely together, she was worried that COT might lose its own distinctive identity.

The way the team has developed it has – at Thamesreach, it’s a voluntary organisation and then we’ve got lots of people from SLaM as well and it has felt a little bit like the SLaM element has become quite a bit stronger within the dynamic of the team [. . .] And I’ve been a bit concerned about the voluntary sector kind of just blending in with the SLaM side of things and not retaining its own kind of separate identity, and the sort of approach that we have might be slightly different, but it all feels like it’s all kind of blending in together a lot more and I think it’s great for us to work together, but I think it’s important for us to still keep – we’re coming from different approaches and we’ve got different things to bring; so yeah, that’s a little bit of a concern for me.

Staff participant 2, COT team worker

Concerns about the future of the PASS service were also raised, and the uncertainty caused by this added further pressure and led to some staff starting to make other employment plans.

On a more positive note, GPs were thought to be more accepting of the service, and appreciated the speed at which the PASS team could respond to referrals by seeing service users at the GP surgery. Communication between GPs and the PASS team had also improved.

I think the GPs appreciate that you’re seeing patients at surgeries, that – instead of them having to wait to hear from the community mental health team patients referred to them. They immediately know they can contact me, they get me the feedback.

Staff participant 4, PASS team worker

Community options team participants also mentioned an increase in caseloads and an expectation to see more clients, and an increase in referrals for housing problems. However, the COT service users were seen as more independent and therefore as not needing to be seen as often as the PASS service users. However, these changes meant increased pressure on resources.

The teams in my office have changed a little bit, so we seemed to have had more staff coming on board maybe. [. . .] Well, I mean for the building for example, has had to manage more people; we need more space; more interview rooms for meeting clients [. . .] Things like that. So practical things like that have become a bit more of an issue I suppose [. . .] Sometimes it’s difficult to – it’s made it more difficult to book meeting spaces and things like that with clients.

Staff participant 2, COT team worker

In response to the increased caseloads, their work had become more focused and boundaried, and periodic understaffing in the team meant that more signposting work came to the fore. The COT team had also expanded, and the new team members were seen as lacking understanding of how the Hub works, making them reluctant to share and pass on information.

I’ve worked with new members of the team and they’ve been hesitant or reluctant to share information and not really understanding that we’re working very closely together jointly, so I’ve asked somebody for some paperwork and they’ve sort of asked me why do I want it and been a bit resistant sharing that [. . .] I think it’s just a bit of a learning curve for people.

Staff participant 2, COT team worker

Once again, there had been changes in team manager (in fact, for a period there had been no manager), which meant that team members were reluctant to discharge service users, and an inevitable impact on supervision, leading to a lack of confidence. The larger caseloads and expansion of the team also meant increased stress on resources, as did the increased recording and administrative work that closer working with the PASS team entailed.

The peer support services also underwent considerable change between time 1 and time 2. PS1 had lost its tender, and was replaced by the organisation that operated PS2. One of the PASS participants particularly commented upon the loss of PS1, as there was great potential for this service to do work with her service users that she was unable to do.

I thought it was absolutely just really, really essential, really important. I’ve linked in quite a few of the clients I was working with to them, so co-working with a peer supporter and just the things that I just couldn’t really do, maybe go and have a coffee with the person, or they were worried about a hospital appointment and the peer support went there [. . .] I think it really helps a lot of the people we’ve had and it’s also made them realise, well actually, if they can do that, maybe I can become a peer supporter, or I can get into this, or I can make friends. So [. . .] I was quite sad when that went and I know they’re trying to look at alternatives, but in the form that we had it, I know there’s nothing that kind of matches that at the moment.

Staff participant 1, PASS team worker

The three participants who had previously worked for PS1 were still involved in peer support work at time 2. When the PS1 workers moved to PS2, they expected a similar set-up to PS1, but there were significant changes. PS2 operates a different model to peer support – crisis support rather than continuing support – and because continuing support was not adopted, many of the former PS1 workers felt that their earlier work was not respected, although in fairness there was an insurance problem for the organisation concerning continuing support. They were also looking for volunteers rather than paid workers and paid less money to their paid workers. One participant resented this drop in income, feeling that her contribution was devalued, and suspected that work would be allocated to volunteers before paid workers, which raised uncertainties about income. She also resented the fact that, unlike mental health and social care professionals, supervision and time for meetings was not paid, so the hourly rate was actually quite low. This concern with pay levels was shared by one of the PS2 workers.

I don’t think I want to stay doing peer support for much longer because it’s just slave labour really. It’s just really little money for really quite hard work. So yeah, and it always only says 1 hour pay per week per client, but then it’s more than that because you always end up having a phone call and then they cancel the meeting with you and you have to rearrange, and then there’s the paperwork you have to fill out and report on afterwards. It actually takes much longer but you only ever get paid for one hour, and it’s quite emotionally draining.

Staff participant 8, former PS1 and PS2 worker

Some concerns were also raised by the former PS1 workers about supervision. Although one of them felt that this was much the same as before (and praised the quality of the training), others were more critical. One complained that there was less supervision than before, although she did not resent this, as it meant that she had to attend fewer meetings (for which she was not paid), and because she got bored in group supervision. Another found that her supervisor did not respond as quickly to telephone calls and e-mails as her former supervisor at PS1 and was not as supportive. She particularly complained that most contact with her new supervisor was by text message and e-mail, although she found that personal contact (even just on the telephone) was more supportive and she was less prone to read things into what was said face to face. She was also anxious because the lack of personal contact meant that it was harder to check things and raise concerns with her supervisor. In contrast, the PS2 workers were quite content with their supervision, and one worker commented positively about the training, saying that it made her more confident in her work and as a carer.

One of the former PS1 workers had obtained funding to set up a peer-mentoring project, which is similar to PS1 in many respects but more goal-oriented and with a longer intervention period. Many changes went along with this, principally around changing from fieldwork to a co-ordinating and supervisory role. A particular problem for him was that changes to benefits and availability for work rules gave his workers some anxieties, which reduced the pool of peer supporters who were available. A former PS1 worker who left PS2 was now working on this new project, and was quite positive about her work. Because the structure was less restrictive than that of PS1, she was able to apply skills and activities that she used in other settings to her work. The longer intervention period also gave more time to work with the service user and to make more detailed plans, which relieved any pressure to produce results quickly. However, this longer intervention period also meant that the relationship developed further, making it difficult to terminate the relationship when the intervention ended, and she also found that it was sometimes hard to prioritise and define what service users wanted into achievable goals. She was also concerned that people were sometimes referred who were not as far along their road to recovery as she had been led to believe, and that working with such challenging service users was draining and affected her confidence. This reflected concerns expressed by the PASS worker who had commented upon the loss of PS1. Although she valued their contribution and wished that the service still operated, she was concerned about the support and supervision offered to PS1 workers, and about safe working practices and role delineation between mental health and social care professionals and volunteers.

The other former PS1 worker found that the change to working with PS2 made little difference to her work, apart from the fact that less face-to-face work was involved (which she preferred). The nature of crisis service work also meant that she was often uncertain about how a caller was coping after the call had ended, whereas with PS1 she was able to watch the service users she worked with change and improve.

Between time 1 and time 2, PS2 increased its hours of operation and had expanded into Southwark and Lewisham. It was also running support groups (e.g. for paranoia) and was planning to have a presence in hospital A&E departments. However, there was resentment about the fact that none of these changes had been raised with the workers, which impacted on team morale because it felt underhand and they were confused about possible role changes.

There’s a lot of confusion, no communication and kind of a sense of secrecy – I don’t understand yeah, also kind of finding out from different sources that there’s a [PS2] advertised in A&E, but you as a member are not being told actually – it’s really bizarre, I don’t know how to make sense of it.

Staff participant 10, PS2 worker

Concerns were also raised about the logistical difficulties in obtaining more telephones for the expanded service, and in recruiting and training new workers who were needed to staff the expanded service and to conduct community visits in Lambeth and Southwark.

For us to be where we are now, we’ve had to undertake many courses, many training, so it’s not going to be recruiting for off the ground, get them on the phone line – easy as that; they have to be trained, they have to do courses like we did, so that at least they’re not half cut, so that they’re spot on the job [. . .] My fear is if they don’t train these people well, they can easily have problems.

Staff participant 11, PS2 worker

The PS2 workers noted an apparent increase in calls relating to alcohol and drug problems and deliberate self-harm. There were concerns that this might mean having to refer callers on to other services (which might be overstepping PS2’s role), and that PS2 was unable to carry out the work that often seemed to be needed by some of the callers.

It was also noted that PASS, COT and GPs were more likely than before to refer service users to peer support services. However, there was also the worry that CMHTs were referring service users on to them who they were unwilling or unable to see, and that PS2 was starting to be seen as a cheap alternative to statutory services.

The service users

The service user sample was predominantly male, and approximately evenly divided between white/white other and black and other minority ethnic service users. More than 75% of these participants lived in rented accommodation, and two-thirds lived either alone (58%) or with at least one child under the age of 18 years (12%), and nearly half of the sample was dissatisfied with the amount of social contact that they had. Only one-third of the sample was in some kind of employment (including voluntary work) or education. Clinically, the average length of time receiving treatment for mental health problems was > 15 years, and less than half of the sample had experienced at least one inpatient psychiatric admission (with two-thirds of these having had at least one formal admission). However, recent episodes of inpatient care were quite rare across the sample, with the mean time elapsed since the most recent admission being 7 years and 7 months.

The most common primary diagnosis among the participants was depression (accounting for nearly 60% of the sample, with nearly one-quarter of these also experiencing clinically significant anxiety).

Service users with a diagnosis of SMI accounted for around one-third of the sample (although this proportion might have been larger if some of those reporting a diagnosis of depression also experienced undisclosed psychotic features of their disorder). This was because of changes in service configuration of the Hub that occurred between the planning and the implementation of the evaluation. These changes were reflected in both the range of diagnoses represented among the service user participants and in comments made by the PASS and COT team participants in their interviews. Although this meant that the evaluation became less concerned with the experience of service users with SMIs than was originally intended, it did examine the experiences of the typical service users who were under the care of the Hub at that time.

The evaluation also diverged from the original plan when it became apparent that too few of the service user participants had recent (or indeed any) experience of care under a secondary mental health service. At the time of interview, just over one-fifth of the participants were actually under the care of secondary mental health care services, and two-thirds of those in primary care had never been seen by secondary services. Among participants who had previously been under the care of secondary services, the mean time since transfer to primary care was 9 years and 6 months. This meant that too few service users would have been able to meaningfully compare their experiences of care under secondary services and the Hub. Furthermore, although the sample is large for a qualitative study, it is too small to examine meaningfully variations in perceptions across different diagnostic and sociodemographic and practice groups.

Although SMI was not particularly strongly represented within the sample, and although the PASS and COT team participants noted in their interviews that the service users in their current caseloads had less clinically acute symptoms and a less serious degree of social and functional impairment and were more able to actively discuss their needs and adhere to their treatment plan than those in their previous caseloads in secondary mental health care and residential settings, this was still a clinically needy group of service users. Almost half of them suffered from significant physical health problems, with almost one-third having chronic health conditions that were either managed by medication or in remission, and 15% of the sample was complicated by at least one additional psychiatric diagnosis.

There were two other differences between how the evaluation was conducted and what was originally planned. First, the original intention was that the focus group would be used to see if the findings of the economic modelling matched the experiences of people who had used the services. As this was substantially different from the purpose of the interviews, it was deemed appropriate to recruit the focus group members from among the service users who had been interviewed. However, when the focus group was run, discussion moved very quickly towards people’s experiences of the services themselves, rather than their experience of the referral pathway, and remained on this topic for a considerable part of the time allocated for the discussion. As there was no further time available to bring the discussion back to its original purpose, it was decided to continue the discussion as it had developed. This meant that a valuable opportunity to elaborate upon the findings of the modelling was missed. A focus group could have been very useful as a means of triangulating the interview findings, but such a group would have to have been recruited from service users who had not been interviewed.

Second, it was originally intended to use the constant comparative method to analyse the interviews. However, the decision was taken to apply the thematic analysis method described by Braun and Clarke,26 as this was felt to be more suitable for the analysis of such a large data set.

Service use

Across the sample, participants had been in contact with their service for approximately 5 months and typically saw a worker from their service once every 14 days for approximately 1 hour. Differences in time in contact and typical frequency and duration of contact were observed between the services. Most COT participants had been in contact with the service for 3 months or less and typically met their caseworker every 1–3 weeks for 60 minutes. However, most PASS participants had been in contact with the service for 6 months or less, typically meeting their caseworker fortnightly for 40–60 minutes. Although the typical frequency and duration of contact with the peer support services are broadly comparable with those of PASS and COT, amalgamating these data for the two services may mask crucial differences between these services owing to their differing purposes. PS2 tends to support people undergoing crises (although some do make contact more regularly), whereas PS3 facilitates access to resources that are based in and targeted towards the wider community, so users typically make few contacts with the service before fully engaging with these resources.

The experience of Hub support

At time 1 the service users were, for the most part, satisfied with the support that they had received from their service. The practical support that they had received from PASS and COT was highly valued, but so were non-specific aspects of support, such as feeling listened to and being given time to explore their problems in a friendly, collaborative, solution-focused setting, which was reiterated at time 2. The emphasis on generating possible solutions to practical problems and the support offered to deal with them was also appreciated, and the nature and quality of the support offered by these services was seen as contrasting with that offered by GPs in respect of mental health problems and secondary mental health services. A division of labour was also created, whereby mental health problems (apart from medication) were often not discussed with GPs, unless there was a crisis. Only one user of these services complained about the support that she received, and she and one other service user found the physical environment of the team base to be unsatisfactory, with its security features being seen as intimidating. It is, unfortunately, likely that some service users will not receive the support that they feel they need. This person wanted help because her husband’s mental state was deteriorating seriously, and the PASS team had no direct access to a psychiatrist who could make an assessment and prescribe appropriate treatment (a problem that was raised in the staff interviews) and her PASS worker went on a long period of leave, thus breaking any continuity of care. Given this, her dissatisfaction is perhaps understandable. Regarding the physical environment, there is, by necessity, a difficult trade-off between security and providing a welcoming physical environment, particularly in buildings that were not originally designed as therapeutic environments. Overall, the level of satisfaction with these services was such that the users of these services overwhelmingly felt that the 12-week intervention period was insufficient. However, this could mean that they had become attached to the service or that the intervention period was too brief a window in which to tackle their specific presenting problems.

Unsurprisingly, the nature of the support offered by PS2 was strongly seen as rooted in mutually shared experience. This was seen as very important, as this made it easier for both the service user and peer supporter to establish rapport, and enhanced the credibility of the peer supporter’s input, all of which was also recognised by PS2 staff and acknowledged by the PASS and COT staff interviewees. PS2 was certainly seen as providing a valuable service, although there were concerns about the lack of continuity that the rota system necessarily imposes and an accompanying variability between the listeners. A paradox was also observed, in that the more a PS2 peer supporter attempted to offer help, the further they could drift from the mutuality and egalitarianism of the helping relationship that was so unique and prized.

Peer support 3’s input with service users who were approaching discharge or who had just left hospital, was highly welcomed. However, the emphasis that PS3 placed on helping its service users to participate in mainstream social groups and activities outside the service user sphere, while laudable, meant that some service users felt that they were signposted towards activities that were pitched at a level that was inappropriate to their level of interest, or were impossible to take up because of their restricted incomes. There was also a sense of a lack of initiative and follow-through by the service, which meant that activities were often so poorly attended that interest in them waned.

Comparatively few service users said that the clinical and social problems they identified at time 1 had improved at time 2, and, for a similar number, their problems had actually worsened. Most who reported particular problems described them as ongoing. However, no standardised assessments of clinical and social problems were made in this component of the study, so it is impossible to know the true extent of any improvement or worsening that is unaffected by recall bias. Such assessments should have been made at time 1 and time 2.

Focus group findings

Given that the focus group selected from among the interviewees, it is not surprising that the findings from it so closely mirror what was said in the interviews. As mentioned above, it was originally intended that the focus group would be used to further explore the findings of the modelling, but the discussion ran very quickly away from this towards the group members’ direct experiences of the services, which had already been explored during the interviews.

No great difference was detected between CSQ ratings for the Hub services and other service. Because the other service was usually the GP, this could reflect the division of labour commented on above (see The experience of Hub support), whereby mental health problems were not usually discussed and the GP was presented with other problems. The participants might therefore be equally satisfied with the help that they receive from the Hub for their mental health problems and with the help that they receive from their GP for other problems. This is an important question and could be better explored by filling in separate ratings for mental health care and other care where the GP is named as the other service.

Similarly, the health quality subscores for the EQ-5D-5L showed little difference between those given for before referral to the Hub and those for the day of the focus group. Given that across the services the average length of time for which the participants had been receiving support was 5 months at time 1 (and given that the focus group was held rather later than the interviews), the pre-referral subscores might have been affected by recall bias. There was a quite marked increase in overall rating of health quality at the time of the focus group compared with the pre-Hub ratings (the sample size was too small to justify testing the significance of this finding). However, because the pre-Hub ratings were made retrospectively, these too might have been affected by recall bias, or the ratings of overall health for the day of the focus group might reflect an increased ability to ignore or cope with problems. Again, this is an important part of an evaluation of the service, and it would have been better to conduct prospective ratings of pre- and post-Hub health quality as part of the Hub’s standard data recording.

The staff

The COT and PASS staff participants were highly experienced in mental health and social care work, and had all worked for their respective services from or almost from their inception. Similarly, the former PS1 and the PS2 staff participants were quite experienced in working (usually in a voluntary capacity) in mental health services in general, and in their Hub service in particular. Perhaps equally importantly, they had a great deal of experience of receiving mental health services, although diagnostically they were different from the service user participants, with psychotic disorders being less prevalent. They could also be considered to be quite far along the road to recovery. Fewer of them had been admitted than in the service user group, and their most recent inpatient treatments occurred almost twice as long ago as those of the service users.

The COT and PASS staff found their work to be markedly different from their experience of working in secondary mental health services or in social care and residential settings, enjoying more autonomy in their work and with clients who were less clinically symptomatic or functionally impaired, and more able to engage collaboratively in setting and working towards treatment goals. This, together with smaller caseloads, meant that they felt able to assess their service users’ needs and to work towards positive goals (as opposed to dealing with crises), which meant that their clients were more satisfied with the service that they received. This in turn increased their job satisfaction. Job satisfaction was also enhanced by the work being less stressful than in previous posts, and by the opportunities that their current roles gave to apply old skills and acquire new ones. For the PASS workers the experience of working in GP surgeries instead of in a ward or CMHT was markedly different and required some adjustment. However, there was no direct access to secondary services, and secondary services were slow to accept referrals to them (which may have contributed to the unsatisfactory service received by one of the PASS participants).

By time 2, this positive note had been soured somewhat by uncertainties that had arisen out of several changes that had occurred in the intervening period, with few of the original team still being in post. Expansion of the teams and extending the teams’ remit to encompass the whole of Lambeth, together with an increase in caseloads and several changes of team managers, led to an adverse effect on morale. The expansion of the teams led to increased pressure on resources, particularly interview rooms and workspace, and the PASS team had experienced ongoing problems in accessing the GPs’ information systems to update their clients’ notes. On a more positive note, the PASS workers reported improved relationships with the GP practices, which appreciated the speed at which referrals were taken up and the quality of work that was undertaken. Another problem identified by both PASS and COT workers was that the 12-week intervention period was insufficient, a point with which the service user interviewees agreed. There were also concerns that CMHTs sometimes referred service users to PASS and COT too early, when they were unable to benefit as much from the service as they might otherwise do.

The peer supporters also valued the opportunities that their work gave them to earn an income and to use their experiences to help others who were in a similar situation, as well as the opportunities that the work gave them in terms of future paid employment. However, there were misgivings. One former PS1 worker resented the fact that they were paid only for 2-hour sessions with their clients, which were often insufficient for anything other than quite local activities. She also resented the fact that her time spent in supervision and meetings, unlike for professional workers, went unpaid. There were also concerns that some service users were referred too soon, which could place them in difficult situations. One of the PS2 workers also worried about the possibility that their role might be expanded to a point beyond that which they could reasonably be expected to provide, and that they would be seen as a cheaper alternative to statutory services.

The former PS1 workers who participated all transferred to PS2, expecting that they would be performing a similar role. However, the new provider preferred to use volunteers rather than paid workers, and also offered lower pay, which bred some resentment. The service model under which PS2 operated (crisis support) was also considerably different from the goal-oriented model pursued by PS1. This need for planned and consistent work with some of their service users was also noted by one of the PS2 staff participants. The style of supervision was less face to face, which also led to problems. A further source of concern for both the regular PS2 and former PS1 workers who worked with them was that proposed changes to the service were not fully shared with them before they took place, which felt underhand, and there were also logistical concerns about the changes to the service.

Although one might expect there to be some tensions between statutory and peer support services working together, both PASS and COT staff commented positively upon the value that peer support services could add. Although there were some initial teething problems and complaints that they were not taken seriously, the peer supporters felt that their contribution was welcomed by the other teams and that, over time, relationships had improved. This improvement was also mentioned by one of the PASS workers at time 1, and seems to be confirmed by the observation at time 2 that GPs, as well as PASS and COT, were more likely to refer people to them than before. However, there were still worries that they were referring service users who they were unable or unwilling to see. Similarly, at time 2 one of the PASS workers, while praising the work of PS1 and regretting that it had been discontinued, worried about the difficult and possibly dangerous situations in which some of its workers could be placed. She was also concerned that safety and coworking arrangements were quite informal, and that role demarcations between peer support and PASS and COT were uncertain. More positively, they reported that the experience of working alongside mental health and social care professionals had increased their respect for them and their appreciation of the help that they had received.

Model overview

Two treatment strategies were compared in this analysis:

1. triage strategy – referred to triage by GP

2. standard care – remain with GP for monitoring.

A NHS and social care perspective was used for this analysis and a relatively simple decision-tree model was used. Figure 5 presents the design of the model and shows how service users are simulated to transition through each service (left to right). The length of lines is not intended to represent time spent using a service. Service users start at either side of the decision node (square). At each chance node (circle) a theoretical service user is faced with a fixed set of mutually exclusive (and jointly exhaustive) transitions. Service users can no longer transition once they have reached an end node (triangle).

FIGURE 5.

Model structure. a, Integrated psychological therapy team, memory clinic and perinatal services, etc.

In particular there are two sets of basic services that compose the model: ‘Hub services’ and ‘other services’. Triage can refer service users to the former; service users can also experience non-engagement with the service to which they are referred. The latter includes services that service users can transition to when discharged from Hub services, but can also form a part of standard care. The following five services are included as Hub services:

1. PASS (12 sessions)

2. COT (12 sessions)

3. PASS + COT (8 + 12 sessions)

4. peer support

5. non-engagement with any of the above.

The numbers of sessions above are estimates that were provided by Hub management. The following eight services are included as other services:

1. GP

2. admission (inpatient user mental health)

3. Improving Access to Psychological Therapies (IAPT)

4. CMHT

5. specialist services (integrated psychological therapy team, memory clinic and perinatal services, etc.)

6. support worker

7. consultant (area mental health consultant psychiatrist)

8. home treatment team (HTT).

The data that informed transitions were taken from triage service records. Service use was theorised based on a typical usage and weighted by relevant time period. Unit costs were obtained from recognised sources. 22,23

The model length was set to 12 months and so discounting was unnecessary. The rationale for this choice of length derived quite naturally from the calculated average time a service user was in contact with each Hub service. This calculation was based on the fixed number of sessions for each Hub service and information on average time between each session provided by the participants of the qualitative evaluation (see Chapter 3). PASS, COT and PASS + COT have a fixed number of sessions (see above). In general, a fixed number of peer support sessions are not provided and, therefore, average time spent in this service was taken directly from the experience of service users in the qualitative sample. Table 29 shows how time spent by each service user in each service type is set in the model. A service user will spend 12 months in each arm of the standard care strategy.

Model inputs

This section explains the model inputs in more detail. Raw data on the number of patients who used the triage services and discharge destinations from records were converted to proportions. Because raw data were available for different time periods (from 2011 to 2015), the final proportions were also confirmed with the Hub manager as reasonable to be used in the model. These proportions can then function as probabilities that determine transitions at each chance node.

Information from the triage service indicated that a small number did transition to a PASS + COT arm, and raw data were available to calculate this probability. Unfortunately, there were no raw data available to calculate the probabilities of transition to other services when service users had been discharged from the PASS + COT arm. Therefore, the assumption was made that these transition probabilities were the same as those for transition from PASS to other services. This assumption was confirmed as reasonable by the Hub manager. To facilitate this assumption an additional assumption had to be made. The model implies that some proportion of service users can be discharged from PASS to COT; clearly it is unlikely that service users will again enter COT when moving from the PASS + COT arm as well. Therefore, the probabilities have been adjusted before being copied so that this proportion of service users is instead discharged to GP monitoring: 0.546 + 0.014 = 0.560 [Prob(PASS to GP) + Prob(PASS to COT) = Prob(PASS + COT to GP)]. Table 30 presents the probabilities of transition for each chance node in the triage strategy.

In addition, no data were provided as regards service users’ transition after 6 months of peer support. As a consequence, the model assumes that all service users who finish peer support are monitored by their GP for the remaining 6 months of the model.

Data derived from the quantitative analyses presented in Chapter 2 for the year 2011 were used to derive probabilities for the chance node that represents standard care. Two data sets could be used to calculate the probabilities: a sample of mental health service users with previous primary care contact only and a sample of service users who also had some secondary care contact. The pooled data were used to calculate probabilities, because this sample was most similar to the service users who are referred to triage. Table 31 shows the raw proportions of the pooled sample and their groupings.

Two key assumptions were made so that these raw proportions could be transformed and entered into the model:

1. 10% of group 1 transition to GP + IAPT and 90% transition to GP (0.066 and 0.594, respectively)

2. an equal proportion of group 2 transition to GP + HTT, GP + specialist and GP + CMHT (0.260/3 = 0.087).

Table 32 presents the probabilities of transition for the single chance node representing standard care.

Costs

Table 33 lists the typical usage assumptions for the base case of the model, which determine per service user resource use for a given service and period of time. These service use assumptions were combined with appropriate unit cost data; the sources of unit costs are also given.

Resource use was combined with unit costs and adjusted appropriately by time period (see Table 29) and entered into the model (see Tables 34 and 35).

Analysis

The base-case analysis presents the results of the deterministic and probabilistic model. The two treatment strategies are compared in terms of cost only, owing to the lack of available efficacy data. As justified above, the time frame for the model is 1 year and results are presented as such.

A decision-tree model (or non-recursive Markov model) was used for this analysis. The quality of data from the triage service – which determined transitions for the triage strategy – placed a severe limit on the analysis. This data source would probably best be described as an incomplete health registry, with signs of bias (lack of internal validity). The limits that the data have placed on the analysis are discussed in more detail below (see Discussion).

The analysis investigated two general types of uncertainty:

1. parameter or stochastic uncertainty

2. clinical and structural uncertainty.

The first was investigated via a Bayesian probabilistic sensitivity analysis (PSA) that involved 4000 Monte Carlo simulations. Each probability input was simultaneously varied according to a pre-assigned Dirichlet distribution. The Dirichlet distribution is the multivariate generalisation of the Beta distribution and is a natural way to characterise probabilities for mutually exclusive events that are restricted to sum to 1. For each of the 4000 simulations, cost outcomes were calculated and used to construct their respective posterior distributions. Parameter uncertainty will be presented using credible intervals (CrIs), which are interval estimates that have a purpose similar to confidence intervals. Calculating the proportion of simulations that imply that the triage strategy is less costly than standard care was used to calculate probability (triage strategy is less costly). Cost inputs were not sampled because Personal Social Services Research Unit costs do not usually provide the necessary parameter information needed to define the relevant distributions. The second type of uncertainty was explored using a one-way sensitivity analysis (OWSA). This was used to investigate the sensitivity of results to each cost input. The OWSA used low and high values based on an arbitrary 20% decrease and increase, respectively, in cost input values. In addition, the OWSA was used to test the sensitivity of the results to clinical and structural assumptions made during the modelling process.

The key assumptions made in the model can be grouped into three categories:

1. probability assumptions owing to limited data

2. cost assumptions based on typical usage assumptions

3. fundamental structural assumptions.

The sensitivity of the model to the second set of assumptions was tested by the OWSA that varies cost input values. The third set is more fundamental and concerns assumptions about general resource use and the length of contact with services. The former would include the base-case assumption that there is no GP use while using triage services or the other (post-triage) services. Assumptions about the length of contact are particularly uncertain because they are based on the very small qualitative sample (see Table 29).

Table 36 lists and describes each OWSA that was conducted.

Main findings

The average cost of treatment over 1 year for the triage strategy is around 40% of the cost of standard care (£926 vs. £2504). The difference in average cost over 1 year is £1578. Table 37 provides the results of the deterministic and probabilistic analyses.

As an example, the 95% CrIs are interpreted in the following manner: there is a 95% probability that the true population difference in average cost lies in the interval –£10,612 to £1480. Therefore, a large amount of the distribution for this difference is in the negative part of the number line (i.e. triage strategy is cheaper). This is consistent with the calculated probability: the probability that the triage strategy is cheaper than standard care is 74.3%.

One-way sensitivity analysis

A list of the OWSAs that were conducted is provided in Table 36.

Figure 6 provides a visual representation of the results from the first OWSA using a tornado plot for each strategy. Decreasing (‘Low’) and increasing (‘High’) the cost inputs by 20% had symmetrical effects on the total average cost of the triage strategy and standard care. Variations in the cost of triage services did not have any effect on the cost of standard care because these services are not inputs into any of the arms of this strategy. The reader should notice the difference in horizontal axes scales between the plots: changes in costs usually impact the average cost of standard care more. This is simply because service users are using these services for longer in the standard care strategy. In general, there are two factors that determined the sensitivity of the model to percentage changes in cost inputs: (1) the proportion of service users who use the service and (2) the size of the original base-case cost.

FIGURE 6.

Results of cost input OWSA: tornado plots. (a) Triage service; and (b) standard care.

Changing the cost of PASS by 20% changes the average cost of triage by £55. PASS is not unusually costly but because so many service users use the service, the cost of the triage strategy is relatively sensitive to changes in this cost. This also explains why the cost of standard care is so sensitive to GP costs. In contrast, admission to inpatient mental health is also a key driver of the model despite very few service users transitioning to this service. This is because typical usage for this service is very expensive.

The results of the first sensitivity analysis in terms of differences in the average cost of strategies are presented in Table 38. Decreasing (‘Low value’) and increasing (‘High value’) the cost inputs by 20% had symmetrical effects on the difference in average cost between the triage strategy and standard care.

None of the results differed enormously from the base case of a £1578 difference, but there were notable differences. The difference in cost was not sensitive to the cost of support workers or consultants. However, changes in GP and admission cost changed the cost difference by the largest amounts: £81 and £191, respectively.

The results were not particularly sensitive to the costs of COT and peer support, despite these inputs entering the triage strategy only. This is because fewer patients transition to COT and very few service users transition to peer support, which also has a very low cost. In contrast, the results are more sensitive to a change in the cost of PASS. This is because most transitions are to PASS instead of to the other triage services.

Table 38 also indicates whether the new cost difference is higher (i.e. more negative) relative to the base-case result. As expected, a decrease in the cost of any triage services will decrease the cost of the triage strategy and so increase the difference (and vice versa). A decrease in the cost of most of the other services decreases the cost difference between strategies (and vice versa). This is because service users in the standard care strategy have longer contacts with these other services and so the drop in cost will be higher in this strategy; thus, the cost difference between strategies will decrease.

Table 39 presents the results of OWSAs 2–5, which are intended to test the sensitivity of the results to fundamental assumptions about resource use and length of service contact. OWSA 2 involves adding a GP charge – consistent with typical usage – to each triage service. Similarly, in OWSA 3, service users are assumed to experience GP usage in addition to other (post-triage) service use. As expected, these changes increase the average cost of the triage strategy and have no effect on the cost of standard care. Therefore, in both cases the difference in cost decreases compared with the base-case scenario. It should be noted that, even with these adjustments, the cost difference is still substantial and comparable to base case (£1369 and £1410 vs. £1578).

The results of OWSA 4 and 5 are harder to predict. In OWSA 4 service users spend 6 months in whatever triage service they experience: service users using COT and PASS + COT experience 6 as opposed to 9 months in these services. This increases the cost of the triage strategy by a small amount (£1121 vs. £926) and so the difference also decreases slightly (£1543 vs. £1578). OWSA 5 is similar but more extreme: service users in the triage strategy spend only 3 months in triage services and 9 months in other services. This increases the average cost of the triage service (£1260 vs. £926) and decreases the difference (£1383 vs. £1578) by even more relative to base case. Notice that the relative difference is stable during both these scenarios; the triage strategy is still substantially cheaper. This is an important result and provides an explanation for why the triage strategy is cheaper: other services are more expensive and triage services withhold the movement of service users into them.