Evidence and methods required to evaluate the impact for patients who use social prescribing: a rapid systematic review and qualitative interviews

Participants

Eligible interview participants were purposively sampled through the NHSE network and contacts of staff members at the University of Warwick (Coventry, UK). Subsequent participants were snowball sampled from these initial participants. We adopted an iterative approach, progressing to the next participants if information was not available from our primary target source. All participants were approached via e-mail by members of the research team (LAK, JH or IG). Targeted participants included (1) national SP leads identified through NHSE, (2) Social Prescribing Network regional leads (for East of England, London, North East, North West, South East, South West, West Midlands and Yorkshire and The Humber) and (3) sites delivering SP, both those with an established/mature link worker social prescribing model scheme and those that did not yet have a mature scheme.

In addition, we interviewed voluntary sector organisations involved in SP to understand what happens after people have been referred to voluntary sector service providers. We contacted people from five voluntary organisations and three people participated. In addition, we contacted project stakeholders, topic experts and academic colleagues to capture opinions and feedback regarding ongoing and completed SP evaluations. In total, we contacted 41 people (plus three academic researchers), of whom 28 participated in interviews (participants, n = 25; academics, n = 3; 68% participation rate).

Data collection

Qualitative data were collected using a semistructured interview topic guide that was informed by our rapid systematic review synthesis, questions set out in the project commissioning brief, and the study protocol (in collaboration with the NIHR and NHSE). The semistructured approach allowed for systematic data collection across participants and sites while ensuring flexibility in structuring the discussion.

The interview guide was peer reviewed by NHSE and colleagues at Warwick Medical School (University of Warwick) and piloted on one participant. Any questions that were not clear were amended as appropriate. Key areas of the interview topic guide included:

• the nature of the service in terms of structure, models implemented, organisations involved and health domains covered

• patient journey throughout the service

• any measured outcomes

• data collection methods and human resources

• volume of service uptake

• type of service utilisation (highest vs. lowest)

• nature and length of follow-up

• potential strengths and limitations of the current service

• major enablers of and challenges to developing and implementing SP service

• costs and savings when recruiting for and implementing SP service (for both PCNs and other organisations involved)

• non-attendance data (e.g. resulting from people who do not take up their SP referral)

• the make-up of people taking up SP (e.g. how they compare to the overall practice population and availability of social class data).

In addition, during the interviews we asked interviewees to comment on how the COVID-19 pandemic was affecting the implementation, provision and uptake of SP.

Interviews were conducted by four members of the research team (LAK, IG, JH or EM; three women and one man; 78% were conducted by IG). Two are senior research fellows, one is a research assistant and one a is medical student; all have previous experience or training in qualitative research methods. Most of the interviews were conducted on Microsoft Teams (Microsoft Corporation, Redmond, WA, USA) and only two were conducted over the telephone. Interviewers did not have a previously established relationship with interviewees at study commencement. A project brief was provided to each potential participant (either via e-mail or verbally) ahead of the interview, which described the research aims and objectives.

Interviews were audio-recorded with the interviewees’ consent, all audio-recorded data were manually transcribed and interview notes were generated by the interviewers. Interviews lasted for a mean of 51 minutes (range 31–147 minutes).

Virtual workshops

Two virtual workshops with researchers, people involved in delivering SP services, and patients and people with lived experience of SP services were undertaken to contribute to the specification and the focus of the recommendations for a future evaluation. The workshop attendees were grouped into two: nine experts focused on process evaluation and eight experts focused on impact evaluation. The workshops also considered the implications of COVID-19 for a future evaluation of SP.

Analysis

Of the 28 interviews that were conducted, 25 [which comprised interviews with link workers, social prescribers and voluntary community and social enterprise (VCSE) workers] were included in the main qualitative data analysis. The remaining three were academic colleagues, two of whom provided descriptions of their research projects to give us further insight into their work and experience in SP evaluations. Data from the academic participants do not follow the main topic guide and are discussed separately in Chapter 3, Ongoing studies of social prescribing: lessons learned from researchers. One academic colleague interview was excluded because their work was not appropriate to the research commissioning brief.

Interview data from the 25 non-academic participants were analysed using a pragmatic framework approach,13,14 using the interview guide as a framework to code and summarise the data into the key areas of interest. The framework was applied to and further developed as the subsequent transcripts were coded, and data were charted and summarised into the framework. Data manipulation and management were supported by NVivo 13 (QSR International, Warrington, UK) software and the framework was constructed in Microsoft Excel^® (Microsoft Corporation). Analysis was performed by three members of the research team (AA, EM and IG). The immaturity of the data prevented any theoretical development and identification of reasons for the emergence of phenomena, beyond presentation of summary data in tables and example quotations.

Randomised controlled trials

We identified one SP-intervention RCT. 16 The study was an evaluation of a smartphone application (app)-based well-being intervention involving 582 participants (54.2% of all selected participants) from Sheffield, UK. The app aimed to encourage appreciation of nature and was intended to be a potential social prescription. Primary outcome measures were the ReQoL and Inclusion of Nature with Self Scale. Secondary outcome measures were the Types of Positive Affect Scale (TPAS), Nature Relatedness Scale and Engaging with Nature Beauty Scale. The study participants were randomised to two different versions of the app, either intervention (green space) (n = 414, 70%) or control (built space) (n = 168, 30%). A total of 322 (55.1%) participants completed baseline measures and 164 (27.4%) completed the 1-month follow-up evaluation. The study reported an improvement in ReQoL score from baseline to follow-up for patients in the intervention group [mean score: baseline, 29.19 points, 95% confidence interval (CI) 28.53 to 29.85 points; follow-up, 32.05 points, 95% CI 30.93 to 33.18 points] but not in the control group (mean score: baseline, 28.67 points, 95% CI 27.69 to 29.65 points; follow-up, 30.69 points, 95% CI 28.90 to 32.47 points). Multivariate analysis showed no significant effect of condition (green vs. built space) [F(7, 118) = 0.964; p = 0.461; ηp² = 0.054]. In summary, no overall clear benefit of the app was identified. The study was poorly designed because the discrepancy in group sizes (70% randomised to the intervention group and 30% to the control group) meant that the power of the study to detect differences was compromised. The minimum clinically important difference for the ReQoL is described as at least 5 points,44 which is larger than the differences that were observed in the paper reported. In addition, the study failed to recruit enough patients with common mental health problems (referred by a GP), had a high attrition rate for follow-up (the rate of retention from post intervention to follow-up was 27.36%) and reported that the app was only ‘moderately engaging’ to subjects.

Trials within cohorts studies

There was one study with a trials within cohorts (TWiCs) design. 17 The study evaluated telephone-based ‘health coaching’ among a cohort of 1306 older people with multimorbidity, of whom 504 were offered the health coaching intervention (intervention group) and the remaining 802 were not offered health coaching (the usual care group). This study measured patient-reported outcomes including patient activation, quality of life, depression, health-care utilisation and self-care. A cost-effectiveness analysis was also performed. There was no statistically significant improvement in patient activation, quality of life, depression and self-care in the intervention group compared with usual care. Use of planned services and overall costs increased and use of emergency care decreased in the intervention group. The incremental cost per quality-adjusted life-year (QALY) was £8049, and the probability that this intervention would be cost-effective at a cost-per-QALY threshold of £20,000–30,000 was 70–79%. The authors reported that the lack of significant improvement in primary outcomes may be due to the low level of uptake among those selected. The intention-to-treat study design included all patients who were offered health coaching in the intervention group regardless of whether or not they declined; therefore, the study estimates the effect of an ‘offer of treatment’ rather than the effect of receiving the treatment. Because only 41% of those selected consented to the intervention, 59% of participants in the intervention group did not receive health coaching, which substantially diluted the treatment effect.

Cohort studies

We identified two relevant cohort studies. 18,19 Munford et al. 18 assessed whether or not community assets participation is associated with better quality of life or lower costs of care among 4377 people aged ≥ 65 years with long-term conditions in Salford, UK. Outcomes assessed included QALYs, health-care costs and net benefits. Starting to participate in community assets was associated with a gain in QALYs of 0.056 (95% CI 0.017 to 0.094) at 18 months’ follow-up. The cumulative effect on care costs was –£453 (95% CI –£1366 to £461), for a net benefit of £1956 (95% CI £209 to £3703) per participant at 18 months. Stopping participation led to a change in QALYs of –0.102 (95% CI –0.173 to –0.031) and an increase in costs of £1335.33 (95% CI £112.85 to £2557.81) at 18 months. Overall, the study suggests that community assets participation was associated with improved quality of life and reduced costs of care. The study’s strengths include its longitudinal cohort design, use of statistical matching to address potential confounding and use of objective administrative data to collect health-care costs. However, it was conducted entirely in Salford, where significant investment in community groups has occurred, so results may not be generalisable to other regions. Furthermore, the study examined community assets participation (which may or may not be associated with SP schemes) rather than SP itself.

Sumner et al. 19 assessed factors associated with attendance, programme engagement and well-being change among 1297 patients with the ‘arts on prescription’ scheme in the south west of England. Higher baseline well-being was associated with successful attendance [odds ratio (OR) 1.030, 95% CI 1.006 to 1.054; p = 0.012] and increased likelihood of being engaged with the programme (OR 1.032, 95% CI 1.007 to 1.057; p = 0.012). However, higher baseline well-being also decreased the likelihood of improved well-being following the intervention (OR 0.961, 95% CI 0.933 to 0.989; p = 0.007). Considering deprivation level, participants in the medium deprivation quintile of the Index of Multiple Deprivation (IMD) reported more successful attendance (OR 2.080, 95% CI 1.092 to 3.963; p = 0.026) than participants of the lowest deprivation quintile of the IMD. Although the study had a large sample size, there was no control group, and the homogeneous socioeconomic status of participants reduced generalisability to other populations.

Before-and-after studies

There were two before-and-after studies. 20–22 Pescheny et al. 20,21 assessed change in mental well-being levels for 63 participants in the Luton SP programme, which involves link workers. Well-being, measured using the Short Warwick–Edinburgh Mental Well-being Scale (SWEMWBS), improved statistically significantly from baseline to post intervention by 2.78 points (95% CI 1.68 to 3.88 points; p < 0.001). However, the study authors noted that the mean difference in scores was less than the minimal clinically significant difference of 3 points. The study assessed mental well-being outcomes by working status, sex and age. It used skew-normal regression, which is better aligned with the data than a paired t-test. However, problems included large loss to follow-up, a short follow-up period and lack of a control group. Elston et al. 22 evaluated the impact of ‘holistic’ link workers on 86 older adults with complex health needs. They assessed well-being, activation and frailty, and use of health and social care services and associated costs. Well-being, measured using the WEMWBS and Well-being Star,52 increased significantly, by 7.9 points (95% CI 6.1 to 9.7 points; p < 0.001) and 13.3 points (95% CI 10.6 to 15.9 points; p < 0.001), respectively. The total costs increased at the end of the study by £4212 (p < 0.001). Elston et al. 22 reported that 59% of this increase was attributable to 13 users with high costs due to morbidity and frailty. The study had high follow-up rates and comprehensive data collection, but with a lack of a control group it is difficult to confidently attribute the changes observed to the intervention.

Mixed-methods approaches

Five studies used a mixed-methods approach. 23–27 Four of these23–26 used patient outcomes and other service-related outcome measures such as service-related outcomes, barriers and facilitators, training needs, potential wider impact and programme effectiveness. Bird et al. 26 performed an evaluation of a community-based, 12-week physical activity programme for inactive adults with (or at risk of) long-term conditions. A total of 326 participants attended at least one 30-minute activity session, which accounted for 30.2% (1080) of the target population. The study reported a significant improvement in WEMWBS scores from baseline to 12-month follow-up (9.09 points, 95% CI 5.65 to 12.53 points; p < 0.001) and weekly physical activity increased from baseline to 12-month follow-up by an average of 158.60 minutes per week (95% CI 103.11 to 214.10 minutes per week; p < 0.001). Strengths of this study are its Reach, Efficacy, Adoption, Implementation, Maintenance (RE-AIM) study approach,45,46 which allows for transparent understanding and reporting of intervention planning, evaluation and outcomes, and multiple quantitative/qualitative measures to assess a broad range of outcomes that are relevant to SP. However, missing data for long-term follow-up (6.8% of participants completed 12 months’ follow-up), self-reported data and lack of a proper control group are the main drawbacks of the study.

Bowden et al. 25 evaluated the effects of asthma control therapy (the BreathStar Project) among children with breathing difficulties and the associated lifestyle and wider community consequences. The study recruited 7- to 12-year-old children from deprived and highly polluted communities. There were no statistically significant improvements in asthma control. However, interviews with four subjects reported improvements in self-esteem, enjoyment of participating in a choir and the importance of a family-centred approach. Lack of GP and NHS involvement, a small sample size (four children at baseline and the end of the study), local cultural factors and difficulties generalising the study findings to other SP settings are problems in this study. A pilot study evaluated the effectiveness of a structured 6-week nature-based intervention (NBI) in improving mental health (anxiety and/or depression) of 16 individuals. 24 The pilot study documented significant improvements in mental well-being (mean WEMWBS score: pre intervention, 37 points, post intervention, 41 points; p-value = 0.009). However, again the small homogeneous sociodemographic study sample limits the generalisability of the findings.

Woodal et al. 23 evaluated the service outcome and the process of delivery of SP provided by ‘well-being coordinators’ in the north of England. 23 The data summarised change in well-being, mental and physical health, loneliness and ability to manage long-term conditions among 342 participants. Well-being scores significantly improved by an average of 3.98 points (95% CI 3.41 to 4.55 points; p < 0.001). The role of well-being coordinators, who directed patients to community groups and services, was considered the key element of SP. Limitations included lack of a control group, the cross-sectional design and the presence of the service manager during interview data collection.

Agaku et al. 27 explored barriers and facilitators related to the delivery and scale-up of the Ask, Advise, Assess, Assist, Arrange (5As) smoking cessation intervention programme. They used semistructured interviews with 21 programme directors and a quantitative survey of 120 clinic staff members. Barriers included time constraints, difficulty engaging patients and documentation challenges. Community referral resources were considered as a facilitator. Staff confidence in discussing treatment options (29%) and supporting relapsed patients was low (30%). Study limitations included the small number of participants, preventing intervention stratification by provider type; possible misclassification of clinics into 5As training status; and self-reported data from clinic staff, leading to misreporting.

Qualitative studies

We identified six qualitative studies. 28–33 Batt-Rawden and Anderson28 reported that singing in a choir can affect social inclusion and women’s perceptions of their own health and well-being. The researchers interviewed 19 female choir members in Norway and found that choir singing can support health and well-being in four ways: choir members can experience (1) joy when singing, (2) singing as essential for survival, (3) group singing as a route to social connection and (4) increased social inclusion. Participation in the study was via self-selection and, therefore, subject to selection bias. With the lack of follow-up, it is not possible to attribute improvement in well-being to taking part in the choir. In addition, men were not included and participants came from only two locations, which limits the generalisability of the findings. In a similar study, Redmond et al. 31 analysed open-ended survey responses from 1297 participants in a longitudinal study of an arts referral programme in general practices in the south-west of England. The study reported benefits across the thematic domains of ‘being with others’, ‘being on my own’, ‘doing something for me’, ‘losing oneself’ and ‘threshold’ (i.e. threshold opportunities to recognise personal growth). The study had a large sample size but generalisability to SP is limited because of the lack of attention to health outcomes.

Wildman et al. 33 conducted semistructured interviews to explore experiences of SP among people with long-term conditions 1–2 years after their initial engagement with a link worker. The study included 24 participants aged 40–74 years living in a socioeconomically deprived area of north-east England. Participants reported less social isolation and improved health-related behaviours and condition management. Barriers to SP were lack of onward referral options, unsuitable location or scheduling of activities, and language and cultural barriers. The authors suggested that an evaluation of SP requires longitudinal data collection because of the range of improvements and their episodic nature.

The remaining three qualitative studies explored organisational experiences of SP among small samples of participants conducted in single sites. 29,30,32 Payne et al. 30 focused on the mechanisms of SP. 30 They recruited 17 participants from a multiactivity SP organisation in Sheffield, UK, and identified five themes: (1) receiving professional support, (2) engaging with other participants, (3) developing new skills, (4) changing perceptions and becoming open to new futures and (5) developing a positive outlook on the present while moving forward. The study reported limitations related to participant recruitment and data coding for qualitative analysis. Bertotti et al. 29 conducted an interview-based realist evaluation of a SP pilot in London, UK, in the boroughs of Hackney and City of London. They aimed to explore the contextual factors and mechanisms that underpin the pathway linking primary care with the voluntary sector, especially via the SP coordinator. The authors subdivided the pathway into three stages: the GP referral process, consultation with the SP coordinator and interaction with the community/statutory organisations. Several challenges relating to the pathway were highlighted. This included ‘buy-in’ from some GPs, branding and funding for third-sector organisations. White et al. 32 interviewed 18 health-care professionals and 15 third-sector organisation workers involved in SP to explore the quality of the relationships with participants. The authors reported different representations of ‘health’ between the two groups, ‘mistrust of unknown third sector organisations’ by health-care professionals and the ‘lack of effective networks connecting the two groups’. The transferability of the study findings is limited by the single organisational recruitment, ‘socially desirable’ self-perceived response and the small number of GPs.

Reviews

We identified four relevant systematic reviews. 34–37 One of these reviews35 aimed to assess the outcomes of SP programmes based on primary care and involving navigators for service users. The review identified 16 studies of variable quality and reported that qualitative studies reported improved health and well-being outcomes but that quantitative studies reported mixed results. The authors suggested there is a need for more high-quality evaluations to accurately assess SP. Bickerdike et al. 37 reviewed 15 studies and reported that most of the studies were small in scale and assessed as being at high risk of bias. A review by Pescheny36 explored the barriers to and facilitators of the implementation and delivery of SP. This review included eight studies and identified barriers and facilitators related to legal agreements, leadership, stakeholder engagement and local infrastructure. The authors reported a lack of high-quality studies relevant to the review. The fourth systematic review sought to measure health and economic outcomes of SP for frail, elderly adults living in the community, but no papers met the selection criteria. 34 Other literature reviews (i.e. non-systematic)38–43 also highlighted the need for more high-quality studies.

Summary

In this rapid systematic review we have identified and summarised the growing literature surrounding SP. We screened 124 papers and included 27 in the final review. We identified various primary and secondary studies that used a range of research designs and methods. In summary, the key findings identified from the included systematic reviews are:

• SP is a clinical priority and a key to the future provision of community care.

• The available evidence reported mixed results; some evidence suggests improvement in well-being and health-related behaviour and some evidence does not.

• The available evidence comes from small-scale studies that are mostly poor in design.

• There is a lack of standardised and validated tools to measure the outcomes.

• Studies have included short follow-up durations and high levels of missing data.

• Service-related barriers include implementation approach, legal agreement, staff engagement, communication between partners and stakeholders, and local infrastructure.

Our review helped to identify appropriate evaluation questions for the evaluation of SP and to inform our qualitative data collection. It fed into the structure and content of the topic guide. Our review also helped the systematic assessment of anticipated and reported outcomes captured in the data collection section (see Qualitative interviews). In addition, the review highlighted a number of research challenges such as the low uptake of SP interventions, relatively small study sample sizes, frequent lack of a control group, heterogeneity of interventions and programmes and geographical variation of the evidence that limits generalisability.

Characteristics of study participants

A total of 25 participants (18 women and seven men) were included in the main analysis; descriptions of ongoing studies from the two academic researchers are provided in Ongoing studies of social prescribing: lessons learned from researchers. Participants’ roles covered several regions in England: the North East (n = 5), South West (n = 5), West Midlands (n = 4), South East (n = 3), North West (n = 2), East of England (n = 2), London (n = 2) and Yorkshire and The Humber (n = 1). One participant had a national role. Participants’ length of experience in SP-related roles at the time of the interview ranged from 4 months to 20 years.

The most common roles were social prescribers/link workers (n = 8), regional leads (n = 5) and learning coordinators (n = 3). The remaining participants covered a range of roles, and included one commissioner, one head of community well-being services, one community-linking project development manager, one chief executive, one operations director and one manager. Three VCSE workers were interviewed, one of whom was a programme manager for the VCSE who had a background as a health trainer and in a ‘health trainers in GP surgeries’ scheme from 2009. The programme manager worked in an organisation that had a link worker structure and served as an ‘umbrella organisation’ for several services, and sometimes supported patients directly or referred patients to an appropriate service. The remaining two VCSE interviewees were from organisations that deliver specific services (i.e. arts on prescription and mental well-being services). They are mainly dependent on volunteers and have their own policies in place in terms of referral acceptance from the link worker, self-referral or directly from general practices. One of them had worked as a freelance director of a voluntary organisation for 4 years and had previously worked as an artist. The other was managing a new service that provided telephone support, risk assessment and secondary referrals during the COVID-19 pandemic. The organisations were all located in the West Midlands. Participant characteristics are summarised in Table 1.

Categories

In this section we provide a summary description of each of the 12 categories, with supporting example quotations from interview data.

Barriers to and enablers of a social prescribing impact evaluation

During the interviews we asked participants about what they considered to be the main barriers to and enablers of an impact evaluation/future research in SP. To facilitate SP impact evaluation, several enablers were reported. Some participants suggested using case studies and routine data collection, and others emphasised the importance of familiarisation with the relevant people beforehand to mitigate the resistance that can result from being continually evaluated/monitored. Building relationships with health-care professionals and personnel from relevant organisations was often referred to. It was reported that organisations like the Social Prescribing Institute would also be good steering groups for future research.

Key enablers of a SP impact evaluation:

• an interested and active workforce of link workers

• a system that is gradually becoming more mature and, therefore, potentially more evaluable

• some robust data systems

• a desire and need by link workers and those providing SP funding to understand the impact of the service.

Key barriers of a SP impact evaluation:

• heterogeneous link worker service provision (participants reported wide variation in services even within the same county)

• large differences in terms of available onward referral services and engagement with the PCN

• the precarious financial and organisational nature of some onward referral services

• heterogeneous data collection, reporting and data monitoring by different social prescribers

• commissioners do not usually ask social prescribers to use standardised assessment tools

• lack of agreement across schemes regarding the key outcomes that need to be measured

• services are individualised because SP filters into so many aspects of health and social care and patients’ well-being.

Participants’ views on evaluating SP can be found in Appendix 5.

Ongoing studies of social prescribing: lessons learned from researchers

To gain insight into ongoing SP research, we interviewed three researchers who had undertaken or were conducting research studies in the field of SP. Unfortunately, one data set was unusable because the research did not align with our commissioning brief (e.g. it provided a generic description of SP, not experiences of SP service provision), so we present data from the remaining two. One researcher was a social scientist who had been working in SP since 2014. Another had worked for Public Health England (PHE) for a decade before moving into research. The researchers gave detailed examples of the types of research they had conducted in SP and lessons they had learned.

Summary

We collected interview data from 25 participants (and three academic researchers; data reported separately) over 6 months (April 2020 to September 2020). In addition, views of topic experts, patient and public representatives, stakeholders and academic colleagues were captured during network meetings (see Appendix 6).

The findings are presented in a pragmatic framework. A brief narrative summary is provided here, with key area in parentheses and related headings in italics.

We identified several delivery sites. Participants were based in different regions (key area 1) in England, including the South West, South East, North West, North East, West Midlands, East Midlands and London, which demonstrates distribution of SP sites. The service (key area 1) has a holistic nature that covers several domains, such as social isolation, housing, healthier living and weight management. SP is a complex intervention and there is heterogeneity in service delivery (e.g. routes to referral, services offered, outcomes assessed, number of link workers and training provided to link workers); services vary widely even in the same county. SP is an open-ended service and link workers have varied backgrounds; variations were also reported in types of services available and engagement with the PCN. The link worker model had been running for approximately 2 years; however, other forms of SP had been operational for longer. Training programmes provided to link workers differed and high turnover of link workers was reported. The voluntary sector plays a major role in service delivery. The patients’ journey (key area 2) starts with referral. Referral routes to link workers include GPs, nurses or self-referral. Following a needs assessment, the link worker can either deliver the intervention or refer the client to the appropriate service.

Outcome measures (key areas 3 and 4) were collected using a wide range of tools in SP services and across sites. We found significant heterogeneity in data availability and outcome assessment. Each locality had its own core data collection method using a wide range of tools and recording databases. We found not consistent criteria for outcome data collection; as a result, there is considerable inconsistency in the data collected. Social prescribers usually collect data on non-attendance as they are contractually obliged, but these are collected in different formats across different sites. Non-attendance data for the services patients are referred to are not routinely collected. Findings from our rapid systematic review and interviews suggest that well-being is an appropriate outcome to capture the impact of SP.

Volumes of service and uptake (key area 5) are not recorded in standardised systems. Sites have their own databases. Link workers have their local referrals and contact records, but do not capture data beyond the point of referral. Mapping of the services relies heavily on the link workers’ efforts and the voluntary sector presence. Services utilised (key area 6) are those that help with social isolation, finance, housing, increasing physical activities, healthier living and weight management. These services are often underfunded or funded for a short period of time. Length of follow-up (key area 7) varied across sites and there were differences in the follow-up measures and frequency of outcome measurement. Follow-ups and outcome measures were seen as client dependent and, therefore, not comparable. Interviewees highlighted that significant challenges to follow-up and data collection were lack of communication and time.

Strength and limitations (key area 8) included the increase in time that link workers can dedicate to address patient needs (compared with GPs) and that SP in general has the potential to reduce GP workload. The service enables health-care service providers to look holistically at the needs of people (e.g. beyond medical health) and to unite communities, identify community needs and develop the voluntary sector. Service implementation was limited by the mismatch between patient needs and what providers considered appropriate. A further limitation was the reported lack of a comprehensive directory of services, making it difficult to know where to signpost patients, although some clearly had substantial local knowledge. Barriers and enablers (key area 9) included the relatively new concept of the link worker, which affected service awareness and acceptance from both patients and those referring. Adequate training of link workers was seen as an important enabler of SP, and peer support as a helpful mechanism to limit link workers’ isolation.

A variety of costs and savings (key area 10) issues were described by participants, including concerns about lack of support for ‘additional costs’ such as overheads, training and voluntary sector costs. The SP service relies heavily on the voluntary sector, where funding models may be unstable. Non-attendance (key area 11) data were not systematically collected for onward referral services. Instead, link workers kept their own limited records. The voluntary sector generally records attendance data, but not systematically, to facilitate comparison and transfer to a third party. People who take up social prescribing (key area 12) were understood as being from more deprived areas and were vulnerable, mostly older adults and individuals living alone.

COVID-19 changed the service in terms of delivery mode, service uptake, client recruitment and client demographics. There was a clear push from service providers, stakeholders, topic experts and academics for mixed-methods research to improve our understanding of whether or not the current model is effective and how.

Stage 1

Row 1 of Figure 2 illustrates potential referral pathways by which the client can access a link worker. For instance, through referral from a health-care provider (such as a GP, nurse or health-care professional). This referral can take place using an established IT system or other modes (such as by telephone or e-mail). The health-care provider can document this process using the relevant SNOMED code (106 for referral made and 103 for declined or not required). In some instances, ‘Nil’ (no code used) may be recorded, which indicates that there was no clear reason available or given for declining the referral. It is worth noting that the ‘reason’ for referral may vary; for instance, a GP might refer a client to control their diabetes but the underlying condition may be a financial issue that is picked up in stage 2 (by a link worker’s assessment of needs).

Stage 2

Row 2 of Figure 2 illustrates the factors (see box f) that may influence link worker service delivery: the covariates that should be considered in an evaluation. Figure 2, box h, describes the client measures that are captured by the link worker at the initial visit, also known as baseline measures or time zero (T0). These measures often include a needs assessment (involving a dialogue with the client to understand their needs and what issues are most important to them) and well-being measures, such as ONS4, WEMWBS and the PAM, which includes 13 questions to quantify four levels of ‘activation’: disengaged and overwhelmed; becoming aware but still struggling; taking action; and maintaining behaviours and pushing further. 56 Additional measures include hopes and goals, and what barriers a client might envisage. These assessments vary across sites and some sites have developed their own assessment tools (e.g. the HAY tool covers aspects such as housing, relationships, finances and physical health). These measures are usually made at the initial contact.

Stage 3

Row 3 of Figure 2 illustrates the service available to the client. The link worker can provide this service themselves or refer the client on to the voluntary sector. The type and number of services a client is referred to may vary depending on the client’s needs. For instance, a client may require a service to support them with weight loss, whereas another may require services for weight loss, housing issues and diabetes. In addition, referral to services is highly dependent on the locally available services, knowledge of services and relationships with the voluntary sector. The link worker usually captures the number of referrals and number of contacts with each client (including the number of contacts across the service). They are also able to capture who has been signposted to what service and may be able to assess if the patient achieved their goals if follow-up data are collected (stage 4).

Stage 4

This is the stage where the client is engaged in the onward referral services, including the voluntary sector. Many link workers do not capture data beyond the point of referral (i.e. after stage 3). However, if they do, they would potentially recapture the same measures collected at stages 2 and 3, such as well-being, number of contacts across services and reasons for referral. There may be some communication between the link worker and the voluntary sector, and this is determined by a number of factors. For instance, if the link worker is based in the voluntary sector, data capture would be different from that in the case of a general practice-based link worker. The relationship between the voluntary sector and the link worker would also influence data capture. The IT infrastructure that connects the link worker to the voluntary sector is also important at this stage. The maturity of the service is important: more mature services will have a stronger infrastructure. The voluntary sector records attendance data regularly. However, how they record non-attendance varies, because there is no standardisation of this process. Post-referral data that may be collected by the voluntary sector before the start of the intervention are related to patient health and well-being, interest and lifestyle. Follow-up data are mostly related to health, well-being and lifestyle to identify possible improvements (using tools such as the ONS4, PAM and Well-being Star). There is also a considerable level of variability in how often outcomes are measured, depending on the service undertaking the measurement, its assessment of patient needs and how often the patient is contacted.

Retrospective matched cohort analysis

Our feasibility study demonstrated the scale and shape of the current link worker SP service. Our data highlighted that outcomes relevant to well-being and SP are currently recorded using different tools and different systems and vary by site and region. Therefore, running a national evaluation informed by current record-keeping is potentially extremely problematic. An alternative approach is to use anonymised individual-level primary health-care data between 2018 and 2019 (i.e. time cut-offs informed by when the link worker model was implemented) using EHRs, such as from the IQVIA Medical Research database (IQVIA Inc., Danbury, CT, USA) (these data sets include participating general practices to capture routine EHRs of patients managed in primary health care). This analysis would need to be condition controlled (e.g. diabetes only or obesity only or depression only). Patients with a social prescription code (e.g. SNOMED 106: social prescribing made) could be compared with matched controls. The two retrospective cohorts (i.e. intervention and matched control) could be matched based on several characteristics, such as age, sex, postcode, ethnicity and socioeconomic status, to generate a less heterogeneous sample. The intervention would be whether or not a referral to a social prescription was made. Outcomes would include clinical outcomes relevant to the health condition of the cohort (e.g. glucose measures, adiposity measures or prescriptions). Exploratory outcomes could include GP appointments and PAM scores. 50 However, the levels of uncertainty around the effect of the link worker per se would be wide. It would be almost impossible to reliably attribute outcomes to the link worker model. This approach will investigate only the effect of a SP referral rather than the uptake of the social prescription. The clinical outcomes (e.g. glucose measures, adiposity measures or prescriptions) that are routinely collected may not reflect the ‘effect’ of the intervention. Our data highlight the holistic nature of SP; therefore, well-being outcomes may be more appropriate to detect change.

It may be unwise to proceed with this approach for two reasons: first, the link worker model is a relatively new model and this will reduce the retrospective time points of the evaluation; second, a premature evaluation may not produce clinically meaningful or statistically significant changes in clinical outcomes (e.g. HbA_1c level or weight). In addition, clinical outcomes may have multifaceted determinants, making it difficult to infer causation.

Colleagues in Oxford, UK, are working on harmonising the SP codes for routine data. 54 This work aims to extrapolate codes related to social needs and social interventions that are more relevant to SP. This work is ongoing, though it may eventually help inform an evaluation using ‘homogeneous’ retrospective routine data.

A mixed-methods pre and post design

Our systematic review and interview data suggest that well-being is a key outcome for the current model (see Figure 2). However, it is not possible to systematically pool available data on a national level. We have demonstrated through this feasibility assessment the complexity of the service, the potential factors that may influence the link worker model and the heterogeneity of service provision (see Figure 2). Therefore, we consider that a mixed-methods case study might be a valuable option, with a pre- and post-test design, with baseline and possibly with 6-week and 6-month data points for quantitative measures of well-being, and with qualitative data collected midway through the follow-up period to understand participants’ lived experience of the services. The time points are suggestive, although they are informed by this feasibility assessment (described in Chapter 3 and Figure 2), reflecting academic colleagues’ experience in primary research in this area and taking into account the balance between an ‘ideal’ scenario and a ‘feasible’ scenario.

The holistic nature of SP would suggest a need for quantitative measures around quality of life, well-being, depression, anxiety, hope, loss of control, social support and social networks. A primary outcome would be identified to inform the sample size calculation and the remaining outcomes would serve as secondary outcomes. Additional data could include baseline characteristics, sociodemographics, type of services used and number of sessions attended. Self-reported outcomes would be collected using validated and unified tools that would be introduced to capture quantitative measures at three separate time points. Outcomes would be completed by the link worker to account for any illiteracy (as demonstrated by our findings) or other barriers and to reduce the research burden on clients. However, link worker incentives may be necessary. Qualitative data would include a semistructured approach to capture both client and link worker experiences of the process. Baseline characteristics of clients can be used for stratification to evaluate the differences across groups. Additional subgroup analysis can be carried out using the number of link worker visits (i.e. dose) or number and type of services referred on to. For instance, subgroup analysis would be considered for the three levels of the intervention (i.e. link worker, single service and multiple services). Factors (i.e. covariates) influencing the link worker service would be captured and accounted for. Non-attendance data would be captured to better understand the reasons behind attrition. Mean differences across outcomes would be reported at different time points. It should be noted that, in the light of COVID-19, the ‘service’ is rapidly changing across England. For instance, the service is now reaching out to people rather than relying on the traditional referral route. The mode of delivery and outcome assessment has switched over the course of 2020 from being face to face to virtual or by telephone. This may negatively affect people with literacy issues or lack of access to devices. In addition, the demographics and priorities of people taking up SP have changed.

A realist evaluation

What we have demonstrated through this feasibility assessment is that we do not yet know what types of SP services are more successful for whom and in what circumstances, nor do we know how successful programmes are over time. Our data highlight the vast heterogeneity in the delivery of SP, which has a range of consequences, with different participants valuing different outcomes.

To acknowledge and attempt to cut through the complexity, and to explore the active ingredients of SP, it would be important to focus on theories that underlie this social intervention. 60 Therefore, we consider that some form of realist evaluation may be beneficial. The aim of the realist evaluation is to provide an explanatory analysis of how and why SP works (or does not work) in particular contexts or settings. 61

The realist evaluation process will be staged, although the process is iterative and would involve significant back and forth between the data, emerging programme theory and literature. 61 The stages involve the identification of a programme theory or theories. Programme theories represent the proposed mechanism of action and give an indication as to the active ingredients of SP programmes and how they are expected to work. This would be informed by other research components, existing realist reviews and a project stakeholder group. Similar to the context–mechanism–outcome (CMO) configurations described by Tierney et al. ,62 the programme theory will be used to guide the evaluation. The later stages will involve the exploration and testing of the programme theory across a range of contexts (e.g. different patient groups and implementation chains) to examine whether or not and how they work, and if it is in the manner anticipated. This will be achieved through the primary data collection stage, where multiple data collection methods (e.g. interviews, focus groups, observations and document analysis) will be performed across multiple sites in England. The aim of the primary data collection is to conduct up-close, in-depth and detailed investigation of each comparative ‘case’ and its related contextual position. The cases (i.e. the units of investigation) could range from one link worker and one general practice to one region delivering SP, depending on the level of analysis required. Building on the findings of our feasibility assessment (see Figure 2), we propose following the route of referral as the justification for the selection of cases. Each of the four types of referral (i.e. self-referral, third party, IT system, e-mail/telephone) could be used as a tracer and examined in detail during the realist evaluations. This approach would provide excellent opportunities to compare and contrast the different modes of service delivery, testing the programme theory across each. The number of comparative cases would be limited by the resources available, but we suggest that at least four would be required to assess each referral route.

Once the data are analysed using appropriate methods for each type of data collected, the evaluation would be completed by refining the original programme theory, which will be informed by the primary data (e.g. how we thought it would work compared with what we found in practice). This would be refined further using mid-range social science theory to support explanations of how and why SP does or does not achieve the intended outcomes. It will be important to build on the findings of Tierney et al. ,62 who suggest areas for further empirical investigation, including the need to focus on link worker integration, sustainability of impact on patient well-being, and cost-effectiveness.

The realist method is not intended to be formulaic, with the goal of determining the effectiveness of an SP intervention. Instead, it is a logic of enquiry that is characteristically pluralist and flexible. 61 The planned realist evaluation could contain both qualitative and quantitative methods, formative and summative analysis and prospective and retrospective understanding. This would be informed by members of a project stakeholder team (including professionals and patients) who act as the ‘insiders’ with knowledge of SP; the project team can then test programme theories and findings. Therefore, the approaches used in the realist evaluation would differ significantly from those used in the other research scenarios (see A mixed-methods pre and post design and A cost–utility analysis). However, this element of the evaluation will maximise learning across policy, disciplinary and organisational boundaries to enable a more in-depth understanding of SP from those directly involved in its delivery.

A cost–utility analysis

This feasibility assessment has demonstrated that assessing the cost-effectiveness of link worker SP in the current systems is likely to be problematic. The health economic objective is to estimate the comparative cost-effectiveness of the link worker model against a ‘traditional/standard care’ model. To achieve this, a cost–utility analysis (CUA) could be conducted to capture resource use costs and quality-of-life outcomes at a patient level. It would then be analysed by intention-to-treat principles for each treatment pathway (depicted in Figure 2). The difference between overall costs and QALYs for each pathway is then used to calculate the incremental cost-effectiveness ratio (ICER) and established cost-effectiveness of the link worker model.

For the purpose of economic evaluation, to facilitate more robust comparison between pathways, the ‘traditional/standard care’ pathway is limited to GP-initiated referrals to external services (see Figure 2, boxes c and d) and compared with the ‘current’ SP pathway of a GP referring directly to a link worker who then refers on to external services. Simplification of this ‘traditional’ route by removal of self-referral by individuals and non-health-care professionals is important, as establishing baseline costs/quality-of-life measures in this cohort would be highly problematic.

Similarly, establishing a condition-controlled cohort, where participation is limited to those with a specific ‘health/issue’ rather than from a holistic approach would improve baseline heterogeneity. For instance, a cost-effectiveness evaluation would be performed of clients presenting with a primary issue of ‘obesity’ who then transition through either of the two pathways and on to appropriate external services.

This would be a resource-intensive research project conducted on a national level and assumes that there are a sufficient number of sites across England still implementing the ‘traditional/standard’ pathway of GP referral direct to the voluntary sector (this can be assessed by a preliminary scoping assessment).

Economic evaluation would ideally be conducted from both an NHS and Personal Social Services perspective, with resource use, and costs of services accessed by clients, collected at each stage. This will include number and length of patient consultations with GP and/or link worker, number of external services referred on to (both NHS funded and in the voluntary sector) and administration time/communication between referrer and service(s). Different approaches will be used for data collection, including routine data sources (e.g. for frequency and length of patient consultations) and appropriate resource use measurement questionnaires, such as those detailed on the Database of Instruments for Resource Use Measurement (DIRUM) website (URL: www.dirum.org; accessed 1 November 2021). Outcome measures in the form of quality-of-life, well-being and clinical measures would also be collected to estimate the difference in QALYs between those who receive SP and those who do not. Both resource use and outcome data would ideally be collected regularly, for example at baseline and 3, 6 and 12 months. This would be completed by trained researchers at each stage to account for literacy barriers and reduce the research burden on both the client and GP/link worker.

It is noted that 1 year’s follow-up may be challenging, but given successful SP ‘research-controlled’ evaluations, such as the W2W evaulation and the Rotherham evaluation, indicate that these timelines may be possible.

A pragmatic cluster randomised controlled trial

A pragmatic cluster randomised controlled trial (cRCT) (potentially plus CUA and realist evaluation) would control for unmeasured confounders between clients who are and are not offered SP, therefore making it more likely that some element of causation can be inferred if differences in outcomes are observed.

General practices would be cluster randomised to either enhanced link worker input or to standard care (which includes the current model with a link worker for each PCN), and the unit of assessment would be the practice. Standardised GP and link worker protocols for all activity would be required and stratification would be needed for practice size, geographical area and practice deprivation.

Adults with GP-assessed social prescription needs would be recruited and the primary outcome would be a measure of well-being (e.g WEMWBS score). Additional outcomes could include, for example, EuroQol-5 dimensions, five-level version (EQ-5D-5L), score with, for example, 18 months’ follow-up. Allocation of general practice would not be revealed to individual patients until their consent and baseline measures had been collected. An intention-to-treat population analysis should be undertaken.

The strengths of this approach are its unbiased nature, and outcomes relevant to SP could be collected. In addition, the cRCT could be undertaken with a simultaneous mixed-methods/realist process evaluation and alongside a CUA, furnishing more accurate data for the understanding of processes and further implementation as well as data on costs and QALYs and, therefore, for an ICER.

One limitation of this approach is that it is highly resource intensive. To work even as a pragmatic trial it would have to set up a slightly artificial model of link worker practice. This is because standardised protocols for GP and link worker activity would be needed and the inevitable substantial heterogeneity of onward referral would be a problem.

Perhaps more importantly for generalisability, the cRCT would need to trial an enhanced service in general practice as each PCN currently has a link worker; a design with controls who had no link worker input would lack equipoise. Patient selection would also be a problem and would need standardisation, but pragmatic evaluation would require an open approach to inclusion criteria that match current practice as far as possible. Inclusion criteria could be based on the 3D trial63 approach to selecting patients with chronic/long-term conditions, although it would be important to restrict the patient groups selected as little as possible from the point of view of clinical as opposed to social prescription need. There may be problems in the statistical design and sample size assessments because the intraclass correlation coefficient (ICC) would not be known [although there may be previous useful trials to build on, e.g. the WISE (whole system informing self management engagement) trial59].

COVID-19 impact

COVID-19 is changing SP in terms of delivery mode, service uptake, client recruitment and client demographics. SP services have transitioned activities and support online and/or are operating at reduced capacity, affecting their ability to respond to referrals. Link workers are more engaged in outreach to identify those most at risk of isolation and providing one-to-one virtual support. Online questionnaires and methods of data collection have caused accessibility issues for some patients, meaning some outcomes go unrecorded. There is a clear push from service providers, stakeholders, topic experts and academics for mixed-methods research to increase understanding about whether or not the current model is effective and how.

COVID-19 impact

As discussed in Chapter 4, COVID-19 is changing SP service in a number of ways: delivery mode, service uptake, client recruitment and client demographics have all changed. Services have moved to online provision. Link workers are more likely to be undertaking outreach. Gathering outcomes data has become more difficult. This transition may be permanent or temporary, depending on how the COVID-19 pandemic continues. However, these changes mean that there has been significant discontinuity in this relatively new service, which may affect methods and optimal timing of evaluation.

1. What are the most important evaluation questions that an impact study could investigate?

We identified the following most important evaluation questions:

• What is the effectiveness of the national link worker SP scheme in terms of its stated outcomes of improving health and well-being?

• What is the cost-effectiveness of the national link worker SP scheme?

• How do geographical and organisational differences in the link worker model of SP (e.g. differences in setting and availability of onward referral sites) impact the effectiveness of SP?

• Do SP referrals and outcomes vary between different deprivation areas or different population groups?

• What outcomes are associated with the link worker role in SP? What mechanisms are at work to produce these outcomes and under what contexts are they triggered?

• Does SP impact routinely collected process measures (such as GP attendance, hospital admission and readmission) and how do these relate to underlying well-being?

• What policy instruments and tools enhance and advance the effectiveness of the SP link worker scheme?

2. What data are already available at a local or national level and what else would be needed?

We identified that the research infrastructure available to allow for an evaluation of the link worker model was not robust. Any evaluation would span health care (particularly primary care), social care and the voluntary sector. The multiplicity of organisations, sites, data and data collection methods (which can vary by individual social prescriber) would make sufficient harmonisation across multiple heterogeneous organisational and geographical sites extremely challenging. Although there is an NHS national outcomes set5 related to the link worker model of SP, this reflects only process measures (e.g. attendance/non-attendance) and these measures are not sufficient to evaluate the link worker model of SP in terms of its stated outcomes of improving health and well-being. A multiplicity of outcome measures is currently used, with few pointers as to the best method for current assessment. Data needed are those that can be collected to measure harmonised well-being outcomes across all link workers and NHS SP organisations. Data are also required on costs and resources use. We recommend that CUA is undertaken to allow for evaluation of cost-effectiveness. The only way to undertake this (even inside a research-controlled environment) will be via a painstaking collation of costs and resource use data across the disparate sites and link worker SP organisational set-ups. Even in mature sites data are not mature because follow-up ranges from between 3 to 6 months and is rarely longer.

It is also the case that additional harmonised data collection would enhance evaluations, for example of patient experience of services.

3. Are there sites delivering at a large enough scale and in a position to take part in an impact study?

There are a number of mature sites delivering the link worker model of SP that would be in a position to take part in an impact study. However, it is clear that the heterogeneous nature of the link worker model of SP (i.e. the variety of organisational set-ups and of relationships with and quality and size of onward referral agencies) means that standardisation to allow for a national evaluation will be extremely difficult and the problem of lack of harmonisation of data collection remains problematic. The precarious financial situation of many onward referral agencies is a concern for the sustainability of services in the future.

4. How could the known challenges to evaluation (e.g. information governance, identifying a control group) be addressed?

We have highlighted throughout this report the problems of harmonising the data currently collected in a way that would allow for meaningful evaluation of the link worker model of SP. At present this process of harmonisation would be hugely costly and time-consuming, although it may eventually be required as interventions such as SP become more mainstream. In the meantime, we consider that the only way to address the known challenges is to use a research-controlled environment to undertake an evaluation. Therefore, we recommend a pragmatic cRCT with an additional realist evaluation to understand the underlying programme theory. This will be extremely valuable and could be undertaken before or simultaneously with an outcome/impact evaluation.

In summary, we have identified that, although an evaluation of the link worker model of SP is possible and that there are a number of different methods, each has its own drawbacks. Only a RCT would allow for adequate attribution of SP outcomes, although the problem of generalisability to the current model of SP is a concern.

We consider that a cost-effectiveness analysis is key because this is the metric with which public programmes such as this are compared in allocating scarce health and social care resources. We also consider that the investigation of relevant processes (e.g using a simultaneous realist evaluation) would be beneficial. Although the use of routine data is an attractive approach, it will not shed light on the cost-effectiveness of the SP link worker model.

Focus for the evaluation

Generally, the attendees suggested that the process evaluation should focus on how SP link workers have been embedded in PCNs and their multidisciplinary teams (MDTs). Another important focus they identified is whether or not the embedding of link workers in primary care is sustainable and has had a transformational effect on the local systems, particularly in areas that are just starting up. They noted that a process evaluation would be invaluable in showing how well NHSE’s investment had been taken up and could be used to continually develop the policy in real time.

Below are the questions raised during the discussion:

• Are the practitioners (and/or patients) doing what they are supposed to be doing?

• What happens to the users when they access activities?

• How frequently do users access what activities and does it work?

• How can we optimise the presentation of referral to a SP link worker to a patient to ensure that the referral is accepted by the patient?

• Do patients know what being referred to a link worker means?

• How do we retain link workers, what support are they being offered and do they feel part of a MDT?

• What helps people take up a referral?

• What are the factors that may encourage people to stay as a link worker?

• What is the effect of SP on primary care?

• What is not working, where is it not working and why?

• Which sites are not engaging in research?

• How will this process be rolled out to communities that do not have many assets such as outdoor areas?

• Has SP had an impact on reducing inequalities?

• How will link workers make SP effective in areas that have a high level of inequalities?

• Has community infrastructure changed since COVID-19?

• What are the gaps within communities?

• Has cultural change been brought to the NHS in delivery of services, in terms of coproduction in partnership with local communities and people?

Potential impact of COVID-19 on the focus of evaluation

With the prevailing COVID-19 pandemic, the group agreed that there is a need to evaluate how link workers have responded to the pandemic, particularly whether or not their role has supported VCSEs to link with primary care better during this time. It was also highlighted that link workers are uniquely placed to reassure those referred that services across the health and care system are still accessible and that this should be considered in an evaluation. It is important to capture the hard work that SP link workers have put in during the COVID-19 period and showcase the impact of their role. A real-time assessment of how SP contributes to the response to COVID-19 (over the next 18 months to 2 years) would be useful.

Accounting for health inequalities

There was also a discussion on how an evaluation should take account of the experiences of ethnic minority groups and other people affected by health inequalities. The group noted that link workers were well placed to work with those with the greatest health inequalities, especially ethnic minority groups. The work of the link worker enables them to work across the breadth of health inequalities and local context should be taken into account (for example urban vs. rural context). There was agreement in the group that there was a need to evaluate whether or not SP is reaching the populations that are known to have health inequalities, if link workers were supporting the right people at the right time and if different approaches are needed for different people. There is a need to consider how SP will be effectively implemented in areas with poorer infrastructure (e.g. no green spaces and community assets). There was also concern that inequalities will become considerably worse as a result of the current national situation (e.g. high levels of unemployment and debt and poor housing and security).

Potential approach to impact evaluation

The group suggested that evaluation should be based on a logic model that clarifies the outcomes that SP is expected to achieve and what to measure. This can also help to understand unintended consequences, for example where initiatives lead to increased secondary care use because they uncover unmet need. It is important to have clearly defined target groups based on referral criteria.

Potential challenges to evaluation and possible solutions

The diverse nature of SP schemes makes evaluation of impact very challenging. The group also highlighted that it will not be possible to evaluate SP in a randomised trial. The outcome measures suitable for research may be of limited value for practitioners. There is a lack of follow-up data on what happens to people after a referral; for example, do they actually take up community-based activities?

Considering the challenges to evaluation, the group suggested that a well-designed process evaluation (rather than an impact evaluation) might be a better way to understand how SP is functioning. This can make use of routinely collected data (such as GP visits, use of medication and what activities and support options people take up through SP) and help us to understand how SP is working. Primary care indicators will be more measurable and realistic to collect compared with the impact on patients’ health and well-being. This includes GP appointments, number of prescriptions, number of visits and number of visits with link workers.

Outcome measures should be extended to cover issues specific to the intended outcomes of SP, such as improvement in confidence, ability to manage one’s health, feeling connected and not feeling isolated. This could be done by creating a bespoke tool. Provision of an interface that links to SP and voluntary sectors will be useful to facilitate data sharing, particularly for feeding data back from the voluntary sector. Having an information management system is also useful in improving the quality and consistency of routine data.

Finally, though it may not be possible to carry out an impact study nationally, there is a possibility of assessing impact on defined client groups in places where service can be structured to have fewer variables.