The impact on health inequalities of approaches to community engagement in the New Deal for Communities regeneration initiative: a mixed-methods evaluation

Article history

The research reported in this issue of the journal was funded by the PHR programme as project number 09/3008/07. The contractual start date was in December 2011. The final report began editorial review in July 2014 and was accepted for publication in February 2015. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The PHR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

none

Copyright statement

© Queen’s Printer and Controller of HMSO 2015. This work was produced by Popay et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Background

This is the final report of a study that has explored the impact on health inequalities and their social determinants of the diverse approaches to community engagement (CE) adopted in the New Deal for Communities (NDC) regeneration initiative implemented between 1998/9 and 2010/11. This is the third of three closely linked studies. The Department of Health Policy Research Programme funded the first and second of these studies. The first, which ran from 2003 to 2005 considered the feasibility of evaluating the impact of the NDC initiative on health inequalities using secondary data sources [Popay J, McLeod A, Kearns A, Nazroo J, Whitehead M. New Deal For Communities And Health Inequalities: The Final Report of a Scoping Study. Submitted to the Department of Health (ref: RDD/018/063) 2005; unpublished report]. The second, which ran from 2010 to 2013, evaluated the impact of local NDC programmes on health inequalities and their social determinants. 1 The third study, reported on here, has built on this previous work in three ways: (1) by considering whether or not NDC approaches to CE had any independent impacts on a range of health and social outcomes; (2) by considering whether or not these approaches to CE contributed to the impacts of local NDC programmes reported in our previous study; and (3) by including an exploratory analysis of the cost-effectiveness of NDC CE approaches.

In this introduction we describe some of the challenges involved in evaluating action with the potential to reduce health inequalities and their social determinants, focusing in particular on initiatives to engage the public in policy decision-making. We then briefly consider the role of theory-based evaluations of complex interventions, such as the NDC, before describing the NDC initiative itself and providing an overview of previous evaluation of the impact of different types of local NDC programmes on a range of health and social outcomes. Some of this work is considered in more detail in Chapter 4. Finally, we outline how the rest of the report is structured.

Conceptualising health inequalities and action to reduce them

Health inequalities are systematic differences in health experiences/status between socioeconomic groups, areas of the country, women and men, and different ethnic groups that have their roots in unjust social arrangements. Repeated enquiries into the scale and causes of these inequalities have adopted a social model of health, which places the individual at the centre surrounded by ‘layers of influence’ relating to lifestyle factors, social and community networks, living and working conditions, and the general socioeconomic and cultural environment. 2 To reduce health inequalities action is required at all levels including the wider determinants of health such as unemployment, poverty, low educational attainment, poor housing and poor physical environments.

Despite the considerable effort and resources that have been invested in action to reduce health inequalities in the UK, these inequalities have remained largely unchanged over recent decades and in some instances have worsened. 3 Over the same period action aimed at reducing these inequalities has focused primarily on changing lifestyles among groups with the poorest health. Interventions have had a particular focus on proximal ‘risky behaviours’ including, in particular, poor diet, high alcohol intake, low physical activity, high smoking rates and risky sexual activity. When attempts are made to change environments public health initiatives typically continue to focus on the same ‘risky’ behaviours by, for example, creating environments to encourage greater physical activity, working with ‘fast food’ outlets to change menus and/or banning smoking in public places.

Evaluative research suggests that, although lifestyle-oriented approaches can improve population health overall, they do not significantly reduce health inequalities,4–6 and in some instances can increase them. 7 Part of the explanation for the failure to reduce health inequalities may be a lack of understanding about the meaning of what are commonly labelled ‘risky behaviours’. Research has shown, for example, that for some groups these behaviours may be better understood as ways of coping with difficult life circumstances. 8 From this perspective people experiencing disadvantage may have the capacity to change behaviours only if their social and economic circumstances change for the better. If the circumstances do not change in the face of lifestyle interventions, they may substitute one coping mechanism for another, which could have equally negative impacts on health and well-being.

Another part of the explanation for the enduring nature of health inequalities may lie with the failure of action to address the more distal causes – the causes of the causes as Marmot and colleagues4,9 have argued. For example, Phelan and colleagues6 have shown that historically new socially patterned proximal threats to health tend to replace the risks previously prioritised by public health policies, so continually reproducing health inequalities. These authors and others9,10 argue that this occurs because the fundamental causes of health inequalities – socioeconomic inequality or social injustice – are not being reduced. These ‘fundamental causes’ are assumed to operate through an unequal distribution of multiple resources, including income, wealth and power, which give groups/individuals differential capacity or ‘control’ to reduce proximal risks.

Control, community engagement and health inequalities

Community engagement is an eclectic arena. In the health field it may mean involving people living in particular areas in delivering (typically lifestyle) interventions designed by professionals. Other initiatives aiming to engage local people in action to improve health seek to give residents of a neighbourhood or group influence over which issues are to be prioritised for action, what the action is to be and who delivers it, and how it is to be delivered. In theory there are a number of possible interlinked pathways between the processes of CE, enhanced community control/influence and social and health outcomes (including both improved population health and reduced health inequalities). Some of these pathways are illustrated in Figure 1, which shows the initial conceptual framework that informed our study.

FIGURE 1.

Pathways from community empowerment and engagement to health improvement. Source: Popay. 11

The model proposes that, at one end of the spectrum, engagement that involves the more or less passive transfer of information between communities, professionals and/or organisations may impact on the appropriateness, accessibility, uptake and ultimately the effectiveness of services but is unlikely to increase social cohesion and solidarity among people living in a neighbourhood or result in significant empowerment of a community. Hence, the impact on health at the population or individual level is likely to be modest.

In contrast, the greater the emphasis on giving communities more power and control over decisions that affect their lives, the more likely there are to be positive impacts on service quality, social cohesion, socioeconomic circumstances, community empowerment and ultimately population health and health inequalities. It is also theoretically possible that engagement initiatives may have negative impacts on service use, social cohesion and individual and population health, for example if ‘engaged’ individuals are not appropriately supported and, at the population level, if community expectations of involvement, influence and/or control are not met. 12

There is a considerable body of research on the health and social outcomes of control at the individual level. Within social psychology, control is conceptualised at the individual level as the power to manage situations in the paid work environment, for example Karasek’s demand–control model13 and Siegrist’s effort–reward imbalance model. 14 There is also a literature on social identity, social relationships and membership of groups that links individual control to well-being and health outcomes. 15

Reviews of research evaluating the impact of CE,16–22 have found evidence of positive health and social outcomes for ‘engaged’ individuals including increased self-efficacy, confidence and self-esteem; improved social networks; a greater sense of cohesion and security; improved access to education leading to increased skills and paid employment; and self-reported improvements in physical and mental health, health-related behaviour and quality of life. There is also some evidence that CE can have unintended negative impacts on ‘engaged’ individuals including physical and emotional health costs, consultation fatigue and disillusionment. Additionally, some evidence points to potentially important relationships between the type or level of CE and intermediate social determinants of health outcomes at a community level including, for example, improved uptake and effectiveness of services,19 improved living conditions including housing quality20 and both ‘bonding’ and bridging social capital. 21

Research by Chandler and Lalonde23,24 found that the greater the degree of cultural continuity in British Columbia’s First Nations the lower the rates of youth and adult suicide. Cultural continuity was operationalised in terms of measures of the collective control that First Nations have over their ‘civic lives’ including securing ownership of their traditional lands; community control over educational services, police and fire protection services, and health care; having dedicated ‘cultural facilities’ to help preserve and enrich their cultural lives; women’s participation in local governance; community control over child custody and child protection services; and the proportion of children removed from parental care.

Emerging work by the New Economics Foundation on the concept of solidarity is also relevant to discussions about collective or community control. Coote and Angel25 argue that solidarity locates the sources of transformative action in civil society and recognises that moments for change arise from popular understanding of the structures and processes that reproduce inequalities. Qualitative research suggests that some of the strategies that people develop to manage their lives in difficult places, for example distancing themselves from others living in the same neighbourhood, can undermine the development of shared narratives and respect based on mutual understanding, which are prerequisites for collective action for change. 26,27 Research suggests that initiatives aiming to engage people in policy-making and implementation may help counter these processes and may have positive health and social outcomes for the people who get engaged. However, there is also a significant body of research highlighting the barriers to effective engagement of communities in policy and practice decision-making. 18,28 Formal evaluations of a number of English high-profile health-related policy initiatives in the first decade of this century, all with a strong emphasis on CE (such as Health Action Zones, Sure Start Centres and Healthy Living Centres), highlighted these barriers, reporting that, although community members participated successfully in specific health improvement initiatives and service delivery, there was little collective community control over the strategic direction of these initiatives. 12,29–38

It is also the case that much available research on the impact of CE is of poor quality and there are major gaps in the evidence, including an absence of evidence on the impact on health inequalities; poor descriptions of the CE interventions; a lack of evidence on the relative effectiveness of different CE approaches in engaging people from different social groups/communities, a lack of evidence on the impact of these different CE approaches on the individuals who are engaged and at the community level; and differential impacts of CE across population groups. There is also an absence of adequate information on resource use and especially the opportunity costs of CE to community members. 39 The research reported here has sought to address these problems and advance the evaluation of CE approaches and their impact on health inequalities.

‘Theory of change’ approaches to evaluation

Over the past two decades, theory-based approaches to the assessment of public policy and interventions have been elaborated in the general evaluation literature. 40–42 The idea, as originally proposed by Wholey,43 is to analyse, for the purposes of evaluation, the logical reasoning that connects intervention programme inputs to intended outcomes and assess whether or not there is any reasonable likelihood that programme goals could be achieved. This logical reasoning, called the ‘theory of change’ refers to how and why an intervention works. 44 This literature acknowledges that all intervention programmes are based on theories, whether implicit or explicit, of how the activities proposed in a programme are expected to have their impact. Making these programme theories explicit helps in the design of an evaluation, but also draws attention to the existing literature on the probable effectiveness of the proposed mechanisms for change.

Such theory-based approaches have been used in particular to evaluate the effectiveness of health promotion and risk prevention interventions,45 and latterly have been adopted for the evaluation of complex community interventions in the UK, such as the English Health Action Zones. 46 Their value lies in assessing the effectiveness of the various components of existing interventions, and also the design of future initiatives, by generating plausible programme theories and designing programmes to test them under real-life conditions. 42 The usefulness of ‘theory of change’ approaches is currently being explored for the assessment of the various endeavours to tackle health inequalities. 47 In this evaluation we have used this approach to help in the development of a typology of NDC approaches to CE and in the interpretation of our findings.

The New Deal for Communities Initiative and community engagement

In 2003, the then Labour government set the NHS and local authorities targets for reducing the gap in life expectancy and infant mortality between the most disadvantaged areas/groups and the average for the population as a whole in England. Although much activity was aimed at lifestyle and behaviour change, a National Strategy for Neighbourhood Renewal48 was also expected to contribute by improving the wider social determinants of these inequalities, for example reducing poverty and increasing housing and improving physical environments. The area-based regeneration initiative NDC was a central part of this strategy.

There were 39 NDC areas, launched in two waves: 17 pathfinder areas in 1998 and a further 22 areas in 1999. Each NDC area had a budget of around £50M (nearly £2B in total). The overall aim was to bridge the gap over a 10-year period between some of the most deprived neighbourhoods in England and the rest of the country in six outcome domains, three intended to improve the areas (crime, community cohesion, and housing and the physical environment) and three intended to improve outcomes for people (education, health and worklessness). However, the form that these local programmes took varied significantly as they sought to address local needs in very different socioeconomic, cultural and political contexts.

Various commentators argue that public participation under New Labour was grounded in theories of democracy and citizenship, and framed in rhetoric about building stronger communities through social cohesion and civic renewal. 49–51 Like other policy initiatives at the time, therefore, the NDC programme had a strong focus on CE, reflecting the wider trend of intensifying citizen participation in both UK and international public policy. 52

The theory of change underpinning CE in the NDC and other policy initiatives emerged from the Social Exclusion Unit in the Cabinet Office and assumed that engaging the communities of NDC areas in developing and delivering local programmes would overcome problems of social exclusion and promote social cohesion, hence reducing crime and incivilities, and would also make services more responsive to local needs and hence increase access and effectiveness. 48 An analysis of NDC policy guidance and strategy undertaken by Wright and colleagues51 midway through the programme positioned the NDC as ‘an attempt to reconnect deprived neighbourhoods with the rest of Britain . . . Herein the NDC partnership expresses the ideal of active citizens asserting their equality of status and demanding accountability from service providers’ (p. 351). To achieve these goals, empowerment was conceived of as a mechanism to support members of the public to develop capabilities to participate in and influence services, and shape the future of their neighbourhoods.

However, commentators have also noted the constraints of an empowerment model in the context of a government-led programme ultimately driven by policy-makers’ strategic goals, with policy-makers criticised for lacking an appreciation of the complexity of ‘community’. On the one hand the NDC’s model of engagement was concerned with the active participation in democratic and civic life of politically aware citizens. On the other hand, Wallace50 argues that policy-makers advocated for a very particular type of participation, creating structures for ‘responsible but apolitical actors’ to participate in decisions about neighbourhood life, with little scope to include social action resistant to or challenging of government agendas (p. 813).

In terms of the practice of engagement in the NDC initiative,53 the team conducting the national evaluation of the NDC reported that 18% of the total expenditure of local NDC partnerships (excluding management and administration) or approximately £248M went on ‘community-related interventions’ in the first 8 years of this 10-year programme. Almost one-fifth of this went on new or improved community facilities, but a substantial amount was also spent on involving local people and developing the skills and infrastructure of the community. 53 Although approaches to engagement varied (as discussed in Chapter 3 of this report), a wide range of engagement activities were implemented, including forums, festivals and events, NDC newsletters, funding for community development and involvement teams, training for resident representatives on NDC boards and providing resources for local action through the ‘Community Chest’ and other small grant programmes.

Predictably, only a minority of residents either had heard of the NDC or got engaged in NDC activities. For example, data from the four Market & Opinion Research International (MORI) household surveys conducted for the national evaluation suggest that the numbers who had heard of the NDC increased over time between 2002 and 2008 from 16% of residents to 22%, but declined a little by 2008. In the last of these surveys in 2008, 17% of all respondents said that they had been involved in NDC activities at some point in time (although this percentage was 44% among those respondents who had lived in a NDC area over the whole of the period). Of the 17% of all survey respondents who reported getting engaged in some NDC activities, 87% played a participative role, with most of them attending events or festivals. Only 14% voted in NDC elections and just over one-quarter (or 4% of all residents) had been involved in volunteering for the NDC partnership, although overall voluntary activity increased over the life of the NDC programme (albeit remaining lower than for England as a whole). However, in all 39 areas residents were a majority on the partnership boards that oversaw local programmes and were involved in most subcommittees overseeing thematic work (e.g. on environmental or health initiatives).

Previous evaluations of the New Deal for Communities

The government’s Neighbourhood Renewal Unit, in the Office of the Deputy Prime Minister, funded a consortium of universities led by Sheffield Hallam University to carry out a national evaluation. The National Evaluation Team (NET) started work at the end of 2001 – almost 2 years after the first wave of NDC partnerships was approved (1999) and 1 year after the second wave was commissioned (2000) – and its funding ended in 2010/11. The evaluation involved an initiative-wide impact assessment, investigating the association between ‘effort’ and ‘change’ during two periods, 2001–5 and 2006–8, utilising 36 indicators developed to measure progress across the six outcome domains. The NET also explored factors influencing programme implementation. 54 The NET evaluation involved the identification of comparator areas matched to each of the 39 NDC areas by deprivation score and local authority area, and the collection and analysis of a wide range of quantitative and qualitative research data – both cross-sectional and longitudinal – some for all of the NDC areas and their comparators and some for selected NDC areas (see Chapter 2). The NET has produced a number of reports, which are available for download from http://extra.shu.ac.uk/ndc and journal papers (accessed November 2014).

In addition to the national evaluation, there has also been a number of smaller-scale evaluations of local NDC programmes, some undertaken by individual NDC partnerships and one by a team based at the Universities of Birmingham, Staffordshire and Central England. The latter study was exploratory, considering the availability and selection of health outcome measures relating to specific activities in six NDC local programmes in the West Midlands, but there have been no publications of impact on health or health inequalities.

Although the NET looked at some health outcomes (see Chapters 2 and 4), neither its work nor any of the smaller-scale evaluations focused explicitly on the impact of the NDC programme as a whole on health inequalities or their social determinants. In 2003, the Department of Health therefore commissioned us to assess the feasibility of undertaking such an evaluation using secondary data sources, including in particular the data sources compiled by the NET. The scoping study concluded that an evaluation of the impact of the NDC initiative on health inequalities using secondary data sources was feasible [Popay J, McLeod A, Kearns A, Nazroo J, Whitehead M. New Deal For Communities And Health Inequalities: The Final Report of a Scoping Study. Submitted to the Department of Health (ref: RDD/018/063) 2005; unpublished report]. The Policy Research Division subsequently funded this evaluation in 2010 and the findings were reported in November 2013. 1 Findings from our evaluation of the health and social impact of different types of NDC local programmes are summarised in Chapter 4.

About this report

The research reported here has built on our previous evaluation of the impact of the NDC on health inequalities and their social determinants. In the research described here we sought to evaluate the impact on a range of health and social outcomes of the approaches that NDC partnerships took to engaging their residents in the design and delivery of local programmes. In Chapter 2 we describe the study design, methods and sources of the largely secondary data that we have used. Chapter 3 describes the development of a typology of NDC approaches to CE, which we subsequently used in our analyses of what impact, if any, CE had on health and social outcomes. The findings of these impact analyses are reported in Chapters 4–6. Findings from our qualitative research are reported in Chapter 7 and our cost-effectiveness work is described in Chapter 8. In Chapter 9 we highlight some of the limitations of the research, summarise key findings and consider the implications for policy and practice of aiming to engage people in shaping and delivering action to improve their lives and the places in which they live.

Introduction and research questions

Our research has involved collaboration between researchers at the Universities of Lancaster, Liverpool, Manchester and St Andrews, and the Medical Research Council Unit for Lifelong Health and Ageing now at University College London. We have also engaged NDC residents and past workers in the research as described in the following section. Originally, the evaluation was to be conducted over 24 months from 1 September 2011. However, a 9-month no-cost extension was agreed to accommodate significant delays because of changes in staff, giving a new end date of 31 May 2014.

Public involvement in our research

The main mechanism for public involvement was through the advisory group. Professor Paul Lawless, who also led the national evaluation of the NDC, chaired this advisory group, which included residents of NDC areas, former staff of NDC areas [including a previous NDC Chief Office who was later the chairperson of the Neighbourhood Regeneration Agency (NRA)], academics with a range of expertise, representatives from the Department of Health and members of the project team. Advisory group members were able to comment on regular updates on the research and the debates and critiques were invaluable, helping to strengthen the study’s methods and interpretation of findings.

In addition to participating as members of the advisory group, NDC residents (including Ann-Marie Pickup who was a named co-applicant) and NDC workers (including Liz Kessler, a former NDC employee who was extremely helpful in our previous evaluation of the NDC initiative) contributed to our research in other ways. An example of this type of contribution (described in detail in Chapter 3) involved two public advisers participating in an exercise designed to help us allocate nine NDC areas, for which we had insufficient information, to one of our CE ‘types’. Some of the NDC resident members of the advisory group also acted as advisers to the fieldwork, helping to identify research participants, proofreading project information sheets, testing research tools (such as the interview schedule), and taking part as research participants. Additionally, our public advisers participated in a workshop to discuss the interim findings from the project and to inform aspects of the work to develop a CE typology for the NDC areas (described in Chapter 3). Specifically, discussions informed the design of the research tools for telephone interviews and helped to refine the dimensions to be used in the typology of approaches to CE. Public advisers were offered an involvement fee for their participation and we developed an information sheet based on INVOLVE guidance55 that set out what advisers could expect in terms of their participation.

The NRA was the successor body to an organisation set up to support and connect the NDC areas. Its board consisted of former chief executives of NDC areas. When the NDCs were running, the NRA organised conferences and workshops that brought representatives from the NDC areas together, enabling them to share knowledge and ideas. The chief executive of a NDC successor body in one of our fieldwork sites put us in touch with the NRA, and the chair, Sam Tarff, agreed to join the study advisory group. The NRA worked with project researchers to plan a workshop to discuss the study findings with a wider group of NDC area residents and former staff. This workshop was postponed because of the low take-up of places, but we continued to involve the former CE of the NRA as a member of the advisory group. We also circulated e-mail updates on study progress to a wider network of members of the public that included former NDC employees, residents of NDC areas and newer successor organisations. We plan to produce a lay summary of this report when it has been approved and will be discussing other dissemination options with relevant funders in due course.

Study design, methods and data sources

The study has considered whether or not different approaches to CE taken by the 39 local programmes that make up the NDC regeneration initiative had different impacts on health and social outcomes and whether or not some approaches were more cost-effective than others. The aim was not to evaluate the multitude of specific techniques and processes of engagement used in local areas, but rather to evaluate the different strategic approaches to engagement adopted at a local programme level. To do this we first had to develop a typology of these approaches and then evaluate the impact of these different ‘types’ of CE approaches. The study has therefore involved mixed methods and consisted of three linked elements: development of a typology of NDC approaches to CE; assessment of the impact of NDC approaches to CE on health and social outcomes; and an exploratory economic evaluation of the cost-effectiveness of NDC approaches to engagement.

Data archiving

In addition to seeking to answer our research questions we aimed to provide a data archive making available programming codes and data for other users so that the typology development and analyses reported here can be replicated. This legacy will allow longer-term follow-up of the impact of the NDC on health inequalities and their social determinants, as well as comparison of health and social outcomes in NDC and similarly deprived areas.

The material in the archive includes:

1. Time series indicators – these include indicators derived from administrative data and are of two types: (a) hospital admission indicators, already produced and analysed, including but not limited to drug misuse, mental health and circulatory diseases for the period 2000–12 for NDC areas and their comparators; (b) social indicators: health and incapacity- and unemployment-related benefit claimant counts for NDC areas and their comparators. A user guide for these time series data sets, with careful documentation of how measures were constructed, is included so that they can be generated in a consistent way in the future.

2. A user guide and documented programming code for deriving the cross-sectional and longitudinal analytical sample and variables drawn from the NET MORI surveys and used in our impact analyses (see Chapter 4). This also includes micro-data for the comparative areas, currently not available.

3. A user guide and documented programming code for deriving the analytical sample and variables from the HSE as well as the derived data sets (see Chapter 4). This includes documentation and a programming code to code the data according to level of area deprivation.

4. Detailed documentation of our approach to classifying NDC areas in terms of local NDC programmes and NDC approaches to CE.

In addition to the above, the feasibility of archiving anonymised interview transcripts from the fieldwork reported in Chapter 3 is being explored. However, the preparation of these data for archiving will require additional funding.

This archive material is now available from http://dx.doi.org/10.17635/lancaster/researchdata/27.

Summary

In this chapter we have set out the questions addressed in our research and our study design. We have described the primary and secondary data sources we have used and the analytical methods adopted. In the next chapter we describe how we developed the typology of NDC approaches to CE, before moving on to report the results of our impact analyses and economic evaluation.

Introduction

To assess the impact of different approaches to CE, the NDC areas first needed to be categorised according to the approaches that they took. Rather than simply describing the myriad of CE activities taking place, the typology sought to understand the values and ethos underpinning different approaches to CE taken by local NDC partnerships and the factors that shaped the approaches that they adopted as well as the context that NDC partnerships operated in. The development of this typology built on the development of a typology of the local programme interventions implemented across the 39 NDC areas as part of our Department of Health-funded study (see Appendix 1).

Development of the typology consisted of three phases of work:

• phase 1: identifying and reviewing secondary data on CE for all 39 NDC areas where possible

• phase 2: primary data collection in 11 NDC areas and telephone interviews in another 10 areas

• phase 3: synthesising findings from phases 1 and 2 to develop a typology, and categorising the 39 NDC areas according to this typology.

Methods for developing a typology of the New Deal for Communities approaches to community engagement

Summary

Introduction

This chapter begins with a brief summary of existing research evidence on the impact of the NDC initiative on health and social outcomes, including the results of our previous research. It then presents the results from our cross-sectional analyses of impact, first considering trends over time in selected outcomes within NDC areas and then focusing on changes in NDC areas relative to comparator areas.

Previous research on the impact of the New Deal for Communities on health and social outcomes

Which community engagement approaches effectively engage which social groups in the New Deal for Communities populations?

Cross-sectional analyses of the impact of approaches to community engagement on the New Deal for Communities populations

Cross-sectional analyses of the impact of approaches to community engagement on the New Deal for Communities populations relative to comparator areas

Summary

This chapter has reported the results of our cross-sectional analyses of the impact of different types of approaches to CE adopted by NDC partnerships on a number of social and health outcomes. These analyses have considered trends over time in selected outcomes within NDC areas and changes in NDC areas relative to comparator areas.

The cross-sectional analyses of trends within NDC populations found a general pattern that residents in areas with type A approaches to CE were more likely to report positively over time across the nine indicators of trust, social cohesion and involvement, after controlling for socioeconomic and demographic differences between areas. In contrast, although statistical significance was not attained in most cases, odds ratios indicated lower levels of trust, social cohesion and involvement by 2008 in areas with type B, type C and type D approaches to CE than in those with type A approaches to CE. Of particular interest is the decreasing likelihood for residents in CE type D areas to think that they could influence decisions in their area. There were no consistent trends in self-reported health outcomes across CE types. The key findings here are the positive trends among respondents from CE type B and type C areas (compared with those in CE type A areas) in terms of reporting eating healthily and the trend for residents in CE type C and type D areas to be less likely to smoke than those in CE type A areas. In contrast, in most cases significant results on the mental health indicators are in the direction of residents in CE type A areas having better mental health.

There was no evidence from the cross-sectional analyses that a NDC area’s approach to CE contributed to the differential impacts of NDC local programmes reported in our previous research.

Analyses of changes over time in NDC areas relative to their comparator areas found only a small number of positive statistically significant difference-in-differences, although NDC areas reported improvements on six social cohesion and social trust indicators compared with more negative changes in the comparator areas. Areas with type A approaches to CE showed a greater increase in the proportion of residents thinking they can influence decisions in their area relative to their comparator areas, whereas residents in areas with type D approaches to CE were the only group to report an absolute decrease on this indicator over time. There were no consistent patterns for self-reported health outcomes but, although areas with type A approaches to CE had worse outcomes for quitting smoking than other NDC areas, these areas did better in terms of the percentage ‘quitting smoking’ than their comparators. The multivariate analyses have also shown the importance of the initial starting values of both the outcome variable itself and the difference in outcome values between the NDC and the comparator areas.

Introduction

The findings reported in Chapter 4 were based on repeated cross-sectional data from the NDC MORI surveys. It is not possible to assess the extent to which individuals experience change over time using these data. In this chapter we therefore present findings from analyses that have used the longitudinal panel from the MORI survey consisting of data on residents who contributed on two or more occasions between 2002 and 2008. The cross-sectional and longitudinal samples may differ in important ways (see Appendix 2, Table 63). Although the panel includes 10,638 observations, the number of complete cases with data at all four waves is 3554. Compared with the cross-sectional samples, the MORI panel sample for four waves includes a lower percentage of people with high educational attainment and a higher percentage of people with a white ethnic background.

Individual change in trust and social cohesion variables by New Deal for Communities approach to community engagement

Tables 17 and 18 show the results for four latent growth models examining differences in trust in neighbours and local services, respectively, in 2002 (baseline intercept) and change over time (slope) by CE type. Model 1 is unadjusted whereas model 2 adjusts for demographic factors, model 3 for socioeconomic factors and model 4 for NDC local programme type. Here, we focus on models 3 and 4.

In 2002, trust in neighbours was higher in areas with a type A approach to CE than in areas with other approaches to CE after adjustment for demographic and socioeconomic characteristics (see model 3 in Baseline trust levels of Table 17). Overall, there was a significant increase in trust in neighbours between 2002 and 2008 (indicated by a statistically significant and positive slope estimate of 0.20 in model 3; see Parameters of the growth curve of Table 17) but this was smaller in areas with a type B approach to CE than in areas with a type A approach to CE (indicated by a statistically significant and negative coefficient of –0.07 in model 3 in Change over time of Table 17).

To assess the contribution of CE to the impact of NDC programme type, NDC programme type indicator variables were introduced in model 4 in Table 17. The results show that NDC areas with type 2 and type 3 local programmes showed a smaller gain in trust in neighbours over time than areas with type 1 local programmes (p < 0.05). In this fully adjusted model, within-person improvement in trust in neighbours over time was greater in areas adopting a type A approach to CE than in areas with type B and type D approaches (indicated by a statistically significant and negative coefficient for CE types B and D in Change over time of model 4 in Table 17). This association was independent of NDC programme type, and type of CE approach did not contribute to explaining differences in trust by NDC programme type.

The results of latent growth models for trust in local services are shown in Table 18. In 2002, residents in areas with type C approaches to CE showed lower trust in local services than those in CE type A areas (p = 0.1; see model 4). There is also evidence of an overall decline in trust in local services over time (indicated by a slope coefficient of –0.21 in model 3 in Parameters of the growth curve of Table 18). However, levels of trust in local services increased among residents in areas with type D approaches to CE relative to those in areas with type A approaches to CE [indicated by a positive Change over time (Table 18) coefficient for CE type D of 0.06 in model 3 and 0.11 in model 4]. Our previous study; found no difference in change in level of trust in local services over time by local NDC programme type. However, model 4 in Table 18 shows that, when the NDC approach to CE is taken into account, levels of trust in local services among residents in NDC areas with type 2 and type 3 local programmes decreased to a lesser extent than levels of trust in local services among residents in NDC type 1 areas [indicated by a Change over time (see Table 18) coefficient for NDC types 2 and 3 of 0.08].

In summary, analyses of within-person change show that trust in neighbours increased between 2002 and 2008 but this improvement was smaller in areas with type B approaches to CE. Trust in local services decreased over this period in most areas, but increased in areas with type D approaches to CE.

Individual change in health, lifestyle and work outcomes by approach to community engagement

We used survival analysis to investigate differences in the probability of residents adopting healthier lifestyles according to the approach to CE adopted in their area. This enabled us to model the factors that influenced individual changes in behaviour at any time between 2002 and 2008. The results for smoking behaviour are shown in Table 19. Smokers in areas with type B approaches to CE were marginally less likely to quit smoking over time after controlling for demographic and socioeconomic characteristics. This analysis also suggests that approach to CE did not contribute to the finding from our previous study; that residents in areas with type 2 local programmes were more likely to quit smoking over time than those in areas with other local programme types.

Table 20 reports the results of the survival models for healthy eating and Table 21 reports the results of the survival models for positive mental health. In terms of healthy eating, residents with a less healthy diet in 2002 in areas with type C and type D approaches to CE were more likely to report eating five or more portions of fruit/vegetables a day at least three times a week over time than residents in areas with type A approaches to CE. Those in areas with type 2 and type 3 NDC local programmes were more likely to report these dietary changes and, although the estimate was attenuated for type 2 programmes, it was magnified for type 3 programmes once CE type was included in the model. Again, the evidence points to somewhat independent effects of CE type and NDC programme type.

Although residents in areas with type B, C and D approaches to CE reported better mental health at baseline, this was explained by demographic and socioeconomic characteristics (models 1 and 2). Overall, there was no significant improvement in mental health between 2002 and 2008 (indicated by the slope estimates shown in the bottom section of the table). However, after accounting for demographic and socioeconomic factors, residents in CE type C areas had a significantly greater worsening of mental health than those in CE type A areas (indicated by the negative and statistically significant slope regression coefficient for this CE type of –0.10 in Slope regression coefficients of the table). Other analyses reported in Appendix 2 (see Table 64) suggest that there was no differential change in self-rated health between 2002 and 2008 across the different CE types.

Table 22 reports the changes in the likelihood of individuals finding employment between 2002 and 2008, adjusting again for demographic and socioeconomic characteristics, and for local programme and CE types. Out-of-work residents in areas with type D approaches to CE, and to a lesser extent those in areas with type B approaches, were less likely to find employment between 2002 and 2008 than those in areas with type A approaches to CE. Our previous evaluation of the impact on health inequalities of NDC local programmes found that residents without work in areas with type 2 and type 3 NDC local programmes were more likely to move into employment between 2002 and 2008, and model 3 in Table 22 suggests that CE type explained some of these differences according to NDC programme type.

In summary, these analyses indicate that residence in NDC areas adopting CE types C and D was associated with an improvement in healthy eating. Although there was no significant improvement in mental health in any NDC area, the mental health of residents in areas adopting a type C approach to CE worsened to a greater extent than the mental health of residents in areas adopting a type A approach to CE. In addition, residence in NDC areas adopting type B and type D approaches to CE was associated with a lower likelihood of finding employment than residence in areas with a type A approach to CE.

Does the association between outcomes and New Deal for Communities approach to community engagement vary across groups defined by socioeconomic circumstances?

Our final analysis in this chapter explored whether or not there was evidence that different approaches to CE adopted by NDC partnerships were associated with a reduction in the gap in health and social outcomes between the least and the most disadvantaged residents of NDC areas. To do this the analyses considered possible interactions between CE type and three socioeconomic indicators: housing tenure (renting vs. owning), educational attainment [National Vocational Qualification (NVQ) 0–1 vs. NVQ ≥ 2] and household employment status (living in a jobless household vs. living in a household with at least one member in paid employment). Table 23 reports on significant interactions only when a main effect was found for the particular type of CE in question. It is important to emphasise that these analyses involved a large number of significance tests, which makes interpretation difficult, and we have not formally controlled for multiple testing.

These analyses suggest that some approaches to CE may have narrowed the gap between the most and the least disadvantaged in NDC areas. For example, trust in neighbours increased over time to a larger extent among residents living in jobless households in 2002 in CE type D areas than among residents living in jobless households in 2002 in CE type A areas. It also improved more among renters in type B and type C areas. Overall, mental health improved at a slower rate in areas with type C approaches to CE than in CE type A areas, but improvement in mental health was particularly high for residents in jobless households in CE type C and type D areas. Residents in rented accommodation in areas with type D approaches were also more likely to have quit smoking between 2002 and 2008.

Summary

This chapter has reported the results of analyses of within-person changes in NDC areas by CE and local programme type. These results suggest that individuals in areas with type A approaches to CE saw greater improvements on some social cohesion, trust and mental health indicators. For example, there was a significant overall increase in trust in neighbours, which was greater in areas with type A approaches and lower in areas with type B approaches. Against the backdrop of a slight, non-significant within-person improvement in mental health, individuals in areas with type C approaches to CE had a significant worsening of mental health than those in CE type A areas, although some more disadvantaged groups in CE type C areas did better. Individuals without paid work in CE type A areas were also more likely to find employment by 2008 than those in CE type D, and to a lesser extent CE type B areas. On the other hand, an overall decline in trust in local services over time in all areas was slower among residents in areas with CE type D approaches and the approach to CE appears to have mitigated this decline in areas with type 2 and type 3 local programmes. Cross-sectional and longitudinal results were consistent in indicating that improvements in fruit and vegetable consumption were better in CE type C and D areas than in CE type A areas.

There is some evidence that CE types impacted differentially on specific population subgroups in ways that could reduce inequalities within NDC areas, but no consistent pattern was identified. When an effect of CE type was seen, this was usually independent of a NDC programme type impact, found in our previous study. For only one outcome (joblessness) did we find that CE type mediated or contributed to explaining differences across NDC local programmes. In relation to trust in services, failure to consider CE type in our previous study appears to have masked differences by NDC type.

Introduction

In this chapter we explore the impact of NDC approaches to CE on outcomes included in the time series originally constructed by the NET for each NDC area and its comparator using routine administrative data, and updated by us. We considered the following health-related outcomes:

• hospital admissions indicators standardised for age and sex from the Hospital Episode Statistics (HES) data set for six conditions: drug misuse, alcohol misuse, cancer, respiratory conditions, heart conditions and mental health

• work-limiting illness based on claimant counts [from the Department for Work and Pensions (DWP)]

• rates of low birthweight from ONS data sets

• mortality among those aged < 75 years from ONS data sets.

Our analyses also considered trends in three claimant count-based indicators related to employment status and income:

1. worklessness: people of working age receiving a benefit relating to their unemployment or inability to work on medical grounds – Incapacity Benefit (IB), Severe Disablement Allowance (SDA) and Employment and Support Allowance (ESA) in addition to Jobseeker’s Allowance (JSA)

2. unemployment: those in receipt of JSA

3. those in low-income households: based on counts of all claimants of IS (Income Support), JSA-IB (Jobseeker’s Allowance – Invalidity Benefit) or ESA aged 0–59 years, plus any dependent children and partners also aged 0–59 years.

These analyses involved the following linear regression approach to test for trends.

• the difference between the values for the indicators relating to these outcomes for each individual NDC area and its comparator area was computed for each of the years in the time series

• this difference was regressed against time

• unstandardised coefficients are reported when they are significant at ≥ 10%.

It is important to emphasise that, with all of the indicators used in these analyses, an improving trend relative to a comparator may not correspond to an absolute improvement in the NDC area relative to the national picture. For example, hospital admissions remained higher than the national average in NDC areas through the life of the programme. Additionally, for some indicators small numbers in single areas meant that results were not robust. This was the case for low birthweight and mortality, and so only the results for hospital admissions and claimant counts are presented here.

Trends in hospital admission rates

In our previous study (see Appendix 1) we found that areas with type 2 local NDC programmes, which involved an incremental approach to regeneration with modest physical redevelopment, performed better in terms of reducing hospital admission rates relative to their comparator areas than areas with either type 1 or type 3 programmes. The full results of this work are reported in Appendix 2 (see Table 65) and are summarised in the bottom row of Table 24. This shows that one-quarter (25%) of the hospital admission indicators for areas with type 2 local programmes show health improvements (i.e. reduction in hospital admissions) relative to their comparator areas between 1999 and 2010. In contrast, only 5.6% of indicator values relating to type 1 local programmes and 7.6% of indicator values relating to type 3 local programmes show improvement above that of their comparator areas.

The far right column labelled ‘All’ summarises the results for the four types of NDC approaches to CE. In contrast to the results for type 2 NDC local programmes, none of the four CE types shows a high level of improvement (i.e. reducing hospital admissions relative to the comparator area). Areas with a type A approach to CE have the highest level of change (35.8%) in hospital admissions, equally divided between indicator values showing improvements, that is, reductions in hospital admission rates relative to comparator areas (17.9%), and those showing deteriorations, that is, increases in hospital admission rates relative to comparator areas (17.9%). NDC areas with a type B approach to CE have only a slightly lower level of change (32%) than those areas with a type A approach to CE, but in these areas there are 4.2% more values indicating improvements (18.1%) than deteriorations (13.9%). NDC areas with type C approaches to CE are unusual in having a small number of changes (only 13% of indicators) and most of these reflect deteriorations (9.3%). NDC areas with type D approaches to CE have average levels of change but with twice as many deteriorations (20%) as improvements (10%). These results suggest that, overall, areas with type A and type B approaches to CE performed better than areas with type C and type D approaches to CE.

Looking at the different combinations of programme type and CE type (see Table 24), the strongest combination effect is seen in group B2 (areas combining a type 2 local programme with a type B approach to CE, i.e. starting out resident led but weakening over time), with 11 (36.7%) of the 30 indicators showing significant improvements in hospital admissions and only 3.3% showing a relative deterioration. Areas combining a type 2 programme with a type A approach to CE (i.e. resident led throughout the programme) also performed well (group A2), with 22.9% of indicators showing an improving trend in hospital admissions over the decade compared with their comparator areas and 18.8% showing deteriorations.

Trends in work-limiting illness or disability

Our analyses considered two health-related indicators based on claimant count data from the DWP. The first (work-limiting illness or disability) is the rate at which people of working age are claiming IB, SDA or ESA. The second is an age- and sex-standardised measure covering people of all ages who receive at least one incapacity-related benefit. It is important to note that claimant count indicators may be affected by national changes in the welfare benefit regulations (e.g. a tightening of the eligibility criteria, which happened over the period covered by these data: 2004–11) and this should be taken into account when interpreting the findings reported here. However, this said, these types of changes might be expected to impact equally on all NDC areas and so, to the extent that we identify differential patterns in the change in numbers of claimants over time across NDC areas with different types of local programme and/or approaches to CE, this could be an indication that these programmes and approaches to CE affected the impact of national programmes (positively or negatively). As noted earlier, an improving trend relative to a comparator may not correspond to an absolute improvement in NDC area conditions. As Figure 2 shows, at both the start and end of the period covered, the prevalence of work-limiting illness or disability in every NDC and comparator area was always higher than the English average (which is set at 1 in Figure 2).

FIGURE 2.

Standardised work-limiting illness ratios for NDC areas in 2004 and 2011 compared with the English average (set at 1).

Figure 2 also shows that standardised work-limiting illness ratios (based on claimant counts) were lower in 2011 than in 2004 in 27 of the 39 NDC areas. The differences in ratios are similar for the comparator areas: standardised illness ratios were lower in 2011 than in 2004 in 28 of the 39 comparator areas. The two sets of changes are quite highly correlated (at 0.59), suggesting that they may be influenced by similar trends, such as wider changes in health or changes in eligibility criteria for health related-benefit claims. Table 25 shows the percentage of trends where numbers of NDC residents claiming benefits was falling or increasing in areas with different combinations of local programme types and CE approach. There is a high level of significant change overall – 68% of the 78 coefficients in the table – reflecting a great deal of movement onto and off these benefits over the period covered.

Areas with type 1 and type 2 local NDC programmes saw the greatest reductions in the numbers claiming benefits relative to their comparator areas, as did areas with type C and type D approaches to CE. In contrast, areas with type A approaches to CE were more likely to see numbers on benefit rising relative to their comparator areas. In terms of combinations of programme type and approach to CE, NDC areas combining type 2 local programmes with type B approaches to CE were more likely than any other combination to see claimant numbers reduce relative to comparator areas, whereas those combining type 2 local programmes with type A approaches to CE were most likely to see claimant numbers increase relative to comparator areas.

Trends in unemployment, worklessness and low income

Our analyses considered trends in three claimant count-based indicators related to employment status and income:

• worklessness: people of working age receiving a benefit relating to their unemployment or inability to work on medical grounds: IB, SDA and ESA in addition to JSA

• unemployment: those in receipt of JSA

• those in low-income households: based on counts of all claimants of IS, JSA-IB or ESA aged 0–59 years, plus any dependent children and partners also aged 0–59 years.

We expected our measures of worklessness and unemployment to be highly correlated as they are both based on the JSA claimant count. The trends in these three indicators are summarised in Table 26.

Areas with a type A approach to CE perform worst, with 10 (38.5%) of the indicator values showing significantly increasing numbers of claimants over time relative to the comparator areas and 15.4% of the indicator values showing an improvement, that is, a reduced number of claimants. In contrast, areas with a type C approach to CE performed particularly well relative to their comparators, as did NDC areas with a type 2 local programme (where the performance was good regardless of the CE approach adopted). However, when a type B approach to CE, which resulted in good performance overall, is combined with a type 3 local programme, the resulting group of areas record more deteriorations in terms of employment than improvements. The group of NDC areas combining the best-performing approach to CE (type B) with the best-performing local NDC programme (type 2) has the highest proportion of improvements in worklessness and unemployment (on eight of the 10 indicators) and no deteriorations. The performance of areas with type 2 local programmes is consistently good, except when combined with a type A approach to CE, when there are few changes and all of them reflect an increasing number of claimants.

The DWP provide estimates of the numbers of people in low-income households in NDC areas by combining data on those in receipt of IS, JSA-IB or ESA with counts of the number of dependent partners and children. Trends are summarised in Table 27.

Overall, 23 (59%) of the NDC areas show significant changes in low income relative to their comparator. Of these, 17 are improvements (numbers of people on low income dropping) and six are deteriorations (numbers of people on low income growing). This ratio of improvements to deteriorations is found across all local programme types, although type 1 local programmes perform slightly better than the other two types of programme. Areas adopting type C approaches to CE perform better in terms of the reduction in the number of people on low income relative to their comparator than those adopting other CE approaches, although areas with type A and type C approaches to CE have ratios close to the average. There are few changes in the numbers on low income in areas with type D approaches to CE. Two of the combinations, containing three or more NDC areas, stand out in terms of change in the number on low income. On the positive side, four of the five NDC areas combining a type 3 local programme with a type B approach to CE, initially resident led but weakening over time (combination group B2), saw reductions in the number on low income relative to their comparators. In contrast, all three NDC areas combining a type 3 local programme with type A resident-led CE (group A3) deteriorated relative to their comparators.

Summary and conclusions

Tables 28 and 29 summarise the most notable trends in the indicators of health, worklessness and low income by NDC approach to CE and type of local NDC programme.

In relation to hospital admissions there were no noteworthy results when areas were grouped by type of approach to CE (there were few or only average levels of change). In contrast, NDC areas with type 2 local programmes performed well on both the hospital admission and work-limiting illness indicators, NDC areas with type 1 programmes improved only in terms of a reduction in the numbers on benefits because of a work-limiting illness, and NDC areas with type 3 local programmes deteriorated on both hospital admission and work-limiting illness indicators. Trends from these two sets of health indicators are difficult to compare because of the differences in the numbers of significant results: 29.1% for the hospital admission indicators and 64% for those based on claimant counts. The two sets of indicators would not be expected to detect identical trends as they address different aspects of ill health in different age groups. Nevertheless, there are only six (of 39) NDC areas in which the two sets of indicators produce consistently contradictory results.

The areas combining NDC type 2 local programmes with a type B approach to CE (combination group B2) performed well on both hospital admissions and work-limiting illness indicators. Combinations C2 and D1 performed less well on the hospital admission measures, but well on the work-limiting illness claimant count measures.

Indicators of unemployment, worklessness and low income are among the more robust measures in terms of numbers in each area. They do, however, have other limitations, particularly their vulnerability to political changes in the eligibility criteria for the benefits, which may reduce the numbers on incapacity-related benefits, irrespective of whether or not there has been a real decrease in ill health. In relation to CE, areas with type A approaches to CE perform worst on unemployment and worklessness indicators. Areas with type 3 local programmes also perform poorly, regardless of their approach to CE. Therefore, when type B CE (which performs well overall) is combined with NDC type 3 local programmes the resulting group of areas records more deteriorations than improvements on these indicators. The combination of type B approaches to CE, which performed well overall, with type 2 local programmes (combination B2) has the highest proportion of improvements and no deteriorations. NDC areas with type 2 interventions performed consistently well relative to their comparators on all of the indicators of unemployment, worklessness and low income except when combined with type A approaches to CE, when there are several changes, all deteriorating.

As noted earlier, the trends for the proportion of people in low-income households do not precisely replicate those for unemployment and worklessness, although the two are never contradictory. Two combinations of programme type and approach to CE with three or more NDC areas stand out in terms of the low-income indicator: group A3, in which all three areas show deterioration, and group B2, in which four of the five areas improve. None of the other combinations is notable for having high ratios of deteriorations to improvements.

Table 30 summarises the relative performance across all of the indicators in all groups of NDC areas. One combination stands out: in the five NDC areas combining a type 2 local programme with a type B approach to CE, six of the seven groups of indicators show improvements. In contrast, the eight NDC areas combining a type 2 local programme with a type A approach to CE were more mixed and the three NDC areas combining a type 3 local programme with a type A approach to CE saw almost all indicators deteriorating.

There are two caveats to these results. First, there is considerable variation within the groups. This can be clearly seen in Table 65 in Appendix 2, which reports the coefficient values, and the tables in this chapter showing the proportions of improving and deteriorating trends. Second, the results could be influenced by initial baseline differences between NDC areas and comparators or changes in the socioeconomic characteristics of the resident population unrelated to the NDC local programme during the period analysed. To test this assumption we compared the age and socioeconomic composition of NDC areas and their comparators using weighted census data for 2001 and 2011. In relation to age differences, we asked, among other things, whether or not an age effect may have contributed to the relatively better performance of areas with NDC type 2 local programmes and found that, although populations in these areas were younger at the start, the age difference did not increase over time. Differences and changes in four sociodemographic characteristics were also explored: housing tenure, household composition, social class and ethnicity. Although NDC areas and their comparators differed in sociodemographic characteristics at the start of the programme, these differences changed relatively little between 2001 and 2011, although there is some evidence of the ethnic and social class make-up of the NDC areas diverging from that of their comparators over time. Changes over time in the social class composition of populations were particularly apparent in NDC areas adopting type 1 local programmes (in line with a prominent aim of these programmes being to change the tenure mix, bringing in more owner-occupiers). However, this does not appear to be reflected in the performance of this programme type, which was generally poor relative to that of other programme types.

Introduction

Although not offering evidence of direct effects, findings from our qualitative research with NDC residents and past workers provide insight into the processes underpinning different types of approaches to CE and highlight potential pathways to social and health outcomes. As there are only a small number of fieldwork areas allocated to a particular CE type and small numbers of informants from each area who were working in roles that could make them identifiable, reference is made to the NDC CE type (i.e. A–D) and whether the informant is a former NDC staff member or a resident, and not to the individual NDC area. A unique identifier is provided for each respondent [e.g. JT3c refers to the initials of the researcher, the fieldwork site (1–11) and the individual participant at each site (a–e)].

Community engagement, trust and cohesion

In all areas people interviewed reported a myriad of issues in relation to the local community at the start of the NDC initiative that could be expected to impact on processes of engagement. These included the presence of multiple faith and cultural identities, living on an estate with minimal social cohesion or towns with strong cohesive identities. Levels of trust at the beginning of the NDC initiative – both between residents and in agencies – were also reported to be relatively low in all areas regardless of their approach to CE.

In areas allocated to type A or type C approaches to CE, informants reported how the NDC approach to engagement had sought to generate opportunities for residents to get involved and become more active in community life. This comment on a NDC programme’s aspirations for CE from a chief executive working in an area allocated to CE type A is illustrative of this:

We had this idea that you needed to create opportunities for lots of people to get involved as much as they could in things that were going on and make a contribution and so there are little bits at the top around governance; well that might have been people in governance roles on the NDC or we encouraged people to become school governors or whatever it might be; decision-making stuff.

Staff member, type A, JT3c

Similarly, a worker from an area allocated to CE type C used an example of a group of older people who had struggled to engage with the NDC programme initially to explain how local residents were supported to develop local groups and networks:

One of the community workers set up a little, you know, tea dance and all those opportunities for people to come to something that they felt more comfortable with and then there was an opportunity then to bring officers to talk to them in a way that they felt more comfortable with. They then went on to become a constituted group, they then campaigned and they got a bus shelter and, you know, they’re a real voice in the community and everybody knows them across the city . . . so we’ve got lots of examples of groups like that.

Staff member, joint interview, type C, EH1a&b

It is not possible from this research to measure the extent to which residents became more involved in the community as a consequence of the NDC programme but respondents in NDC areas allocated to CE type A or type C gave several examples of such transitions in which residents had set up community activities, had taken on roles such as a school governor or in some cases had gone on to become a local councillor. New parish council structures were also set up in one type A NDC.

Respondents in NDC areas allocated to CE type B similarly provided examples of community activities being instigated by committed residents. But for some respondents in type B areas the NDC programme lacked a more strategic approach to developing the capacity of local residents and groups, undermining its sustainability.

So, for example, we supported local people to set up loads and loads of community groups. When the NDC programme came to an end there were 50–60 community groups and there isn’t now obviously because all the money disappeared.

Resident, type B, JT9a

In contrast to the more positive accounts of community activity during the NDC programme provided by respondents in NDC areas allocated to CE types A and C, those in areas allocated to CE type D frequently commented on the way that NDC plans for redevelopment had dominated engagement activities. One resident of a CE type D area, for example, expressed disappointment that the NDC programme had lacked a more empowering ethos:

It was disappointing to me, it was really, because I really thought that was going to be . . . because of the directive of it, it was supposed to be about investing in the people and the engagement and empowerment of people within the NDC communities.

Resident, type D, SP2b

Although this NDC partnership funded community activities such as trips and festivals, both staff and residents acknowledged that the time pressures and scale of the redevelopment had resulted in a predominantly instrumental approach to engagement.

Respondents’ accounts in areas allocated to CE types A and C also highlighted approaches to engagement that sought to invest in strengthening social cohesion:

I don’t know if we had any proof, that this would be a better area for it. You just know if people are looking out for themselves and each other and people working in small groups together it is going to be a stronger area.

Staff member, type A, JT3c

I would hope that in terms of the was a bit of a sense of [neighbourhood 1 and 2] were two separate areas at the start of New Deal and I hope that the work we did [in neighbourhood 1 and 2] are perceived more as one area rather than two competing areas, ‘cause there was a little bit of rivalry at the beginning.

Staff member, type C, EH1d

According to the staff member quoted above, the type A NDC area in which he worked had historically very low levels of trust and social cohesion between different groups in the population. This had led to deliberate activity with community groups and members to work more closely together:

The tensions that used to be there perhaps between the older white and Asian [populations] are kind of just disappearing a little bit. I think NDC was something to do with that; I think we did quite a lot of work with older white people as well.

Staff member, type A, JT3c

In other areas examples were given in which the wider NDC redevelopment activities were perceived to impact negatively on community cohesion. For example, one resident of a NDC area allocated to CE type D expressed disillusionment about the impact of the new housing development on the community:

It [is] like with houses built and everything they are all facing outwards; those people don’t come in; they get in their car and they drive out; it’s a transient community. Their money is spent outside of this area so it doesn’t benefit this area.

Resident, type D, SP2b

A worker in a NDC area with a type B approach to CE similarly suggested that the redevelopment in the neighbourhood impacted negatively on the community’s identity:

I just find it tragic; looking at it now I do find it absolutely tragic and some of the community’s comments during the time about designing out crime families and removing the heart of the community . . . I’m seeing it; I am absolutely seeing it; it’s so sad.

Staff member, type B, SP11d

However, not all respondents viewed area redevelopment negatively. Another resident in the same type D area as the resident quoted above commented on the same development more positively:

So although they are living in a new housing environment they have still got the neighbourly feel that they had when they lived there before because they have still got their next door neighbour that they had lived next to for 25 years or so; and there is a lot of new people and a lot of younger people moving into the area.

Resident, type D, SP2d

Finally, many NDC areas experienced significant population change during the life of the programme with high numbers of new residents moving in (and others moving out). There were some suggestions in the qualitative data that those NDC partnerships that were guided by a community empowerment ethos (types A and C) may have been better positioned to support and respond to new residents’ needs and involvement, so helping to reduce tensions. However, the qualitative data available on this are very limited and do not allow firm distinctions to be drawn between CE types.

Contribution of engagement to the New Deal for Communities outcomes

Resident informants from NDC areas adopting type A and type C approaches to CE generally reported positive changes in their neighbourhoods and felt that aspirations for the regeneration of their areas were largely achieved. These improvements covered a range of areas including major structural changes in the neighbourhood and improvements to housing, streets and estates as well as the development of shopping areas and open/green spaces. Other improvements arose from investment in new facilities and amenities in the area, such as local businesses, community hubs and services. For some residents, being involved in producing these improvements had generated a considerable sense of pride:

I think with NDC erm I was physically involved and everything was, it was physically, mentally everything, you were involved in it all and you feel proud because you’ve had, you‘ve taken part in something good, something that, you know, there’s an awful lot of these projects have come to fruition and you can see, you know, you can actually see the difference that it’s made.

Resident, joint interview, type A, EH4d&e

Such improvements were often contextualised in terms of addressing the consequences of long-term decline and high levels of deprivation. For example, in one instance both residents and workers presented a bleak picture of their NDC area (which was allocated to CE type A) at the start of the NDC programme, describing rapid economic decline after a major industry closed down and characterising the NDC area as physically neglected and unsafe with high rates of crime, social problems and poverty. A resident living in the area for decades described his disillusionment over this decline:

I have lived here a long time and I saw it when it was called ‘Corn[ed] Beef Island’, that was just after the war because all the houses were all neat and tidy and had hedges around them and corn beef in them days it was a sort of a luxury . . . I have watched it go from that and I have watched it go downhill really, really badly.

Resident, type A, JT5c

Going on to reflect positively on the changes that the NDC programme achieved, this resident highlighted the importance of residents’ involvement in the achievement of these changes:

I am quite happy now, quite pleased that we have got this far and we’ve done what we’ve done . . . If you get plenty of likeminded people I am sure you can make a difference to an estate.

Resident, type A, JT5c

Similar reflections on positive change were evident among informants in type C areas. There was, however, more evidence of mixed views between informants in these areas over tensions concerning strategic decisions related to the housing stock or redevelopment plans. For one participant this was perceived to have led to a declining focus on the community:

It moved away from gathering the views of the people and acting on the views of the people to involving the people in New Deal for Communities. So I might actually say that New Deal for Communities became more important than community engagement. And I . . . thinking now . . . I’m actually very clear about that.

Resident, type C, EH1c

Although informants in NDC areas with a type B approach to CE did report positive impacts resulting from the NDC programme, greater disillusionment about aspirations having not been met was expressed, with improvements perceived to lack sustainability. In one area, for example, resident informants spoke positively about the impact of a warden scheme. However, the fate of the scheme was perceived to reflect an absence of attention (or commitment) to sustaining the programme after NDC funding ended:

Some of them are sadly missed; our Warden Programme for example, our Community Wardens; absolutely loved by everybody but unfortunately the city wouldn’t continue it after the programme; in fact the city didn’t continue anything after the programme; everything went . . . It was very noticeable as well; everything just shut overnight.

Resident, type B, JT9a

Major redevelopments (often undertaken in NDC areas with type D approaches to CE) resulted in less-positive views of the success of the NDC programme. One factor for this was the lengthy timescales and scale of disruption caused by the physical regeneration. In one type D NDC area, more than 1000 houses were demolished with the intention of constructing at least 1500 new homes, but much of the rebuilding remained to be completed by the end of the programme. One resident expressed a particular frustration with the process:

We were highlighting these kinds of things and saying you are going to have this because you are going to have pockets of wasteland, it’s just going to attract fly tipping and unless you do something to stop that what are you going to do is; . . . So when people were saying these things it was like ‘oh no, that’s not going to happen’; but it has.

Resident, type D, SP2b

Some residents in other NDC areas where physical regeneration involving housing demolition/clearance was also planned reported similar experiences of stress and anxiety. For a resident of a type B NDC area, for example, this was attributed to high expectations and subsequent disappointment over the timescales for change:

One thing that residents don’t realise, I didn’t myself for a long time, is how long it takes to go from planning a clearance and rebuild of an area to actually doing it; all the legalities and the time and everything that’s involved . . . I think this is one of the main disappointments that I think residents should have been informed from the outset that there could be a very long period.

Resident, type B, JT9b

In a type C NDC where more modest redevelopment was planned, one resident explained that the NDC staff and resident board members experienced hostility from the wider community regarding the uncertainty of redevelopment plans when housing had not been delivered as ‘promised’ by the end of the programme. However, the same resident informant also suggested that housing and environmental improvements achieved alongside these planned housing redevelopments helped to mitigate the negativity when the new housing plans stalled.

The impact of political and policy contexts on approaches to engagement

All NDC areas were operating in a continually evolving policy context and experienced many external pressures – local and national. The impacts of these pressures may have masked any benefits that could have accrued from adopting particular approaches to CE.

Locally these challenges could include working relationships between the NDC partnership and the local council. Some respondents reported tensions because residents believed that the NDC money was ‘their money’ and were reluctant to share control of this with local agencies. As the following quote illustrates, some workers felt that the lack of experience among residents in planning for, managing and spending such large sums of money was a major problem, at least in the early stages of the programme:

Because at the end of the day government put so many barriers in the way of the, you know, you were never going to go into an area with £53 million and say ‘here’s a suitcase, spend it’ so you know there was always going to be so many strings attached to it and so there had to be structure and somebody had to be accountable and all of that and actually that wasn’t what the community thought they were buying into in the early days.

Staff member, joint interview, type C, EH1a&b

Conversely, there were other areas where council representatives refused to work with the NDC members, also creating tensions, as is evidenced in the following quote:

I think I’ve still got regrets about the fact that we just couldn’t get the councillors and the resident[s] . . . to talk to each other for the last 3 years because of this schism which was largely about one councillor . . . we never really understood it, but he just developed a visceral hatred of everything to do with NDC and just and it was a very difficult situation for the last 3 years, I regret that and the fact that that then led to, distorted some of the programme in the last few years because things that we wanted to do, it was almost opposition for its own sake.

Staff member, type A, EH4a

Changes at the national level also added pressures and complications to the way that the NDC partnerships worked. There were changes in emphasis in the way that the NDC programme was managed by central government, with the government introducing new policies and institutions that were felt to have added to the burden of the NDC (e.g. local strategic partnerships). Respondents were also aware that over time the NDC programme lost importance and visibility within central government, a process also commented on by the NET:52

I think early doors it was very much a community leading the NDC programme and I think over time perhaps some of the rhetoric at a national level changed, communities being part of that.

Staff member, type C, EH1d

In the early years of the NDC the emphasis was on planning for a 10-year delivery programme but over time the emphasis changed with pressure from central government to ‘quick wins’ to demonstrate impact. There was a greater emphasis on performance management and NDC partnerships had to demonstrate how they had spent their money each year (personal communication from public advisers confirmed by recent research public adviser workshop, May 2013). The pressures that this created for NDC areas are summarised by a resident in a type A NDC:

And it’s easy to forget these kinds of things but constantly through NDC there was (a) question of whether it would continue, (b) whether any particular NDC would continue, whether it would get stopped because it was mismanaging its responsibilities. Those things were ever present real threats erm, you know, once you’ve got to the end you can quickly forget those things but that’s the environment we were operating in. Not only that but also one which was contradictory at times; when I was first involved I went to National Conference and the only message we’d been given was ‘do your needs assessments before you spend money for 10 years’ and a year, 2 years in for first phase NDCs and whoever was the minister was saying ‘spend, spend, spend’ because there was this pressure to see results and so we were getting these contradictory pressures.

Resident, type A, SP6e

The extent to which individual NDC areas were able to negotiate these challenges will have been dependent on a number of factors, including the commitment of the various agencies to the local programmes and the strength of the NDC board and key individuals. For example, the resident quoted above noted how the chief executive in their NDC area with a type A approach to CE urged the board to ‘hold our nerve’ and they continued to place emphasis on the development of the overall programme during the time when there were pressures for ‘quick wins’. In another NDC, a respondent described how the board had placed greater emphasis on spending, in keeping with national shifts in emphasis:

Because the pressure throughout was spend, spend, don’t really matter about what you are spending the money on; this is your allocation and it’s got to be spent and we were very good at doing that. And I’ve kind of always put, my strategy when I was chair was spend, spend, spend and try and grab everybody else’s money that they hadn’t spent and how we’ll spend it and I’m not always sure that it was spent in the best possible way but we were very good at spending money and the government seemed to like that, you know. I’m not really sure that was what it was all about but that’s what it ends up being about and it’s almost projects for projects sake, we were just chucking stuff and desperately trying to do all these things, all at the same time with and still generate the reports the government wanted in terms of the changes we were making, you know, a huge pressure for that.

Resident, type C, JT8b

By the time that the NDC programmes were drawing to a close, the economic crisis meant that there were limited funds available in the agencies to support NDC succession plans:

Yeah, oh definitely, yes, yeah, the whole climate has changed really. The need hasn’t changed but obviously erm, you know, in terms of re-contract, we were quite successful in re-contracting the NHS and the borough and all of that comes to an end next year and new tender arrangements, local commissioning groups huge sea change locally erm so I think every voluntary sector organisation at the moment is just really rethinking [laughs] its sustainability strategy and what’s going to happen past April next year so I’m hoping that we are going to, you know, be able to survive through that.

Staff member, type C, JT8c

These external challenges to the operation of NDC partnerships will almost certainly have impacted on their strategy for, and implementation of, CE and on their activities in other outcome domains. In turn this can be expected to have influenced the health and social outcomes that local NDC programmes were able to achieve. However, although national pressures will have affected all of the NDC areas, the interaction with local contexts will have meant that the NDC areas had different challenges and will have managed their particular challenges differently. Unfortunately, our qualitative data are not able to illuminate how these processes unfolded in areas with different types of local programmes and different approaches to engagement.

Summary and discussion

Findings from our qualitative research confirm that NDC areas that adopted type A and type C approaches to CE were most closely associated with an approach underpinned by empowerment and community development values. There was also evidence in these areas that resident involvement had helped to achieve demonstrable improvements related to social and physical assets in the neighbourhood. However, it was residents in NDC areas with type A approaches to CE who most consistently reported that a positive difference had been made through the involvement of local people in the change process, and those closely involved reported pride from being part of achieving these changes. Respondents in type B NDC areas described how a resident-led ethos fell away during the course of the NDC programme and that the agendas of external agencies involved, such as the council, came to dominate. In contrast, type D NDC areas represented a model of NDC that predominantly engaged the community around the external agency’s agenda, often as a consequence of progressing major redevelopment plans. Although not directly related to ‘engagement’, residents from NDC areas undergoing major redevelopment often reported on the upheaval and anxieties that ensued in the wider community. This was not necessarily just because of the immediate disruption caused by demolition and redevelopment (sometimes seen to be an expected part of the process) but because of the very lengthy timescales it took for redevelopment to be completed and subsequent delays in the latter years as a result of the housing market downturn.

The quantitative impact analyses reported earlier (see Chapters 3 and 4) suggest that trust, social cohesion and involvement indicators were higher in 2008 in areas with type A approaches to CE than in areas with other CE types. The qualitative evidence appears to support these findings but suggests that there are potentially different explanations for why a type A approach to CE, and to a lesser extent a type C approach, may have been more likely than other approaches to CE to have contributed to these positive outcomes.

First, respondents noted that NDC areas adopting type A and type C approaches to CE invested more heavily than other areas in activities that deliberately sought the empowerment of the wider community as well as residents actively involved in the NDC programme. These NDC areas employed community development teams that nurtured local grassroots community action as well as undertaking outreach in communities in an attempt to develop trust and encourage involvement. In this regard, type A and type C engagement processes may have had direct and indirect effects on social cohesion by bringing the community together, investing in the community infrastructure and increasing collective action to address issues in the neighbourhoods. An alternative explanation is that areas that adopted a type A approach to engagement had a starting point that was enabling to support a resident-led ethos. Chapter 5 highlighted, for example, based on the longitudinal data, that trust in neighbours was higher in areas with a type A approach to CE than in areas with other approaches to CE. However, our qualitative data suggest that these results should be interpreted carefully as the findings suggest that most NDC areas started from a position of relatively low levels of trust between residents and in local agencies.

The qualitative research also points to potential impacts of the NDC programme that are not readily detected in the quantitative analyses that we have undertaken. One issue is the extent to which changes achieved by NDC local programmes could be sustained beyond the end of the initiative. In areas with type A and type C approaches to CE, the legacy of resident engagement with the NDC was typically reported by respondents to lie with the residents and the community, including residents who felt that they had a stronger voice to challenge decision-making by statutory organisations. Respondents in NDC areas with type A approaches to engagement were also more likely to report that their NDC area had invested in structures that supported longer-term regeneration beyond the NDC funding period, such as social enterprises or community assets that would deliver a financial return. Although the scale of such ventures appears relatively modest (with returns likely to have been affected by the economic downturn), these activities suggest an approach strongly orientated towards a model of engagement/empowerment aiming for greater social and economic security for the community living within the local area.

Introduction

In this chapter we report on the cost-effectiveness analyses that we have undertaken. However, before presenting these analyses there is an important caveat to be made. The right to participate is one of the building blocks of a democratic society. The presumption of the exercise reported here is therefore that CE in a democratic society is universally a good thing. Such a presumption is not subject to economic analysis (see Okun76 1975). The issue that we are concerned with here is only whether or not some approaches to CE as a vehicle for improving the community’s health are better than others. Cost-effectiveness analysis compares the relative costs and outcomes (effects) of two or more courses of action. It differs from a cost–benefit analysis in which a monetary value is assigned to the measure(s) of effect(s). Typically, cost-effectiveness is expressed in terms of a ratio in which the denominator is a gain in health from a measure (e.g. years of life, premature births averted, sight-years gained) and the numerator is the cost associated with the health gain. The course of action with the highest ratio is the most cost-effective.

This is the procedure we followed: we did not attempt to monetise – put a monetary value on – the effects of (different forms of) engagement/involvement. In the analyses reported we considered the comparisons between NDC areas and their comparator areas; different types of local NDC programmes; and different types of approaches to CE. The work was divided into three phases: assessing cost, measuring effectiveness and comparing cost and effectiveness.

Assessing the cost of community engagement

We explored two ways of assessing the cost of CE. First, we analysed the expenditure data relevant to CE collected through the ‘official’ NDC accounting system. Second, we examined documents and interviewed people to assess the extent of contributions of volunteers during the period of the NDC programme. The first approach to assessing the costs of CE, reported in Analyses of the Hanlon System K expenditure data, is based only on money transactions – the expenditure data entered into the accounting system. The second approach takes an ‘economic’ perspective, attempting to take into account all of the resources involved in what is meant to be, at least partly, a community-driven process. More specifically, the economic approach involves trying to find monetary value equivalents for the in-kind resources committed to CE in NDC areas, for which no charge (or a highly subsidised charge) has been made, and find ways of accounting for the value of voluntary labour time input. The latter raises the issue of the opportunity cost concept used by economists. These issues are dealt with in Costing in-kind and volunteer time inputs.

Analyses of the Hanlon System K expenditure data

The Hanlon System K was developed to monitor the NDC and associated project expenditure. Expenditure data were collected for all NDC areas from 2004 to 2005 (and retrospectively for some of the NDC areas for previous years). Initially, data were collected at the level of the six outcome domains (community involvement, crime and safety, housing and physical environment, education, employment and health) but this was seen as insufficiently detailed by the evaluation team and 70 specific project types under these headings (with four other miscellaneous categories) were defined and used both retrospectively and prospectively for accounting purposes. The accounting system also includes codes for indicating when a project type contributes to another subsidiary outcome domain so that one can analyse expenditure on the community involvement domain, or expenditure on community-related projects, and restrict the analysis to NDC expenditure or NDC expenditure plus expenditure related to the NDC projects.

Our analyses considered the variability in spend related to CE across the 39 NDC areas and whether or not there were systematic differences between the NDC areas that spent the least, and those that spent the most, in each of the six outcome domains and each of the 74 project types, focusing in particular on activities that are most likely to include CE with the potential to impact on health and/or inequalities. As the following examples illustrate, the patterns of reported percentage spend by each of the NDC areas across the six outcome domains areas varied widely:

• community development: from 7.9% in Knowsley to 41.3% in Walsall

• crime and community safety: from 4.1% in Hackney and Hull to 19.7% in Islington

• education: from 4.4% in Knowsley to 29.5% in Sunderland

• employment and business: from 4.1% in Haringey to 19.3% in Nottingham

• health: from 1.7% in Rochdale to 21.1% in Aston

• housing/physical environment: from 2.3% in Southampton to 47.0% in Manchester.

Areas in the North West were much more likely to spend on housing. Otherwise there were smaller differences with the Midlands spending more on community involvement and employment, the North West spending more on health and the North East spending more on education (Figure 3).

FIGURE 3.

Ratio of spend on each of six sectors in each of five regions compared with the total ( = 100). E, North East; L, London; M, Midlands; S, South; W, North West.

Table 31 shows the percentage spend on each of the six main outcome domains broken down by groups of NDC areas according to (1) their approach to CE and (2) the type of local NDC programme. The highest percentage spend in all areas was on housing, except in the group of areas adopting a type B approach to CE and the group of areas adopting a type 2 local programme. In these two groups the highest average spend was on activities under the community development heading, the second highest spend in all other areas. NDC areas with a resident-led approach to CE (type A) or which began with a resident-led approach that weakened over time (type B) spent about half as much again on activities labelled as community development as areas with type C and D approaches to CE. The average expenditure under the community development heading across NDC local programme types also varied, from 17.3% in areas with type 1 local programmes to 23.3% in those with type 2 local programmes. The highest average percentage spend under the community development theme (25.6%) was in the group of five NDC areas combining a type 2 local programme with a type B approach to CE. The breakdown of expenditure for each NDC area is provided in Table 67 in Appendix 2.

TABLE 31 - Percentage spend on each NDC outcome area by CE typology and programme type

CE/programme type	Community development	Crime and community safety	Education	Employment and business	Health	Housing/physical environment
CE type
A (13 areas)	21.5	9.7	11.7	11.4	8.2	21.9
B (13 areas)	22.3	11.0	13.6	9.5	7.4	16.8
C (eight areas)	15.4	7.8	14.4	7.4	8.2	29.7
D (five areas)	14.9	7.4	11.8	8.0	6.6	26.1
Local programme type
1	17.3	8.0	12.4	8.4	7.0	24.5
2	23.3	10.3	10.4	8.8	7.3	23.0
3	18.2	10.5	15.6	11.0	9.2	18.8

Although information about spending across the broad outcome domains provides some information on the costs of CE, the range of project types included within each domain was very broad (with the partial exception of worklessness). In particular, it cannot be assumed that all of the costs of CE were reported under the community development heading. It was not possible, however, to identify with any precision expenditure contributing to CE but reported under other outcome domains.

More information about the nature of expenditure on CE is provided by looking at how the total spend is spread across specific projects within the community development outcome domain. As the data in Table 32 show, perhaps not surprisingly the highest expenditure was on capital development of community facilities, but substantial amounts were also spent on involving local people and developing the skills and infrastructure of the community: £32M on general capacity building, £19M on communications, marketing and raising awareness, and £9M on capacity building relating to the governance of NDC partnerships.

TABLE 32 - New Deal for Communities community spend according to the 10 categories accounting for most spend, 1999/2000 to 2007/8 (current prices)

Note

Some categories fit into more than one of the NDC’s six outcomes; this table displays only expenditure identified as relating to the community outcome and therefore not necessarily the total spend for each category.

Source: Hanlon System K expenditure data.

Expenditure category	NDC community spend (×1000) (£)	% of all NDC community spend
New/improved use/access to community facilities	45,774	18
Capacity building general	32,072	13
Community development workers/officers	26,764	11
Reports/research/studies/professional fees	18,898	8
Promotion/communications/marketing/raising public awareness	18,873	8
Community Chest – general/youth	18,444	7
Youth support/services provision	10,244	4
Capacity building NDC governance	9318	4
Land/asset acquisition/demolitions/stock transfer	7162	3
Other NDC posts	6561	3

Calculating cost-effectiveness

The difficulties of collecting cost data have been extensively documented in the previous sections and the results in Table 33 do not show any systematic pattern. Importantly, we thought that there would be more information available about those who were engaged in different activities for us to be able to confidently extrapolate from the 11 fieldwork sites to the 39 areas; instead, the only possibility was to apply the estimates in Table 33 for the 11 fieldwork sites directly to the other 28 areas based only on the local programme type and CE typology. For this reason, despite the effort put into collecting the bottom-up data, the System K data have been preferred for the purposes of calculating cost-effectiveness, and the pattern of spend within each outcome area is taken as a marker indicator of the approach within that area. The cost-effectiveness analyses therefore had two components:

• an analysis of the administrative outcome data reported in Chapter 6 compared with System K expenditure data and the CE typology

• an analysis of the MORI outcome data reported in Chapter 4 compared with System K expenditure data and the CE typology.

The NET’s final81 report used respondents’ assessments of their overall quality of life and specifically their satisfaction with the area as the measure of final outcome in assessing value for money of the whole programme. Given our specific focus on CE, we have concentrated on health and social cohesion outcomes, and on hospital admission indices for drugs, alcohol and mental health-related conditions. However, given the strong relationship between feeling safe when walking alone after dark and the quality of life score, we have also included four marker crime indicators.

Summary and conclusions

Introduction

The research reported here has built on our previous study funded by the Policy Research Programme,1 which explored the impact of the NDC programme on health inequalities and their social determinants. The NDC was a major neighbourhood regeneration initiative implemented in 39 of the poorest neighbourhoods in England at a cost of around £2B over 10 years from 1999/2000. Our NDC study found a range of social and health outcomes associated with different types of local NDC programmes. The study reported here has explored what, if any, outcomes related to health inequalities and their social determinants are associated with different approaches to CE within the NDC and considered what, if any, contribution these CE approaches made to the differential outcomes that we reported for NDC local programmes types.

In this final chapter of our report we identify some of the limitations of our research. We then move on to consider the new insights that we believe this study has to offer to research evaluating interventions involving some element of CE and aimed at reducing health inequalities. Lastly, we consider some of the implications for research and policy.

Limitations of the research

Our research has used rich cross-sectional and longitudinal survey and time series data sources compiled by the NDC NET to develop an innovative approach to the evaluation of the social and health impacts of different approaches to CE implemented within the same policy context, that is, implemented in the 39 local programmes that made up the national NDC regeneration initiative. However, there are several limitations to the data and methods that we used.

The outcome measures available to us were more limited than anticipated because the form of some measures in the MORI survey meant that we were unable to use them (e.g. measures of income) and because of constraints imposed by the size of the MORI longitudinal panel. Additionally, available measures of the experience of engagement and of community control and influence, and data for the effectiveness work, were particularly limited. On the costing side the main limits arose from inconsistencies in the volume and quality of available documentation, the retrospective nature of documentary data and recollections from a small number of interviewees. On the effectiveness side, for example, as noted above, we were not able to capture quantitatively some dimensions of the experience and impact of engagement expressed qualitatively by interviewees. Additionally, although there were some positive findings from the cost-effectiveness analyses, there was an almost equal number of negative findings and there was no obvious metric for combining them.

We were also aware of other limitations commonly associated with the use of secondary data sources: the fact that they were generated for other purposes; the often inadequate meta-data describing how the data have been collected and from whom; and their precise coding under different circumstances. In particular, some of the data sets (e.g. HES) have confidentiality restrictions; others (e.g. MORI) arise from the limitations of self-report data such as socially stratified reporting biases.

Bias may also have been introduced into our findings through residential mobility. Some of our results are based on repeat cross-sectional studies measuring change over time in different people; other analyses assess within-person change over time. The use of cross-sectional data reduces the problem of attrition found in the longitudinal data set but means that differences in the sample and residential mobility could contribute to the observed trends over time. To address this we undertook sensitivity analysis, excluding those who had recently moved and adjusting for a range of demographic characteristics to make the samples as similar as possible. For the longitudinal data, we compared results with and results without respondents who had recently moved. This comparison suggests that resident mobility effects seem to have been limited, at least in terms of the main outcomes of interest for this project. In all of our analyses we also adjusted for a range of demographic characteristics to help compensate for differences in the samples.

Findings from our longitudinal and cross-sectional analyses do not always agree. Although the sensitivity analyses with and without movers do not point to substantial bias when movers and non-movers are included, they do not throw much light on why this is the case. Numerous other studies show that longitudinal and cross-sectional approaches to estimating change frequently yield different estimates and different conclusions about the size and direction of change. Although longitudinal data are typically considered more precise within epidemiology, it is not necessarily the case that either of the two MORI data sources provides stronger evidence of impact in this study. Changing the profile of the resident population (e.g. through mixed-tenure redevelopment) was an explicit aim of some of the local NDC programmes, and between 2002 and 2008 there was substantial population turnover in some areas. The NET did attempt to follow up people moving out of NDC intervention areas but this was too expensive to pursue. Differences in the way that the health and social cohesion outcomes were treated in the longitudinal and cross-sectional approaches may also have contributed to different results across data sources. In the longitudinal analysis we analysed latent factors of social cohesion, which take better account of the measurement error associated with combining multiple items of trust in the community, trust in local services, mental health, etc. Despite the difficulties of interpretation introduced by using both longitudinal and cross-sectional data in the same study, this triangulation undoubtedly helps evaluations such as ours to capture more of the complexity of impacts associated with interventions such as the NDC in general and the approaches to engagement in particular.

As described in Chapters 3 and 7, alongside the secondary data that we used we also collected new primary qualitative data from interviews with residents of NDC areas and past workers. Although these data provide important insights into the processes of engagement, discussed further in the following section, the heavy reliance of our CE typology on the recollections of interviewees is a potential weakness. These recollections could have become distorted over time, particularly in NDC areas adopting a type D approach to engagement. These areas all involved redevelopment initiatives, which were more likely to have stalled after the economic problems of 2007 slowed the economy and in particular private sector house building. Additionally, being restricted to those residents and past NDC workers who were easiest to contact would have inevitably biased our sample of interviewees to those who were most engaged in the NDC (positively or negatively). Documents, another key source of data for the typology, also have problems of bias, typically being written to meet the requirements of particular audiences.

A final limitation was that we faced the usual challenges of developing a theoretically robust approach to conceptualising ‘community engagement’, challenges that are well documented in existing literature. As in all qualitative research, another team may have made very different decisions on this front.

New insights from the study

Our evaluation of NDC approaches to CE has sought to answer five research questions:

1. Which approaches to CE effectively engage which social groups in NDC populations?

2. Do different approaches to CE have different health and social outcomes for NDC populations?

3. Does the association between these outcomes and NDC approach to CE vary across groups defined by age, ethnicity, gender and material circumstances?

4. Do different approaches to CE have any impact on the health gap between NDC areas and areas from across the socioeconomic spectrum?

5. Does the approach to CE help to explain any of the differential outcomes of local NDC programmes identified in our previous research?

We also undertook an exploratory economic evaluation of the cost-effectiveness of different approaches to engagement in the NDC areas.

In seeking to answer these questions our evaluation has provided new methodological insights relevant to the evaluation of complex social interventions and new evidential insights into the pathways that might be expected to link CE in decision-making to positive social and health outcomes, and the factors that may support or undermine engagement processes. The study also provides insights into the processes of public engagement in evaluative research in the public health field, a field in which public engagement in research is relatively underdeveloped.

Research implications

Our research represents a serious attempt to advance theoretical and methodological thinking on how complex social interventions involving CE can best be conceptualised and evaluated for their health and health inequalities impact. This included innovative use of secondary data sources in order to distinguishing different types of CE and different ways in which interventions are operationalised in different contexts. We have also tested ways of constructing comparators from across the social spectrum and explored ways of costing in-kind contributions to engagement. This work now needs to be taken forward and refined.

The relative lack of statistically significant results at a population level is not unusual in this field. For example, two recent reviews89,90 of diverse ‘CE’ interventions found that, although trends in quantitative evidence suggested that different approaches could have different impacts, the lack of statistically significant results made it impossible to conclude that a particular approach was more ‘effective’ than another. This points to the need for research to explore the sources – theoretical, conceptual and methodological – of this ubiquitous uncertainty in evaluations of CE interventions in the public health field. It also points to the need for evaluative research to be more tightly focused on the effectiveness of engagement, running concurrently with engagement processes (i.e. factored into the programme from the beginning), and to the need for methodological research on measures of community control and influence and on the economics of CE.

Our research also points to the importance of designing evaluations of CE interventions that are able to enhance understanding of the shaping influence of context and implementation processes. We have argued that the patterns of impacts that we have identified, relating in particular to community cohesion, control/influence and mental health, are consistent with theoretical understandings of the potential positive impacts of empowerment approaches to community engagement. In particular, the divergent patterns we have identified associated with a type A empowerment approach and a type D instrumental approach fit with a theoretical position which posits that if people are engaged only around the system's agenda rather than their own,86 community engagement approaches will have relatively little positive impact and may actually undermine some dimensions of social cohesion and well-being.

We have shown that a majority of the 39 NDC areas began with a commitment to empowerment and a resident-led model of change but that over time many were diverted to a more instrumental approach. In research on other regeneration initiatives, Lawson and Kearns91 have highlighted how empowerment has been used as a label that has ‘enabled other stakeholders to legitimate what they wanted to do in furthering more important objectives’ (p. 78). Head61 similarly concludes that engagement has rarely demonstrated increased power sharing with communities. This issue is highly relevant to the NDC initiative. Other researchers evaluating the NDC have argued that the initiative was ‘a tightly controlled policy space (p. 358)51 that had negative impacts on engagement processes, as our qualitative research confirms. Interestingly, a recent realist review of the evidence on CE initiatives involving lay health workers similarly found that contexts in which professionals and/or agencies maintain control limited the ability of lay health workers to deliver culturally tailored support. 92

Our study has not provided firm evidence of the effectiveness of empowerment models of engagement, but it does suggest the need for more careful application of theory to public health interventions that aim to utilise CE as a vehicle to deliver external objectives. As Hart and colleagues93 argue, instrumental engagement may even result in an amplification of disempowerment. In this context it is important to illuminate the processes underpinning both empowerment and instrumental approaches to engagement and their impacts. To do so requires greater theoretical sophistication in the development and evaluation of CE interventions.

Contribution of authors

Jennie Popay (Professor, Sociology and Public Health) was the project’s lead investigator, jointly led the typology development in work package 1, contributed to all work packages and led the writing up of the results.

Margaret Whitehead (Professor, Public Health) was the joint lead for the typology development in work package 1, led the updating of the administrative data sets and contributed to all work packages and the writing up of the results.

Roy Carr-Hill (Independent Consultant, Economics) led the cost-effectiveness work and contributed to the impact analyses and to the writing up of the results.

Chris Dibben (Professor, Health Geography) provided advice on the NDC administrative database resource and contributed to its updating and the impact analyses.

Paul Dixon (Statistician, Independent Consultant) undertook updating of the administrative data sets and impact analyses based on these and writing up of results.

Emma Halliday (Senior Research Fellow, Public Health) undertook typology development and contributed to the conduct/analysis of fieldwork and the writing up of the results.

James Nazroo (Professor, Medicine and Sociology) co-supervised the impact analyses and contributed to the writing up of the results.

Edwina Peart (Independent Consultant, Economics) contributed to the economic evaluation including data collection and analyses, and the writing up of the results of the cost-effectiveness analyses.

Sue Povall (Senior Researcher, Public Health) undertook typology development and contributed to the conduct/analysis of fieldwork and the writing up of the results.

Mai Stafford (Reader, Social Epidemiology) co-supervised the impact analyses and contributed to the writing up of the results.

Jill Turner (Research Associate, Public Health) undertook typology development and contributed to the conduct/analysis of fieldwork and the writing up of the results.

Pierre Walthery (Research Fellow, Applied Social Statistics/Sociology) undertook the impact analyses and contributed to the writing up of the results.

Data sharing statement

The NDC secondary quantitative data used in the study are available from http://extra.shu.ac.uk/ndc/ndc_data.htm and detailed information on the approach to data analysis we used is obtained from http://dx.doi.org/10.17635/lancaster/researchdata/27. Further information about this and about our qualitative data, which is not being archived, can be obtained from Professor Jennie Popay (j.popay@lancaster.ac.uk).

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the PHR programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the PHR programme or the Department of Health.

Background and aims

The research reported here focused on the nature and scale of the NDC regeneration initiative and its distributional impact on inequalities in health and their social determinants. The overall aim of the NDC policy initiative was to bridge the gap between some of the most deprived neighbourhoods in England and the rest of the country in six outcome domains over a 10-year period: crime, the local community, housing and the physical environment, education, health and worklessness. There were 39 NDC areas, each receiving public funds of around £50M (£2B in total) from 1999 to 2011. This report builds on the work produced by the NDC NET to address the following research questions:

1. Is there any evidence that inequalities in health and their social determinants improved in NDC areas to a greater extent than in non-NDC areas of comparable baseline deprivation?

2. Is there any evidence that health and its social determinants have improved in NDC areas to a greater extent than in areas drawn from across the social gradient?

3. Do the impacts of the NDC initiative on health inequalities and their social determinants vary across types of NDC local programmes being implemented in different ‘types’ of areas, and if so what characterises NDC areas and local programmes that have seen the greatest improvements in health inequalities and their determinants?

4. What are the lessons for future initiatives aiming to reduce health inequalities?

Evaluation design and data sources

The evaluation comprised three work strands:

• Work strand 1 involved the development of measures of the local context in which the NDC programmes were implemented and a typology of local NDC programmes. For this work we used secondary quantitative and qualitative and documentary data collected by the NET, publicly available quantitative data from other sources and new primary data collected from interviews and documents in a sample of NDC areas

• Work strand 2 evaluated the impact of the NDC initiative as a whole, and different types of local programmes, on health inequalities and their social determinants. This work used data collated by the NET for NDC areas and their ‘matched’ comparator areas, selected from within the same local authority boundary and with broadly the same Index of Multiple Deprivation. These data included repeat cross-sectional and panel data from MORI surveys conducted in 2002, 2004, 2006 and 2008 for the NDC areas and their comparators; area-level time series data on hospital admissions and welfare claimant numbers for NDC areas and their comparators; and data from the HSE household surveys for 2002, 2004, 2006 and 2008 used to identify areas with high, medium and low levels of deprivation.

• Work strand 3 involved the creation of a data archive making available programming codes and data for other users so that our analyses can be replicated. This legacy will allow longer-term follow-up of the impact of the NDC on health inequalities and their social determinants, as well as comparison of health and social outcomes in NDC and similarly deprived areas. We hope that this material will be hosted as part of the NET’s archive on the Sheffield Hallam University website.

Measuring the local context in which New Deal for Communities programmes were implemented

Although all NDC local programmes had the same purpose (closing the gap in the six outcome domains between the NDC areas and the rest of the country), the form that they took varied significantly as they sought to address local needs in very different contexts. We therefore developed new measures of these contextual factors including current and historical levels of deprivation and employment, and patterns of industry and migration. However, only the deprivation measured had any predictive power, possibly because of constraints in the data that we were able to use to capture historical context.

Developing a typology of local New Deal for Communities programmes

Our typology of local NDC programmes reflects the relative emphasis that they gave to three core dimensions: housing and environment, human capital and CE. Three types of programmes were identified:

• type 1 – transforming environments: diversifying the social composition of the population through major redevelopment and changing tenure patterns

• type 2 – incremental: increasing neighbourhood resources and moderate neighbourhood redevelopment

• type 3 – strengthening people: building residents’ human capital and improving living conditions with little or no redevelopment.

The impact of the New Deal for Communities on the health of the ‘poorest’

Predictably, socioeconomic disadvantage was associated with poorer health in NDC areas, their comparator areas and high and medium deprivation areas derived from the HSE. Hospital admission rates were also much higher than the national average in NDC and comparator areas, particularly for alcohol- and drug-related conditions, and remained so over the 10 years of the programme. Most differences in health and social outcomes between NDC areas and their comparator areas were explained by the relatively more disadvantaged circumstances of populations in the NDC areas. However, there were a few exceptions, with respondents in NDC areas reporting higher levels of life satisfaction in 2002.

Markedly different pictures of changes over time are provided by our analyses of longitudinal and cross-sectional data. The longitudinal data paint a less positive picture, showing no overall trend of improvement on most outcomes in either the NDC areas or the comparator areas, with two exceptions. First, levels of satisfaction with the area actually declined over time between 2002 and 2008 in comparator areas compared with NDC areas. Second, NDC areas experienced a greater reduction in benefit receipt among lone parents than comparator areas, suggesting that the NDC programme may have added value to national programmes supporting lone parents to get back into employment.

In contrast, the cross-sectional analyses using MORI and HSE data found a general trend of improvement between 2002 and 2008 in smoking, mental health, educational attainment and employment rates in areas across the social spectrum. Importantly, levels of reported poor health appeared to decline faster in NDC and comparator areas than in areas with low deprivation and these improvements were greater in NDC areas than in areas with high and medium deprivation in other parts of the country.

There is also some evidence of a NDC ‘effect’ on the social determinants of health inequalities, with the proportion of people in NDC areas with no qualifications falling more steeply than that in the HSE low and high deprivation areas. This more rapid decline was also seen in the NDC comparator areas but these were relatively more advantaged and hence were facing a less severe challenge. Similarly, although the gap between HSE high and low deprivation areas widened on five of the six social determinants of health that we considered, only the gap in smoking widened between NDC areas and HSE low deprivation areas. In other words, the NDC intervention may have helped to prevent a further widening of the gap, or even narrowed it, in some of these outcomes, which is an important achievement.

Finally, our study suggests that the NDC programme may have reduced some inequalities within NDC areas, with the most disadvantaged respondents within NDC areas faring better in terms of positive changes in mental health and life satisfaction than similarly disadvantaged groups in comparator areas.

Do different types of New Deal for Communities programme have different impacts?

This general picture masks some differential impacts of different types of local programmes. Type 1 local programmes (involving major redevelopment) consistently underperformed on most health and social outcomes compared with both type 2 (incremental) and type 3 (strengthening people) local programmes. Overall, type 2 local programmes were most often associated with improvements over time. Compared with type 1 areas, in type 2 areas:

• social capital and smoking advantages emerged or were maintained between 2002 and 2008

• respondents had a greater probability of reporting improved mental health over time

• respondents without work were significantly more likely to find employment over time

• hospital admission rates fell faster than in comparator areas, particularly for alcohol- and drug-related conditions, which are closely linked to mental health problems

• the better performance on hospital admission rates was maintained in type 2 programme areas with very different social and economic contexts.

There is also some evidence that type 1 programmes may be associated with increased inequalities. Although less successful at reducing hospital admissions than the other types of local programmes, they appear to have performed better on these indicators in the least deprived areas.

Explaining the patterns that we have found: implications for policy and research

Our findings add to previous research showing that area-based interventions such as the NDC can have positive impacts on the health and social circumstances of residents of disadvantaged neighbourhoods. More importantly, the NDC also reduced the gap between more and less disadvantaged residents of NDC areas, and between NDC residents and those living in less disadvantaged areas of England – albeit that these effects were relatively modest, confined to a small number of health and social outcomes and not always consistent across the multiple outcomes and analytical approaches used. Impacts are most obvious for mental health improvements (a finding reinforced by the reduction in mental health hospital admissions in NDC areas that we also identified), aspects of social cohesion/capital and educational attainment.

The relatively better performance of type 2 local programmes suggests that future area-based interventions designed and delivered to achieve a better balance between redevelopment of the physical environment and investment in releasing the capacity of residents in terms of both human capital and genuine community governance could be better able to impact on a range of health and social outcomes at the same time as reducing inequalities, albeit modestly.

There is also the intriguing suggestion that 6 years into the NDC initiative, a time when political priorities were changing nationally, improvements in outcomes for disadvantaged communities began to slow. This underscores the potential health equity benefits of long-term investment in area regeneration programmes but also highlights the importance of understanding how local programmes are best enabled to weather changes in policy expectations and interests over time.

Our project has developed, and tested, more nuanced research processes, utilising secondary data sources to help tease out why some area-based investments do or do not work and in which contexts. We will be archiving material that will allow other researchers to access the data sets and variables that we have constructed to replicate the methods we have used. The findings emphasise the need for prospective evaluative studies that track both the form and the function of interventions, as well as local context. We were not funded to undertake an economic evaluation of the NDC initiative and with the data available this would not have been possible, highlighting the need for further methodological innovations in this field.