Transforming community health services for children and young people who are ill: a quasi-experimental evaluation

Realising children’s health care ‘closer to home’ in practice

Designed to care for ill and disabled children in the community, CCN teams have been recognised by national policy as the services to achieve care closer to home. 1,5,7 More recently, the Royal College of Nursing described them as the ‘bedrock to integrated care closer to home’. 8 Despite this, not every health community in England has access to a CCN team. 9,10 CCNs have a long history in the NHS11 and terminology has varied over time; paediatric home care, hospital at home and, to some extent, ambulatory care can all describe types of services known in the contemporary NHS as CCN teams.

Variation in models of care closer to home are evident12,13 and typologies have been conceptualised in a number of ways. For example, dichotomies of generic/specialist, inreach/outreach, community based/hospital based, and long-term/short-term input are all described in the literature. 10,12,14,15 However conceptualised, the key components of such services are that they are led by children’s nurses and deliver nursing interventions and care in community settings, such as home and school.

Evidence shows CCN services provide health care at home for a wide range of needs,9 including acute (e.g. see Callery et al. ,16 Sartain et al. ,17,18 Davies and Dale19,20), long term and complex (e.g. see Carter,21 Carter et al. ,22 Hewitt-Talor23 and Stein and Jessop24) and end of life (Parker et al. 9). They undertake a range of clinical care activities (e.g. home chemotherapy regimens) and, depending on skill mix, may be able to offer advanced nurse practice, such as assessing and prescribing. 25 They can also play a central role in non-clinical aspects of care, such as care co-ordination and empowering parents to support an ill child at home. 16,26

Although CCN services play a key role in facilitating care closer to home, they do not work in isolation. Often, they link with primary and secondary care services, particularly for urgent care pathways. 8 However, the success of these partnerships can be challenged by poor visibility and a lack of understanding more generally about the role of CCN teams in the NHS. 9

What outcomes could children’s community nursing services achieve?

Given the function and nature of CCN services, it is possible that they might influence outcomes such as secondary care use (and the associated costs), and quality of care for families. Secondary care use may change as a result of CCN service provision to families in two ways. First, admission avoidance is a function for some CCN services. 9 This, in turn, may result in a reduced rate of admissions for children. Admission avoidance may be achieved through immediate diversions from the inpatient ward following referral from accident and emergency (A&E) or primary care. It could also be achieved as a long-term consequence of CCN services empowering and educating parents, over time, to manage acute childhood illness at home. For example, in dealing with acute illness in children, parents have anxieties about making ‘wrong’ decisions27 and the risks of longer-term harm to the child. 28 Parents can feel disempowered29 and recent studies have shown a need for reliable information about recognising and managing acute childhood illness. 30,31 Recent evaluations of CCN services show community nurses can counteract these issues by acting as a source of support, advice and reassurance, and increasing parents’ confidence to support an ill child at home. 9,16,17,20,26

Second, CCN services may reduce length of hospital stay. Although children with common conditions that resolve rapidly may not stay long in hospital, their large numbers mean that reductions in length of stay (LOS) of only small amounts, and particularly avoiding a single overnight stay, would add up to changes in the individual average and total LOS. There are also possibilities for shorter hospital stays when a child’s condition is likely to resolve completely after treatment, or for complex care. Children with complex care needs may have longer LOS in hospital when admitted for common acute illnesses, and very long LOS for issues related to their complex care. For example, a study of 15 children in the South West region dependent on long-term ventilation found that they experienced a mean length of hospital stay of 513 days before being discharged home, with one child having stayed for a total of 1460 days. 32

Despite the theoretical possibility of impact of CCN services on secondary care, evidence for this is only just emerging, and is tentative at best. 9,33 Continued increases in emergency admissions lasting < 1 day34 mean efforts by CCN teams to fulfil admission avoidance functions are taking place against greater demand for urgent care. It is therefore important that future service evaluations try to account for this, although doing so is difficult outside the design of randomised controlled trials, an evaluation method inappropriate for this type of intervention.

Evidence for service quality is largely, but not consistently, positive. For example, studies of parental caring for ill children at home, particularly those with complex and ongoing needs, demonstrate a consistent picture of the demands associated with this. 35–40 The support provided by CCN services plays a critical role in supporting families in this situation,26 although other evidence suggests that existing provision is not always sufficient in this respect. 22 However, evaluations of CCN services typically show that parents highly value these teams and are largely satisfied with provision. 16,17,20,26

Despite the evidence trend towards positive evaluations of CCN services, there remains a gap in understanding how views of service quality change over time. For example, the importance parents place on different aspects of service provision may change over time as families become accustomed to being supported by a CCN team at home. Such considerations may be important to managers and commissioners as they plan services. In addition, in Parker et al. ’s evaluation,9 evidence suggested that perceptions of service quality were strongly linked to the quality of relationships with health-care staff,9,26 a finding replicated elsewhere in the context of other services. 41,42 Parent–provider relationships inevitably change and develop over time, and again, this presents questions about the longitudinal experience of CCN service quality for families.

Organisational change in health care

An extensive literature on organisational change indicates a range of factors that influence both the acceptability and sustainability of service change. For example, enabling and sustaining change requires a stable environment with capacity to acquire new knowledge. 43,44 A ‘receptive context’,43 reflected in strong leadership, allowance for risk taking, clear objectives, and dedicated funding is also important. 43,45,46 Consideration should be given to the innovation itself and how it is implemented. For example, there must be compatibility between the change and the organisation in which it is taking place, and changes need to be seen as credible and vital to organisational success, with thought given to timing and order. 43,45 The influence of key stakeholders can support or inhibit efforts to enact change, with the motivations, commitment, skills and values of others all playing a critical role. 43,45,47

The evidence base regarding organisational change in health-care spans both acute and community services, although no work has examined the salience of this evidence specifically for implementing CCN services. Given the national agenda to move care closer to home, which requires a degree of organisational change to introduce and expand new CCN services, understanding the factors that influence efforts to implement change in this particular context may be useful.

The need for research and background to the National Institute for Health Research call

Despite the national policy direction of moving care closer to home, commissioners have stated a need for evidence about the costs and secondary care outcomes of these types of services. 9 The tentative evidence about such outcomes warrants a need for further research in this respect. In addition, questions remain about the longitudinal aspects of perceived service quality for families.

Typically, CCN services have not been formally evaluated using ‘gold standard’ experimental approaches, not least because of the methodological and ethical challenges of doing so. A 2011 National Institute for Health Research (NIHR) call for quasi-experimental evaluations presented the opportunity for robust quasi-experimental evaluation of organisational innovations to develop CCN teams. Such evaluation would make a substantial and timely contribution to evidence, addressing the gaps outlined above. In response to this call, we designed a research project to answer the following questions:

1. Does redesigning children’s health-care services by introducing CCN teams affect acute hospital admission rates for common childhood illnesses and LOS for all conditions?

2. What changes in the quality of care do families caring for children with complex health-care needs experience when CCN teams are introduced?

3. What benefits and challenges do commissioners and providers experience, as the new services are planned, implemented and established?

4. What are the costs and outcomes of the new services compared with those achieved by alternative service configurations?

As the research was taking place during structural changes in commissioning arrangements, a supplementary research aim addressed how these changes affected the planning, implementation and establishment of services (as part of question 3).

Strand 1: a longitudinal qualitative study of service change over time

• Longitudinal in-depth, semistructured interviews and focus groups with 41 commissioners, managers, practitioners.

• Documents pertaining to the local service reconfigurations.

• Field notes.

• Thematic analysis using the Framework approach. 51

Strand 2: an interrupted time series analysis to explore the impact of introducing children’s community nursing services on acute hospital admission and length of stay

• Use of Hospital Episode Statistics (HES) data to examine trends in secondary care activity by case site.

• Time series using between 60 and 84 monthly data points.

• Looking at impact on acute admissions for common childhood illnesses, bed-days and LOS for all paediatric conditions (including a subset of complex conditions).

Strand 3: a cost–consequence analysis to assess whether or not the introduction of children’s community nursing teams is likely to provide value for money compared with current service provision

• Data from resource-use questionnaires and the translated Medical Home Family Index (MHFI) (see below) from 32 parents using the case site CCN services.

• Cost questionnaires for case site CCN teams.

• Data on acute activity from the interrupted time series (ITS) analysis.

Strand 4: a longitudinal qualitative study of changes in quality of care, with parents of children with complex health care receiving children’s community nursing services

• Longitudinal in-depth, semistructured interviews with 31 parents of children using the case site CCN services.

• Thematic analysis using the Framework approach. 51

In addition to the four studies, we translated two service assessment tools, the Medical Home Index (MHI) (paediatric) and the MHFI,52 as part of the research. The purpose and process of translating these tools are described in Appendix 1, Medical Home Index: overview of translation and Medical Home Family Index: overview of translation.

Detailed methods and analysis for each strand are described in the chapters that follow.

Ethical review

The study was approved by a NHS ethics committee in 2012, and all relevant local governance approvals obtained.

Summary of changes to the original protocol

Five key changes were made to the original protocol:

1. One case site (site A) represented a regional approach to service change and there were several CCN services introduced within the same period. Although our site contact (a NHS manager) had identified one team to participate, a second team were also keen to take part in the research. Thus, this case site resulted in two ‘arms’, with two CCN services being studied. One of these teams, however, declined participation in the qualitative study with practitioners and parents (although their managers and commissioners participated). Table 1 summarises the research activity across sites.

2. We had developed the research in partnership with three health communities, and thus, had intended to carry out the research with three case sites. However, because of delays with service reconfiguration in one case site (site C), a fourth (site D) was recruited so that we could undertake the parts of the research we were unable to in the delayed site. Because the service changes in site D were complete with no further expansions or developments, we did not undertake the longitudinal qualitative component with service staff, instead carrying out one set of interviews.

3. We had intended to look at time of admission and discharge as part of the ITS analysis. However, this information was not available within the HES data, and thus we could not examine this as intended.

4. Owing to difficulties recruiting newly referred parents, we amended our approach to recruit ‘established’ parent users of the CCN teams. Details of these changes are described in Chapter 5.

5. We had intended to collect data with commissioners and managers from the translated MHI52 (now the CCN development tool). However, during the process of translating the tool with our NHS partners, quantification of responses was removed, as this was felt to make the translated tool more useful and meaningful in practice. Thus, we were no longer able to collect data using the tool. This did not affect the wider project, as originally data collected with the MHI were to be analysed descriptively, and compared with interview data to examine face validity. Thus, although we could not provide a descriptive analysis of change to compare with interview data, our ability to meet the wider project aims was not affected. The intensive feedback gained from NHS staff about the tool has enhanced its validity as a service development instrument. Full details of the translation and final tool are in Appendix 1.

Patient and public involvement

A dedicated project steering group, which consisted of representatives from the Royal College of Nursing and the voluntary sector, a paediatrician and community nursing practitioners, advised on the research process as well as the development of the MHI tool (see Appendix 1, Medical Home Index: overview of translation). The group met five times throughout the 3-year project. Social Policy Research Unit’s parent consultation group (PCG) also advised on the materials developed for the qualitative study with parents.

Learning days

Three learning days were also incorporated into the management of the project, for which representatives from the case sites came together with the research team to discuss the research and developments in services. These learning days also provided an opportunity to develop and translate the MHI tool.

Production of an analytical toolkit

Any area undertaking a review of health services for children is likely to have the aim of preventing unplanned hospital admissions for children or reducing the length of time children spend in hospital following admission.

The Improving Services Toolkit: improving services for children and young people who are ill is aimed at commissioners and service managers who are aiming to redesign acute services for ill children. It provides evidence and data that can be used to help develop strategies and business cases. It examines emergency hospital admissions for children with common childhood conditions, such as respiratory and gastric conditions. It also looks at how long children spend in hospital and presents similar information specifically for children who have complex conditions, such as congenital heart conditions, cerebral palsy or metabolic disorders. The toolkit is available at the following link: www.chimat.org.uk/istoolkits.

Site A: generic children’s community nursing teams

Prior to service change in the region, a number of inpatient units and small community or outreach nursing services existed. Across the region, workforce changes meant that inpatient services had to be reduced in order to be sustainable and this, in turn, meant that more care was to be provided in the community. Thus, existing CCN teams were expanded and new CCN teams introduced to support nursing care in the community for children with acute and chronic conditions, reduce multiplicity of care and prevent acute admissions to, and facilitate early discharge from, hospital. Newly expanded or introduced community services were expected to be in place with evidence of competency prior to the closure of inpatient services. In addition to the changes in inpatient and community services, a new children’s hospital was built for the region. We studied two health communities within this region, both of which introduced either new (area 1) or expanded existing (area 2) CCN provision in line with the regional reconfigurations outlined in Chapter 1.

Site B: acute and complex care teams

Prior to the service changes, this site, a children’s hospital, had a number of specialist services with outreach nurses and a complex home care team for children with neurological conditions, but no generic CCN team or provision of acute care ‘closer to home’. Continuing health care (CHC) packages for children were an ad hoc commissioned arrangement, drawing on agency resources and, as a result, were seen to delay discharge from hospital. These two issues, an absence of an acute home nursing team and the ad hoc commissioning of CHC packages, led to two proposed service changes. The first was the introduction of an acute home nursing team, intended to reduce hospital admissions, attendances and LOS, and offer families a choice of care at home, school or nursery. The team would support a strategic aim of the commissioner and provider in providing care closer to home.

The second proposed service change was the expansion of the complex home-care team, through an increase in nurses and clinical support workers, to accommodate ‘in-house’ provision of CHC packages. It was intended that this change would allow more nursing care (for CHC packages) to be provided in the community through the existing home-care team rather than through an agency. It was expected that this would enable packages to be in place much more quickly (as staff would be already in place), therefore reducing LOS. Alongside this, a discharge co-ordinator role was created for the high-dependency unit in the hospital, in which children with CHC packages tended to reside prior to discharge. The co-ordinator was intended to work closely with the expanded team to facilitate discharge of the child. Finally, it was proposed that the assessment of CHC would take place through a single point of assessment, in the form of a newly employed nurse assessor and co-ordinator.

The area served by the PCT here is a large urban community with a population of just over half a million. Just over 10% of this population are from black and minority ethnic groups, and the PCT rank falls within the high deprivation quintile (2). In addition to the children’s hospital, there were two NHS providers locally: a hospital trust and a health and social care trust.

Site C: children’s community services in development

This site, which at the start of the research was a large PCT, had historically comprised a number of smaller PCTs, not all of which had CCN provision. When these trusts came together to form one larger PCT, the different models of CCN provision were highlighted. It also resulted in a ‘patch’ that had no access to CCN services except for an acute trust discharge team for children under the care of the trust’s consultant. Thus, children with a consultant in another trust (e.g. a regional centre) would not access this service. This led to plans to standardise existing CCN provision and ensure equity of access for all children and families by introducing a fourth team to cover the patch that had no provision. These proposed changes were embedded within a wider drive to reconfigure a number of other children’s community services in the local area, including continuing health care, school nursing, health visiting and advanced paediatric nurse practitioner-led urgent care pathways.

The area served by the PCT is a mix of both urban and rural geographies, with a population of around 750,000. It has low levels of deprivation, falling within the ‘very low deprivation’ classification (5), and fewer than 10% of the population are from black and minority ethnic groups. Within the PCT boundary there are a number of NHS provider trusts, all acute hospital trusts except one, which is a community provider. The community provider was leading the bid to transform the local CCN services.

Site D: nurse practitioners for complex conditions

In this site, prior to service change, there was an existing CCN team providing clinical nursing care for children. A team of advanced nurse practitioners was proposed to complement this existing CCN team and support children with complex health-care needs who were frequently attending the local hospital for acute care. There was no case management of children with long-term complex conditions, who had tended to rely on the local hospital instead of primary care for this. The new team of advanced nurse practitioners would address this by taking on the case management role, alongside assessing and prescribing. It was also intended that the case management would reduce multiplicity of care for families. The model was to be based on a model of adult community matrons that had previously been introduced, and which was felt to be successful by trust staff.

The area served by the PCT is a major urban community with a population of around 440,000, fewer than 10% of which are from black and minority ethnic groups. The PCT rank falls within the ‘very high deprivation’ quintile (5). Within the PCT boundary there are two NHS provider trusts: one an acute hospital trust and one a community provider. The service change was located in the community provider trust.

Table 2 summarises the intended changes to CCN services across each case site.

In-depth, semistructured interviews with NHS staff involved in developing children’s community nursing services

In-depth, semistructured interviews are a widely used qualitative data collection tool. 55 Semistructured interviews allow set topics to be covered, but with the participants’ responses determining ‘the kinds of information produced about those topics, and the relative importance of each of them’. 56 They were used here to capture the experiences and views of staff involved in the strategic development of CCN services. These staff included NHS commissioners, senior managers (e.g. heads of nursing, heads of children’s services) and, in one case site (C), a senior nurse. We undertook these interviews longitudinally to understand the drivers, context and planning for service change, the factors enabling and inhibiting service change and perceived outcomes of service change (when applicable) as they occurred. A topic guide was used for interviews (see Appendix 1, Topic guides for interviews and focus groups with staff, parents and children), and all interviews were audio-recorded and transcribed. A record of informed consent was taken prior to all interviews (see Appendix 1, Sample consent form). We treated consent as ‘ongoing’ and, thus, a record of consent was sought prior to each interview when a participant took part in multiple interviews across data collection waves.

Sampling and recruitment

A snowball approach to sampling was used, where an initial interview was undertaken with our key contact in each case site. This person then identified, either independently or as part of the interview, other relevant people to approach. Subsequent participants were also asked to identify any other NHS staff that we could approach for interview. All staff invited to participate in an interview were sent an information sheet about the research and a response form to record their participation decision (see Appendix 1, Sample information sheet). Staff members were contacted by either e-mail or post, depending on the contact details provided. Responses to invitations were given either by response form (returned to the researcher) or via e-mail. If no response was received after 2–3 weeks, a reminder letter/e-mail was sent. No further reminders were sent if a response was not received. In one case, a reminder letter was not sent because of organisational sensitivity in the case site at the time. We felt that sending a reminder letter about the research and possible participation in an interview might aggravate researcher-observed tensions between staff.

In line with the longitudinal approach, all staff members who participated in an interview were contacted for a further interview approximately 6 months later. The information leaflet was resent along with an opt-out form. If an opt-out form was received, or a participant made contact to decline a subsequent interview, we asked if they could recommend anyone else for us to approach for an interview. If no opt-out form was received after approximately 2 weeks, a researcher made contact with the participant to arrange an interview.

We intended to carry out interviews with willing participants at 6-monthly intervals. Based on the timetable, we expected approximately five waves of interviews. However, sometimes the intervals between waves of data collection were longer than 6 months. There were three reasons for this:

1. Sometimes participants asked us to delay the interview until after a particular meeting, when they felt they would be in a better position to tell us about service change plans.

2. The 6-month interval expanded if a participant declined a further interview but recommended another person, or persons, to speak to, as this meant having to approach the new individual and arrange an interview.

3. There were sometimes delays in participants responding to contact about the subsequent interview.

Owing to expanded intervals (as well as longer than expected processes of getting local research governance approvals in some NHS trusts, which delayed the start of data collection with staff), we were able to undertake three waves of interviews with staff involved in developing CCN services.

Owing to the variation in service structures in each case site, we did not set out to recruit a defined number of commissioners and managers involved in service development. Rather, we set out to recruit and interview as many key informants57 as necessary in order to build up a picture of service change in each case site. In the first wave of interviews, we approached 17 individuals to participate in an interview (four of these staff were in site D, for which we did not undertake longitudinal interviews – see Chapter 1). Of these, 13 agreed to participate and 12 took part in an interview. One participant who originally agreed to participate did not respond to later e-mails attempting to arrange the interview. We received no responses from three individuals and a decline from one individual (who recommended another person to approach).

In the second wave of interviews, five of the participants interviewed in the first wave declined participation. In three cases, this was because a change of job following the reconfiguration of NHS commissioning structures. In one case, the participant felt unable to add anything further, but was happy to be contacted later if needed. Three individuals who participated at wave one gave a further interview at wave two, and a further three new individuals were approached and agreed to participate. This gave a total of six interviews (from eleven approached) at the second wave of data collection.

At wave three, nine individuals were approached for an interview. Five of these were those who had been interviewed at wave two, and one individual who participated at wave one but not two (but was happy to be contacted again later). Three were new individuals identified at this wave. Of the nine individuals approached, four agreed and took part in an interview, and five declined or gave no response. We attempted a fourth wave of interviews to further explore the perceived outcomes of service change in sites A and B (partly explored at wave 3). However, we did not receive any response to the invitations to participate in site B, and in site A we were informed there was nothing new to add. Table 25 in Appendix 2 summarises this recruitment and attrition across waves 1 to 3.

Across all waves of data collection, a total of 22 interviews were undertaken with 17 staff involved in strategic service development across sites. Participants included those involved in commissioning children’s health services within the trust, and those with a management role (e.g. heads of CCN services, operational leads and associate directors of children’s services).

Table 5 summarises the recruitment by site across waves, Table 6 gives details of the number of staff approached and interviewed in total by case site and Table 7 gives this information by job designation.

Use of documents

Documents are valuable sources of data for case study designs,51 as they ‘play an important role in organizational life’. 58 We used them here to provide contextual insight into the case site area and its service developments. Alongside interviews, we asked participants for copies of any documents that would help us to understand the development and reconfiguration of services and wider contextual developments in children’s health services in their locale. We also asked for copies of documents that participants referenced in interviews. All documents obtained were numbered and read. Any information in the documents that would provide helpful context was used to supplement the analysis. Across the sites, we obtained 34 documents.

Field notes

Field notes are used to provide additional context to data collected from participants in interview studies. 51 They were used here primarily to record observations about service changes between staff interviews and to record notes from any informal discussions or meetings the researchers held with case site staff. These notes were sometimes used to inform later interviews with staff in the case sites.

Non-participant observations

We had intended to use this method as part of the qualitative study, to observe key meetings about the services being developed. Although we identified some meetings that we could potentially observe that might offer some insight into the development of services, these often involved staff members from multiple trusts, including trusts for which we did not have local governance approvals (we had governance approvals from the key trusts involved, but meetings often involved those from other trusts beyond the immediate coverage of the research case site). Thus, in order to carry out these observations, we would have needed to obtain governance approvals from these other trusts, which was not possible given the time this process takes. As an alternative, we followed up the content and outcomes of these meetings with participants in subsequent interviews, when possible. In another case, we identified a meeting that we could observe and we contacted those attending to seek permission. However, despite reminders, not all of those attending responded or gave permission for the observation. Thus, we were unable to carry out the observations as planned.

Analysis

The longitudinal data were managed using the Framework approach51 and analysed thematically. For full details of the analysis, see the analytical plan and summary in Appendix 1, Analytical process and framework for study 2.

Financing service change

A theme emerging in the data across all case sites was that of financing service change. In sites A and D, staff reported having money to invest in the service transformations and, therefore, initially they did not face financial challenges:

Obviously we did this growth when we had money, in PCTs, we don’t have money now.

Commissioner, site A

I think, you know, fortunate we’re, we’re, at that time, in a position where we, we did have money to invest.

Manager, site D

Accessing additional investment for further CCN service expansions at a later stage was reported as difficult for some teams in site A, because there was no longer an allocated budget as there had been initially. Instead, they had to go through the process of bidding to local commissioners and not all were successful in securing such additional funds.

In site C, where the initial stages of service change were taking place during the research, it was reported that there was no dedicated money to finance the proposed service change. The lack of available new funds to invest meant existing resources in the local NHS had to be reinvested. This meant a lengthy process of getting different NHS organisations in the local area together to review these existing resources. This was reportedly challenging, as organisations were trying to protect their income in a constrained financial climate. Tensions around withdrawing funds from acute trusts to fund community services were also reported:

So we need to find a system that will enable us to have that open dialogue and find a way forward because the secret to success is releasing funding from the acute sector to support children in their communities.

Manager, site C

In the later stages of planning in site C, the commissioners’ consideration of the proposed service plans for children’s community services was thought to be triggered by the financial viability of some parts of the local health service, although this, in the end, did not facilitate the intended service change.

In site B, there was initial caution about funding expansions to the complex home-care team because of concerns about potential fluctuations of demand for the service. This evidently did not prevent the team expansion, but it was reported that such caution did delay getting the workforce in place. We later learned that the trust had responded to these concerns through an agreement in which, if support workers in the expanded team were released through the termination of a package, they would transfer to the high-dependency unit and intensive care unit until a new package was agreed. Therefore, this staff resource would always be in use, regardless of demand for the complex home-care team. Further funding requests from the team to managers and commissioners for packages were apparently supported by the trust’s clinical director.

Medical support for children’s community nursing services

The support of medical colleagues appeared to mediate the process of service transformation, with such buy-in supporting the planning, development and implementing of CCN services, and a lack of buy-in inhibiting plans for change. For example, in site B, there were plans to pilot and introduce an acute home nursing team to reduce secondary care activity and deliver care closer to home for families. However, this intended service did not materialise because of a lack of ‘medical’ support for it locally. Concerns were raised by the medical directorate about the risks of this care being delivered through a community nursing function. At a later stage of data collection in site B, the theme of medical support arose again, but in relation to the complex home-care team. Here, one nurse reported the value of having a senior consultant that gave legitimacy to the team by supporting them with bids for further funding and working closely with, and having confidence in, the team nurses:

He [consultant] values our opinions and gives us that, you know, confidence to be able to provide the service that we provide … Plus, if there’s any issue with the team, or we want to go forward for further funding or anything like that, then [he] is our clinical lead who will support us in doing that.

Senior nurse, site B

It is not explicitly clear in the available data why there was this contrast in reported level of medical support for the intended acute home nursing service and that for the current complex home-care team. One possible reason may be because the complex home-care team was already in existence (and therefore known and trusted) when it expanded to incorporate continuing care. In contrast, the intended acute home nursing team was a complete innovation, and to our knowledge, this site had never had a service like this before. The lack of support for it may have stemmed from a lack of familiarity with, and trust in, this kind of service.

In site A, the network that oversaw the reconfigurations reportedly won backing from the medical workforce for the proposed changes during the planning stages. In site C, the proposed service changes had some medical support, with a GP lead working closely with commissioners and practitioners to support change. However, this support did not overcome the other barriers this site faced, such as the impact of the NHS reforms and a lack of dedicated finance combined with local competition for business (see NHS reforms). Issues were also highlighted in this site around how some local acute trusts were supportive of CCN services, but others were not, and that this had prevented a cross-provider partnership working to develop a possible service. In site D, it was reported that while GPs were not involved in the development of the new nurse practitioner team, they did support it.

NHS reforms

The changes to CCN services in sites B and C were taking place during the recent NHS reforms, in which new commissioning arrangements, in the form of CCGs, were being set up. The planning stages of transformation in site C, and some aspects of service development in site B, straddled both sides of 1 April 2013, when these new commissioning arrangements officially came into place.

In site C, the NHS reforms were felt to be particularly problematic for the planning of new CCN services in the period before 1 April 2013. Commissioning in this preceding period was characterised as being in a state of flux, with changes to job roles and people. This, it was felt, contributed to the delays in progressing with plans for service change. Others reported the reforms were ‘stifling’, making people less open to change. In subsequent data collection that took place in this site after the 1 April 2013 (between July and September 2013), these challenges continued. With new people coming into post as a result of the reforms, it was felt to be difficult to know who to talk to and when regarding the proposed new services:

I think what seems to have made, made it sort of not quite so clear, in terms of moving forward, is, you know, all the new CCG setting up and people getting into posts and knowing who, who we need to talk to when.

Manager, site C

At the later stage of data collection, it was also apparent how the broader set of plans for changing CCN services (e.g. continuing care, CCNs, urgent care pathway) had been split following the changing commissioning arrangements. The urgent care pathway changes now sat with the CCG and the remainder with the joint commissioning unit (JCU). While the two were reportedly working closely on these changes, this, we were told, could sometimes be challenging. Around November 2013, 7 months after 1 April, there was still uncertainty about who was leading the commissioning arrangements locally, with concerns expressed that this would further delay service change plans. Approximately 1 year later in October 2014, the complexity of local commissioning arrangements between the local CCGs and JCU persisted, with the new arrangements described as a ‘minefield’.

In site B, the NHS reforms were reported to have affected the original plans for supporting children receiving CHC packages (through the complex care team) in schools. Originally, it was intended that the local special school nursing resource would be used to support these children in schools. However, following the NHS reforms, the contracts for special school nursing moved to the local authority and it was no longer possible to go ahead with these plans. At the first round of data collection, it was reported that discussions about this issue between health and education commissioners were ‘uncomfortable’, stalling progress in securing the special school nursing resource. However, at later data collection it was reported that this was now progressing and the two commissioners were working together to develop the school support for children with CHC packages. Later, the team suggested school support was not particularly problematic, with support workers and teaching assistants in school working together for the child’s care.

Managing change across multiple NHS providers

In both sites A and C, the service reconfigurations were taking place across a number of provider trusts, but the nature of this multiple trust involvement differed between the two. In site A, the multiple provider trusts involved were each the recipient of financial investment and their own service change was led by a network. In contrast, the proposed changes in site C were being advanced by one particular provider trust. However, with no new money to invest, existing provision across a range of providers in the local area had to be reviewed, leading to cross-provider talks to address this. These cross-provider talks appeared to be in the form of a clinical group established by a former commissioner who had initiated the original proposal for service change. A lack of clinical consensus was cited as a factor delaying progress with service plans, which may reflect competition for business between the different provider trusts, a factor cited in later data collection in this site. Such competition may have been exacerbated by the lack of new funds to invest and the need to use and review existing NHS resource (see Financing service change).

In site A, a network had centralised the service change and oversaw all contracting, (initial) recruitment and monitoring. The network was felt to be particularly beneficial to the reconfigurations. It was reported to help standardise the process across participating NHS trusts, it created a high profile locally to communicate the changes effectively (‘the network was very key in it getting the priority it needed locally’) and it enabled children’s commissioners to come together, creating opportunities for shared learning.

Such macro-level structures were not reported in sites B and D, and this probably reflects that, in these sites, service changes were isolated to one provider trust and thus no macro structures were required.

Termination of reconfigurations in the planning stage (sites B and C)

As noted earlier, service change was not achieved in site C, and only partially in site B. In site B, a lack of medical support was cited as the reason why the introduction of the acute home nursing team was not achieved. We learned of this early in the research, but it was unclear from interviews and available documentation how much work had been invested in the planning for this service. We know it got as far as a service specification from the commissioner, but as the planning for this appeared to take place before the research we were unable to capture it in detail and in ‘real time’.

By contrast, in site C much of the planning for the intended service change took place during the course of the research, and we were able to capture this ‘as it happened’, including the decision to terminate much of the reconfiguration plan. This provided an opportunity to describe the extent of the work, time and effort invested in the planning of a service change that was not achieved. In previous sections, we have described the factors that participants in site C felt influenced the planning. In this section, we describe these to provide a chronology of events and factors and offer an overarching picture of the difficulties in achieving service change in this case site.

Prior to the research, we had been in touch with this case site in 2010, when the then commissioner was proposing plans to redesign CCN services. The commissioner knew of our previous work in this area9 and wanted to use the learning from the research to inform planning in the site. By the time the current project got under way in 2012, these proposals were ongoing and with the appointment of a new head of nursing, work was under way to plan and research the proposed redesign. For example, we received copies of draft commissioning intentions, and the first round of interviews in late 2012 indicated that there was work being done on business cases, auditing existing services and workforce planning with other local providers. In 2013, further planning work was being done, including a review of services, a public consultation, continued discussions and workforce planning with other provider trusts, and the development and submission of a business case following a service specification from commissioners.

Across these first and second rounds of interviews, there was discussion of tensions and competition with other providers. These discussions indicated that it was not simply a case of one provider negotiating with a commissioner, but multiple providers negotiating with each other as well as commissioners about resources and future service provision. Perhaps most prominent in the discussions were the difficulties of continuing such service planning with the change in commissioning arrangements. This was felt to play a significant role in delaying service change, and was mentioned at each stage of data collection (see NHS reforms for more detail).

It was also apparent that there was much interdependency between stages of planning, with certain aspects or events holding up the progression of others. For example, in 2013 the planning of the new service model became dependent on the outcomes of a public consultation on existing provision. This was because the new service model partly depended on the resources released by the decommissioning of the existing provision. Another example was that the new intended service was dependent on the appointment of a new nursing post in a different trust, to facilitate discussions across providers about workforce planning.

Despite these difficulties, in late 2013, it was reported that the submitted business case had verbal go-ahead, with the provider now planning the phased implementation of the proposed service. These service changes aligned with outcomes of the public consultation regarding existing provision in the area. However, by early 2014, we learnt that this decision had been reversed by the commissioners. It was reported this was because of a lack of financial planning around the redesign. In October 2014, it was reported that despite persisting complexities around the new commissioning arrangements, there was some progress. A number of factors were felt to be responsible. First, the appointment of a new commissioner allowed someone to look at the proposed redesign and the issue of inequity of CCN provision with ‘fresh eyes’. Second, the CCGs were felt to be giving more thought to the redesign. As noted earlier in this chapter, following the new commissioning arrangements, it was the JCU that had largely led the commissioning for the redesign of CCN services in this area. However, by October 2014, it was reported that the CCGs were beginning to grapple with the issues around CCN services, although it was still felt that children’s services more generally was a low priority for them. Third, neighbouring health communities were also redesigning their CCN services as a priority, and this was felt to have influenced the commissioners in this case site in thinking about the possibilities of their own CCN services.

The plans and intentions for the service change in this site had started at least as far back as 2010, and continued into 2014. This represents 4 years of planning for a service change that was eventually abandoned.

Influence of changing commissioning arrangements

As this research was taking place during major NHS reforms, we included a secondary research question about whether or not there would be an impact of changing commissioning arrangements on the intended service reconfigurations. An impact was most evident in site C, and to some extent site B, which we have already described above. As expected, in sites A and D, where the services were established, there was no evidence of impact on service reconfigurations in terms of planning and early implementation (such as that seen in sites C and B, respectively). However, when we asked staff in these sites about the changing commissioning arrangements on existing CCN provision, a common issue raised in site A was how urgent and secondary care was a priority for the CCGs. Findings about this are presented in Chapter 4, in which we discuss the issues teams faced in practice.

Focus groups and in-depth, semistructured interviews with children’s community nursing team practitioners

Focus groups are particularly useful for exploiting group interaction to generate information. 59 To understand issues associated with the implementation of new or expanded services, we undertook focus groups, and some supplementary in-depth interviews, with the practitioners in the CCN teams in sites A1, B and D. As with the data collection with managers and commissioners, we planned to undertake longitudinal focus groups with practitioners to understand expectations and reflections about the changes to practice. For sites B and C, for which the teams were to be introduced during the research period, we expected to undertake the first focus group soon after implementation and the second approximately 6 months later. We were able to do this for site B, in which we explored issues associated with their recent expansion to incorporate CHC provision. However, as the intended new CCN team in site C did not go ahead, it meant we were unable to collect data in this site for this component.

For site A, for which the team was already established, we also undertook two focus groups. The first retrospectively examined the earlier phase of implementation as well as expectations about new developments in the team. The second, approximately 6 months later, examined reflections about these new developments. Topic guides were used for each focus group (see Appendix 1, Topic guides for interviews and focus groups with staff, parents and children). In site D, which was a late addition to the research to compensate for the loss of research activity in site C (because of the delays to service change), we undertook one focus group to explore issues associated implementation retrospectively. A second focus group was not necessary, given that there were no further service changes or developments taking place.

Sampling and recruitment

Owing to variation in local service structures, a set sample size was not feasible. Rather, we intended to recruit as many team members as possible across the three sites. Thus, all available practitioners (n = 35) across sites were approached to participate in focus groups at wave 1. Of these, 21 agreed and took part, and a further individual who had not been approached joined a focus group on the day (site B). Fourteen individuals either declined participation or did not respond. In site A we were advised that it would be difficult to find a date where everyone could attend because of rostering of shift work. Instead, we were advised to set a date for the focus group and those who were present and willing to participate could join in. At wave 2, we approached all those who had participated at wave one in sites A and B (n = 18). Of these, 12 agreed to participate (thus, there was an attrition rate of one-third of the original sample). An additional three individuals, who had not been approached because they had not taken part at wave 1, joined the focus group in site A on the day. At wave 2 in site B, two participants requested an individual interview because of scheduling conflicts with the date set for the focus group.

Participants across the sites were nurses (n = 20) and clinical support workers (n = 4), all of whom worked as part of the teams. Nurses included generic CCNs, specialist nurses and those with advanced practice qualifications. Table 8 details the number of participants approached and the number who took part in a focus group or interview, at each wave of data collection, by site.

A record of consent was obtained prior to data collection for participants taking part in both the focus groups and interviews, and for each wave of data collection.

As described in Chapter 1, we also drew on relevant documents for context and contemporaneous field notes.

Analysis

The focus group and interviews were audio-recorded in all cases, transcribed, and data managed and analysed using the Framework approach. 51 In one case, the audio-recording of an interview failed and notes of the interview were made and sent to the participant for confirmation. For full details of the analysis, please see the analytical plan (see Appendix 1, Analytical process and framework for study 2).

Visibility and legitimacy of children’s community nursing services

Once services became operational, there were immediate and ongoing issues about how visible the teams were to others. For example, when the nurse practitioner team in site D was initially introduced, it sat alongside a CCN team in the acute trust. Later, when acquired by a different NHS trust, their base was moved into the community. Although the team preferred being based in the community, visibility to acute trust consultants was felt to be compromised, affecting consultant referrals to the team:

[W]e’re not utmost in people’s, in consultants’ minds for when they’re looking at, you know, care for these children.

Focus group, site D

Problems with visibility were thought to be partly a result of the team being so small at just three advanced nurses, and partly a lack of marketing on their behalf. While the introduction of a similar team in adult services was preceded by a marketing exercise at the senior management level, this did not happen for the nurse practitioner team. When the service was first set up, there was ‘suspicion’ from other services and a lack of clarity about what the team would do. In response, they had to raise their own profile with others through educating them about the service. The team reported that others’ learning about their team continued in everyday practice through joint working, but they still struggled to find time to continually maintain their profile while delivering the service.

Similar accounts were found with the site A generic CCN team, particularly with regard to how they were used and understood by GPs, for whom referral rates had been low. In response, the team implemented a liaison post to increase visibility of the service to, referrals from, and to improve relationships with, primary care. This involved close working with local GP practices, sometimes sitting in on GP consultations to advise on which children could be safely managed by the team. This post was felt to have worked well: referrals from GPs had doubled, and GPs better understood and valued the role of the team. Face-to-face contact between the team nurses and GPs was thought to be a significant part of the liaison post’s success.

Despite this success, visibility of the service to others remained a concern for the team. Earlier implementation was accompanied by a publicity drive, co-ordinated by the network that had overseen the wider reconfigurations (see Chapter 3). Thus, the new service was ‘marketed’, which the team felt was valuable for creating awareness of what they could provide to others in the NHS as well as to the public. However, the network ceased during the period of research, and there was no longer anyone to do this marketing. Having a dedicated person to do this was deemed important to raise and maintain awareness of the service to ensure its continued use:

[W]e need a marketing bod for everything we do really.

Focus group 2, site A

Both the accounts of those in the nurse practitioner and the generic CCN team highlight not only difficulties with being visible to and understood by others but also a wider issue about how the teams struggle to achieve legitimacy within the local NHS. They had to prove themselves to the GPs and consultants that would refer to them, in order to ensure their use and success.

Problems regarding visibility and understanding were also reported with the complex care team in site B. However, unlike those reported in sites A and D, these appeared to be temporary problems following the expansion of the service to incorporate the provision of CHC packages alongside their original remit (home nursing and respite for children with complex medical conditions). In the first focus group, the team saw a division between the two sorts of provision, with ‘the same staff . . . doing two different roles’. There was, they felt, some confusion among other professionals and families about this division in the newly expanded team and what it provided. However, when we followed this up 6 months on, it was clear that this issue had largely disappeared, with the team no longer seeing any sort of division in the care they provided. Other professionals were becoming more informed about them as a team because more were initiating contact with them. This suggests that the service expansion had become embedded in a relatively short period of time and did not face the same longer-term pressures to establish themselves as seen in sites A and D. One explanation for this may be that the complex care team was not brand new (like the teams in sites A and D), but an expansion of an existing team that was already largely visible to those who used it.

Balancing different caseload demands

Another implementation challenge faced by the generic CCN team in site A was that of balancing the needs of different groups of children on their caseload. The intention of the new team was to offer provision for those with acute, long-term and complex needs. However, there was, immediately, a substantial population of children with complex needs being referred, resulting in a large caseload for this group. Balancing the needs of this group with the needs of those with acute illness could be particularly challenging in winter, when referrals for acute cases typically increased. It was expected that a newly created post, dedicated to children with complex, continuing and palliative care needs, would help to address this.

Indeed, later data collection with the team confirmed that the appointment of a complex needs co-ordinator had been beneficial in this respect, although the large caseload of children with complex needs still presented an ongoing concern. Co-ordinating care for children with complex needs was felt to be time-consuming, but the nurse who had taken on the post was felt to be particularly knowledgeable about local services for this group of children because of her extensive experience, and could make these links much more quickly than a nurse without this experience:

I found it dead helpful, ’cos I’ve got a few that are complex needs, and it, I could spend days and days and days on just like that patient, and she’s [complex needs co-ordinator] come in and took a lot off me so I can carry on with all my acute stuff as well.

Focus group 2, site A

The post was also seen as a resource for the wider team, with the nurse being able to offer advice and training regarding care of children with complex conditions.

The teams in sites B and D did not face issues about balancing demands between those with acute and complex needs, but that is likely because their caseloads included only those with complex needs. However, the expansion of the site B complex care team had resulted in two different caseloads, those with and without NHS-funded CHC packages, and concerns were raised about balancing the needs of both within finite team resources. There were suggestions that the newly incorporated CHC service component could potentially dominate provision, with those families on the team’s original caseload who did not have a CHC package being ‘neglected’. On following this up at a later data collection point, there were mixed views among team staff. Some felt this was not problematic currently, whereas others felt it was an issue in times of staff absence, when priority was given to children with CHC packages. Some suggested addressing staff absence by having a floating nurse to step in and provide cover, but also noted that it was unlikely such a post would be funded.

Demonstrating impact and value

The generic CCN team in site A believed that they were helping to keep children with complex needs out of hospital by intensively managing them at home, but reported that it was difficult to demonstrate or evidence this. They collected data about the activities they undertook, but they also required data about secondary care use. However, this was difficult to achieve. First, they did not have the resources to audit the hospital use of every child on their caseload. Second, population-level secondary care data (i.e. HES) were deemed problematic, as potentially high-volume attendances at the local POAU were counted as inpatient admissions within these data. Thus, staff felt that it was not accurately representing trends in inpatient admissions (an issue also seen to be part of a wider trend nationally) and was a poor measure of their impact on this activity. Other forms of data collection, designed to show impact on secondary care, had been attempted at the caseload level. While this showed a patient-reported decreased use of secondary care, this had not corresponded with decreases in activity in the secondary care data used by commissioners (typically HES data). Similarly, while the team could demonstrate an impact on primary care through increased referrals from this source, which was felt to be a precursor to deflecting hospital attendance, there was no visible corresponding effect in the secondary care data.

Team members felt that there was also too much focus on contact activity, rather than service quality, for demonstrating impact. However, this had begun to change by the time of our second data collection, in which the team reported plans to start collecting data on quality outcomes and patient satisfaction. Despite this, there were still concerns that patient satisfaction outcomes would not be sufficient to demonstrate the team’s value to commissioners.

Similar concerns were raised by the nurse practitioner team in site D. The team were keen to collect data to evaluate their service, but it was felt that the current information technology system did not adequately capture their activity. It was also expected that attributing outcomes to the team could be a challenge, given the close working and interdependency between the team and other services. Impacts on A&E and school attendance were outcomes being examined using data reported from families using the service. Work was under way with the acute trust to seek more robust data on A&E outcomes.

Concerns around demonstrating impact and value were not raised by the complex care team in site B. However, it was raised by the commissioner, who was unable to access patient-level data, because of recent legislation, to examine the impact of increasing team activity on LOS locally.

Site B: complex care team

The expansion of the complex care team in site B to incorporate provision of CHC packages meant that new band 3 clinical support workers were brought into the team to staff these packages. Initially, the team used temporary 1-year contracts for support workers, which offered flexibility to determine ‘fit’ between the worker and the family to whom they were allocated for a package. We later learned, however, that this caused problems in attracting high-calibre applicants and created job insecurity, meaning that existing support workers would look for other work towards the end of the contract. To address this, permanent contracts were introduced, and ‘fit’ between families and support workers was addressed through moving the worker to a different package. It was reported they now had enough packages in place to allow this. Securing the permanent contracts was difficult initially, because of the financial implications. However, this was addressed through an agreement whereby if a package ended, thus releasing the support worker, they would work at the children’s hospital in the interim until a new package became available. Retaining staff through these permanent contracts reportedly contributed to more efficient processes in getting care packages in place. It was also thought to save money in training, as the team no longer needed to train new workers replacing those on temporary contracts.

Another issue faced by the team was about how best to staff the care packages. A model of core care teams was used, in which a core set of staff in the team was allocated to a package and peripheral staff acted as a back-up in the event of staff sickness. This offered continuity to, and was reportedly desired by, parents. However, this approach also had challenges, in that staff could become very knowledgeable about one particular child to whom they are allocated and less knowledgeable about others. This, in turn, made it difficult to move staff around at times of staff sickness and absence:

Because if, if you, you know a lot about one particular child because you work with them a lot, it does mean that you tend to know less about others, and some of our children are really complex so it means that we can’t always move them around quite as flexibly, you know, as we could before.

Nurse, site B

Practitioners had also expressed a desire to work with different children to allow them to maintain a professional relationship with the family, avoiding relationships that might become too emotionally close with one family, which may tax their own emotional resilience. The use of core care teams thus continued, but with more crossover of staff onto different care packages to address these issues:

[I]t works for both, parents become, don’t come as dependent on other small groups of people and you, the support workers keep their skills up.

Focus group 2, site B

Site D: nurse practitioner team for children with complex conditions

The development and introduction of the nurse practitioner team required staff to have advanced practice qualifications, either generic or paediatric. All staff that had worked in this team did so, and while it was felt that having this qualification was necessary, clinical practice and experience were also important to undertake the role. Having an experienced advanced paediatric nurse practitioner in the team was thought to be beneficial in helping newly qualified advanced practitioners develop into their role through supervision, support and mentoring:

[W]e wouldn’t have been able to do the post without [other nurse practitioners] already having come in with their [Advanced Paediatric Nurse Practitioner] because they acted as, you know, supervisors, mentors, etc., supporting us to, for us to develop into the role.

Focus group, site D

Maintaining an advanced skill set for assessing, diagnosing and treating acute illness required regular use of such skills, but time spent on team management and tasks for ongoing nursing needs (such as managing portacaths, gastrostomy tubes) could potentially limit opportunities for this. However, in this case, the team felt that the existence of a local CCN team, which dealt with these other ongoing nursing needs, enabled them to focus on assessing, diagnosing and treating, and thus helped them to maintain their skills.

One of the most prominent issues that arose in the discussion with this team was the role of GPs in the care of children on their caseload. The team described positively their working arrangements with primary care services such as health visitors and school nurses. However, while they tried to keep GPs informed about the child’s care, they noted that GPs had little actual involvement with the child. The team attributed this partly to the fact that parents tended to contact the nurse practitioner team for supporting their child during acute episodes of illness, instead of the GP. Given that the focus of the team is to address acute illness through assessing and prescribing, this would make sense. Indeed, as we see in Chapter 5, parents told us they typically chose the nurse practitioner team over GPs when their child was acutely ill. The consequence of this, however, was that when young people were ready to transition to adult services, a GP may have seen the child very few times and thus would be unfamiliar with their care needs. The nurse practitioner team were keen to do more to involve GPs in preparation for when the young person transferred to their care in adult services. However, even if the nurse practitioner team did not exist, it is possible that GP involvement in the child’s care would still be limited. The team were designed in response to a population of children with very complex health needs, who typically used the local hospital for acute illnesses, such as fever and infections. Thus, parents were bypassing primary care anyway, suggesting that the absence of GP involvement is not a result of the nurse practitioner team, but a demonstration of the difficulties primary care has in dealing with a group of very complex children with highly specialised needs.

In-depth, semistructured interviews

We undertook two waves of in-depth interviews with parents of children with complex health needs using the teams. The first interview explored the parents’ early experiences of using the team for their child, from when they were referred, including how the team was used, views of care quality and expectations for future use of the team. The second interview explored if and how their use of the team had changed since the first interview, and again views on quality of care. Topic guides are in Appendix 1, Topic guides for interviews and focus groups with staff, parents and children. Interviews were audio-recorded with permission and transcribed in all but two cases, in which detailed notes were taken instead (at the request of the participant). At the same time as the interview, the researcher administered the cost questionnaire for study 3 (see Chapter 7 and Appendix 1, Resource-use questionnaire for parents, time 1 and 2), and parents self-completed the MHFI. A record of informed consent was obtained prior to each interview.

Intended sample

For this part of the study, we chose to recruit parents of children with complex health needs. There were two reasons for this. First, this group of children are those who are likely to be intense users of secondary care and for whom the support of a CCN team may be particularly important. Second, we wanted to understand changes in experience of care over time, and it was most likely that those with complex needs would be using the team over longer periods of time, as opposed to those using the team for one-off episodes of care (e.g. children with acute needs with no underlying complexity). As we wanted to understand change over time, we intended to recruit parents who were newly referred to the services. This would enable us to capture their early experiences and their expectations of using the team for their child. A second interview approximately 6 months later would allow us to see if and how use of the team had changed over time, and if and how experience of quality of care changed over time.

Parents were eligible to participate if they had a child with a complex health condition, were newly referred to the team (within 6 months of referral) and were likely to use the team again one or more times in the 6 months from referral. We excluded those who were receiving end-of-life care. Children with complex health needs constituted just one part of the caseload for the team in site A1, who also provided care for children with chronic conditions and acute conditions with no underlying complexity. The teams in sites B and D catered only for children with complex health needs. We intended to recruit between 25 and 35 parents across the three case sites. At the start of the research, the case sites were planned to be A1, B and C. Owing to the delays in site C, described in Chapters 1 and 3, the research activity transferred to the newly recruited site D. However, because new referrals per month were few in site D, we compromised and recruited the most recent referrals.

In addition to interviewing parents, we intended to interview, when possible, their child. This would draw on the interview facilitation materials that were developed, piloted and used in the previous project. 9 Children were eligible to participate if they were aged > 5 years and were cognitively able to give informed consent and participate in an interview. We intended to adapt the mode of interview to suit the young person’s communication style.

Recruitment

For parents, we prepared invitation-to-participate packs, which included a covering letter, an information sheet explaining the research, a response form that parents could return directly to the research team and a pre-paid return envelope. We asked the teams to distribute these packs to parents meeting the eligibility criteria. If we received no response from the parent after 3 weeks, we prepared reminder letters and asked the team to send these on our behalf. For data protection reasons, we were not able to directly contact families who were using the teams. For children, we explored with parents at the second interview whether or not it would be possible for their child to take part in an interview, and if we could approach them with an invitation to participate. We explained that we would adapt the interview to suit the young person’s communication needs, and information packs for the parent and child were given. These included a response form to return to us. As we could not approach children directly, we do not know for certain whether or not parents chose to pass on the information to their child. All final recruitment materials used are in Appendix 1.

Access to and initial experience of the team

For the majority of those interviewed, their child’s entry to the CCN service followed a period in hospital. When this was the case, some, but not all, made comparisons in terms of care quality between hospital and CCN care. These comparisons suggest that experienced quality of care changes in response to the shifting needs of the child and family when moving from hospital to home. In hospital, input is focused on the more immediate medical intervention required to address the child’s illness. Once home, input centres on managing ongoing nursing and health needs. Parents valued different aspects of both hospital-based and CCN care, indicating that mostly, quality of care was experienced as no better or worse between the two.

Not all parents accessed the CCN service following hospital discharge. There were some parents and children who had, in a sense, slipped through the net, and were managing their child in the community with no nursing assistance or support before being referred to the teams. When this was the case, the input of CCN services helped them get the support they needed to manage their child’s care. Being without such support was described as a ‘struggle’ and a ‘no-man’s land’. These cases highlight what it is like to care for a child at home with little support and the role a community nursing service can play:

It’s a weight off my shoulders because we just literally felt that we’d been abandoned and it feels like there’s somebody there that I can call on.

Interview 3

A minority of parents reported coming into contact with the CCN team after a period of using another team after moving from another area.

Once referred to the teams, parents used them for their child’s care in different ways, as summarised in Table 10. These different uses reflect the nature of the service models (see Chapter 2 for a summary of these). For example, parents used the nurse practitioner team (site D) typically when their child was acutely ill, whereas parents used the complex care team (site B) as part of an allocated nursing and home respite care package.

Ongoing experiences and outcomes

The accounts of those interviewed suggest a largely positive experience, in terms of both the positive outcomes that occurred or were achieved as a result of the team’s input and what they were able to offer families. The most predominant theme was about how perceived quality of the CCN services, that is positive experiences of care, were largely rooted in whether or not parents felt confident and supported by the teams to manage their child’s care in the community. CCN services were vital in this respect, and the support they offered helped to avoid the feeling of being ‘abandoned’ that we saw with those who previously had no support in the community (see Access to and initial experience of the team). Secondary to this was how using the teams enabled the child to remain at home by avoiding planned and unplanned hospital visits. Other outcomes across sites were also reported, but to a much lesser extent. In this section, we present findings about parents’ experiences of what the teams did to achieve these outcomes.

Feeling supported and confident to manage the child’s care in the community

The accounts of parents in each of the three sites show how feeling supported to manage their child’s care in the community was a key outcome of using these services, in terms both of what was expected and what was, in most cases, achieved. However, the aspects of provision that were offered to achieve this outcome differed somewhat across the three services studied. Not only did the services appear to differ in what they did to achieve this outcome, but they also differed in how these valued aspects of service provision changed, or not, over time. Each of these is described in turn below.

Avoiding planned and unplanned hospital visits and stays

A second outcome was discussed by parents – avoiding planned and unplanned hospital visits and stays. As with the first outcome, the value of being able to avoid hospital when possible was described positively, but the ways in which the CCN services facilitated this differed by model.

This outcome was typically described in sites A and D, and to a much lesser extent site B. Across all sites, being able to contact the team for advice and support, as described above, helped parents decide whether or not to take their child to hospital. Beyond this, the teams in site A and D each offered different approaches to preventing unnecessary hospital visits and stays. In site A, this reflected their role in providing clinical care at home or school for the child; this was, for most, a key component of care by the CCN team in site A. This clinical input could be planned administration of treatment at home, maintaining technological care (e.g. passing a nasogastric tube, caring for gastrostomy tubes), monitoring and observations, or assessing and advising parents whether or not to take the child to hospital. Some described the team’s clinical input as helping to avoid unplanned visits to hospital, while others observed their input as being in place of planned hospital care:

Rather than me take him back to [hospital] daily, for his daily dose of i.v. [intravenous] antibiotics, the community nurses’ll come out and do that. So that, that helps, ‘cos [it] stops us from, either him being in hospital for days when he doesn’t really need to be, and he’s bored and at risk of catching something else, and then it saves me having to drive to [hospital].

Interview 12

By having this clinical input in the community, parents felt it avoided hospital visits, which could be time-consuming, costly, logistically difficult in terms of travelling, harmful to the child’s physical (e.g. exposure to infections) and emotional (e.g. stressful) health, and result in family separation, lost work and missed school:

If we hadn’t have had it, like I say, it would have just meant more and more trips down to the hospital for what takes all of 5/10 minutes in total to do, but it just seems a total waste really. So it, it, it was just really, I found it beneficial to us.

Interview 2

It’s distressing for [child] when she’s in hospital . . . they mess about with her, it’s like even when she does sleep they’re here prodding her, taking her temperature and things like that, it’s, then I’m there up with her, it just turns everything upside down. So being able to care for her actually in the home is so much easier, and [child’s] more comfortable at home, so she gets everything that she needs.

Interview 14

Being trained to manage clinical care at home was also a part of this, helping to avoid trips to hospital for the parent and child:

I was glad that they did the training [for gastrostomy button], because before they actually trained me, if [child’s] button went or anything like that, or the feeds weren’t going down, I’d have to take her into hospital, which then meant a stay in hospital; now that I can change the button myself and things like that it does make life a lot easier.

Interview 14

By contrast, in site D, it was the role of the team in assessing and prescribing (see Other service features) that parents valued, and which they felt largely helped to avoid visits to A&E when their child showed signs of illness.

Other positive outcomes from using the children’s community nursing services

Feeling confident and supported to manage the child’s care in the community and avoiding planned and unplanned hospital visits were the main outcomes reported by parents across the three sites. Other outcomes were reported to a lesser degree, and only in relation to certain teams. For example, some parents in site A felt the team’s active management of the child’s health and nursing needs and their availability in the evening had enabled children to remain in school. In site B, the home nursing care packages provided by the team gave parents respite, something that they greatly valued.

Change in experienced quality of care over time

So far, we have described the service features that parents valued when using the CCN services and the positive outcomes they felt were achieved as a result of this input. We were also interested in how these positive experiences, as indicators of care quality, changed over time. The accounts of parents suggest that there are two key issues here: the first is about change in what is valued and when, and the second is about change (or absence of it) in perceived level of overall care quality over time. Each is discussed further below.

Changes in what is valued and when, over time

First, the importance and value that parents attach to some features of CCN provision appears to change depending on ‘where’ they are at in their journey of care and how confident they feel in managing their child at home. This is not a change in the perceived ‘level’ of quality in relation to the valued service features, but simply a change in the ways they are important. This was most noticeable in site A, in relation to being able to contact the team for advice and support, care co-ordination, frequency of contact and training. Being able to contact the team for advice and support was important in the early stages of using the CCN team, when the child was recently discharged. This was a time of anxiety and the ‘unknown’, and being able to contact the team was a way of accessing reassurance and advice when parents were not yet feeling confident about managing their child at home. Knowing the team were there at the end of the telephone continued to be important for parents even as they grew more confident in their child’s care, but it seemed to act as more of a back-up, a ‘just-in-case’ rather than an actively used resource:

I needed her services more initially, but now we’re on an even keel and we’re just plodding along, things are fine, but I know that she’s just there at the end of a phone if I need any advice or help.

Interview 11

Similarly, frequent contact was valued at the ‘daunting’ initial stages of being at home, and became less important when parents grew confident in care.

Co-ordination of care was valued when the child was initially referred to the team (typically following hospital discharge), as this helped to get services in place for the child in the community. Over time and once services were in place, this became less important: ‘. . . [named nurse’s] probably stepped back a little bit more ‘cos obviously she’s set everybody up, everybody’s on board kinda thing now’ (interview 5). Training was also valued for its ability to empower parents and take control of their child’s care, and this continued to be important in the form of ‘refresher training’ in the longer-term. Figure 1 summarises these changes in what was valued and when for the site A1 team.

FIGURE 1.

What is valued and when (site A1).

In site B, there were, for some, similar discourses about the importance attached to being able to contact the team for advice and support; it was important at all stages of care but was used more actively in the earlier stages of being at home, and was valued as a back-up later in care. However, this was the only service feature in this site for which the value attached differed over time. For other service features in this site, and all service features in site D, parents’ accounts suggested that there were no changes in what was valued and when in their journeys of care. In site B, this may be explained by the nature of their input, which was intended to be ‘static’, reflecting the allocated care package each family received. For example, a child might be allocated 7 hours a week of nursing care, which would remain unchanged until they were reassessed. Thus, unlike the site A team in which input typically decreased over time, input from the site B team meant that the intended level of contact remained more or less the same over time. This might explain why there was little difference in what parents valued and when in site B, because there was little change in the basic component of provision over time. In site D, the absence of change in what was valued and when may be explained by the fact that parents were using the team for a very particular aspect of their child’s care (acute illness), which tends to be episodic rather than ongoing.

Changes in perceived level of quality of care over time

As described above, for most of the parents we interviewed using the generic CCN team in site A, the team was more involved when parents first began using the service for their child, when the team fulfilled clinical, care co-ordination and advisory roles for the family. Over time, the team’s involvement changed to be one more of ‘maintenance’, in which less care co-ordination was required, visits became less frequent and they were used mainly for supplies and advice as and when needed. For a smaller number of parents, use of the team did not follow this trajectory, with some reporting that the team had become more involved over time and others reporting that their input had remained steady. Team involvement also reflected significant changes in the child’s needs, for example, for end-of-life care.

Where the team’s involvement had gradually decreased over time, this was mostly viewed positively. Parents reported that they had become more confident in managing their child’s care over time but valued that they still had access to the team for support and advice when needed. In these cases, parents’ accounts indicated that despite the reduced involvement of the service, they still felt supported by the team and were still satisfied with the input received. This suggests that the perceived level of care quality remained positive and static over time, even though input had changed, and was linked to the ‘empowering’ role played by the team. This may explain why certain service features are valued more at the earlier and later stages of care.

However, there were two cases in site A1 when the diminished involvement of the team over time was seen to be a result of heavy workloads, rather than a planned reduction of input. In these two cases, both parents felt that they did not need the team for their child as much as they had at first and still valued them as a source of reassurance and advice. Still, these two parents had concerns and frustrations about the impact of these perceived heavy workloads. For example, one parent described how the quality of the site A1 team was better in the beginning, when they were ‘very on the ball’ with her child’s care (interview 7). More recently, this had been lost somewhat because the team had become so busy, meaning that she was chasing them for information and test results: ‘a lot of the time now it’s me phoning, me checking and me making sure’. In another case, a parent described how the ‘personal touch’ of the team had waned, which was attributed to the team’s heavy workload. This ‘personal touch’ was reflected in having a regular nurse that knew her son’s health and needs. Without this, the mother was concerned about how it would affect his care:

I don’t want them to forget [son], I don’t want them to come and go, oh hang on, let me read his notes. Who he is, what’s he got again, what does he need, what have we give him in the past. I think it’s [continuity] good, I think it’s good for [son], so he won’t be frightened.

Interview 12

As noted above, the level of input from the team in site B was intended to be fixed, reflecting an allocated package of care. But even within this fixed allocation of hours, there were reports of losses and gains in what the team delivered to the child and parent, and these were closely tied to experiences of satisfaction with the service. Thus, valuations of quality tended to fluctuate, depending on whether or not parents felt the service was doing what it was meant to and to a sufficient standard. For example, there were reports of both lost and enhanced continuity of staffing. Some felt continuity had been lost over time as the team grew bigger, which could be a source of frustration, while others reported their ‘core care’ teams had become more established over time, which was viewed positively. There were reports of losing key workers, which were then not replaced, as well as reports of getting new key workers who were able to co-ordinate their child’s care. Both were linked with negative and positive views, respectively. There was also a loss of input and support at times when allocated hours of respite care were not achieved because of what was apparently staff absence or shortages. Thus, while parents in this site had a global view of the service, that, overall, it played a critical support role, assessments of quality depended on whether or not they felt individual service processes, such as staffing continuity, were being adequately delivered.

For those interviewed in site D, service use either increased over time as the child’s condition worsened or remained constant and matched fluctuations in the child’s acute health needs. Use of the team was mostly for when the child was acutely ill, and so largely reflected patterns of the child’s illness. Some parents reported that they had developed positive, trusting relationships with the nurses over time, and that the nurses had grown familiar with the child’s clinical signs and symptoms when acutely ill. This, combined with other factors (e.g. the perceived inaccessibility of primary care) sometimes appeared to correspond to a preference, developed over time, for using the team when their child was acutely ill where possible. Evaluations of the service were all positive and none of those interviewed described any loss in quality of care in their use of the team over time. One felt the team was less ‘hands on’ than it used to be but still felt the quality of the service was excellent, while another felt that quality had improved over time as they used the service more for their child. Thus, the perceived level of care quality appears to be linked to what the service offers to parents in terms of managing acute illness.

In summary, the three services each offered different trajectories of service use, as summarised in Figure 2. Typically, input from the generic CCN team in site A decreased over time; input from the complex care team in site B was intended to be static, reflecting an allocated package of care; contact from the nurse practitioner team in site D fluctuated depending on acute episodes of illness. Despite these changing trajectories of input, assessments of quality of care did not align to these. That is, parents mostly held positive views of the site A team, despite decreasing input over time. In site D, fluctuating, ‘irregular’ input was also matched by positive evaluations. In site B, despite the set allocated hours of input, perceived quality was closely tied to whether or not aspects of care within the allocated input were delivered and, thus, tended to be more variable.

FIGURE 2.

Trajectories of service use.

Outcome measures

Data were extracted from HES to explore the impact of introducing CCN services on acute hospital activity. The measures of hospital activity were:

• Rate of admissions per 100,000 for the 12 most frequently occurring common childhood conditions requiring an emergency admission to hospital. For the purposes of our analysis we identified the ‘top 12’ primary diagnoses in each site for the 24-month period prior to implementation of the CCN services and used this grouping throughout. Data were extracted on the number of finished emergency admissions for infants (excluding neonates/newborn infants) and children (the age range was between 14 days and 15 years, inclusive). The rate of admissions was calculated by dividing the number of admissions in each month by the appropriate Office for National Statistics (ONS) mid-year population estimates for PCTs and multiplying by 100,000. The resident population figure was used, rather than the population of PCT/commissioning area service users, in order to ensure that all hospital activity for residents was captured rather than just activity in that locality.

• Bed-days per 100,000 and average LOS for all diagnoses, whether elective or non-elective, that were reported in HES inpatient care data. Data were extracted for infants (excluding neonates/newborn infants) and children (the age range was between 14 days and 15 years, inclusive). Bed-days per 100,000 was calculated by dividing the total number of bed-days in each month by the appropriate ONS mid-year population estimates for PCTs and multiplying by 100,000. LOS was calculated by dividing the total number of bed-days by the number of discharges in that month.

• Childhood conditions associated with highest average and highest total LOS in inpatient care (complex care). For each site, HES data for a period of 24 months before implementation of CCN services were used to identify these conditions and use them for subgroup analysis to explore the impact of CCN services on patients requiring complex care. The primary and all secondary diagnoses codes were used to identify children with complex conditions. Rate of bed-days per 100,000 and LOS were the outcome measures used in this analysis, with the exception of site D, where the rate of admission per 100,000 was the only outcome measure as the aim of the team was to reduce admissions. The age range was between 14 days and 15 years, inclusive.

• For the analysis of impact in children with complex conditions, we explored the HES data and discussed the choice of a group of conditions with experts and with our project advisory group. We identified three main groups usually found in CCN teams that served children with complex conditions:

  • congenital malformations, deformations and chromosomal abnormalities

  • diseases of the nervous system

  • metabolic disorders (E70 to E90).

In addition, the above data were extracted for following subgroups:

• age on admission (rates were based on actual age profile of resident population):

  • 2 weeks to 1 year

  • 1–4 years

  • 5–11 years

  • 12–15 years

• ethnic group:

  • white (British, Irish, any other white background)

  • Asian or Asian British, Black or Black British, mixed, other

  • not stated

• sex: male/female (rates based on actual sex profile)

• local authority lower super output area data to provide a link to area deprivation data [1 (most deprived), 2, 3, 4 and 5 (least deprived)].

• primary diagnosis using International Classification of Diseases, Tenth Edition (ICD-10)60 block (admission rates only)

• specialty:

  • paediatrics

  • other

• day of admission

• day of discharge

• elective or non-elective (for bed-days and LOS).

In the subgroup analyses, data are suppressed when there are between one and five admissions in a particular month. This is to ensure that individual children cannot be identified. When categories were supressed, a figure of 2.5 was inputted. This was to ensure that there were no missing data in the time series.

Statistical analysis

Interrupted time series analysis is arguably the strongest quasi-experimental research design and is particularly useful when a randomised trial is infeasible or unethical. 61 This design is appropriate when randomisation is not feasible – for example, when evaluating organisational change of health-care delivery – and when a time series is available, as here, from HES data. 62 Although randomised trials may be considered the gold standard of causal evidence, quasi-experimental designs informed by extensive qualitative work about decision-making, it is argued, are probably the best way to progress the discipline of quality improvement and implementation science. 61 ITS analysis allows assessment, in statistical terms, of how much an intervention changed an outcome of interest, immediately and over time. When a separate control group is not available, the analysis of the outcome of interest in the study group does not allow control for other events that may have influenced the outcome. 63 Nonetheless, the level and trend of the pre-intervention segment serve as control for the post-intervention segment in single-group time series, still address important threats to internal validity and represent a methodologically acceptable design for measuring the impact of interventions. 63 Figure 3 demonstrates graphically the ITS analysis.

FIGURE 3.

Interrupted time series analysis.

The first step in ITS studies is visual inspection of the series over time. Visually, this allows comparison of the time series pattern before the intervention with the pattern after the intervention to assess if, after the intervention, the time series pattern has changed noticeably in relation to the pre-intervention pattern.

This is followed by a segmented regression model to fit a least squares regression line to each segment of the independent variable, time, and thus assumes a linear relationship between time and the outcome within each segment. The following linear segmented regression model (Equation 1) is used to estimate the level and trend before the intervention and the changes in level and trend following intervention.

Segmented regression model:

The components of the model outlined above are:

• Y_t is the outcome in month t.

• Time is a continuous variable indicating time in months at time t from the start of the observation period.

• Intervention is an indicator for time t occurring before (intervention = 0) or after (intervention = 1) the intervention, which was implemented at a specific month in the series.

• Time after intervention is a continuous variable counting the number of months after the intervention at time t, coded 0 before the intervention and (1, 2, 3 . . .) after the intervention.

In the model the parameter estimates can be explained as follows (see Equation 1):

• β₀ estimates the baseline level of the outcome at time zero.

• β₁ estimates the change in the outcome that occurs with each month before the intervention (i.e. the pre-intervention slope).

• β₂ estimates the change in level in the outcome immediately after the intervention, that is from the end of the preceding segment.

• β₃ estimates the change in slope after the intervention, compared with the slope before the intervention.

• The sum of β₁ and β₃ is the post-intervention slope.

• The error term e_t at time t represents the random variability not explained by the model. It consists of a normally distributed random error and an error term at time t that may be correlated to errors at preceding or subsequent time points.

Using the model to estimate level and slope changes associated with the intervention, this allows control for baseline level and slope, a major strength of segmented regression analysis.

In the analysis undertaken here, the parameter estimates and p-values from the linear segmented regression model are presented for the total models and subgroup analyses. The 95% confidence intervals (CIs) of the parameter estimates were calculated using the methodology of Zhang et al. 64

Interrupted time series analysis was undertaken for each outcome measure at each site (total model), followed by ITS for each subgroup outlined above. The subgroup analysis was undertaken to investigate further whether or not certain subgroups had more or less change on the outcome measures; for example, were certain diagnostic groups more likely to see a fall in admission rate after the introduction of a CCN team?

To summarise, the key outcomes of interest in this ITS analysis are:

1. change in level immediately after the intervention

2. difference between pre- and post-intervention slopes.

The findings from the segmented regression were used to predict the post-CCN monthly values that would have been expected if the pre-CCN trends had continued. This was calculated using Equation 2. It was then possible to compare the values derived from the Equation 1 model to investigate the difference between what was predicted without CCN intervention and predicted with CCN.

Model for prediction of post CCN, assuming pre-CCN trend:

Data were extracted for each site for at least 2 years before and at least 2 years after the intervention in order to minimise the effect of seasonal variation. 65 Early analysis indicated that there were serious discontinuities in the HES data before April 2006, owing to the introduction of Payment by Results, so no data were extracted before this date for any site. Data were extracted up to March 2013 for all but one site: in site B data were extracted from April 2009 to March 2014, owing to the later intervention date (October 2011).

Data were adjusted for autocorrelation if required. Correcting for autocorrelation avoids underestimating standard errors and overestimating significance of the effects of an intervention. There may be seasonal patterns in monthly time series, in which the rate of admissions in January of 1 year is more similar to rate of admissions 1 year previously than to rate of admissions in other months. This is an example of higher-order autocorrelation. For estimating seasonal autocorrelation, the autoregression model needs to evaluate correlations between error terms separated by multiples of 12 months. Accounting for seasonally correlated errors usually requires at least 24 monthly data points. 65

To assess the fit of the final model, the residuals were examined around the predicted regression lines. Residuals that are normally distributed and that follow no observable pattern over time indicate that the assumptions underlying the linear model are met. Partial autocorrelation function and autocorrelation function residual plots were examined. The Durbin–Watson statistic was then used to assess remaining autocorrelation. This statistic tests for serial autocorrelation of the error terms in the regression model; values close to 2 indicate no serious autocorrelation.

Table 11 shows the specific analyses undertaken for each site for the main analysis and Table 12 for the complex condition analysis.

A p-value of < 0.05 was considered to indicate statistical significance. The p-values for each model are presented with no adjustments for multiple significance testing. Owing to the large number of models, the analysis has resulted in a large numbers of significance tests and these may be difficult to interpret because, if we go on testing long enough, we will inevitably find something that is ‘significant’. Therefore, caution should be made in attaching too much importance to a lone significant result among a mass of non-significant ones. It may be the 1 in 20 that we expect by chance alone. 66 Simply describing what was done and why, and discussing the possible interpretations of each result informed by the qualitative data to understand the nature of and changes to services, should enable the reader to reach a reasonable conclusion without the help of Bonferroni adjustments. 67

All analyses were undertaken on IBM SPSS Statistics version 22 (IBM Corporation, Armonk, NY, USA). A sample of the IBM SPSS Statistics syntax used is shown in Appendix 3.

Site A1

The introduction of the CCN team was in April 2008. Data were extracted from April 2006 through to March 2013.

Rate of admissions per 100,000

The number of admissions was examined between April 2006 and March 2008, covering the 2-year timeframe prior to the introduction of CCN. In total, there were 8267 admissions. Table 13 shows the top 12 reasons for admission. These are based on the primary diagnoses grouped into ICD-10 code blocks and account for 63.5% of the total non-elective admissions.

TABLE 13 - Site A1: top 12 reasons for finished emergency admissions in the 12 months prior to introduction of CCN

Primary diagnosis	Number of finished emergency admissions	%
J00–J06: acute upper respiratory infections	798	15.2
R50–R69: general symptoms and signs	689	13.1
R10–R19: symptoms and signs involving the digestive system and abdomen	615	11.7
B25–B34: other viral diseases	567	10.8
J20–J22: other acute lower respiratory infections	430	8.2
R00–R09: symptoms and signs involving the circulatory and respiratory systems	381	7.3
S00–S09: injuries to the head	376	7.2
J40–J47: chronic lower respiratory diseases	362	6.9
A00–A09: intestinal infectious diseases	327	6.2
K50–K52: non-infective enteritis and colitis	312	5.9
S50–S59: injuries to the elbow and forearm	211	4.0
R20–R23: symptoms and signs involving the skin and subcutaneous tissue	179	3.4
Total top 12	5247	100.0

The most common reason for admission was acute upper respiratory infection, which accounts for 15% of the ‘top 12’. Injuries to the head and injuries to the elbow and forearm were also in the top 12 primary diagnoses. The research team did discuss whether or not these should be retained, as they may or may not be within the remit of the CCN teams. For example, minor head injuries not requiring sustained observation might be susceptible to CCN intervention, while major head injuries clearly would not, at least in relation to non-elective admission. However, CCN teams might influence how long a child with a significant head injury remains in acute care. We decided to retain these conditions in the top 12 analysis as a form of control, allowing the subgroup analysis to demonstrate whether or not admission rates for these might change, alongside those for other conditions.

Figure 4 shows the seasonalised monthly rate of admissions per 100,000, along with the regression lines before and after the introduction of CCN (April 2008). It also shows the national data and the national regression line. The overall trend in the national rate shows an increasing admission rate of 0.49 per month (95% CI 0.30 to 0.68 per month) over the whole time period, which was a statistically significant increase (p < 0.001). The rate of admissions per 100,000 in site A1 was higher than the national rate throughout the period covered.

FIGURE 4.

Site A1: overall admission rate by month (per 100,000).

At the start of the analysis period in April 2006, the rate of admissions per 100,000 was 455 per month. Immediately prior to the introduction of CCN in April 2008, there was a rise in admissions of 2.28 per 100,000 per month (95% CI –0.91 to 5.47 per 100,000 per month), but this increase was not statistically significant (p = 0.165). There was a statistically significant fall of 60.70 (95% CI –144.67 to –6.73) in the admission rate immediately after the intervention (p = 0.030). Overall, however, after the intervention, the magnitude of the underlying trend increased by 1.89 (95% CI –1.42 to 5.20; p = 0.226), with no statistically significant difference between pre- and post-intervention slopes. Following the introduction of CCN, the rate of increase of admissions was, thus, 4.17 (2.28 + 1.89) per 100,000 per month.

As can be seen in Figure 4, from April 2008 through to January 2011, the regression line with CCN was below the regression line predicted without CCN. This indicates a lower number of admissions with CCN than would have been expected without CCN. Just looking at a 2-year time frame post CCN, this difference represents 399 fewer admissions than would have been predicted without CCN.

In the overall model, the Durbin–Watson statistic was 1.54, which is close to 2, indicating no serious autocorrelation in this site. There were statistically significant falls in the admission rates immediately after CCN for:

• males: statistically significant fall in admissions immediately after the intervention (p = 0.035)

• age group 1–4 years: statistically significant fall in admissions immediately after the intervention (p = 0.012)

• other specialty: statistically significant fall in admissions immediately after the intervention (p = 0.001)

• white ethnic group: statistically significant fall in admissions immediately after the intervention (p = 0.019)

• A00–A09: intestinal infectious diseases – statistically significant fall in admissions immediately after the intervention (p = 0.005)

• R50–R69: general symptoms and signs – statistically significant fall in admissions immediately after the intervention (p = 0.003)

• Sunday admissions: statistically significant fall in admissions immediately after the intervention (p = 0.011)

• Sunday discharges: statistically significant fall in admissions immediately after the intervention (p = 0.010).

Within the other subgroups, there were no statistically significant findings.

Bed-days per 100,000

Figure 5 shows the monthly number of bed-days per 100,000, along with the regression lines before and after the introduction of CCN (April 2008). It also shows the national data and the national regression line. The overall trend in the national rate shows a flat rate, with the regression giving a small fall in the monthly number of bed-days per 100,000 of 0.20 (95% CI –1.00 to 0.60), which was not statistically significant (p = 0.623). The number of bed-days per 100,000 per month in site A1 was higher than the national rate throughout the reporting period.

FIGURE 5.

Site A1: analysis for bed-days by month (per 100,000).

In the overall model, the Durbin–Watson statistic was 1.874, which is close to 2, indicating no serious autocorrelation.

At the start of the analysis period in April 2006, the number of bed-days in the site was 1390 per 100,000 per month. Immediately prior to the introduction of CCN there was a rise of 10.49 (95% CI –0.33 to 21.31) in the number of bed-days per 100,000 per month (p = 0.335). There was a fall of 22.98 (95% CI –160.18 to 206.15) bed-days per 100,000 per month immediately after the intervention, but this was not a statistically significant change (p = 0.900). After the intervention, the magnitude of the underlying trend in bed-days per 100,000 per month was reduced by 13.34 (95% CI –35.34 to 8.67; p = 0.238), bringing it much closer to the national average. Following the introduction of CCN, there was thus a decrease of 2.65 bed-days per 100,000 per month (10.49 – 13.14), although this change did not reach statistical significance. There were no statistically significant findings in any of the subgroup analyses.

In the 2-year time frame after the introduction of CCN, the regression models suggest that there were 1548 fewer bed-days than would have been predicted without CCN.

Length of stay

Figure 6 shows the average LOS in this site, along with the regression lines before and after the introduction of CCN (April 2008). It also shows the national data and the national regression line. The overall trend in the national rate is flat, with the regression giving a small fall in the mean monthly LOS of 0.002 days (95% CI –0.002 to –0.001 days) over the study period, which was a statistically significant fall (p < 0.001). The average LOS in site A1 was similar to the national rate throughout the reporting period.

FIGURE 6.

Site A1: average LOS by month.

At the start of the analysis period (April 2006), the average LOS in the site was 1.561 days. Prior to the introduction of CCN there was very small increase in LOS of 0.006 days per month (95% CI –0.013 to 0.024 days per month; p = 0.558). There was an increase of 0.086 days (95% CI –0.228 to 0.401 days) immediately after the intervention, but this was not statistically significant (p = 0.593). After the intervention, the magnitude of the underlying trend was reduced by 0.013 days per month (95% CI –0.032 to 0.007 days per month; p = 0.183), but this change was not statistically significant. Following the introduction of CCN there was, thus, a decrease of 0.007 days in average LOS per month, bringing it below the national average.

In the overall model, the Durbin–Watson statistic was 1.561, which is close to 2, indicating no serious autocorrelation. None of the subgroup analyses showed any statistically significant findings.

Site A2

The introduction of the CCN team in this site was in February 2008. Data were extracted from April 2006 through to March 2013.

Rate of admissions per 100,000

Between February 2006 and January 2008, the 2-year time frame prior to the introduction of CCN, the total number of non-elective admissions was 9147. Table 14 shows the top 12 reasons for admission, which account for 63.1% of the total. Although in a slightly different order, these top 12 are the same as were seen in site A1.

TABLE 14 - Site A2: top 12 reasons for finished emergency admissions

Primary diagnosis	Number of finished emergency admissions	%
J00–J06: acute upper respiratory infections	1329	23.0
B25–B34: other viral diseases	600	10.4
A00–A09: intestinal infectious diseases	541	9.4
R50–R69: general symptoms and signs	511	8.9
J20–J22: other acute lower respiratory infections	498	8.6
R10–R19: symptoms and signs involving the digestive system and abdomen	454	7.9
S00–S09: injuries to the head	431	7.5
J40–J47: chronic lower respiratory diseases	381	6.6
R00–R09: symptoms and signs involving the circulatory and respiratory systems	329	5.7
S50–S59: injuries to the elbow and forearm	251	4.4
K50–K52: non-infective enteritis and colitis	232	4.0
R20–R23: symptoms and signs involving the skin and subcutaneous tissue	212	3.7
Total top 12	5769	100.0

Figure 7 shows the seasonalised monthly rate of admissions per 100,000, along with the regression lines before and after the introduction of CCN. It also shows the national data and the national regression line. As can be seen, the rate of admissions per 100,000 in this site was higher than the national rates throughout the reporting period. For analysis for regression model for the national data, see Site A1, Rate of admissions per 100,000.

FIGURE 7.

Site A2: overall admission rate by month (per 100,000).

The total rate of admissions per 100,000 in this site, at the start of the analysis period in April 2006, was 498 per 100,000. Prior to the introduction of CCN in February 2008 there was an annual fall of 0.25 (95% CI –3.80 to 3.30) in the rate of admissions per 100,000 per month, but this was not statistically significant (p = 0.891). There was a small fall in the rate of admissions immediately after the intervention of 8.46 (95% CI –62.58 to 43.66) but this was not statistically significant (p = 0.760). Overall, after the intervention, the magnitude of the underlying trend was increased by 1.70 (95% CI –1.94 to 5.34; p = 0.363). Thus, after the intervention there was a rise in the rate of increase of admissions of 1.45 (–0.25 + 1.70) per 100,000 per month.

As can be seen in Figure 7, the post-intervention regression line was above the regression line predicted without CCN. This indicates a higher number of admissions with CCN than would have been expected without CCN. In the 2-year time frame after the CCN team was introduced, this represents 150 more admissions than would have been predicted without CCN.

Despite the overall absence of statistically significant change in total admissions, there were statistically significant findings for the change immediately after the intervention for some subgroups:

• unknown ethnic group: statistically significant rise in admissions immediately after the intervention (p < 0.001)

• A00–A09: intestinal infectious diseases – statistically significant fall in admissions immediately after the intervention (p = 0.016)

• J00–J06: acute upper respiratory infections – statistically significant fall in admissions immediately after the intervention (p = 0.041)

• R00–R09: symptoms and signs involving the circulatory and respiratory systems – statistically significant rise in admissions immediately after the intervention (p < 0.001)

• Tuesday admission: statistically significant fall in admissions immediately after the intervention (p = 0.005).

Within the other subgroups there were no statistically significant findings.

Bed-days per 100,000

Figure 8 shows the monthly number of bed-days per 100,000 in site A2, along with the regression lines before and after the introduction of CCN (February 2008). It also shows the national data and the national regression line. The number of bed-days per 100,000 was slightly higher in site A2 than the national rate throughout the reporting period.

FIGURE 8.

Site A2: bed-days by month (per 100,000).

At the start of the analysis period in April 2006, the total bed-days was 1068 per 100,000 per month. Prior to the introduction of CCN in February 2008 there was an annual rise of 10.62 (95% CI –3.33 to 24.57) in the number of bed-days per 100,000 per month, but this trend was not statistically significant (p = 0.140). There was a fall in bed-days immediately after the intervention of 58.20 (95% CI –271.15 to 154.75), but this was not statistically significant (p = 0.594). Overall, after the intervention, the magnitude of the underlying trend was decreased by 8.05 (95% CI –22.37 to 6.27; p = 0.274). Thus, after the intervention, bed-days per 100,000 were increasing by 2.57 (10.62 – 8.05) per month, that is, much more slowly than before the intervention.

As can be seen in Figure 8, the post-intervention regression line was below the regression line predicted without CCN. This indicates a lower number of bed-days per 100,000 with CCN than would have been expected without CCN. In the 2-year time frame after the introduction of CCN, this represents 1864 fewer bed-days than would have been predicted without CCN.

There were no statistically significant changes evident in any of the subgroup analyses.

Length of stay

Figure 9 shows the average LOS for site A2, along with the regression lines before and after the introduction of CCN (February 2008). It also shows the national data and the national regression line. The average LOS in site A2 was lower than the national LOS throughout the reporting period.

FIGURE 9.

Site A2: average LOS by month.

At the start of the analysis period in April 2006, the average LOS was 1.321 days. Prior to the introduction of CCN in February 2008 there was a very small increase in mean LOS of 0.004 days (95% CI –0.010 to 0.018 days), but this was not statistically significant (p = 0.588). There was a slight rise of 0.050 days (95% CI –0.159 to 0.260 days) immediately after the intervention, but this was not statistically significant (p = 0.639). After the intervention, the magnitude of the underlying trend was decreased by 0.004 days (95% CI –0.018 to 0.010 days; p = 0.550). Overall, therefore, following the introduction of CCN there was no increase in average LOS (0.004 – 0.004).

A single subgroup analysis (for deprivation group 3) showed a small increase in average LOS of 0.047 days after the introduction of CCN (p = 0.049).

Site C

This site was used as a control site as it had not been able to introduce the planned CCN service change during the period of our project. A dummy intervention date, which was the same as site A1 (April 2008) was used. The rationale for this was to explore change in a site that had not implemented change with those that had.

Rate of admissions per 100,000

The number of admissions was examined during the time period between April 2006 and March 2008, which is the 2-year time frame prior to the dummy introduction of CCN in April 2008. The total number of admissions was 20,434. Table 15 shows the top 12 reasons for finished emergency admissions by primary diagnosis, which represents 57.9% of the total. The top 12 were the same as seen in sites A1 and A2.

TABLE 15 - Site C: top 12 reasons for finished emergency admissions

Primary diagnosis	Number of finished emergency admissions	%
J00–J06: acute upper respiratory infections	2002	16.9
R50–R69: general symptoms and signs	1593	13.5
R00–R09: symptoms and signs involving the circulatory and respiratory systems	1293	10.9
R10–R19: symptoms and signs involving the digestive system and abdomen	1153	9.7
S00–S09: injuries to the head	1017	8.6
J20–J22: other acute lower respiratory infections	1015	8.6
J40–J47: chronic lower respiratory diseases	772	6.5
B25–B34: other viral diseases	665	5.6
S50–S59: injuries to the elbow and forearm	659	5.6
A00–A09: intestinal infectious diseases	647	5.5
K50–K52: non-infective enteritis and colitis	589	5.0
R20–R23: symptoms and signs involving the skin and subcutaneous tissue	426	3.6
Total top 12	11,831	100.0

Figure 10 shows the seasonalised monthly rate of admissions per 100,000, along with the regression lines before and after the dummy introduction of CCN in April 2008. It also shows the national data and the national regression line. The admission rate per 100,000 was similar to the national rate throughout the study period.

FIGURE 10.

Site C: overall admission rate by month (per 100,000).

At the beginning of the observation period, there were 290 admissions per 100,000 per month. Before the intervention date, there was a statistically significant month-to-month increase in the rate of admissions of 4.44 per month (95% CI 2.66 to 6.21 per month; p < 0.001). Immediately after the dummy intervention date, the rate of admissions dropped abruptly, by 22.06 (95% CI –52.09 to 7.96) per month, but this was not statistically significant (p = 0.154). After the mock intervention point, it is estimated that the magnitude of the underlying trend decreased by 5.26 admissions per month (95% CI –7.10 to 3.42 admissions per month; p < 0.001), indicating a statistically significant difference between pre- and post-intervention slopes. Overall, following the mock intervention point, there was a decrease of 0.82 admissions per month per 100,000 (4.44 – 5.26).

Within all subgroups there were some statistically significant findings for the pre-intervention increase in the rate of admissions per 100,000 and between pre- and post-intervention slopes, but not for the change after the mock intervention point. These subgroup results, alongside the total results, suggest some change in the way that HES data were recorded in site C before and after the mock intervention point of April 2008.

Bed-days by month (per 100,000)

Figure 11 shows the monthly number of bed-days per 100,000 per month, along with the regression lines before and after the dummy introduction of CCN (April 2008) in site C. It also shows the national data and the national regression line. The number of bed-days per 100,000 per month was slightly lower than the national rate throughout.

FIGURE 11.

Site C: bed-days by month (per 100,000).

Before the dummy intervention date, there was a slight but not statistically significant increase month to month in the number of bed-days per 100,000 (0.95 bed-days, 95% CI –5.56 to 7.46 bed-days; p = 0.776). Immediately after the dummy intervention, the number of bed-days per 100,000 per month fell by 49.98 (95% CI –160.14 to 60.19), but this was not statistically significant (p = 0.377). After April 2008, the magnitude of the underlying trend was increased by 0.47 bed-days per month (95% CI –6.69 to 7.22 bed-days; p = 0.893). Post intervention there was an increase in the number of bed-days per month per 100,000 of 1.42 (0.95 + 0.47).

Unlike the analysis of the admissions data for this site, there was no evidence of any statistically significant change in any of the subgroup analyses.

Length of stay

Figure 12 shows the average LOS, along with the regression lines before and after the mock intervention point (April 2008). It also shows the national data and the national regression line. The average LOS in site C was lower than the national LOS throughout.

FIGURE 12.

Site C: average LOS by month.

At the beginning of the observation period, the average LOS was 1.481 days. Before the mock intervention point LOS was decreasing by 0.014 days per month (95% CI –0.022 to –0.007 days) during the pre-intervention period, which was statistically significant (p = 0.001). Immediately after this point, the average LOS rose by 0.101 days (95% CI –0.033 to 0.236 days) but this was not statistically significant (p = 0.142). After April 2008, the magnitude of the underlying trend increased by 0.019 days (95% CI 0.011 to 0.027 days; p < 0.001), showing a statistically significant difference between pre- and post-intervention slopes. Overall, then, after the mock intervention point, average LOS increased by 0.005 days (–0.014 + 0.019).

As with the admission data, within each subgroup there were some statistically significant findings for the pre-intervention increase in LOS and between mock pre- and post-intervention slopes, but not for after April 2008.

Site B

This was the second site where a planned CCN team targeting acute activity had not been implemented. We used the same dummy intervention date of April 2008 as in the site C analysis. The ITS was undertaken only on rate of admissions per 100,000 as this site already had a CCN service for children with complex conditions, the impact of which it was felt might be evident on bed-days and LOS.

Rate of admissions per 100,000

The number of admissions was examined during the time period between April 2006 and March 2008, which is the 2-year time frame prior to the dummy introduction of CCN in April 2008. In total there were 14,189 admissions. Table 16 shows the top 12 primary diagnoses leading to admission, which together account for 60.6% of the total. There is one difference in the top 12 compared with sites A1, A2 and C. R20–R23: symptoms and signs involving the skin and subcutaneous tissue is replaced by J10–J18: influenza and pneumonia.

TABLE 16 - Site B: top 12 reasons for finished emergency admissions

Primary diagnosis	Number of finished emergency admissions	%
J00–J06: acute upper respiratory infections	1506	17.5
J20–J22: other acute lower respiratory infections	998	11.6
B25–B34: other viral diseases	916	10.7
R50–R69: general symptoms and signs	852	9.9
R10–R19: symptoms and signs involving the digestive system and abdomen	847	9.9
R00–R09: symptoms and signs involving the circulatory and respiratory systems	716	8.3
A00–A09: intestinal infectious diseases	500	5.8
S00–S09: injuries to the head	494	5.7
J10–J18: influenza and pneumonia	492	5.7
S50–S59: injuries to the elbow and forearm	474	5.5
K50–K52: non-infective enteritis and colitis	408	4.7
J40–J47: chronic lower respiratory diseases	394	4.6
Total top 12	8597	100.0

Figure 13 shows the seasonalised monthly rate of admissions per 100,000, along with the regression lines before and after the dummy intervention point in April 2008. It also shows the national data and the national regression line. The admission rate per 100,000 in site B was slightly higher than the national rate throughout.

FIGURE 13.

Site B: overall admission rate by month (per 100,000).

At the beginning of the observation period, the rate of admissions was 381 per 100,000 per month. Just before the intervention date, there was a small but not statistically significant fall in the monthly rate of admissions per 100,000 (–0.60, 95% CI –3.005 to 1.80; p = 0.62). Immediately after the mock intervention date, the rate of admissions rose abruptly, by 36.90 (95% CI –3.7 to 77.5) per 100,000 per month, but this was not statistically significant (p = 0.079). After April 2008, the magnitude of the underlying trend increased by 1.44 per 100,000 per month (95% CI –1.05 to 3.93 per 100,000 per month; p = 0.260). Overall, after April 2008, the rate of admissions rose by 0.84 per 100,000 per month (–0.604 + 1.441).

There were some statistically significant findings within subgroups: increases for those aged 1–4 years, for those from black and minority ethnic groups, for Saturday, Sunday and Monday admissions and for Thursday discharges, and for three disease groups (infective enteritis and colitis; circulation and respiratory signs and symptoms; and digestive system/abdomen signs and symptoms).

Site A1

Bed-days for complex conditions

Figure 14 shows the monthly number of bed-days for children with complex conditions, expressed per 100,000 children, along with the regression lines, before and after the introduction of CCN (April 2008). It also shows the national data and the national regression line. The overall trend in the national rate for these conditions, with the regression model, shows a rise in the number of bed-days per 100,000 per month of 0.76 (95% CI –0.48 to 1.04), which was statistically significant (p < 0.001). The number of bed-days per 100,000 per month in site A1 was higher than the national rate throughout the study period.

FIGURE 14.

Site A1: bed-days by month (per 100,000) – complex conditions.

At the start of the analysis period in April 2006, there were 314.932 bed-days per 100,000 per month. Prior to the introduction of CCN in February 2008 there was an annual rise of 11.92 (95% CI 0.07 to 23.77) in the number of bed-days per 100,000 per month (p = 0.052). There was a fall immediately after the intervention of 50.40 bed-days (95% CI –250.93 to 150.14 bed-days) but this was not statistically significant (p = 0.624). After the intervention, it is estimated that the magnitude of the underlying trend was decreased by 12.94 bed-days (95% CI –25.23 to –0.65 bed-days; p < 0.001), hence this demonstrates there was a statistically significant difference between pre- and post-intervention slopes. Following the introduction of CCN there was a decrease of 1.02 per month (11.92 – 12.94).

Owing to the need to suppress data to preserve anonymity, day of admission and day of discharge were grouped as Monday–Thursday and Friday–Sunday, and deprivation as ‘most deprived’ and ‘other’. Ethnicity was not analysed at all, for the same reason. There were some statistically significant findings within remaining subgroups. Statistically significant differences (reductions) between before and after intervention slopes were found for children aged 2 weeks to 1 year, paediatric admissions (so described), children from the most deprived areas, Monday–Thursday admissions and discharges, and elective admissions.

Length of stay

Figure 15 shows the monthly average LOS for the chosen complex conditions, along with the regression lines before and after the introduction of CCN (April 2008). It also shows the national data and the national regression line. The overall trend in the national rate is flat, with the regression giving a small monthly fall in the LOS of 0.003 days per month (95% CI –0.004 to –0.002 days), which was statistically significant (p < 0.0012). The average LOS was similar to the national rate throughout the study period.

FIGURE 15.

Site A1: average LOS by month – complex conditions.

At the start of the analysis period in April 2006, the average LOS was 2.54 days. Prior to the introduction of CCN in April 2008 there was an annual rise of 0.064 days (95% CI –0.01 to 0.14 days) in LOS per month (p = 0.112). There was no statistically significant change immediately after the intervention (–0.381 days, 95% CI –1.71 to 0.94 days; p = 0.575). After the intervention, it is estimated that the magnitude of the underlying trend was decreased by 0.056 days (95% CI –0.136 to 0.024 days; p = 0.184) but there was no overall statistically significant difference between pre- and post-intervention slopes. Overall, after the introduction of CCN there was a small increase of 0.008 per month (0.064 – 0.056).

There was one statistically significant difference (reduction) between pre- and post-intervention slopes, for children aged between 2 weeks and 1 year, of –0.013 bed-days [0.096 + (–0.109)].

Site A2

Bed-days for complex conditions

Figure 16 shows the monthly number of bed-days per 100,000 children, along with the regression lines before and after the introduction of CCN (February 2008). It also shows the national data and the national regression line. The number of bed-days per 100,000 was similar to the national rate throughout the study period.

FIGURE 16.

Site A2: bed-days by month (per 100,000) – complex conditions.

In the overall model, the Durbin–Watson statistic was 2.070, which is close to 2, indicating no serious autocorrelation.

At the start of the analysis period in April 2006, there were 336.30 bed-days per month per 100,000. Prior to the introduction of CCN in February 2008 there was an annual rise of 3.51 (95% CI –7.28 to 14.29) in the number of bed-days per 100,000 per month (p = 0.526). There was a fall of 86.59 bed-days (95% CI –251.18 to 78.01 bed-days) immediately after the intervention, but this was not statistically significant (p = 0.306). After the intervention, the magnitude of the underlying trend was decreased by 0.82 bed-days (95% CI –11.89 to 10.25 bed-days; p = 0.885), thus there was no statistically significant difference between pre- and post-intervention slopes. Overall, following the introduction of CCN there was an increase of 2.69 bed-days per month (3.506 – 0.818).

There were no statistically significant findings within subgroups, except in relation to weekend admissions, which showed a statistically significant reduction immediately after introduction of CCN (–10.06; p = 0.041) but this was not sustained over time.

Length of stay

Figure 17 shows the monthly average LOS for children with complex conditions, along with the regression lines before and after the introduction of CCN (February 2008). It also shows the national data and the national regression line. The average LOS was similar to the national rate throughout the study.

FIGURE 17.

Site A2: average LOS by month – complex conditions.

At the start of the analysis period in April 2006, the average LOS was 3.18 days. Prior to the introduction of CCN in February 2008 there was an annual rise of 0.02 days (95% CI –0.06 to 0.10 days) in LOS per month. There was fall of 0.11 (95% CI –1.34 to 1.12) immediately after the intervention, but this was not statistically significant (p = 0.863). After the intervention, the magnitude of the underlying trend decreased by 0.018 (95% CI –0.100 to 0.064; p = 0.668), thus there was no statistically significant difference between pre- and post-intervention slopes. Overall, following the introduction of CCN there was an increase of 0.002 per month in average LOS.

There were no statistically significant findings within subgroups.

Site B

Bed-days for complex conditions

Figure 18 shows the monthly number of bed-days per 100,000, along with the regression lines before and after the introduction of CCN (October 2011). It also shows the national data and the national regression line. The number of bed-days per 100,000 in site B was higher than the national rate throughout the study period.

FIGURE 18.

Site B: bed-days by month (per 100,000) – complex conditions.

At the start of the analysis period in April 2009, there were 561 bed-days per 100,000. Prior to the introduction of CCN there was an annual rise of 0.41 (95% CI –7.63 to 8.45) in the number of bed-days per 100,000 per month (p = 0.920). There was a rise of 48.76 (95% CI –158.62 to 256.15) immediately after the intervention, but this was not statistically significant (p = 0.647). After the intervention, the underlying trend increased by 4.34 (95% CI –7.65 to 16.32; p = 0.481), showing no statistically significant difference between pre- and post-intervention slopes. Overall, following the introduction of CCN there was an increase of 4.75 per month (0.41 + 4.34).

There were no statistically significant findings within subgroups, except in relation to the least deprived group when there was a statistically significant difference in the pre- and post-intervention slopes.

Length of stay

Figure 19 shows the monthly average LOS, along with the regression lines before and after the introduction of CCN (October 2011). It also shows the national data and the national regression line. The average LOS was similar to the national rate throughout the study period.

FIGURE 19.

Site B: LOS by month – complex conditions.

At the start of the analysis period in April 2009, the average LOS was 3.174 days. Prior to the introduction of CCN in October 2011 there was an annual rise of 0.004 (95% CI –0.032 to 0.039) in LOS per month (p = 0.842). There was no statistically significant change immediately after the intervention (–0.340, 95% CI –1.254 to 0.575; p = 0.470). After the intervention, it is estimated that the magnitude of the underlying trend was increased by 0.009 (95% CI –0.044 to 0.062; p = 0.743), showing no statistically significant difference between pre- and post-intervention slopes. Overall, following the introduction of CCN there was an increase of 0.013 per month (0.004 + 0.009).

There were no statistically significant findings within subgroups.

Site D: acute hospital admission avoidance for children with complex conditions

Rate of admissions per 100,000 children with complex conditions

Analysis for this site was undertaken on G codes only, diseases of the nervous system, as described in Outcome measures. There was no subgroup analysis owing to suppression of data in order to preserve anonymity.

Figure 20 shows the rate of admissions per 100,000, along with the regression lines before and after the introduction of CCN (March 2008). It also shows the national data and the national regression line. The overall trend in the national data shows an increasing admission rate of 0.07 per month (95% CI 0.06 to 0.07 admissions) over the whole study period, which was statistically significant (p < 0.001). The rate of admissions per 100,000 was higher in site D than the national rate until 2011. From 2011, there was a dip in the rate of admissions per 100,000, with lower rates than the national rates, which might suggest possible discontinuity in the data.

FIGURE 20.

Site D: G-code rate of admissions by month (per 100,000).

At the start of the analysis period in April 2006, there were 25.91 admissions per 100,000. Prior to the introduction of CCN in March 2008 there was an annual rise of 0.42 (95% CI –0.13 to 0.97) in the rate of admissions per 100,000 per month, but this was not statistically significant (p = 0.135). There was no statistically significant change in the rate of admissions immediately after the intervention (6.61 admissions, 95% CI –2.24 to 15.45 admissions; p = 0.147). After the intervention, the magnitude of the underlying trend decreased by 0.81 admissions (95% CI –1.38 to –0.24 admissions; p = 0.006), showing a statistically significant difference between pre- and post-intervention slopes. Overall, following the introduction of CCN there was a decrease of 0.39 admissions per 100,000 per month (0.423 – 0.809).

As discussed above, there was a fall in the rate of admissions in 2011 suggesting some possible discontinuity in the data. We therefore re-ran the analysis using only data up to 2 years post intervention (April 2008 to March 2010).

As before, the overall trend in the national rate shows an increasing admission rate of 0.06 per month (95% CI 0.04 to 0.07 per month) over the reduced time period, which was statistically significant (p < 0.001) (Figure 21). Table 17 shows the parameter estimates from the linear segmented regression models. At the start of the analysis period in April 2006, there were 25.41 admissions per 100,000. Prior to the introduction of CCN in March 2008 there was an annual rise of 0.52 (95% CI –0.04 to 1.08) in the rate of admissions per 100,000 per month, but this was not statistically significant (p = 0.077). There was no statistically significant change in the rate of admissions immediately after the intervention (–1.90 admissions, 95% CI –12.93 to 9.13 admissions; p = 0.738). After the intervention, the magnitude of the underlying trend was decreased by 0.47 admissions (95% CI –1.27 to 0.33 admissions; p = 0.254), showing no statistically significant difference between pre- and post-intervention slopes. Overall, following the introduction of CCN, there was an increase of 0.05 per month in admissions of children with complex conditions (0.52 – 0.47).

FIGURE 21.

Site D: G-code rate of admissions (per 100,000) – 2 years post intervention only.

TABLE 17 - Site D: G-code rate of admissions (per 100,000) – 2 years post intervention only

Durbin–Watson = 1.973.

Parameter estimates	Coefficient	Standard error	Beta	t-statistic	p-value
Intercept	25.405	4.102		6.193	< 0.001
Baseline trend	0.519	0.287	0.723	1.809	0.077
Level change after CCN	–1.897	5.628	–0.095	–0.337	0.738
Trend after CCN	–0.470	0.406	–0.375	–1.157	0.254

No subgroup analysis was undertaken because of the need to suppress data, owing to the relatively small number of admissions for G codes.

Non-elective admissions

1. The overall trend in the national rate shows an increasing non-elective admission rate of 0.49 per month (95% CI 0.30 to 0.68 per month) over the whole period of analysis, which was a statistically significant increase (p < 0.001).

2. Site A1, which is a generic CCN team (community based), had an admission rate that was higher than the national rate. The analysis shows a statistically significant fall in admissions immediately after CCN, but no statistically significant change in the slopes pre-CCN team compared with post-CCN team. The admission rate was increasing monthly pre CCN and post CCN. However, the abrupt fall in admissions post CCN suggests that CCN may have had an impact with an estimated 399 fewer admissions in the 2 years post CCN than predicted without CCN.

3. Site A2 was a generic CCN team (hospital based) and had an admission rate that was higher than the national rate. The analysis shows no statistically significant changes, although the monthly rate of admissions was falling slightly prior to CCN, there was a slight increase immediately post CCN, and a rise monthly post CCN. This suggests that CCN had no impact on the rate of non-elective hospital admissions.

Bed-days

1. The overall trend in bed-days per 100,000 nationally shows a flat rate, with the regression giving a small monthly fall in the number of bed-days per 100,000 of 0.20 per month (95% CI –1.00 to 0.60 bed-days), which was not statistically significant (p = 0.623).

2. In site A1 the rate of bed-days per 100,000 was increasing prior to CCN, with an annual rise of 10.49 (95% CI –0.33 to 21.31) in the number of bed-days per 100,000 per month (not statistically significant). There was a fall of 22.98 bed-days (95% CI –160.18 to 206.15 bed-days) immediately after the intervention, but this was also not statistically significant. Following the introduction of CCN there was a decrease of 2.85 bed-days per month, but this was not a statistically significant change. In the 2-year time frame post CCN, using the regression lines, this change in slope represents 1548 fewer bed-days than that predicted without CCN. Hence, although not statistically significant, there does seem to be some effect of CCN on bed-days.

3. Site A2, which was a generic CCN team (hospital based), shows no statistically significant changes, although there was an increase in admission rates. The rate of bed-days per 100,000 was rising prior to CCN (10.62, 95% CI –3.33 to 24.57) and, although there was an increase immediately post CCN (58.20, 95% CI –271.15 to 154.75), there was a slower monthly increase post CCN (2.57 per month). Hence, although not statistically significant, there does seem to be some effect of CCN on the rate of bed-days per 100,000. In the 2-year time frame post CCN, using the regression lines, this represents 1864 fewer bed-days than that predicted without CCN.

Length of stay

1. The overall trend in LOS nationally shows a flat rate, with the regression giving a small monthly fall in the LOS of 0.0018 days per month (95% CI –0.0023 to –0.0012 days), which was a statistically significant fall (p < 0.001).

2. In site A1 there were no statistically significant changes in slope or immediately after CCN. Although the LOS was similar to those seen nationally, by the end of the period the LOS had fallen to below the national average.

3. In site A2, the LOS was lower than the national rate, with no statistically significant changes in slope or immediately after CCN and no net change over the whole study period.

Complex care

Overview

The methodological approach taken to considering the impacts on costs and consequences of the CCN team is presented in this section. The cost implications are considered in two parts throughout: the cost to the NHS of having a CCN team in place in the different sites (part 1) and the cost implications as a result of a change in the resource use within the NHS (part 2).

We address these topics in the following order:

1. the cost to the NHS of having a CCN team in place

2. the cost implications as a result of a change in the resource use within the NHS

   1. the cost implications of a change in bed-days

   2. the cost implications of a change in non-elective admissions for common conditions

   3. the cost implications from wider resource-use changes

   4. the cost implications to parents and wider society

3. the health and well-being consequences

   1. MHFI questionnaires

   2. threshold analysis.

The primary analysis takes the perspective of the NHS and Personal Social Services, with all costs being discounted at a rate of 3.5% per annum, to ensure consistency with best evaluative practice as outlined by the National Institute for Health and Care Excellence (NICE) methods guide. 68 As outlined in Table 12 (see Chapter 6), the CCN teams have a different set of defining caseloads, and, as such, the analytical approach needed to be tailored to each team. Table 18 presents an overview of the types of analysis relevant to each of the sites. Note that site C is not included here as it was included in Chapter 6 as a control site only.

Part 1: the cost to the NHS of having a children’s community nursing team in place

We used a ‘bottom-up’ approach to ensure that differences in the CCN teams were reflected in the cost estimate.

To achieve this, each CCN team completed three questionnaires. A sample of these questionnaires can be found in Appendix 1. Multiple questionnaires were completed to account for changes in the team over the analytical period, typically as a response to changes in caseload.

This analysis utilises the staffing whole-time equivalent (WTE) and estimates of the distance travelled by the CCN team to attend patients to derive an average cost of the CCN team per annum, estimated over the entire analysis period. Details of the WTE were collected for all members of the team (which corresponded to thier working hours). Distance travelled was reported as total miles travelled by the team. All fields covered the period of the past week to ensure accuracy of reporting. The estimates across the three questionnaires were averaged to provide a mean team structure per annum across the analysis period.

Additional details of the methods and unit costs applied to derive the cost of the CCN teams can be found in Appendix 4 (methods applied in Chapter 7).

Part 2: the cost implications as a result of a change in resource use within the NHS

This section considers how these observed changes from Chapter 6 can be transformed into a total cost saved or incurred over the period of analysis. The primary analysis presented focuses on bed-days. This section also considers whether or not the cost implications of considering change in admissions, rather than bed-days, would affect any conclusions drawn from the bed-day analysis. Finally, there is a limited analysis of questionnaires completed by parents of children with complex conditions cared for by the CCN teams to ascertain if meaningful conclusions can be drawn as to the impact of the CCN teams on a wider definition of resource use in the NHS and out-of-pocket costs to the parents themselves.

The pros and cons of the two approaches (use of the ITS analyses from Chapter 6 compared with information from the parent questionnaires) are presented in Table 19. The table shows that each approach has strengths and weaknesses, and neither would be expected to show the full picture around resource use. Although the parent questionnaire considered a much wider definition of the resource use affected by the CCN teams, the limitations in the scale and scope of the questionnaires collected has led us to present the ITS analysis as the primary source, with bed-days selected for the reasons given above.

In order to compare the cost of having a CCN team in place (discussed in part 1) and the change in resource use, it is necessary to convert the rates of bed-days and admissions per 100,000 population reported in Chapter 6 to absolute estimates over the full analysis period. To do this the rates are adjusted by the size of the paediatric population in each of the four sites. The population sizes were drawn from the latest ONS estimates. 69

Part 3: the health and well-being consequences

An important additional consideration, beyond the net cost of the CCN team to the NHS, is the impact of the team on the health and well-being of the children being cared for and their parents. To consider the impact of the CCN teams on these factors, two approaches are considered: first, the use of a series of questionnaires conducted during the evaluation and, second, a threshold-type analysis considering the impact the CCN teams would have to have on population health to be considered a cost-effective allocation of NHS resources.

Site A1

The wide scope of the CCN team, reflected in their relatively broad caseload, coupled with the engagement of the site in the questionnaires, allows for results to be presented across all four of the cost scenarios, in addition to both evaluative investigations of the health and well-being consequences of the CCN team in that area.

Total cost implications

Table 21 reports both parts of the cost analysis across the primary analysis and the three additional analyses. All costs are presented as an average cost across the full analysis period for site A1 (59 months) taking into account all uncertainty. As such, the costs presented in the table may differ from the direct calculations made earlier in this section. In addition, the table presents estimates of the probability of the CCN team being cost saving given each of the scenarios in isolation. This provides a representation of the level of uncertainty underlying the mean estimates provided above and in Table 21.

TABLE 21 - Combined cost analyses across all scenarios, site A1

Evaluative scenario	Direct cost of the CCN team (£)	Cost saving (NHS/parents) (£)	Expected total cost saving (£)	Probability team being cost saving
Preferred analysis: bed-days (analyses 1 and 2a)	1,111,049	578,272	–532,777	0.26
Scenario 1: admissions (analyses 1 and 2b)	1,111,049	99,116	–1,011,933	0.09
Scenario 2: parent questionnaires (analyses 1 and 2c)	1,111,049	2,278,287	1,167,236	0.49
Scenario 3: parent questionnaires (analyses 1, 2c and 2d)	1,111,049	2,475,752	1,364,702	0.53

The table shows a number of important findings for site A1. First, there is a very large level of uncertainty in the estimates of annual cost saving, with estimates ranging from £99,115 to £2,475,752. This variation is indicative of the failure of a single measure, such as admissions, to demonstrate the full impact of the CCN team, as well as the significant limitations in the data available to demonstrate the impact of the teams, both from HES and from the questionnaires. Second, while the results of scenarios 2 and 3 may initially appear the most appealing and representative of the full impact of the CCN team, the high expected annual total cost saving is not reflected in a high probability of the teams being cost saving (shown in the final column). This is as a direct result of the very large level of variation and small sample size in the parent-completed questionnaires used to inform these scenarios. Figure 22 gives a graphical representation of this uncertainty in scenario 2. Each point in the figure represents a single realisation of the uncertainty from the 1000 draws from the Monte Carlo simulations, that is, given the uncertainty in the data each point is a realisation of what the true cost results could be. While many of these observations represent extreme cases, it highlights the important limitation in scenarios 2 and 3 and indicates why we chose to use the ITS analyses from Chapter 6 as the preferred analysis rather than the broader, but hugely uncertain, parent questionnaires.

FIGURE 22.

Graphical representation of the uncertainty in scenario 2, site A1.

Site A2

While the site A2 CCN team has a broad caseload matching site A1 they did not participate in the parent questionnaire section of this project. Therefore, the cost analysis of this team is limited to the preferred analysis and scenario 1.

Site B

As noted in Table 18, the site B CCN team has a more restrictive caseload than sites A1 and A2, covering only children with complex conditions (mostly neurological). As a result, and as discussed earlier in this chapter, the analysis conducted is limited to the preferred analysis (bed-days), considering only bed-days related to complex conditions. Although site B did return a number of parent questionnaires (detailed further in The cost implications to parents and wider society), there were too few to conduct the full cost analysis.

Site D

As noted in Table 18, the site D CCN team had a more restrictive caseload than sites A1, A2 and B, covering only children with complex neurological conditions. As a result, the analyses conducted are limited. Specifically, the analysis is limited to scenario 1, considering only admissions related to complex conditions. Although site D did return a number of parent questionnaires (detailed in The cost implications to parents and wider society), as with site B, these were deemed too few to conduct the full cost analysis.

The study as a whole

The main strength of our work is its contribution to a very small evidence base on the key policy imperative of delivering care ‘closer to home’. Transforming Community Services4 encouraged commissioners ‘to develop services so that children with support from family members can choose to be cared for at home at all stages of their illness or disability’ and suggested that this would deliver ‘high impact’ change. Community health services in general and those for children in particular are under-researched; the substantial challenges that Transforming Community Services4 outlined reflect this.

Our work constitutes the first formal evaluation of CCN services using quasi-experimental methods and has generated new knowledge about impact, quality, outcomes, activity and achievements in CCN services. It was also innovative in its design, using a formal quasi-experimental design with embedded longitudinal qualitative elements, allowing it to explore what changes work, for whom, and in what contexts.

As predicted in our proposal, our work took place during a period of radical change in health service commissioning and delivery. Because of this, we included an element in the project to explore how new commissioning arrangements affected service redesign that was already happening and that which was planned. This has given us insight into the challenges faced by those attempting to bring about ‘high impact’ change in community services and to maintain the impact of those already introduced, at a time of restructuring at higher levels in the organisation.

A further strength of our work was the regular input provided by our project advisory group. This allowed us to ground our work in the realities of the NHS context. For example, the group members provided helpful insight into how HES data are coded in day-to-day practice, as well as guiding us in our choice of disease groups and variable splits for the ITS analysis.

The main limitations of our work mirror, to some degree, its strengths. Before submitting our proposal we had identified three areas of the country that had already introduced CCN services (sites A1 and A2) or were well on with planning their introduction or development (sites B and C). By the end of the project, one site (C) had abandoned its plans altogether, despite the considerable time, effort and resources that had already gone into planning, and another (B) had not implemented the acute admission avoidance service that would have partnered its CCN services for children with complex conditions. As a result we had to recruit a new site (D) that had implemented a service some time previously and could carry out only a more restricted range of analyses on sites B, C and D. These changes allowed us the insight into the challenges of developing community health services that we outlined above, and we adapted our research plans to reflect the changes. However, only in site A1 were we able to carry out all the work and analyses that we had originally planned.

Given this limitation, future researchers might consider evaluating only services that are already contracted, rather than those that are still in development. However, this may make it more difficult for researchers to respond to the timing of NIHR-commissioned calls for research. Researchers cannot control the fact that the NHS environment in which services are developed changes frequently, but they can develop research strategies to help deal with such change. For example, we added a supplementary research question about the impact of changing commissioning arrangements and this proved helpful in understanding service development.

The qualitative elements

As described in Chapter 5, we struggled to recruit newly referred parents to our study. On reflection, this was perhaps an ambitious approach, both because it is a small population from which to draw and also because of the challenges parents face during this difficult period soon after onset or diagnosis. As a result, we were unable to carry out the longitudinal element to the full extent we intended. Instead, we also drew on established users, of whom there were far more who were ready and able to participate, and carried out a single point of data collection to examine their experience of care and change retrospectively. This amendment to the longitudinal approach is a limitation, but was necessary in order to generate sufficient qualitative material to allow meaningful analysis of the data. We thus undertook a ‘true’ longitudinal analysis on two data collection points with just under half of the sample, but were still able to draw meaningful accounts of change from those who participated in one retrospective interview.

A strength of the study was the process evaluation of how the services were planned and implemented. We were able to build a detailed picture illuminating the complexity of this process for some, and the relative simplicity of it for others. Although we experienced attrition in some staff taking part, we were still able to attain sufficient information about service change.

As noted earlier, we were unable to undertake observations as planned. This presents a further, although minor, limitation to the qualitative work. Had we been successful in undertaking these observations, this may have added a descriptive layer to our understanding of the process of developing CCN services.

Interrupted time series analysis of impact on acute hospital activity

The statistical strengths and limitations of the ITS analysis as an approach in quasi-experimental evaluation are discussed in detail in Chapter 7. Here we discuss the more general issues related to the use of HES data for our work.

The main strengths of HES data are both their specificity and their potential universality. The data are gathered at a local level and therefore should provide a clear log of local activity in hospital settings that would be difficult to generate cost-effectively in any other way. At the same time, because data collection uses the same framework in all service sites, the data can be aggregated to provide national pictures of activity. This allowed us to see how our chosen case sites differed from or were similar to what was going on in England as a whole over the periods we selected for analysis. In both local and national applications, the long-running nature of the data collection opens up opportunities for longitudinal analysis and therefore, potentially, the ability to track change in activity alongside change in policy and practice.

In reality, however, our experience was that HES data needed to be used and interpreted with care. First, we found unexpected discontinuities in runs of data. In some cases we could propose credible explanations for these (e.g. the introduction of payment by results) and take this into account by analysing only data from after the introduction of this change. In other cases we struggled to identify possible causes (e.g. the spikes in acute activity in site D; see Chapter 6) and therefore needed to interpret the data cautiously. Second, the smaller the group in question, the more cautious we need to be. HES use requires suppression of data when numbers are small, and small numbers, by definition, introduce added uncertainty into results. This was particularly the case for site D where the analysis was based on a single disease category group. Third, service commissioners and managers interviewed in the qualitative elements of the work often told us that standards of HES data collection were not as high as they might have wished, particularly in relation to diagnostic categories for children, whose symptoms may be more vague than in adults, making a secure diagnosis difficult, even after investigation.

Overall, however, we believe that ITS analysis of HES data of the sort we carried out is a powerful addition to the quasi-experimental tool kit in service delivery and organisational research, in which fully experimental evaluation is unethical and/or impractical. However, it is important that analysis of this sort is based on a very thorough understanding both of the underlying structure of the data over time and of the quality of collection at local levels.

The costs and consequences study

In this element of our work, we used the analysis of the HES data to examine the cost implications of introducing or expanding CCN services, by looking at changes in NHS activity, as measured in this instance by bed-days, and wider resource use. We also looked at parents’ satisfaction with CCN services. There were some inevitable limitations with this part of our work, over and above those outlined above in relation to using HES data.

The CCN teams themselves provided good information about the costs of running the teams, which we used in the health economics modelling. However, this modelling did not take into account any disinvestment in acute services, which we know happened in site A and which might have been expected in other areas planning to move services into the community. Thus, our comparison of the costs of CCN teams with the costs of hospital activity does not consider any such disinvestment. Second, the small number of questionnaires completed by parents about resource use and the very high level of variation in the data within them limited the extent to which we could draw conclusions about cost implications for wider resource use and costs to society.

Third, because of the design of the study, resource-use questionnaires were limited to parents of children with complex needs. Although these children form a core component of CCN team caseloads, they are not the only children served by generic CCN teams (e.g. sites A1 and A2). Therefore, any conclusions made are limited to this population only. Together, these issues underline the importance of treating the conclusions of the cost study with caution.

Finally, the additional analysis utilising parental questionnaires highlights that a broader estimate of costs beyond simple measures of bed-days and admissions may be required to define fully the NHS activities influenced by CCN services. However, the complex nature of the range of services provided in a hospital and community setting makes any attempt to disaggregate all interactions with the NHS into constituent parts challenging.

These challenges in constructing a robust study represent significant limitations to the cost–consequence analyses. Two main limitations of this type have been identified: reliance on provider questionnaires to inform cost of the team rather than the true cost of setting up and maintaining a team, and the use of average rather than marginal costs to represent change in bed-days and admissions. First, the cost–consequence analyses utilise questionnaires completed by the providers to estimate an average cost for each site. Although this approach will accurately reflect the WTE of the teams when sampled, it is potentially biased if the sampled periods do not represent a true reflection of the average. Furthermore, this approach is likely to overlook broader costs of maintaining a CCN team and is unable to reflect potentially high, upfront costs of implementing a team. Second, the unit cost estimates of bed-days and admissions saved as a result of the CCN teams are based on average costs estimates produced for the NHS. The use of an average, rather than a marginal, cost implies that the cost savings estimated as a result of reduced hospital activity are overestimates of the true cost. The scale of this overestimate is impossible to estimate given existing data. However, as none of the sites was found to be associated with a cost-saving, this limitation does not affect the conclusions of the analysis. Further research is needed on the marginal cost of displaced bed-days and hospital admissions before a robust estimate of the cost impact of CCN teams can be assessed.

Site A

These two sites had introduced (site A1) or expanded (site A2) CCN services as part of a major redesign of children’s services across a large health and social care community.

Site B

Site B was selected for our study because, well before we submitted our proposal, plans were being developed to introduce a CCN team that would target acute admissions and LOS for all children. This service would complement an already existing CCN service for children with complex needs that was also planned to expand its remit in order to meet the needs of children with continuing care packages. In the end, the acute home-care team was not introduced and our work therefore focused on the complex care team and the impact of the changes it had undergone. However, ITS analysis of admissions for common conditions, using a dummy intervention point the same as that in site A1, allowed us to explore what happened when a planned change targeting acute activity did not actually take place. The health economics work, however, was confined to the impact of changed services on children with complex conditions.

Site B was an area of high deprivation but with non-elective admission rates for common conditions only slightly higher than the national average. Admissions were falling slightly before the dummy intervention point but rose substantially (although not statistically significantly) after, and this rise was maintained over the analysis period. Significant increases were seen for children aged between 1 and 4 years, for weekend admissions and for specific conditions: infective enteritis and colitis, circulation and respiratory signs and symptoms, and signs and symptoms of the digestive system/abdomen. These increases mirror some of the decreases seen in site A1.

For complex conditions, the number of bed-days was higher than the national average before the expansion of the CCN team, and remained so throughout the analysis period. There was a slight rise in bed-days for complex conditions immediately after the expansion, and this continued throughout, meaning that bed-days were at a higher level than would have been expected if no change had occurred. However, none of the changes was large enough to reach statistical significance.

The annual cost of the site B team for children with complex conditions was £865,120, and the cost of the observed increase in bed-days for children with such conditions was £606,151 per year.

The qualitative work highlighted the challenges of expanding existing teams, particularly in terms of clarity and understanding, but these appeared to be short-lived for the site B team. Thus, their expansion appeared to become embedded in a relatively short period of time. Processes for getting care packages in place were seen to have become more efficient, and as a result, were believed by the team to be enabling earlier discharges. However, because of problems accessing patient-level data, the commissioner could not confirm any reduction in LOS. Our ITS analysis would suggest otherwise, although it is important to note that the categories of complex conditions used may reflect a wider population than that served by the team. From the perspectives of families using the complex care team, there were consistent messages that it served an important support function, but assessments of quality were closely tied to whether or not certain aspects of the service were adequately delivered. These included getting allocated hours of care, having a key worker and continuity of staffing.

Site C

We chose this site for the project because, when we wrote the proposal, it was at an advanced stage of planning for standardising and enhancing its CCN provision across a complex network of historical PCTs that had been amalgamated into a single PCT. This amalgamation had left gaps in services across the area and created substantial geographical differences in access to CCN services. The planned changes were part of a wider strategy to reconfigure other community-based children’s services. This area was very different from sites A and B, both in terms of its rural/urban mix and its relative affluence.

Although we recruited this site at this advanced stage of planning, by the end of the project the changes had not taken place; indeed, it would be fair to say that they had been abandoned, despite the considerable investment of time and resources that had gone into planning for them. We were, thus, limited in which aspects of the project we could pursue in this site. We carried out the qualitative elements with service commissioners and providers and used the ITS analysis to explore what happens to acute activity when planned changes do not materialise. We did not explore any aspect of provision for children with complex conditions.

As in site B, we used a dummy intervention point that was the same as the actual intervention point in site A1 for the ITS analysis.

The rate of non-elective admissions for common conditions in site C was similar to that found nationally, but there was a statistically significant month-to-month increase in this rate before the dummy intervention date. The analysis then showed an abrupt, although not statistically significant, reduction in rate that was sustained during the period of analysis. Further analysis of the data, including subgroup analysis, suggested that there were considerable discontinuities in the data, suggesting change in the ways in which HES data were recorded in site C, before and after the mock intervention point. Indeed, a former commissioner in this site confirmed that coding of POAU attendances at one trust in site C had been changed at some point to be counted as inpatient admissions. Other work elsewhere highlights this as a problem nationally. 72

The number of bed-days was slightly lower in site C than the national average but was increasing slightly before the dummy intervention date. As with admissions, there was a substantial (although not significant) decrease in bed-days after the dummy intervention date. This reduction was not sustained, and over the whole period of analysis the number of bed-days increased, although not to such an extent as to reach statistical significance.

Average LOS in site C was below the national average before the dummy intervention point and was reducing significantly, and remained below the national average throughout the period of analysis. However, after the dummy intervention point, there was an increase in average LOS that was maintained through the period of analysis. This increase over time was statistically significant.

The longitudinal qualitative work with commissioners and managers in this site highlighted the process of attempting service change and the challenges associated this, especially when taking place in the context of organisational flux (i.e. the NHS reforms). The absence of dedicated finance for the intended change required competing providers to audit, review and negotiate resources and services. This was a lengthy process, which itself was not aided by changed leadership at the commissioning level, brought about by the changes in commissioning arrangements. Thus, despite the best efforts of managers and some commissioners, who between them had extensive expertise in CCN development, other contextual factors ultimately thwarted plans to reconfigure their services as planned.

Site D

As described in earlier chapters, site D was recruited to the project at a relatively late stage, to compensate for the loss of data caused by the abandoned change in site C. This team of paediatric nurse practitioners ran in parallel with a wider range of services in the area that delivered generic CCN care. The specific focus of the new site D team that we researched was to complement the existing provision by supporting children with complex neurological needs who were frequent users of acute hospital care. Given the tight focus of the team, we confined our ITS and health economics analysis to children in the relevant disease group in HES (code G) and to non-elective admissions only.

Site D was an area of high deprivation and its non-elective admission rates for the group in question were higher than the national rate and growing, until half-way through our analysis period (July 2011), when the data showed a substantial reduction. Overall, while there was no immediate reduction in non-elective admissions for the G-code children, there was a sustained and significant decrease across the period of analysis, bringing the rate below the national average by the end of the period.

Given the large swings in the HES data, we suspected some discontinuity in the data and therefore re-ran the analysis, restricting it to the 2 years before and 2 years after introduction of the team. This analysis still showed a reduction in the underlying trend of non-elective admissions but, over this shorter period and without the effect of the potential discontinuity, this trend was no longer statistically significant.

This reduction in admissions tallies with what we learnt about this site in the qualitative study: that its intention was to reduce acute admissions for children with complex neurological conditions and that such intentions were achieved from the perspectives of the limited number of parents we interviewed in this site.

Over the whole period of analysis, the health economics work estimated a saving of 994 admissions for children with complex conditions, which translated into an estimated annual saving of £215,954.

The estimated annual cost to the NHS of the team in site D was £229,727 and the estimated cost saving of the reduction in bed-days was £215,954. This analysis did not include any changes in bed-days or LOS.

Even though the ITS analysis suggests this area was successful in reducing acute admissions for the target group, the qualitative work with staff indicates that there were still challenges being faced in practice in achieving their intended outcome. The main challenge was maintaining visibility to the acute trust since they had moved their service base into the community. This was thought to affect consultant referrals to the team.

While we were able to interview only a small number of parents from this site, messages were consistent about the team helping them to assess the need for acute admission and in some cases avoid it. Overall satisfaction with the service was also high, which was linked to its ability to respond quickly and assess and prescribe in home and school, which were settings seen as more accessible than primary care and A&E, the nurses’ clinical familiarity with the child and the option for parents of contacting the team for advice when needed.

Does redesigning children’s health-care services and introducing children’s community nursing teams affect acute hospital admission rates for common childhood illnesses and length of stay for all conditions?

As explained in Chapter 8, bed-days per 10,000 children is a more granular, and therefore more sensitive, indicator of impact on acute activity. Thus, although we had not originally planned to, we used bed-days for the health economics work. However, one team (site D) was specifically intended to reduce acute admissions so this remained the main outcome measure for this site.

Across the sites, we saw a mixed picture of change in all three possible indicators of activity. However, it is possible to develop a narrative that helps to bring these different patterns together and that may allow us to think about a theory of change for the introduction of CCN teams. First, in making this first step towards developing a theory of change, we draw together the evidence on acute activity for all children.

Site A1 was a large, community-based CCN team working in an area of highest deprivation and multiethnicity and which, before the team was introduced, had non-elective admission rates for common childhood illnesses that were substantially higher than the national average and which were growing rapidly. The introduction of the team appeared to affect this rate immediately and continued to do so until for around 3 years. We do not know conclusively what caused the effect to wane, but possible factors identified during our work include later organisational change in children’s urgent care in the locality and issue with team visibility, which may have affected referral to the team. The reduction in admissions was accompanied by a reduction in bed-days for all children, which had also been substantially higher than the national average before the team was in place.

Site A2 was part of the same redesign of children’s services as site A1; it was in the second to highest deprivation group but had a lower rate of multiethnicity than site A1. Its non-elective admission rates were also substantially higher than the national rates and growing rapidly before the team was introduced. Unlike site A1, however, site A2 saw no reduction in admission rates (indeed, they increased over the study period), but did see a sustained reduction in bed-days over time. What could explain these differences?

There are three factors to consider.

First, the A1 team took parent self-referrals, whereas the A2 team did not. We do not know the exact rate of self-referrals for team A1, but we were told by practitioners in this site that they tended to have a number for conditions such as chronic constipation, eczema and asthma, for which word of mouth (e.g. at school playgrounds) led some parents to contact the team directly. The option of self-referral increases accessibility to the team, some of which may be replacing use of A&E. This may help to explain some of the lowered rate of admissions in this site.

Second, the A1 team was community (clinic) based, whereas the A2 team was hospital based. We wondered if this might explain these differences to some extent; given its base, did the A2 team have a predisposition to triage children to short-term admission, which might explain both the increase in admissions and the reduction in bed-days in this site? The commissioner and manager in this site both said that the team’s priority was admission avoidance, and they had developed referral pathways from consultants and advanced paediatric nurse practitioners in A&E to the CCN team. This would suggest no particular predisposition to triage children to short-term admission. However, if children were referred first from A&E to the POAU prior to referral to the CCN team, and if POAU attendances are coded as admissions, this could explain the increased rate of admissions and the lowered bed-days.

Third, our qualitative work with parents in site A1 highlighted the role of the team in empowering parents to care for their child, and the role the team played in assisting with acute episodes of illness. Although this work focused on those with complex conditions, it is not improbable that this philosophy extended across the wider caseload of those with less complex conditions. Thus, it is possible that an empowering role helps to change help-seeking behaviours for episodes of acute illness. Not only is this something that we saw in our earlier work on CCN teams,9 but past work has repeatedly demonstrated the importance of empowerment (e.g. through provision of information, training) for parents dealing with acutely ill children. 17,28,29,73 However, there is no evidence such parent empowerment reduces acute admissions for children with acute illness.

Without the in-depth qualitative work with practitioners, which we were unable to undertake in site A2, we are not able to put any further flesh onto these possibilities.

We explored changes in our three indicators of acute activity in site C and acute admissions only in site B, both of which had been unable to implement elements of planned change. This was to provide a degree of control for our findings in sites A1 and A2. Despite its high levels of deprivation, site B had admission rates close to the national average at the start of the analysis period, which were falling slightly. This fall was not sustained and admissions increased over time. Site C was different from all the other case sites in that it had low levels of deprivation and also had admission rates for common conditions similar to those found nationally. The HES data for site C were difficult to interpret, with a suggestion that there had been significant changes in the way HES data were recorded during the period of analysis, which showed falls in admissions and bed-days but increases in average LOS.

Overall, then, it may be that when rates of admission for common conditions are much higher than the national average there is more scope for admissions to fall, thus making it more likely that the ITS would identify an effect. However, it is also possible that the model of CCN team also has an impact: the more community-based and parent-facing (as in site A1), the more scope there is for reducing both admissions and bed-days.

Turning now to children with complex conditions, there was evidence of significant impact in both site A1 (on bed-days) and site D (on non-elective acute admission, the main outcome for this site). In both cases, rates before implementation of the CCN service had been substantially above national rates. By contrast, site A2, where level of bed-days was close to the national average, showed a smaller level of reduction. Without the qualitative insight in site A2, it is difficult to elucidate the reasons for this difference. We do know, however, that site A1 acted not only to empower parents of children with complex conditions to feel confident about caring for their children at home, but also played a key role in care management in the early days after diagnosis or discharge from hospital. This may explain the reduction in bed-days for complex conditions in this site. It may also be that the population of children with complex conditions in site A1 was always larger than that in site A2 and there may, thus, have been more scope for a reduction in bed-days, but we are not able to confirm this in the absence of any national statistics on children with complex conditions.

Site B showed a slight increase in bed-days over the period of analysis, despite already having a rate higher than the national average.

Again, then, it seems that the original level of activity may be important in whether or not an effect is possible. However, these findings again suggest that the model of CCN service is important. The teams in both sites A1 and A2 were ‘integrated’, in the sense that their work to avoid acute admission for all children was of a piece with their work to support children with complex conditions. Site D was very different, but was highly targeted on a specific group of children and on a single outcome (avoiding acute admission for children with complex neurological conditions). By contrast, site B had not developed much from its origins as a respite service, even when it had taken on responsibility for children with continuing care packages. The inability of the area to deliver the planned CCN service that would focus on acute admission may have meant that the site B team had no opportunity to develop the more active work with parents of children with complex conditions that we saw was an important part of the work of the site A1 team.

As noted in the introductory chapter, evidence about the impact of CCN services in secondary care is tentative. 9,33 Our work here adds to this evidence by highlighting the potential for either a generic or highly focused acute avoidance model to reduce admissions. However, given the limitations outlined above, this is a cautious and tentative conclusion. Although our work adds to the limited work on impact of CCN services on secondary care, there remains no firm evidence about this.

What are the costs and outcomes of the new services compared with those achieved by alternative service reconfigurations?

Chapter 7 gives a full account of how we examined the costs and consequences of the introduction or development of CCN teams in our case sites and we have summarised some of this material above.

Given the type of health economics analysis that was possible based on the ITS findings and the inclusion of other costs only for children with complex conditions whose parents took part in the qualitative work, it is difficult to be secure about any of the individual findings. Furthermore, given the very different type and costs of the four services included in this part of the work, it is difficult to synthesise the findings in any meaningful way. Individually, it seems that site D had the service in which costs most closely matched savings. This was a highly targeted service with only one main outcome and the small numbers involved in the ITS analysis mean that there is greater potential variability around the outcome. Given this, however, the overall conclusion is that this service has a relatively high probability of being cost saving. Parents valued the service, too.

For the site A1 and A2 teams, the health economics verdict remains unclear. We were not able to assess the impact of the disinvestment that had taken place in the acute sector before the teams were introduced, neither were we able to assign value to benefits or costs that might arise from all the other work that these teams did. For example, both sites ran a number of long-term condition clinics in the community to provide care for children who would otherwise have attended hospital outpatient clinics.

Site B is unusual, both in terms of our case sites and in terms of CCN services more generally. The high proportion of children funded via continuing care packages means that an analysis based on bed-days is slightly out of kilter with the aims of the team. The question that should, perhaps, be asked about this team’s activities is whether or not better value is achieved for continuing care expenditure from this model of CCN team than with other ways of delivering continuing care (e.g. via spot purchasing of services). The team was established because spot purchasing had been problematic in the past, so that perhaps provides part of the answer.

This is the first attempt, to our knowledge, to examine the costs and consequences of a range of different types of CCN team, meeting diverse types of needs. There is clearly much still to be done.

What changes in the quality of care do families caring for children with complex health-care needs experience when children’s community nursing teams are introduced?

The three services used by parents we interviewed (sites A1, B and D) identified different trajectories of service use, reflecting provision for different types of needs, for a very similar group of children, those with complex conditions. A decreasing trajectory of ‘hands on’ input was seen in which the team played an empowering role, spanning clinical and care co-ordination, and gradually stepped back as the responsibility of care shifted to parents and/or other services were set up and in place (A1). A constant trajectory was seen in which input reflected a designated allocation of hours for a continuing care package, with less evidence of parents being encouraged to increase their confidence and input (site B). Finally, we saw an episodic trajectory, in which input fluctuated in response to acute episodes of illness (site D, and to some extent in site A1). These patterns of service use have not previously been identified in evaluations of CCN services. Although the role of nurses in empowering parents has been repeatedly demonstrated in other studies,9,16,17,20 its association with a decreasing trajectory of service use is less apparent in current evidence around CCN services and has implications for practice (see Chapter 9).

When CCN service provision and use decreases over time (as typically seen in site A1), change is evident in what aspects of provision are important to families for feeling supported over time, but this is not the case for ‘constant’ or episodic trajectories. In addition, changes in the perceived quality of care do not necessarily follow these trajectories. Despite decreasing or episodic use of teams in site A and D, assessments of quality were overall positive. Despite the intended regular shift-like pattern of service input in site B, assessments of quality were closely tied to service processes such as staffing continuity. Thus, input does not need to be resource intensive to result in positive experiences of care quality. What appeared to be the critical service feature in positive accounts was having the team as a potential back-stop, as someone to call on, if needed. This probably reflects the anxieties parents have when caring for acutely ill children, whether or not associated with the additional demands associated with caring for children with more complex conditions. 16,27–29,35,36,40 It is also consistent with findings from other studies of community and telehealth interventions, in which telephone access to health professionals is a way of seeking reassurance in times of uncertainty. 74–76

What benefits and challenges do commissioners and providers experience, as the new services are planned, implemented and established?

In Chapters 3 and 4 we synthesised the material from across our case sites in relation to the experience of planning and implementing (or not) the planned changes to CCN services. In this chapter, we examine this material against the factors that other research has shown may help or hinder service change.

Leadership and the ability of leaders to articulate and communicate a vision for and the goals and purpose of change to all stakeholders are seen as key factors in the success of service redesign. 45 In site A, in a very large programme of service redesign that went well beyond the introduction of CCN teams, a multidisciplinary network led all aspects of the change programme and, crucially, helped to ensure that professionals in the NHS gave their support to it. This included those who might be considered particularly powerful in times of change: doctors, who are seen to be at the top of the NHS hierarchy. 77 In site B, by contrast, there was clear leadership for a much smaller change (introduction of a CCN team targeting acute admissions) at commissioner and provider levels. However, the medical directorate at the acute trust opposed plans for a CCN team to take on acute care for ambulatory-sensitive conditions. By contrast, a neurology consultant supported the bids for funding for the expansion of the complex care team in this site and this was felt to have been crucial in their success. In site C, in which nothing changed despite very advanced plans, there was clear leadership and a vision for the future of children’s services across the existing health and social care community, but this was disrupted at a crucial stage by wider NHS restructuring.

Thus, while committed leadership is undoubtedly important, both individual stakeholders43,45,47 and wider change can scupper even the best planned and most advanced strategies. Even in site A, we became aware that there had been some loss of focus, over time, on the key objectives of the system-wide service redesign, as individuals moved on, other policy preoccupations came to the fore, and nationally led restructuring created competition between providers. Others have argued that change needs a stable environment43,44 and our work serves to reinforce this, and to highlight the wasted resources that continued restructuring might lead to.

Dedicated funding for change is another factor identified in earlier work as crucial to the success of change. 43 This was in place in site A (both teams) and site D, where change was successfully implemented, was applied for successfully when the complex care team in site B was expanded but was not available in site C. The planned changes in site C required review and reallocation of existing resources and, in an environment for which competition between trusts had come to the fore, this became increasingly difficult to achieve.

This last point links to others’ suggestions that the timing and pacing of change can also be important for their sustainability. 45 We saw evidence of this in relation both to wider policy and to the scheduling of implementation. In the case of site C, the planned implementation of change came just too late in terms of wider NHS restructuring. In site A, by contrast, the European Working Time Directive and anxieties about the sustainability of existing services provided a fertile ground for planting the seeds of change. Site A also paced its programme carefully, ensuring that public and service support was largely in place before change began, and, once it did, setting up the CCN teams before reducing inpatient beds and recruiting to the teams over time, so as not to destabilise acute services.

Longitudinal analysis of parent interview data

Longitudinal analysis of staff interview and focus group data

Translating and using the children’s community nursing development tool (formerly the Medical Home Index)

Translating and trialling the Medical Home Family Index

The MHFI is a service tool developed in the states to assess use of the medical home. 52 As outlined in the protocol, there are similarities between the medical home and CCN teams as service models. Thus, we used this project as an opportunity to explore whether or not the MHFI could capture parents’ experiences of using CCN teams. To do this, we:

1. ‘translated’ the tool items to reflect CCN service delivery and English NHS context (changes summarised in Table 29).

2. pre-piloted the translated tool with a member of SPRU’s PCG who regularly uses a CCN team.

3. made further changes based on this pre-piloting (see Table 29)

4. administered the translated MHFI as part of data collection with parents after interview.

5. used parent interview data to explore whether or not there were any aspects of their experience of using the CCN service that were not covered in the MHFI

6. monitored participants’ responses and written notes to identify potentially difficult to answer items.

This was not intended as a full validation exercise. Rather, we translated and trialled the MHFI as a ‘pre-validation’ exercise. The MHFI questionnaires (times 1 and 2) used as part of the data collection are at the end of this appendix. Some data from the MHFI was also used to inform the costs study.

Using parent interview data to explore whether or not there are any aspects of experience not covered by the Medical Home Family Index

The MHFI covered most of the issues parents discussed when recounting their experience of using the CCN services. However, five key issues/themes emerged from the data about using CCN services that are not covered explicitly by the MHFI. These are being able to access and use the team for advice when needed; assessing and prescribing for acute illness (mainly in relation to the nurse practitioner team); nursing care and treatment at home (e.g. tube care, i.v. treatment); training for parent and other family members; and arranging the child’s supplies. However, it is possible that existing items on the questionnaire could be revised to incorporate most of these issues:

1. ‘Being able to access and use the team for advice’ when needed could be incorporated into item 6 (‘The CCN team listen to my concerns and questions’). Possible revision: I am able to contact the CCN team to ask questions and advice, and to raise concerns, when needed.

2. ‘Assessing and prescribing for acute illness’ and ‘nursing care and treatment at home’ could be made explicit as a subsection of item 1 (‘The CCN team provides the support that my child needs when we need it’). Possible revision: ‘The CCN team provides the support that my child needs when we need it. This includes: nursing support for my child’s ongoing needs (e.g. feeding tube care); dealing with my child’s acute illness (e.g. assessing and prescribing)’.

3. ‘Arranging the child’s supplies’ could be added as a subsection of item 11 (‘My nurse, or someone else in the team, will:. . . (e) Arrange my child’s supplies’).

For the issue of training the parent and other family members in the child’s care, a new item could be added, for example ‘The team provide training for myself and others (where applicable) in my child’s care’.

Other changes to consider

Next steps

Based on our trialling of the MHFI, we have identified a number of possible revisions that we believe may enhance its validity as an assessment tool for CCN services. Once these revisions are complete, a full validation exercise is required. This is beyond the scope of this project and was not part of protocol. Thus, we suggest it as future avenue of research.

Medical Home Family index, time 1 and 2

(PDF download)

Methods