Nurse staffing levels, missed vital signs and mortality in hospitals: retrospective longitudinal observational study

Nurse staffing levels and adverse outcomes

A large international body of evidence has explored links between low nurse staffing levels and adverse outcomes in hospitals, most notably mortality. One of the early seminal studies in the field, the Aiken et al. 32 study of 10,184 staff nurses and 232,342 surgical patients in 168 general hospitals in Pennsylvania, USA, found that for each additional patient cared for by one RN the odds of death among surgical patients increased by 7%.

This early study established a model of research that can be used to typify the field. It is dominated by large, cross-sectional studies with associations measured at the hospital level. Most evidence relates to RN staffing, with staffing levels drawn from a variety of sources but typically averaged to hospital level over an extended period of time. Most, but by no means all, of the evidence has originated in North America, although this limitation is fast being rectified, with major studies undertaken in countries including Australia,33 China,34 England35,36 and Thailand37 and across 12 European countries (Belgium, England, Finland, Germany, Greece, Ireland, the Netherlands, Norway, Poland, Spain, Sweden and Switzerland). 5,38 Generally, studies give little account of staffing levels in other professional groups, although mix within the wider nursing team (e.g. nursing assistants or the proportion of nurses with a degree qualification) has been considered in a number of them. 36,39 Some have demonstrated relationships between outcomes and nurse staffing levels while also considering or controlling for staffing by other groups, including doctors. 36,39

An early systematic review and meta-analysis by Kane et al. 4 found 101 studies published up to 2006, mainly from the USA. A meta-analysis based on 28 studies that reported adjusted odds ratios (ORs) showed a significant association with a range of adverse outcomes including mortality and infections (Table 1). The authors did, however, conclude that evidence for associations with outcomes that were regarded as potentially more sensitive to nursing, such as falls and pressure ulcers, was less consistent.

Although large cross-sectional studies have dominated the field, studies with a longitudinal element have also been undertaken. Kane et al. 4 concluded that overall results from cross-sectional studies were consistent with those from studies that considered temporality, although there was evidence that cross-sectional estimates were larger for failure to rescue. A direct comparison of estimates from longitudinal and cross-sectional analyses of the same data set found that cross-sectional estimates of associations between nurse staffing and patient experience were more often significant than longitudinal ones, although there was no clear pattern in relation to effect size. 40 A systematic review published in 201331 considered new studies in addition to those in the Kane et al. review. Of the 15 new studies considered, nine were classified as longitudinal. Although the review came to similar conclusions to Kane et al. ,4 the evidence from longitudinal studies was mixed, although these studies tended to consider changes in staffing over long time periods (typically from year to year), and so still made no direct link between patients and the staffing experienced.

Although most research has involved cross-sectional or longitudinal studies comparing year-on-year changes, one exception was a study undertaken in a single US hospital. This study measured exposure to periods of low staffing at the individual patient level for 197,961 admissions in 43 acute (adult) care units over 3 years. When patients were exposed to a shift in which the available nursing time was < 8 hours below the target (effectively one staff member short), the hazard of death was increased by 2% [hazard ratio (HR) 1.02]. 27 Although it was undertaken in only one hospital, this study is significant because it is the first large-scale study to show that the increased risk of death is directly associated with staffing experienced by that patient, making it far more likely that the observed association is causal.

The systematic review of evidence undertaken for NICE14 focused on 35 primary studies that had incorporated skill mix and measured staffing actually deployed on the relevant wards. The results of these studies confirmed the conclusions of other reviews in relation to RN staffing and/or measures of total nurse staffing. Although the evidence suggested that having more nurses was associated with lower rates of falls, findings for other outcomes, often regarded as nurse sensitive, were inconsistent. For example, 12 studies reported the association between staffing and pressure ulcers. Three studies found that higher staffing was significantly associated with lower rates of ulcers,41–43 but two found a significant association in the opposite direction, with units/hospitals with more staff having higher rates of pressure ulcers. 44,45

Skill mix and nursing assistants

The evidence review for NICE14 further explored the contribution of assistants or aides (also referred to as support workers) as part of the nursing team. Eight, mostly weak, studies gave no strong evidence of beneficial associations between nursing assistant staffing levels and patient safety. Studies found no association between assistant staffing level and mortality,46 failure to rescue,47 length of stay,46 venous thromboembolism48 or missed care. 19 However, higher assistant staffing was associated with higher rates of falls,43,49 pressure ulcers,50 readmission rates,51 medication errors50 and use of physical restraints43 and lower levels of patient satisfaction. 50 One study found that higher assistant staffing levels were associated with lower rates of pressure ulcers. 48 Subsequently, a cross-sectional study in 137 English hospital trusts36 found that higher levels of support worker staffing were associated with significant increases in mortality rates among medical patients when using whole Trust staffing figures, although the relationship was not observed in a subsample that looked at nursing assistants deployed on wards. A before-and-after study exploring the effect of introducing nursing assistants on acute care wards in Australia52 found mixed results but reached an overall conclusion that, for every 10% of extra time patients spent on wards using nursing assistants, the odds of developing a urinary tract infection or pneumonia increased by 1% and 2%, respectively.

The review also identified 22 studies reporting relationships between skill mix (typically, proportions of RNs to total nursing workforce) and outcomes. A number of these studies found an association between a nursing skill mix and a higher proportion of RNs and better outcomes including lower mortality or failure to rescue,53–55 lower rates of infections,44,55,56 falls,41,42,57,58 pressure ulcers42,48,55 and higher patient satisfaction. 59 The pattern of results is largely consistent, with the only significant contradictory evidence coming from one of the weaker studies that showed that a higher proportion of RNs was associated with a higher nurse-reported incidence of pneumonia. 60

Overall, there appears to be little evidence suggesting that adding nursing assistants makes a positive contribution to patient safety, although studies that report skill mix as a variable, rather than modelling the numbers in each group, risk confounding between absolute numbers and proportions. This is because skill mix can be lowered by both decreasing the number of RNs and increasing the number of assistants. So far, only one study19 has considered staffing levels by both staff groups as independent variables and modelled the interaction between the two in order to understand if the effect of adding care assistants is the same at all levels of RN staffing. Interestingly, this study found no effect from assistant staffing and no interaction between assistant staffing and RN staffing.

Economic evidence

A number of studies have explored the economic consequences of variation in nurse staffing by attempting to estimate the costs associated with staffing and the consequences, such as reduced length of stay or costs of treating complications. A review (up to 2013) found nine studies exploring either cost benefit or cost-effectiveness. 13 The conclusions varied depending on the costs considered, the context and the economic perspective taken (e.g. hospital vs. societal). In general, savings from better patient or system outcomes did not offset the costs to the hospital of increased nurse staffing. Using the studies considered in NICE’s evidence review,14 we estimated the cost per life saved, which varied hugely between studies. Cost per life saved in studies taking a hospital-cost perspective ranged from over US$9M61 to AU$62,522 (approximately US$46,000 at 2017 exchange rates). 45 Whereas studies that took a wider societal perspective suggested a net economic benefit from lost productivity avoided,61,62 only one scenario modelled in one study63 suggested a net cost saving to hospitals, which arose from increasing the proportion of RNs. A more recent study using longitudinal data over 4 years from 421 US hospitals confirms this analysis. 64 Increases in staffing were associated with increased costs, although the finding was not significant. Increasing the proportion of RNs in the licensed nursing workforce was associated with a net reduction in cost. The one UK study15 we identified, an economic assessment commissioned by NICE, estimated an incremental cost-effectiveness ratio for holding staffing levels constant and increasing skill mix by 10% (from 64% RN to 74%). The estimated costs were £1412 per fall averted and £128,779 per drug error avoided. This study did not explore associations with mortality.

Causation

Although all of the studies we reviewed here are observational, an assessment against the so-called Bradford Hill criteria65 largely supports the case that nurse staffing is related to mortality in a causal manner because of the overall consistency of results as shown in meta-analyses, the invariance of the conclusions to specific features of study design and features such as dose–response relationships. 4

Nonetheless, uncertainty remains. For outcomes most directly linked to nursing, such as pressure ulcers, inconsistency in evidence is likely to be a product of the challenges of recording outcomes in routine data, the possibility that higher staffing leads to higher detection for some outcomes and the lack of well-developed risk adjustment methods. 14 The Needleman et al. 27 study demonstrates that increased risk of mortality follows periods in which patients are exposed to nurse staffing below that deemed necessary, confirming the temporal order of events. However, the observed associations are typically small, making causal conclusions more difficult. Even assuming cause, the proportion of adverse events, such as mortality, that can be attributed to variation in nurse staffing is low. Kane et al. 4 estimated the proportion of deaths associated with variation in nurse staffing (as reported in Table 1) to be 4.2% for all patients at a hospital level, although estimates were higher for population subgroups (e.g. surgical patients, 16%) and outcomes (e.g. hospital-acquired pneumonia, 19%).

Since 2010, research has begun to focus on the mechanisms that must intervene between staffing levels and outcomes to cause the observed associations. It has been hypothesised that it is the nurses’ ability to monitor patients and initiate timely intervention in the face of deterioration that is the key mechanism linking nurse staffing levels to mortality. 17 In simple terms, there must be sufficient capacity in the workforce to maintain adequate surveillance in the face of numerous other demands on nurses’ time. Furthermore, if omissions in nursing care can be linked to important patient outcomes, not only does this provide support for a causal mechanism, it also generates a potential indicator of quality that is more directly linked to the nursing workforce and that can be observed and monitored without relying on the occurrence of rare and severe adverse outcomes. In one study,66 over 40% of variation in nurses’ ratings of care quality was associated with reports of care being left undone.

Missed nursing care

Although evidence for the association between nurse staffing levels and patient outcomes is considerable, and has been extensively reviewed, research on missed nursing care (variously referred to as ‘care left undone’, ‘missed care’ and ‘implicit rationing’) is more limited, in part because nursing activities can be difficult to measure and are often not routinely recorded in a useable format by health-care providers. 67 However, there is now a growing number of studies exploring the link between nurse staffing and missed care, and the subsequent impact on outcomes. We undertook an extensive database search using a core strategy for locating nurse-staffing research that we had developed previously. 30 We supplemented this with focused searches for terms related to missed care (‘missed nursing care’, ‘care rationing’, ‘care left undone’, ‘unfinished care’). We included any quantitative study reporting the association between any measure or report of missed nursing care and staffing levels, skill mix or patient outcomes. We found 18 papers reporting associations with staffing and 15 reporting associations with outcomes. Although we looked for studies with objective measures of missed care or measures recorded in routine clinical practice (e.g. omissions in drug administration), we could find none in which the specific omission of nursing care was reported separately. Instead, all reports relied on intermittent surveys primarily of nurses, or in some cases patients. All studies analysed data in a cross-sectional fashion, although, often, the temporal link between missed care and staffing was clear because it related to a specific shift. Searching was completed in September 2016. Subsequently, citation alerts were used to identify any significant new publications that might alter conclusions.

What care is missed?

In a large pan-European study,68 the frequency with which nurses reported that some care was left undone on the last shift ranged from 75% in England19 to 93% in Germany,69 with an overall estimate across 12 European countries of 88%, based on a survey of 31,627 nurses in 488 hospitals. 68 Studies from Korea, Kuwait and Switzerland also suggest that missed care is a frequent occurrence. 70–72 Although clinical care is less often reported as missed than aspects of planning, communication and psychosocial care, the rates of omission were still substantial. Although monitoring of vital signs was one of the least likely aspects to be reported as missed, omissions of care in this area were still relatively frequent, with up to 37% of nurses reporting some care missed on the last shift. 69

Association with staffing

Of the 18 studies, 15 found lower nurse staffing levels to be significantly associated with higher levels of missed nursing care. Two studies51,73 found no significant effects and one69 found mixed effects (Table 2). The magnitude of the effect varied across studies, but comparison was difficult because of the variety of measures used for staffing and for missed care. In the pan-European study,68 the odds of nurses leaving care undone were increased by 26% when nurses were caring for > 11.5 patients compared with when nurses were caring for ≤ 6 patients.

Of the 18 studies, the two using patient (as opposed to nurse) reports of missed care provided mixed results. RN hours per patient day (HPPD) were weakly correlated with patient reports of not receiving timely care, although not with overall patient reports of missed care. 76 A study focusing on discharge planning in four US hospitals51 found no significant association between non-overtime RN HPPD and patient-reported delivery of necessary discharge information.

Four studies explored associations between skill mix and missed care either directly or indirectly. The results suggest that adding support workers to the workforce does not generally reduce the level of missed nursing care and may even increase it where skill mix is diluted. Two studies19,74 found that greater numbers of support workers were not associated with reductions in the rate of care left undone, except with very high levels of support worker staffing in one study (fewer than four patients per support worker). Patient-reported timeliness of care was significantly correlated with increased RN skill mix in one study. 76 One study81 found that more daily care provided by support workers was associated with increased nurse-reported frequency of missed care [OR 1.04, 95% confidence interval (CI) 1.01 to 1.07].

Consequences of missed care

We found four studies reporting associations between missed care and patient mortality. Two studies found a significant association between levels of nurse-reported missed care and mortality23,82 and two did not. 67,83 Nurse reports of missing treatments and procedures were associated with hospital readmissions in one study. 84 Other studies have shown associations between nurse-reported missed care and falls,67,85 infections,60,67,85–87 pressure ulcers,85,86 pneumonia60 and medication administration errors,60,67 but several of these studies relied on nurse reports of adverse events and some studies found no significant associations between missed care and the rates of hospital-acquired pressure ulcers60,88 or urinary tract infections. 60 Four studies found nurse-reported missed care to be associated with significantly decreased patient satisfaction in hospitals24,85,89 and four found that nurses’ ratings of quality were higher when less care was reported as missed. 19,22,66,90

Does missed care mediate the relationship between nurse staffing levels and outcomes?

Although limited by reliance on nurse reports, the evidence for an association between nurse staffing levels and missed care is largely consistent. When staffing is lower, more care is reported as missed. The evidence for a link between these nurse reports of missed care and patient outcomes is less consistent, although the association between reports of missed care and more adverse perceptions of care by both patients and nurses seems clear. However, although the broad pattern of results supports a similar association with other outcomes, the evidence is more mixed. Crucially, if missed care is at least part of the mechanism through which low staffing affects patient outcomes, its role as a mediating variable needs to be directly tested.

In simple terms, it is proposed that the mechanism by which nurse staffing affects patient care is because necessary care is more often missed when staffing is low. Because this care is necessary to achieve desired patient outcomes (or to prevent adverse outcomes), outcomes are worse if more care is missed. Thus, missed care mediates the relationship between adverse outcomes and staffing. This is illustrated in Figure 1.

FIGURE 1.

Staffing: missed care outcome mediation model.

According to Baron and Kenny,91 if this is the case the following must apply:

1. There must be an association between staffing levels and the outcome. This is generally supported by the existing evidence.

2. There must be an association between staffing levels and missed care. Again, this is generally supported by the existing evidence.

3. There must be an association between missed care and outcomes. There is currently limited evidence to support this.

4. Finally, because the relationship between staffing and outcome is explained by the effect of staffing on missed care, the association between staffing and outcome is reduced when the outcome is regressed on staffing and missed care.

5. If the relationship is entirely explained by missed care (i.e. there is no other mechanism through which low staffing affects outcome), there is no residual association between staffing and outcome when both variables are included as independent variables in a regression analysis. Otherwise, the mediation is partial.

The final two points above illustrate that although existing evidence is largely consistent with mediation, this can be assessed directly only by modelling the relevant variables at the same time in a single study.

To date such analysis has been undertaken in four studies, demonstrating that missed care partially mediates observed relationships between staffing and patient satisfaction,24,26 falls25 and mortality. 23 However, although this provides general support for the mediation hypothesis, all of the studies relied on subjective reports of missed care, and only Kalisch et al. 25 focused specifically on the elements of care that were determined to be likely to affect the outcome (as opposed to a composite measure of all care missed). In relation to mortality, where evidence for the association between nurse staffing and outcomes is strongest, the support for missed care as a mediator is derived from a cross-sectional study (albeit a large multinational one) in which staffing levels and patient outcomes are measured at a hospital level, and both staffing levels and missed care are determined through a nurse survey, thus creating a risk of common method bias. 23

Aims and objectives

The monitoring of vital signs is a fundamental component of the ‘Chain of Prevention’, a structure that describes the interventions necessary to prevent patient deterioration. 92 This study examines whether and how variation in nurse staffing levels on general hospital wards are associated with omissions or delays in delivering necessary nursing care. Our specific focus is on monitoring and acting on vital signs, and whether or not variation in staffing levels and vital signs observations is associated with variation in patient death. Secondary outcomes explore the impact of staffing on cardiac arrest and unanticipated intensive care unit (ICU) admissions, timely nutritional risk assessment and an inferred ‘failure to respond’ to deterioration. The current study is the first to use objective measures of missed care to explore the relationship between staffing levels, missed care and adverse outcomes for patients.

Crucially, this study adds significantly to the body of evidence on associations between nurse staffing and outcomes because it:

1. determines whether or not variation in the presumed causal factor (nurse staffing) precedes the presumed effect (adverse outcomes)

2. explores an intervening care process (vital signs observation) that is a direct result of actions by nurses

3. provides evidence that is directly relevant to the UK context.

Thus, the study aims to provide a basis for identifying the nurse staffing levels and skill mix required to ensure adequate patient surveillance, and to assess whether rates of missed vital signs observations can be used to identify when or where care is falling below accepted standards and putting patients at risk.

To do this we:

• modelled the relationship between the available nursing workforce (including hours of care by both RNs and HCAs per day or per shift), controlling for patient risk and ward-level factors (as appropriate), to identify the association between RN staffing levels and HCA staffing levels and:

  • hazard of death

  • ICU admission and cardiac arrest

  • length of stay

  • missed vital signs observations

  • timely response or resolution of deterioration

  • missed screening for nutritional risk

• explored data to determine whether any relationships are linear or whether performance deteriorates beyond a threshold of staffing level, and whether or not relationships vary across ward types and by time of day

• undertook secondary analyses to determine whether or not the use of bank and agency staff is related to adverse outcomes

• undertook analysis to determine whether or not missed vital signs observations mediate any relationship between staffing and mortality

• used coefficients from the regression models to identify staffing policies that are associated with desired outcomes and model the staffing costs associated with these policies.

Nurse staffing

Nurse staffing data were accessed from an electronic rostering system. The database contained records of shifts worked, location, hours (dates, start and end times), ward and grade for all nurses employed by the hospital. A second database recorded all bank (extracontractual work within the hospital by staff employed by the Trust) and agency shifts (shifts worked by staff employed through an external agency).

We considered shifts worked on adult medical and surgical wards including admissions units and care of older people. ICU staffing was not considered because the staffing levels differ fundamentally from those on general wards and vital signs are not recorded using the same electronic system as elsewhere in the hospital. We did not consider paediatric or maternity services for similar reasons.

A total of 764,008 shifts were recorded in the eRoster, of which 633,525 were rostered to permanent staff (RNs and HCAs) and 130,483 were rostered to temporary staff (RNs or HCAs employed via the bank or agency). The eRoster and bank and agency data were subject to extensive checks and validation by the Trust prior to transfer, as these are used as the source of data for payroll. Shifts with codes indicating sick leave or other absence, non-ward based or non-clinical roles (e.g. ward clerks) or study leave were removed prior to calculating ward staffing levels. A small number of shifts by staff working at an unknown grade (n = 1608) were also removed because the human resources department and senior nursing managers advised that these staff would be unlikely to have been clinical nursing staff.

We included all ward-based shifts (including supervisory hours) and calculated nursing hours based on the start and end times recorded in the eRoster. Although it might be of interest to explore whether or not supervisory hours made a distinct contribution to ward staffing levels, the coding used by the Trust meant that we could not reliably separate these from other hours worked on the ward.

In total, 538,238 shifts were classified as ‘worked’. There were periods when ward closures, mergers and moves created a mismatch between wards recorded for staff on the eRoster and those recorded on the patient administration system (PAS), resulting in ambiguity that meant that we could not reliably match records of staffing to patients. These periods were dropped from the study, resulting in a small reduction in both the available study days and the patient population.

Vital signs and other nutritional assessments

The hospital uses a handheld electronic system to record vital signs and some other nursing information. This is the Vitalpac system (System C Healthcare Ltd, Maidstone, Kent; formerly The Learning Clinic Limited). Vital signs observations and nutritional risk assessments, undertaken using the Malnutrition Universal Screening Tool (MUST) (see www.bapen.org.uk/screening-and-must/must/introducing-must), were derived from a database of records made using this system, where nurses record clinical data on handheld devices at the bedside. This system constitutes the only official record of observations that are maintained on the wards in the study, and the use of the system is mandated by Trust policy. The frequency of vital signs observations required is determined by a protocol based on the National Early Warning Score (NEWS). 97 NEWS uses parameters from vital signs observations to assign a score to the patient, with a higher score indicating a higher level of acuity and potential need for escalation of care. This in turn is used to identify the required frequency of vital signs observation, which identifies when the next observation is due. 97 In total, 3,702,717 sets of observations were extracted, of which 3,367,791 were complete.

Given that the Trust’s policy on recording nutritional assessments was implemented in early 2013, we used these data only from 1 April 2013. In total, we extracted 301,172 MUST assessments. As the policy requires that the first MUST score is recorded within 24 hours of admission, we considered MUST scores only in patients who had an initial stay of ≥ 24 hours on one of the study wards.

Patients

Patient data were extracted from the hospital’s PAS. These included patient demographic and diagnostic data, including reason for admission and comorbidities, which were used to control analyses for patient-level risk factors, and detail of transfers within the hospital, which were used to calculate length of stay and determine the wards where a patient received care. We also obtained records of cardiac arrests and unanticipated ICU admissions from the cardiac arrest audit and ICU WardWatcher (WardWatcher, Critical Care Audit Ltd, Yorkshire, UK) databases. Patient data were, in turn, used to estimate the number of new admissions to the ward for each day/nursing shift and the numbers of ‘patient hours’ per day or nursing shift. From 387,009 records in the PAS data set, 148,994 satisfied the following inclusion criteria: aged ≥ 16 years; discharge date after 1 April 2012; admission date before 31 March 2015; and admitted to a general medical/surgical ward. Patients aged 16 or 17 years were occasionally admitted to adult wards and were therefore included to allow accurate calculation of patient hours.

Patient-level outcomes

The primary patient outcome was in-hospital mortality, determined from the PAS.

For secondary outcomes, we also considered death in hospital within 30 days of admission and a composite adverse event outcome, defined as death, cardiac arrest or unplanned ICU admission. For this outcome, each patient was assigned the outcome based on the time of the first event to occur. Length of stay was calculated from the PAS, from first admission to hospital to discharge (or death) or transfer out of the hospital.

Omissions or delays in recording vital signs and other assessments

The protocol that determines the frequency of observations in the Trust is adapted from a national protocol based on the NEWS, which assigns scores to abnormal vital signs observations and then specifies the observation interval (see Appendix 2, Tables 22 and 23). For example, if a patient is assigned a NEWS of 7, the protocol requires a maximum interval between observations of 1 hour. This provides a clear basis on which to determine whether or not observations are completed and recorded in a timely fashion.

The primary outcome for this aspect of the study was missed vital signs observations. A ‘missed’ observation was assigned when a full set of vital signs observations was not recorded within the scheduled interval plus two-thirds of the interval (i.e. more than five-thirds of the scheduled interval). For example, if the next observation is scheduled to be in 1 hour, an observation is classified as missed after 1 hour and 40 minutes has elapsed, in effect when another set of observations is nearly due. Similarly, we classified observations as ‘late’ if the scheduled interval plus one-third of the interval had passed (i.e. more than four-thirds of the scheduled interval), and ‘not on time’ if not taken on or before the scheduled time.

Formally, we classified observations as:

• on time: T ≤ T₀ + I

• not on time: T > T₀ + I

• late: T > T₀ + (I × 4/3)

• missed: T > T₀ + (I × 5/3),

where T is the time of the observation, T₀ is the time the previous observation was taken and I is the interval between observations determined by the observation taken at T₀.

Our primary analysis considered a daily rate of missed observations. An observation was assigned to the time at which it was due, and rates of missed observations were determined using the total number of observations due during that period as the denominator. A similar approach was taken for ‘not on time’ and ‘late’ observations and for missed nutritional assessments. In addition to daily rates, we calculated missed observations within 6-hourly and hourly intervals to more closely match staffing to the rate of observations. In each case, the denominator was the number of observations due within the specified time period. As observation frequency is related to patient acuity, and, because the significance of missed or late observation may differ in different patient groups, we undertook subgroup analyses focusing only on patients classified as high or very high acuity (NEWS of ≥ 6). According to the current monitoring protocol, observations for these patients are scheduled every 4 hours or more frequently.

Although the focus of the study was the link between staffing levels, missed vital signs and mortality, we also took the opportunity to determine whether or not other electronically recorded assessments could be used to derive objective measures of missed care. Trust protocol specifies that nutritional assessments (MUST) be recorded in the Vitalpac system within 24 hours of admission for all patients; therefore, we were also able to determine whether or not these observations were undertaken within the time specified by the protocol.

Failure to respond

Timely observation is only the first step in preventing or mitigating the effects of patient deterioration. Consequently, we also calculated a variable which might be indicative of a wider failure to respond. According to the protocol, patients with a NEWS of > 7 are to be observed at least hourly and be seen by a doctor within 30 minutes (see Appendix 2). Based on this, we defined a variable indicating ‘failure to respond’ if a patient with a NEWS of ≥ 7 does not meet one of the following conditions within 4 hours:

• a NEWS of < 7 recorded (indicating improvement)

• admission to the ICU

• indication that the patient has been placed on the end-of-life care pathway (flag in Vitalpac).

Similarly, a patient with a NEWS of ≥ 6 is to be observed at least every 4 hours and be seen by a doctor within 2 hours. Based on this, we defined a failure to respond indicator if the following did not occur within 16 hours:

• a NEWS of < 6 recorded (indicating improvement)

• admission to the ICU

• indication that the patient has been placed on the end-of-life care pathway (flag in Vitalpac).

Hours per patient day/shift/hour

Our primary approach to measuring staffing was to calculate care hours per patient day (CHPPD) (i.e. a 24-hour period starting at midnight) for RNs who are trained for a minimum of 3 years [Agenda for Change (www.nhsemployers.org/your-workforce/pay-and-reward/agenda-for-change) bands 5+], and unregistered HCAs with no standardised training requirement (Agenda for Change bands 2–4). For each ward, the nursing hours for each day were calculated from the eRoster with all hours between the shift start time and end time contributing (i.e. from midnight to midnight). In general, nurses would take a 1-hour break for every long shift worked (approximately 12.5 hours from start to finish), and so paid hours are approximately 8% less than the calculated HPPD (1 out of 12.5). These were summed for each ward for each day.

For each day, we also calculated ‘patient-days’, with 1 patient-day equivalent to one patient occupying one bed for 24 hours. The daily number of patient-days for each ward was calculated using the admission, discharge and transfer information during a 24-hour period (i.e. from midnight to midnight). For example, a patient occupying the bed for 12 hours would be assigned a value of 0.5 (patient hours ÷ 24). Consequently, patient-days represent the average number of occupied beds in a 24-hour period.

By combining these data with the staffing variables, we calculated RN HPPD and HCA HPPD as the sum of hours worked by each group divided by patient-days. Using the same approach, we were able to calculate hours provided by temporary nursing staff employed via the hospital’s bank or from an outside agency.

We also calculated staffing hours by shift and by hour. Shift patterns varied both within and between wards, with a mix of two ’12-hour’ shifts (day/night, each lasting 12.5 hours from start to finish) or a traditional three-shift system [with two 8-hour day shifts (early/late) and a 10-hour night shift] with other minor variations. Therefore, we divided the day into 6-hour period ‘pseudoshifts’ in an attempt to provide a degree of consistency that was congruent to major changes in staffing levels throughout the day:

• from 01.00 to 07.00 ‘late night’

• from 07.00 to 13.00 ‘early’

• from 13.00 to 19.00 ‘late’

• from 19.00 to 01.00 ‘twilight/night’.

Hereafter, for brevity, we refer to these four periods as ‘shifts’. For each shift we calculated hours of nursing per patient, taking the same approach to patient hours as for the day-level analysis (patient hours ÷ 4). Consequently, the shift-level staffing variable is on the same scale as the day-level variable, such that, if patient numbers were unchanged across each shift and staffing levels were constant, nursing hours per patient per shift = nursing HPPD. Finally, we took a similar approach to calculating staffing levels for each hour of the day.

For our analyses, we explored these variables in a number of ways. The Trust determined staffing establishments (target/planned staffing levels) for the wards using the Safer Nursing Care Tool,98 a validated acuity/dependency staffing tool endorsed by NICE. These establishments (planned staffing levels) varied by ward, reflecting variation in the typical staffing requirement. As patients experience care on different wards, this presents a challenge that makes it difficult to use absolute HPPD as a measure of staffing as the staffing required for each ward differed. Therefore, we determined days when staffing for each ward was low in terms of the following thresholds/approaches:

• below the mean staffing for the ward (measured in HPPD or equivalent)

• days when staffing fell below 80% of the mean

• below the HPPD estimated from the establishment

• HPPD below the mean.

We also considered the absolute HPPD relative to the mean: a patient who spends a day on a ward where mean staffing is 5 HPPD records a score of 0 for a day when staffing is at the mean level, a score of + 1 for a day when staffing is 6 HPPD and a score of – 1 on days when staffing is 4 HPPD, and so on.

When a patient was transferred during the day, we used staffing levels from the first ward of the day.

The information we received regarding establishments was provided in terms of nurse numbers on each shift, generally using a traditional early, late and night shift pattern, so calculating the established nursing HPPD required a number of assumptions to be made about shift patterns and overlaps. Therefore, we initially focused our analysis using thresholds on the mean staffing level because this required no assumptions on our part. However, mean staffing and established staffing were closely correlated (r = 0.97 for RN HPPD, r = 0.81 for HCA HPPD), with mean staffing levels for RN 6% lower than the HPPD estimated from the planned establishments, and mean staffing for HCA 12% higher than in planned establishments.

When exploring the association between staffing levels and rates of missed observations, we used absolute HPPD, as observations were clustered within wards and therefore ward differences in staffing levels could be accounted for within the statistical models directly.

Admissions per nurse

We calculated the ratio of admissions per nursing hour, to reflect the variation in patient turnover on wards, which also contributes to nurses’ workload. This variable was calculated by identifying the number of admissions to the ward (including transfers from other wards) in the time period and dividing by the number of nursing days or fractional days available during that period (i.e. nursing hours ÷ 24).

Patient acuity and risk

To account for patient-level variation in acuity and risk of death, we calculated two variables based on administrative data. Although the use of administrative data to assess outcomes, in particular risk-adjusted mortality, remains controversial, there is evidence that shows that routine administrative data performs well when compared with clinical data from a national speciality audit database. 99 The arguments against the use of routine administrative data to derive risk-adjusted mortality estimates have been extensively rehearsed in the literature. 100–104

The major criticisms are that:

• the administrative data do not necessarily appropriately or completely describe the patient population

• the process is perverted by the fact that the administrative data used to assign risk may also be used to determine payment

• the risk generated can be ‘gamed’, for example by more extensive coding of comorbidities in those who die or by the use of codes indicating palliative care.

Such factors are likely to have more significance when comparing different hospitals, as practice between hospitals is likely to differ. They are less likely to have detrimental effect when used to examine performance through time in a single hospital (as in the current project), where one would expect coding practices to be (relatively) constant over time. However, inadequate coding could, of course, reduce the accuracy with which underlying risk is estimated. Crucially, there is no a priori reason to suppose that errors might be correlated with the independent variables of interest (i.e. staffing levels on wards) because coding is not undertaken on wards or by ward staff. Furthermore, our chosen approach combines the advantages of a nationally validated risk adjustment model using administrative data with additional risk adjustment using clinical data.

To account for patient-level risk factors, we also used diagnostic and demographic factors (including age and comorbidity) to calculate a predicted risk of death. To do this, we used coefficients derived from the nationally validated SHMI model. 96 We used the June 2015 model, which had been developed from national data for the previous 3 years, approximately coinciding with the study period. 105 This approach allowed us to apply robust estimates of risk associated with a wide range of diagnostic groups (including the effect of interaction between diagnosis and age/comorbidity/admission route) derived from a national population, whereas directly adjusting for mortality risk using the same factors and using only local data would inevitably mean that risks associated with some diagnoses could not be accurately estimated.

Given that the resulting SHMI risk score reflects the average risk for patients with a given diagnosis and comorbidities, we also used the patient’s first NEWS as a measure of their acuity on or near the point of admission. NEWS has been shown to be highly accurate in predicting death in hospital (area under the ROC curve for death within 24 hours of 0.89). 106 As this measure is taken at the beginning of the patient’s stay, it is largely independent of any effects that may result from variation in nurse staffing, whereas subsequent scores may be influenced by care received, including the effects of nurse staffing.

To control for variations in patient acuity on wards when assessing the association between missed observations and staffing, we also calculated the daily proportion of patient observations in categories indicating some higher degree of concern (i.e. NEWS of > 3).

Staffing/outcomes

For all aspects of the analysis, we used multilevel/hierarchical mixed-effects regression models to explore the association between staffing levels and outcomes. In all models, outcomes were regressed onto independent variables for RN staffing levels and HCA staffing levels as the main variables of interest. For patient outcomes, models included independent variables controlling for patient condition and risk. All models also included variables to control for the number of admissions to the ward, also considered as a nursing workload variable (admissions per nurse). Patient outcomes were assessed at the patient level, with patients being exposed to varying staffing levels throughout their stay in hospital. These variables were typically expressed as a time-varying covariate representing the cumulative sum of staffing levels experienced up to that point in the stay. Given that patients could experience staffing on different wards, with different baseline staffing requirements, staffing variables for all patient-level analyses were normalised for each ward.

Missed observations were assessed at the ward level, with the rate of missed observations over a given time period (e.g. day, shift) regressed on staffing levels. For these models, absolute nurse staffing levels were used as predictors, with ward included as a random effect. Where appropriate, and possible, we explored for non-linear effects of staffing variables by including quadratic and cubic terms in models. We also tested for evidence of interactions between RN staffing and HCA staffing variables.

The modelling framework is described in more detail in the next section.

Modelling framework: patient outcomes

We used multilevel/hierarchical mixed-effects survival models to explore associations between nurse staffing and mortality/adverse events. We have repeated observations on the same patient because we observed them over a period of time from admission (i.e. onset of risk) until death/adverse event or discharge date. Covariates of interest (such as staffing levels) change over time and, thus, we had to specify a survival model able to account for repeated measurements and time-varying covariates, and unobserved patient-specific characteristics.

We defined a trivariate response (T₀, T, d) where T₀ is the starting time (i.e. admission date), T is the ending time and d is an indicator for death. Let us consider i = 1, . . ., I patients for which we have j = 1, . . ., T_i observations and let b_i be unobservable patient-level random effects that are independent and identically distributed as a Gaussian distribution with zero mean and unknown variance.

A proportional hazard model was specified. The covariates have a multiplicative effect on the hazard function, for example:

for some baseline hazard function h₀(t_ij). Here, a parametric (i.e. exponential) distribution is assumed for the baseline hazard function, selected as it provided a superior model fit to the Cox proportional hazards, for which no distribution is assumed. The resulting model leads to a likelihood function that is not closed and thus must be approximated. Here, the adaptive Gauss–Hermite quadrature method was employed.

Length of stay is measured on a continuous scale because day and time of admission and discharge for each patient are recorded. Thus, we are able to measure the time spent since admission, not only the ‘rounded’ number of days. Typically, modelling continuous outcomes call for the linear model (i.e. for a Gaussian distributed outcome). However, length of stay exhibits a right-skewed distribution with mode near zero and heavy tails. The Gaussian distribution says that values to the left and right of its mean are equally likely to be seen, by virtue of the symmetry inherent in the form of the probability density. Therefore, to better represent length of stay data features, the gamma distribution was used here [i.e. length of stay ∼ gamma(µ;σ²)]. The gamma distribution describes the probabilities with which a random variable Y takes on values, where Y can only be positive. More precisely, let y_i be the length of stay for the ith patient, the probability density function for which is given by:

where Γ() is the so-called ‘gamma-function’. In the regression framework, the interest lies in modelling the expected value of the outcome, in this case µ. Here a log-linear model, based on the generalised linear model theory, is proposed. Formally:

where x_i collects all patient-specific covariates.

Model variables

For all patient outcomes, we included as covariates the first NEWS, the mode of admission (emergency vs. elective) and a SHMI score indicating the patient’s underlying risk of death given their age, diagnosis and comorbidities [Charlson Comorbidity Index (CCI)]. 107 In addition, we included a random effect for ward, with each ward that a patient was admitted to during their stay assigned a dummy variable (0 or 1), with an additional dummy variable should the patient spend time on a ward where staffing data were unavailable.

The first focus of our analysis was to determine whether or not staffing has an effect on increasing death risk, taking into account the patient’s characteristics. In the core model, we included two time-varying variables measuring the daily exposure to both RN and HCA staffing levels. We extended this in a number of ways to investigate for the presence of further staff-related variables effects, for example by including different exposure variables and/or permanent compared with temporary staffing measures. In general, we considered the effect of RN and HCA staffing separately, reflecting the potentially distinct contribution of each staff group, but also allowing the possibility of staffing substitutions.

Following the approach taken by Needleman et al. 27 we generally restricted our analysis of the association between staffing levels and patient outcomes to consider the effect of staffing experienced over the first 5 days of the hospital stay. Thus, the analysis focused on the period of hospital stay when the patient is most likely to be acutely ill,27 while still including staffing levels experienced by the majority of patients for the majority of their stay (median stay is < 3 days).

Analyses were undertaken using the statistical software Stata^®, release 14 (StataCorp LP, College Station, TX, USA).

Modelling framework: missed care

We used multilevel/hierarchical mixed-effects regression models to explore associations between nurse staffing and missed care in the hospital ward. Let y = (y₁₁, y₂₁, . . ., y_tj, . . ., y_T-1,J, y_TJ) be the observed data vector that consists of clustered data, where y_tj (t = 1 . . . T, j = 1 . . . J) denotes the number of missed observations on day t from ward j, and µ_tj is the expected count given the values of a number of explanatory variables (i.e. covariates) and a set of ward-specific random effects b = (b₁, . . ., b_j, . . ., b_J). Then µ_tj is necessarily a non-negative number, which could lead to difficulties if we considered using a linear model in this framework. Thus, as is often done when modelling counts in a regression framework, the natural logarithm is used as the function to link µ_tj with the covariates. For independent (i.e. not clustered) data, this leads to the Poisson regression model for the natural logarithm of the counts, log(µ_tj). In our setting, as we have clustered data (daily counts of missed care clustered in wards), mixed Poisson models are considered for the logarithm of µ_tj. When modelling at an hourly/shift level, we took an analogous approach with hours or shifts representing the level one unit nested within wards.

Using the logarithmic link function and P covariates x₁, x₂, . . ., x_P, the model may be written as:

where b_j is a random intercept following some known distribution and βs, p = 0, . . ., P, are the regression coefficients associated with the covariates x_ptj plus the intercept term. The term log(m_tj) is included in the model as an offset to account for the total number of observations due at day t in ward j. The most commonly used estimation method for mixed Poisson models is the maximum likelihood. The marginal log likelihood for the considered mixed Poisson model can be written as:

where φ is the distribution of the random effects. The Gaussian distribution with zero mean and unknown variance (to be estimated with all other model parameters) is assumed for φ and the adaptive Gaussian quadrature is used to approximate the integral. A large number of quadrature points (e.g. 11) is used to approximate the likelihood because we have a high number of level 1 units and large intraclass correlations can be assumed.

To check the assumption that our counts of missed observations were adequately captured by the Poisson framework above, we examined the residuals visually for all models using quantile–quantile plots. For the ‘failure to respond’ analysis, we noted the presence of overdispersion and consequently refit a negative binomial model using the same logarithmic link function, although substantive conclusions were largely unaltered. Analyses were undertaken using the statistical software package R version 3.4.0 (The R Foundation for Statistical Computing, Vienna, Austria) using the lme4 and gamlss packages. 108–110

Other modelling considerations and planned subgroup analyses

Given that we were unable to access staffing data for medical teams (and other staff) that were comparable with those for nurse staffing, and indeed because medical staff are generally not clearly allocated to individual wards, it was not possible to directly consider staffing levels from other staff as a potential confounding variable in the analyses. However, we were able to consider this issue by exploring the effects of weekend admissions and stays over the weekend, when medical staffing is known to vary and is suspected to be related to variation in mortality. 111–114 Specifically, we looked to determine whether or not any observed relationships between nurse staffing levels and outcomes are attenuated when patient stays on weekend days are considered as covariates in the model, and also determine whether or not nurse staffing effects are limited to low staffing on night shifts (which might be confounded by low medical staffing at night).

Given that the likelihood and significance of missed observations for an individual may be related to the planned observation frequency, we undertook a subgroup analysis for the association between staffing and missed observations, considering only patients with a high NEWS (therefore requiring frequent observation). We explored for the effect of staffing at a given point in time on missed observations at later points in time, and, because observations are ‘missed’ over extended periods of time, we also explored the effects of the previous shift on subsequent missed observations. We undertook planned analyses on subgroups of wards (general medical, surgical and older people) and subgroups of patients (emergency admissions only).

In addition to assessing the magnitude and significance of the effects estimated for staffing variables in the models, we used the Akaike information criterion (AIC) and the Bayesian information criterion (BIC) to assess relative model fit, preferring models with lower values of AIC/BIC. The first one is known as an estimator of the Kullback–Leibler discrepancy between the data-generating model and the fitted model. The BIC provides an approximation of the Bayesian posterior probability of the candidate model. Compared with methods based on the likelihood ratio test between nested models, these criteria have the advantage of not requiring a bootstrap resampling procedure.

Mediation

Our approach to exploring mediation was based on that outlined by Baron and Kenny. 91 A mediator variable is a variable that accounts, in whole or in part, for the relationship between the independent and dependent variables. 115 First, we hypothesise that low staffing levels reduce the ability of nurses to undertake timely vital signs observations. Second, we hypothesise that timely vital signs observation is a mechanism by which nurses can reduce the risk of death. Thus, we hypothesise that the relationship between nurse staffing levels and mortality is mediated by timely vital signs observation.

If this is the case, there must be a significant relationship between the independent variable of interest (nurse staffing levels) and the dependent variable of mortality (after controlling for other confounders such as patient age and comorbidity). There must also be a significant relationship between the proposed mediator variable (vital signs observation) and the independent variable (staffing levels). Assuming this to be the case, when both the independent variable (staffing) and the mediator (vital signs observation) are included in the regression model, the relationship between the independent variable (staffing) and the dependent (mortality) is significantly reduced or, in the case of full mediation, eliminated. 116

Therefore, to be identified as a mediator, the missed observation variable must satisfy two conditions. First, it must be significantly correlated with the low staffing variable. Second, the missed observation variable must be significantly related with mortality, given that all other related factors are included in the model. If both conditions are satisfied, the variable will be included in the data set as a mediator.

A number of popular methods for the testing of an indirect effect have been proposed in the literature. 116–118 Here, the non-parametric bootstrap was used for inferences. A recommended method to test the hypothesis about a specific indirect effect is to calculate a CI for an indirect effect as the CI gives a range of plausible values for the indirect effect. 119 In general, as long as computing power is not an issue, CIs should be estimated via the bootstrap with > 1000 resamples, which is the used number of simulations in this analysis.

Economic analysis

The aim of the economic analysis was to identify the costs of variation in staffing levels and mixes, and their consequences in terms of reduced rates of missed observations and adverse events. We undertook extensive consultation among patients, the public and professional stakeholders to identify issues and select realistic scenarios to model, with a view to modelling the costs and consequences of staffing levels and skill mix that were associated with ‘acceptable’ levels of compliance with vital signs observations.

To ensure that they are based on an agreed methodology and are nationally representative, we used nationally validated reference and unit values to estimate costs, rather than local figures. For each cost, we determined an appropriate unit cost. We also estimated ‘high’ and ‘low’ costs to determine sensitivity to variation in costs and assumption. For RN staffing we used the unit cost of a band 5 nurse, and for HCA we used the unit cost of a band 2 nurse. For high estimates, we assumed a mix with the staffing band above (50 : 50) and included training costs, using estimates and calculated according to the methodology described in Curtis and Burns,120 although these costs are unlikely to fall directly on the employing trust. As no estimate for training costs is given for HCAs, we estimated these to be half the cost of RN training costs, which is a likely overestimate, although it may reflect the reality of a well-trained assistant workforce. For low estimates of staffing costs, we applied the NHS Improvement agency cap (155% of salary) applied to the hourly salary and applied on costs, but excluding estates and other overheads. We estimated the value of reduced length of stay using the NHS reference cost121 for excess bed-days, which provides a conservative estimate of potential savings from reduced stays. This assumes that no additional treatments costs are incurred associated with extended stays. High and low estimates of cost are taken from the upper/lower quartile estimates. All costs are based on 2015/16 estimates. 121 Sources of cost information are described in detail in Table 3.

The consequences of each scenario were estimated using coefficients from relevant regression models. For estimations involving mortality, we assumed that the HR approximated to a risk ratio, an assumption that is generally valid in the short run and when probabilities of events are small, which is the case with mortality in this study. 123,124 In each case, high and low estimates were based on the upper and lower 95% CIs. We estimated incremental costs and marginal gross costs per life saved associated with each staffing policy, based on staffing costs alone. Net costs were estimated by factoring in the value created by alterations in length of stay. Although reductions in length of stay do not automatically generate cost savings, the resource that is released is of value if it meets the needs of other patients who would otherwise not be able to avail themselves of a bed or of a service that would otherwise not be provided, or hospitals that would need additional bed capacity to meet patient need. Although in some cases fractional days of stay may not be of use, most of our patients were emergency admissions and so we assume that all reductions in stay have value.

Patient and public stakeholder consultations

We undertook a series of consultations with public, patient and clinical experts/stakeholders (including health services managers and ward-based nurses) to identify issues that should be considered when determining scenarios for economic models.

These included:

• determining an acceptable level of vital signs observations compliance (completing a full set of observations in time)

• acceptable ranges of skill mix, assuming that the results demonstrated some potential for substitution between HCA and RN.

The work was done in stages so that different groups were approached separately, thus providing the most conducive environment to engage in conversation and share views relevant to each individual group. We began with consultations with expert and professional groups to elucidate a concrete set of preliminary questions and scenarios that could be presented to other groups. Our approach, co-designed with our patient and public involvement representative/lay co-researcher (ADI), was intended to make the key issues to be discussed tangible for both a lay audience and general professional stakeholders such as staff nurses who might not consider themselves ‘experts’ in the specific topics. Thus, we began with consultations with health-care professionals through an online questionnaire and with members of the Study Advisory Group in order to identify preliminary indicators.

This led to the development of some preliminary propositions about skill mix and observation compliance that were used to structure future consultations.

In these consultations, each group was presented with an outline of the project and related research. Questions focused on:

1. current recommended skill mix and, specifically, whether or not the Royal College of Nursing’s125 recommendation of a minimum 65% RN should be regarded as an absolute minimum, and whether or not higher/lower thresholds should be considered

2. the acceptability of a target for compliance levels, using 90% as a starting point to stimulate discussion.

The ensuing responses were used to ‘sense check’ the propositions, to determine whether or not these scenarios made sense to participants and to capture alternative propositions that we might consider. The consultations included:

• face-to-face engagement with patient and public representative groups

  • Experts by Experience patient groups – co-ordinated directly with members

  • Healthwatch Southampton – co-ordinated by a health community development worker

• online consultation (link distributed at engagement events, to patient groups and on social media)

  • A total of 77 people provided partial responses to the questionnaire and 14 provided complete responses

• face-to-face engagement with senior nursing and management forums

  • Wessex Directors of Nursing Network Meeting, December 2015 (directors of nursing/deputies from all regional NHS organisations; approximately 30 people attended)

  • Nursing and Midwifery Advisory Committee in the Portsmouth Hospital NHS Trust (ward managers and senior nurses; approximately 20 people attended)

  • Deteriorating patient group at Portsmouth Hospital (approximately 15 people attended)

• hosting a Twitter (Twitter, Inc., San Francisco, CA, USA; www.twitter.com) chat (@wenurses)

  • A total of 60 people participated in the Twitter chat.

Nursing hours per patient day

The average observed staffing level across all wards was 7.75 CHPPD, comprising 4.75 RN HPPD and 2.99 HCA HPPPD, with an average skill mix of 60% RN. Staffing levels varied considerably both between and within wards. Mean RN HPPD varied from 2.91 (medical respiratory ward 2) to 9.61 (medical renal high care). Skill mix (the proportion of RNs) varied from 86% (medical renal ward) to 46% (medical general ward 2). Within-ward variability was also high, with the average standard deviation (SD) representing 18% of the mean for RN HPPD, 29% of the mean of HCA HPPD, 10% of skill mix and 16% of total CHPPD. Figure 2 illustrates the variation of RN HPPD across wards. Ward labels and full data are given in Appendix 4, Tables 25 and 26.

FIGURE 2.

Mean RN HPPD by ward.

Although our estimated care hours were based on shift start and end time, and thus made no allowance for breaks (estimated as 8% of care hours), the average CHPPD across these wards falls at the low end of the range observed in the Carter Review of productivity in the NHS127 (where the range was from 6.3 to 15.48). After adjusting for breaks, the average CHPPD we observed was 7.13. The mean RN HPPD was approximately equivalent to a patient-to-nurse ratio of 5.5. This approximate patient-to-nurse ratio is based on the inverse of RN HPPD ÷ 24, which gives the nurse-to-patient ratio (after reducing hours to reflect approximately 8% of shift time as breaks). In the wards we studied, the average RN HPPD was lower by 8% at weekends than on weekdays. Reductions in RN hours at weekends were observed on all wards except surgical admissions, including those that handled exclusively emergency admissions (e.g. medical admissions, where weekend RN staffing was lower by 10%). There was no significant overall difference in HCA staffing levels between weekends and weekdays (mean difference 0.2%).

Mean RN HPPD was highly correlated with the estimated RN HPPD estimated from the planned establishment (Pearson’s r = 0.97), with average RN HPPD 95% of the establishment level. Similarly, mean HCA staffing was closely correlated with the establishment (Pearson’s r = 0.81), although there was more variation, with mean HCA staffing 115% of establishment.

The use of temporary staffing was common, with only 24% of the 30,980 ward days having no bank or agency RN staff and 13% having no bank or agency HCAs. However, the use of high numbers of temporary staff was relatively rare, with 4% of days having 1.5 temporary RN HPPD, although less so for HCAs, with 15% of days with ≥ 1.5 HCA HPPD (Figure 3). The use of temporary staff varied by ward, with one ward showing fewer than 1% of days with no temporary staff (see Appendix 4, Table 28).

FIGURE 3.

Days with temporary staffing.

Exposure to low staffing levels

During a patient’s hospital stay, the median number of days of low staffing (below the mean of whichever ward they were on) was 2 (mean for low RN staffing 1.93; mean for low HCA staffing 1.94), with a mean cumulative shortfall of 0.39 RN hours per patient and 0.25 HCA hours per patient (see Appendix 4, Table 30). Out of a total of 30,980 ‘ward-days’, staffing fell below the mean HPPD on 54% of occasions for both RNs and HCAs. For RN HPPD, staffing was below the estimated establishment on 67% of days and fell below 80% of the mean on 10% of days. For HCA HPPD, staffing fell below establishment on 38% of days and below 80% of the mean on 19% of days (see Appendix 4, Table 29).

Admissions

Admissions per staff member varied between wards, ranging from 0.15 (rehabilitation-neurological) to 5.45 (surgical admissions) admissions per RN. On average, 1.4 patients were admitted per RN and 2.67 were admitted per HCA. Given that typical admissions will vary by ward, and ward establishments should be set to accommodate typical admission levels, we identified days when admissions per nurse exceeded 125% of the mean, reflecting a mismatch between demand and available staff, resulting from either a low number of staff or a high number of admissions. Overall, this occurred on around one-quarter of the days (27% for admissions per RN and 26% for admissions per HCA), with patients exposed to a mean of 0.88 days of high admissions per RN and 0.91 days of high admissions per HCA (see Appendix 4, Table 31).

Missed vital signs observations

In total, 2,869,963 complete sets of vital signs observations were available for the study from eligible wards and were matched to staffing data. Of these, 400,194 (14%) were taken significantly late (between one-third and two-thirds of the scheduled observation interval past time due) and 464,017 (16%) were classified as missed (more than two-thirds of the schedule observation period past time due). Rates of missed observations varied between ward, with the highest rates seen on the rehabilitation-neurological ward (45%) and the lowest on surgical/high care and admission wards (6%). Stroke rehabilitation and respiratory wards also showed high rates of missed observations (between 39% and 31%) (Figure 4).

FIGURE 4.

Percentage of observations missed by ward. See Appendix 4, Table 26 for a key to ward labels.

There are perceived problems with complying with observations scheduled by NEWS in respiratory patients because some patients have persistently high scores as a ‘norm’. 128 Although the two respiratory wards did have a high proportion of observations taken in patients with a NEWS of ≥ 6 (21% and 12%, compared with 6% overall), overall there was a low correlation between the proportion of observations required in high-acuity patients on the ward and the overall level of compliance on that ward (r = 0.24). Of all observations, 2,042,792 (71%) were taken for patients classified as ‘low-acuity’ based on their previous NEWS (NEWS of ≤ 3). Observations in high-acuity patients (NEWS of ≥ 6) accounted for 6% (184,628) of observations. Observations were more likely to be missed in patients classified as high acuity (44% missed) than in those classified as low acuity (9% missed). This pattern was observed on all wards, although the disparity was relatively small in some wards, particularly rehabilitation wards (see Appendix 4, Figure 14).

Who undertakes vital signs observations?

We scrutinised the observation records to determine which staff groups were recorded as taking vital signs observations. Practice appeared to vary by ward, with between 1% and 39% of all observations recorded as being taken by HCAs (see Appendix 4, Table 32). Overall, the average across all wards was 15%. This is likely to under-represent the proportion of ‘nursing’ observations taken by HCAs, as observations taken by other groups (including doctors) are also recorded. Although a sizeable majority of observations appeared to be taken by nursing staff, the lack of standardised coding for staff groups and the variety of groups listed (including a sizeable group of ‘unknown’) rendered more detailed analyses of these data impossible. However, while the proportion of observations recorded as taken by HCAs for high-acuity patients (NEWS of ≥ 6) is lower than for low-acuity patients (NEWS of 0–2), the difference is small (1.8%). The proportion of observations recorded by HCAs on a ward was not significantly associated with average RN HPPD (Pearson’s r = 0.19; p = 0.31) or skill mix (r = 0.25; p = 0.17).

Missed nutritional assessments

We scrutinised the Vitalpac data for other records of care whose completeness could be judged against a clear protocol that would define whether or not the required care was done on time. Although these care items would not necessarily fall on the causal path linking low staffing to mortality, there is an opportunity to explore whether or not these data might prove useful for assessing other aspects of care, and, specifically, to see whether or not there is evidence of associations between low staffing and omissions in other aspects of care. The clearest example of a policy that defined when an assessment was due (and therefore could be used to determine omissions) was the policy for nutritional screening using the MUST tool. Trust policy stated that a MUST assessment must be undertaken for every patient within 24 hours of admission to hospital.

We identified 43,451 instances where we could link staffing levels to a patient for whom a nutritional screening assessment was due within 24 hours of first admission. On 9312 (21%) occasions the assessment was recorded later than this or none was recorded. Compliance with the policy of assessment within 24 hours varied from 100% (rehabilitation-neurological ward) to 8% (surgical–gynaecology, admissions and medical/surgical wards) (see Appendix 4, Figure 14), although some wards had very few direct admissions and so were rarely required to undertake a patient’s first assessment.

Low staffing thresholds

For each day that a patient was on a ward with RN staffing below the mean for that ward, the hazard of death was increased by 3% (HR 1.03, 95% CI 1.01 to 1.06). Each day of exposure to HCA staffing below the mean was associated with a 4% increase in the hazard of death (HR 1.04, 95% CI 1.02 to 1.07). When patients were on wards where admissions per RN exceeded 125% of the mean for that ward, the hazard of death was increased by 5% (HR 1.05, 95% CI 1.01 to 1.09), although there was no association between admissions per HCA and hazard of death (HR 1, 95% CI 0.96 to 1.04) (Table 6).

When considering only deaths within 30 days of admission, the model coefficients were almost identical. Similarly, restricting the model to emergency admissions only had no effect on the coefficients for the staffing variables (see Appendix 5, Tables 36 and 37). Considering staffing on all days of a patient’s stay showed significant adverse effects from low RN staffing only, although the coefficients were lower (HR 1.01 per day of low RN), which is probably a product of increased risk being primarily associated with days early in the stay (see Appendix 5, Table 38).

Patients admitted at the weekend experienced an increased hazard of death (HR 1.06), but the effect of low RN/HCA staffing was unchanged. When 1 or more days of a patient’s stay in hospital were over at a weekend, the hazard of death was reduced (HR 0.58) and the effect of low nurse staffing was marginally strengthened (each day of RN staffing below the mean, HR 1.04; HCA staffing below the mean, HR 1.05). As model fit was not clearly improved by including variables for weekend admissions/stay, we did not include these variables in other models (see Appendix 5, Tables 39 and 40).

Using alternative thresholds to define ‘low’ staffing provides somewhat contrasting results. For each day that RN HPPD fell below the estimated RN establishment, the hazard of death was increased by 9% (HR 1.09, 95% CI 1.06 to 1.11). However, the effect of HCA staffing below establishment was not statistically significant (HR 1.01, 95% CI 0.99 to 1.04; Table 7). There was no association between RN staffing below 80% of the ward mean and the hazard of death (HR 0.98, 95% CI 0.94 to 1.02), whereas HCA staffing below 80% of the ward mean was associated with an increased hazard of death (HR 1.04, 95% CI 1.01 to 1.07; see Table 7).

Although the models using the 80% threshold provide worse model fit (in terms of AIC and BIC), these results highlight the possibility that the staffing outcome relationships are non-linear, or that there could be interaction between the effects of RN and HCA staffing. We explore this further in the subsequent section.

Staffing levels

To further quantify the effect of variation in staffing levels, we explored the effects of absolute variations by calculating the cumulative sum of staffing in hours per patient relative to the mean for the ward for each patient for each of the first 5 days. This gives an indication of the average staffing experienced by the patient over these 5 days, relative to what was normal for each ward, effectively ‘standardising’ and allowing for variation in baseline staffing requirements.

Every additional RN hour was associated with a 3% reduction in the hazard of death (HR 0.97, 95% CI 0.94 to 1.00). However, additional HCA hours were not associated with a reduced hazard of death (HR 1.01, 95% CI 0.98 to 1.04; Table 8). Considering only hours below the mean provided similar estimates, with a 3% increase in the hazard of death for every RN hour below the mean, but no significant association with HCA HPPD (see Appendix 5, Table 44).

The pattern of results across different thresholds, and with staffing as a continuous variable, raised the possibility that the underlying relationship was non-linear, particularly in relation to HCA staffing levels. We explored this by introducing quadratic and cubic terms for the staffing variables (hours ± mean) into the models.

Introducing only quadratic terms left the linear coefficients unchanged, giving no indication of a non-linear effect (HR 1.00 for the quadratic term for both RN and HCA hours). Introducing a cubic term produced a significant quadratic coefficient for HCA hours and a near-significant coefficient for the cubic term. RN hours coefficients were non-significant, although the linear term was similar to the linear-only model. The plot of this model can be seen in Figure 5, with the full model in Appendix 5, Table 45.

FIGURE 5.

Care HPPD and hazard of death: non-linear effects.

There was a slight tendency for the effect of RN staffing to reduce as staffing levels increased, but, as none of the non-linear terms was close to significance, a null hypothesis of linearity cannot be rejected. However, there seems to be some evidence that, as the cumulative level of HCA staffing (hours) experienced by a patient over the first 5 days of hospitalisation varies from the mean in either direction, the hazard of death may be increased. However, as these models do not give an improved fit in relation to the linear-only models (AIC 61920.19, BIC 62453.88), any conclusions about non-linear relationships must be made with caution.

We also considered whether there was evidence that variation in levels of HCA staffing could alter the ‘effectiveness’ of RN staffing (or vice versa) by introducing interaction terms into our models. There was no significant interaction between the cumulative sums of hours of RN staffing and HCA staffing for hazard of death. However, the interaction between these two cumulative sum variables gives no direct indication of how the staffing levels ‘on the day’ might interact, so we generated variables to indicates days of low RN staffing, low HCA staffing or both. This model provided no evidence of interaction with the sum of the coefficients for low staffing by one or the other group, yielding a combined HR very similar to that associated with low staffing by both (see Appendix 5, Table 46).

Temporary staffing

We considered the effect of temporary nurse staffing by adding a variable to indicate the number of CHPPD that were provided by temporary (bank or agency) staff. We considered three different thresholds for a ‘high’ level of temporary nurse staffing, and calculated a variable to reflect the cumulative sum of days the patient was exposed to temporary staffing above that threshold. In general, adding these variables for temporary staffing did not substantially alter the coefficients for other staffing variables, although levels of significance did alter. Temporary staffing was associated with significant increases in the hazard of death, particularly at higher levels (Table 9).

When patients experienced days with ≥ 1.5 HPPD of temporary RN staffing, the hazard of death was increased by 12% (HR 1.12, 95% CI 1.03 to 1.21). Exposure to lower levels of temporary RN staffing was not associated with significant increases in the hazard of death. Although the hazard of death was significantly increased with exposure to days with ≥ 30 minutes of temporary HCA staffing (HR 1.05), the trend was inconsistent across the different thresholds: exposure to days of ≥ 1.5 HPPD of temporary HCA staffing was associated with a similar increase in hazard of death (HR 1.04). There was no clear benefit in terms of improved model fit compared with models without the inclusion of temporary staffing (AIC was marginally lower and BIC was marginally higher) and, therefore, we did not routinely include this variable in other models.

Adverse events

We calculated a composite outcome to identify patients who experienced a severe deterioration resulting in the first occurrence of any ICU admission, cardiac arrest or death. Every additional RN HPPD over the first 5 days was associated with a 2% decrease in the hazard of an adverse event (HR 0.98, 95% CI 0.96 to 0.99; Table 10).

Length of stay

To explore the association between staffing levels and length of stay, we averaged staffing relative to the mean over the first 5 days of the patient’s hospital stay (Table 11).

Higher RN staffing levels are associated with a small but significant reduction in length of stay. Every 1-hour increase in average RN HPPD was associated with a reduction in length of stay of 0.23 days (95% CI –0.30 to –0.13 days; p < 0.001). Additional hours of HCA staffing were associated with a small but significant increase in length of stay of 0.08 days (see Table 11). We tested for non-linear effects by considering quadratic terms. This was not significant for RN staffing. For HCA staffing, the quadratic terms were significant but had little impact on the overall relationship for a plausible range of values.

Ward subgroups

We explored ward subtypes focusing on older people, surgery and medicine (including all relevant specialties but excluding mixed medical/surgical, high dependency and admission units). Full results are reported in Appendix 5, Table 50.

Compared with the overall analysis, when analysing only medical wards the association between RN staffing levels and the overall rate of missed observations was stronger (IRR 0.94) and the association with HCA staffing levels was weaker (IRR 0.97). The findings for surgical wards were similar to the overall results: RN IRR per additional hour was 0.98 and HCA IRR per additional hour was 0.95. However, for care of older people wards, increased HCA HPPD was associated with lower rates of missed observations (IRR 0.91), but a higher level of RN staffing was associated with an increased rate of missed observations (IRR 1.06). In all cases there was a significant interaction between RN and HCA staffing. Plotting these interactions gave broadly similar results to that for the combined sample (Figure 7).

FIGURE 7.

Interaction of RN and HCA staffing in medical wards: missed high-acuity observations.

Focusing on high-acuity observations only in these subgroups yielded few significant results, although the pattern broadly mirrored that found in the combined sample. There were two exceptions: a significant interaction between RN and HCA staffing for medical wards and a significant reduction in the rate of missed observations in high-acuity patients associated with higher HCA HPPD in older people’s wards. Plotting the interaction effect for medical wards provided a ‘classic’ picture of HCA staff acting as labour complements by increasing the effect of RN staffing, while having no direct effect in isolation and not acting as a substitute for RN staffing (see Figure 7). The only evidence that HCAs might substitute for RNs in taking high-acuity observations came from care of older people wards, where HCA staffing was already high (average HCA HPPD of 3.93 hours vs. 2.85 hours in the surgical wards and 2.93 hours in the medical wards). The relatively small number of wards on a single site limited our ability to further explore these subgroups, although, among other issues, these findings do raise the possibility of non-linear effects; these will be explored further.

Shift and hourly level

We calculated separate models for the association between staffing levels and missed observations on four shifts (night, early, late and twilight). First, we considered associations with staffing levels at the time when the observation was due to occur. Given that observations are missed over a period of time rather than in a single moment, we also calculated models to explore the associations between missed observations and staffing on the previous shift and the subsequent shift.

There was evidence for associations between staffing and rates of missed observations on all shifts, which was broadly in line with the pattern observed at the day level. Although night RN and HCA HPPD were not significantly associated with the rate of missed observations on a night shift, the number of admissions per RN was. There was also evidence that the rate of missed observations on most shifts was significantly associated with staffing levels on both previous and subsequent shifts.

Focusing on observations in high-acuity patients, only a few associations were significant, although the pattern of results broadly resembles the results from a day-level analysis. Higher RN HPPD were associated with significant reductions in the rate of missed observations in high-acuity patients on early (IRR 0.99) and twilight (IRR 0.97) shifts. The non-significant relationships on night and late shifts were similar (IRR 0.99). HCA HPPD were associated with significantly higher missed observations in high-acuity patients on early shifts (IRR 1.02) and lower rates on a late shift (IRR 0.98). Missed observations on an early shift were significantly associated with RN HPPD on the previous shift (IRR 0.98) and the subsequent shift (IRR 0.98). See Appendix 5, Table 51, for full results.

We also analysed the association between staffing levels and missed observations at an hourly level. These models also included control for time of day. For models considering all observations, the direction and significance of the relationships were similar to the day-level analysis (RN HPPD IRR 0.98; HCA HPPD IRR 0.97; p < 0.01). For observations in high-acuity patients, the main effect relationships between staffing and the rate of missed observations were not significant, although event rates for many observation periods were very low.

Non-linear effects

We explored the potential for non-linear associations between HPPD and the rate of missed observations by including quadratic and cubic terms in the models for associations between staffing and missed observations at the day level. The full models are reported in Appendix 5, Table 52.

There was clear evidence for significant non-linear effects for both RN HPPD and HCA HPPD in relation to the overall rate of missed observations, with significant linear cubic and quadratic terms for both staff groups. For both RN and HCA HPPD, the incremental effects of adding staff diminish at higher staffing levels, with a clear indication that no further benefits are achieved by adding staff to increase HPPD above 7. The relationships are plotted in Figure 8.

FIGURE 8.

Non-linear associations between staffing and rates of missed observations.

However, for observations in high-acuity patients only, there was no evidence of non-linear effects for RN HPPD, with incremental benefits continuing at higher staffing levels. However, significant quadratic effects suggest that increased HCA staffing is associated with increases in the rate of missed observations until its peak at approximately 3 HPPD, after which the effect of adding staff is to reduce the level of missed observations from this peak (Figure 9).

FIGURE 9.

Non-linear associations between staffing and rates of missed observations: high-acuity patients only.

Although there was no significant interaction effect for RN and HCA HPPD in relation to missed observations in high-acuity patients, there was a significant interaction effect for the overall rate of missed observations. In very simple terms this showed that the incremental benefits of adding staff in one group was highly dependent on both the level of staffing by the other group and the base staffing level.

To further explore and clarify this complex relationship, we modelled the relationship as total CHPPD and skill mix, that is, the proportion of RN HPPD of all CHPPD, including non-linear terms (quadratic and cubic). These results do indeed shed light on the underlying relationship. There was a significant but non-linear association between total CHPPD and the rate of missed observations (see Appendix 5, Table 53). The effect of skill mix is not significant and there is no interaction between skill mix and staffing levels. The interaction between the two staffing levels observed in the previous models is a consequence of decreasing incremental effectiveness of adding more staff of any group, with no additional benefit once CHPPD reaches 10. The relationship is illustrated in Figure 10.

FIGURE 10.

Incremental effectiveness of 1 CHPPD at different staffing levels.

Failure to respond

Timely observation is only the first step in preventing or mitigating the effects of deterioration and therefore we also calculated two variables that might be indicative of a wider failure to respond. Both of these variables were based on identifying patients with a high NEWS, indicating the need for intervention and identified in the response protocol as requiring review by a doctor. We then examined patient records and vital signs observations to determine evidence that a patient’s high NEWS was reduced or action taken (ICU admission or end-of-life care pathway) in a period determined as four times the response time identified in the protocol (4 hours for NEWS of ≥ 7; 16 hours for NEWS of ≥ 6). Where the outcome was not achieved, this might be indicative of a failure to properly respond to an identified deterioration. We calculated rates of response and determined associations with staffing levels.

‘Trigger’ events were common, with 189,123 NEWS of ≥ 6 and 114,504 NEWS of ≥ 7 events affecting 28,098 patients in total. At the lower threshold (NEWS of ≥ 6, response within 16 hours), the average level of response was 84%. For the higher threshold (NEWS of ≥ 7, response within 4 hours), rates of response were lower (50%). The death rate was high among affected patients, reflecting the high level of mortality predicted by these NEWSs (18% for those triggering a NEWS of ≥ 6; 23% for those triggering a NEWS of ≥ 7).

Neither RN HPPD nor HCA HPPD were significantly associated with rates of response for the lower threshold/longer response (NEWS of ≥ 6) criteria, but higher levels of RN HPPD were significantly associated with reductions in ‘failure to respond’ using the higher threshold/shorter response criteria (IRR 0.98, 95% CI 0.96 to 0.99; see Appendix 5, Table 54).

Nutritional risk assessments

Higher HCA HPPD were associated with significantly reduced rates of missed nutritional assessments in the first 24 hours of admission (OR 0.80). There was no significant main effect for RN HPPD, although higher admissions per RN were associated with significantly more missed assessments (OR 1.08), and the RN by HCA HPPD interaction was significant. This interaction suggested that associations between RN staffing levels and timely MUST assessments may exist when HCA staffing is low, but this association diminishes as HCA staffing levels increase (Table 13 and Figure 11).

FIGURE 11.

Interaction between RN and HCA HPPD and missed nutritional assessments.

Staffing levels and mortality

Although the association between nurse staffing levels and patient outcomes has been extensively studied, questions have remained about whether the observed associations were causal. 14 This study has addressed many of the shortcomings of previous research by directly linking patient outcomes to the level of staffing experienced by individual patients, showing that variation in a patient’s experience preceded their outcome and was associated with the staffing that they had directly experienced. The only comparable study of which we are aware is that by Needleman et al. 27 In a study of 197,861 patients in 42 units in a single US hospital, which was described as having low mortality for the case mix and high success in meeting planned staffing levels, the authors found that, for each shift that was ≥ 8 RN hours below target staffing, the hazard of death was increased by 2%. 27 High turnover shifts were associated with a 4% increase. Although the measurement units differ from those in our study, these findings are remarkably similar to ours.

Although there is considerable extrapolation involved, some idea of the comparability of the results in these two studies can be gained by identifying the shortfall from target RN staffing for our ‘days below the mean’ measure of exposure to low staffing, which was, on average, 0.32 HPPD below the target staffing level. On a ward with 30 patients, a day with staffing below this level would be ≥ 9.6 hours below target (HPPD × patients), and in our study this was associated with a 3% increase in the hazard of death. In common with the Needleman et al. study,27 we also found that mortality was increased during periods of high patient turnover relative to the norm for the ward and available RNs.

Studies that have directly linked outcomes to patient exposure to low staffing are rare, but Kuntz et al. 129 explored the effect of exposure to periods of high bed occupancy on mortality among 82,280 patients in 83 German hospitals. They demonstrated a tipping point, with mortality increase occurring when bed occupancy exceeded 92.5%, and no effect below this point. Until the tipping point is reached, the resources available, including staff, are able to absorb fluctuations in demand with no adverse effects. Once the tipping point is passed, the capacity to buffer is exceeded and so additional demand is associated with worse outcomes. 129

Although average occupancy for the Trust was close to or above this level for much of the current study, we did not observe a ‘tipping point’ in relation to the association between RN staffing levels and mortality. The relationship appeared to be linear. Other studies have observed non-linear relationships between nurse staffing levels and mortality. 130 However, the estimated non-linear relationships appear to be of the nature of a maximum benefit arising at staffing levels that are very high compared with those observed typically in the UK, rather than a ‘tipping point’ beyond which adverse outcomes occur. If there is indeed a tipping point, it may be that typical staffing levels observed in this study have already passed it, so there is no buffering effect to observe, and fluctuation in demand cannot be accommodated within the current capacity.

Few previous studies in this field have taken medical staffing levels into account, although a nurse staffing effect independent of medical staffing levels has been demonstrated in cross-sectional studies in the NHS and elsewhere. 36,39 Although we were unable to directly account for variation in medical staff levels in this study, the nature of the design means that it is not confounded by hospital-level correlations between medical and nursing staff sizes. Unless medical staffing variation is significantly correlated with ward-level nurse staffing levels, the relationships we observed are unlikely to be generated by variation in medical staff cover. We have managed to eliminate one significant source of correlation between medical and nurse staffing levels as an explanation of these results, given that the nurse staffing effect was independent of a ‘weekend’ effect.

Temporary staffing and flexible capacity

Our findings on the effects of temporary staffing are relevant to the issue of ‘buffering’ capacity and response to fluctuating demand. We found that, when temporary RN staffing was more than 1.5 HPPD, the hazard of death was substantially elevated. Clearly, the use of temporary staff is an important strategy for dealing with fluctuation in demand or imbalance between demand for and supply of staff caused by staff shortages. Current evidence around the effects of temporary nurse staffing is largely from the USA. A cross-sectional study131 in 665 US hospitals found that the use of agency supplemental nurses was associated with a 4% increase in the odds of death when controlling for staffing levels, but that the effect reduced and became non-significant when including a variable measuring the perceived quality of the nursing work environment. An earlier US study132 recorded similar findings in relation to nurse-reported adverse outcomes, such as job dissatisfaction and burnout, although nurses’ intention to leave the job was significantly associated with higher use of temporary staff even after adjustment for practice environment.

Although the design of our study is generally stronger than that of these cross-sectional studies, our study using routinely collected data was unable to control for nurse perceptions of the practice environment. However, this variable is generally considered to be a hospital characteristic and so the variation observed in our single-hospital study cannot be accounted for in this way. Furthermore, a nurse’s negative perception of the practice environment could itself be related to the use of temporary staff and so the extent to which this should be ‘controlled’ is unclear.

Other studies with an analysis at a unit level give results that are consistent with ours, albeit using other outcomes. We found that the modest use of temporary RN staff (below 0.5 or even 1 HPPD) was not associated with adverse outcomes. A study in 277 units in 142 US hospitals133 found that that the high use of temporary staff was associated with increased rates of falls and nurse injuries, but only where use of temporary staff exceeded 15% of the nursing workforce. While this study differentiated between internal and external temporary staff, the effects appeared to be associated with both groups.

Studies from both the UK and the USA indicate that, in general, wards that use more temporary nursing staff are more expensive to run than wards with primarily permanent nurses,134,135 although one study from the USA found that a modest use of supplemental (temporary) nurses (up to 0.2 nursing HPPD) was associated with slightly reduced total staffing costs. 136 Although our findings point to no harm from the modest use of temporary RNs, and even hint at a benefit because it maintains the overall staffing levels to meet patient need, our findings about the use of temporary HCA staffing suggest that there is risk in even relatively low levels (although it should be noted that 0.5 HPPD is a substantial proportion of the typical hours provided by HCAs). It is unclear whether or not the adverse effects are limited to external (agency) staff and whether or not strategies designed to mitigate the negative effects of temporary staffing (such as specially trained float pools) might be more effective.

Health-care assistant staffing levels

Previous research has provided a nearly uniformly negative conclusion in relation to staffing by assistant personnel. 14 Studies have made explicit conclusions that additional assistant staff have an adverse effect, irrespective of the level of RN staffing, because of the dilution of skill mix. 137 However, this may be an artefact that results from failing to examine the contribution of each staff group independently. Skill mix can be affected by changes in staffing levels by one group, or the other, or both, which is masked by a conclusion based on skill mix alone. One of the few studies to address each group separately, our cross-sectional study in English hospitals,36 in which HCA and RN staffing levels were modelled as independent effects, found that an apparent adverse association with higher HCA staffing at a hospital (trust) level was not present when looking at relationships with ward-based staffing. Where possible, in the current study, we attempted to model the effect of each group separately.

Although the relationship between lower RN staffing levels and increased mortality was generally clear, the relationship between HCA staffing levels and mortality was not. A large number of admissions per RN was associated with increased mortality, but we found no relationship between admissions per HCA and mortality. Although estimates of the additional time associated with admitting or discharging patients (likely to be highly correlated with admissions given high bed occupancy) have not been reported, admissions and discharges are widely recognised as significantly contributing to nursing workload; however, this is not recognised in workload measures based solely on patient census (e.g. HPPD). 138 However, as completing documentation and patient assessment for new admissions is primarily a RN role, the absence of a relationship with HCA staffing is unsurprising here given that we have formulated this as a ‘workload’ variable.

When patients experienced below-average HCA staffing, their risk of death was increased. However, when we modelled HCA HPPD relative to the mean as a continuous variable, we found no significant effect. The relationship was somewhat clarified by the significant non-linear relationship that showed mortality increasing as HCA HPPD deviated from the mean in either direction, suggesting that there is an appropriate and necessary level of HCA staffing that is close to current means or establishments.

To make sense of such a finding requires an exploration of possible mechanisms. The effect of low staffing seems to be potentially explained by mechanisms already articulated to explain associations between low staffing and adverse outcomes. In simple terms, there is insufficient capacity to deliver the required care completely and with quality. Low HCA assistant staffing may have direct effects or indirect effects by virtue of the absence of sufficient support for RN work. However, the mechanism by which a higher number of HCA hours per patient could increase the risk of death is less clear. One possible mechanism is through a combination of an increasing requirement for RN to delegate tasks to make full use of the capacity provided by additional HCA hours per patient, combined with an accompanying increase in the RN supervisory load. Recent research from the UK139 makes it clear that newly qualified RNs struggle to properly manage this process of delegation and supervision.

Low HCA staffing adversely affects mortality because the overall capacity to deliver work is low, with the negative effects arising partly because of the inability of HCAs to deliver their own work (demonstrated by the partial mediation of this relationship by missed vital signs observations), but also, potentially, by the effect this has on RNs’ ability to complete their own work.

However, when HCA HPPD is high, the amount of work that must be delegated to them in order to make use of available time is high. Even when the demand for supervision is balanced by RN capacity, this could result in inappropriate or ineffective delegation of tasks. When RN capacity is low, the ability to properly supervise the delegated work may also be compromised. Hypothetical configurations are illustrated in Table 21, with green and dark green indicating configurations in which the contribution of HCA HPPD could be negative – because of either the low capacity or the demands placed on RNs for delegation and supervision.

Clearly, this hypothetical mechanism requires further exploration, but it does provide a potential pathway whereby an apparent increase in the overall workforce capacity could have negative consequences. The use of non-registered assistants to undertake vital signs observations is one area where the effectiveness of the delegated work has been widely questioned. 140

Missed vital signs observations

The monitoring of vital signs is a fundamental component of the ‘Chain of Prevention’,92 a structure that describes the interventions necessary to identify and prevent avoidable patient deterioration (Figure 12).

FIGURE 12.

Chain of Prevention. 92 Reproduced with permission. © Gary Smith.

Reproduced with permission. © Gary Smith

The nursing roles in this ‘Chain of Prevention’17,18 and the extensive evidence of an association between nurses’ reports that they are unable to complete all necessary care and low staffing141 make the timely observation of vital signs a strong candidate mechanism through which low staffing may lead to adverse patient outcomes, including death. In this respect, this study breaks new ground in the use of objective measures of nurse behaviour in recording vital signs. Although many studies have shown that compliance with vital sign monitoring protocols is often poor,142 studies looking at nurse staffing have relied on nurse reports of missed care based on quite general questions.

As hypothesised, we found that the rate of missed and late vital signs observations increased when nurse staffing levels were lower. We also found some evidence that this relationship mediated the relationship between low RN staffing and mortality.

However, our findings were complex. We found that the total staff complement in terms of CHPPD (RN+HCA HPPD) was associated with the overall rate of missed observations, with no association between the rate of missed observations and skill mix (proportion of RN hours), whereas overall only RN HPPD were significantly associated with the rate of missed observations in a subgroup of observations in ‘high-acuity’ patients (NEWS of ≥ 6). The rate of missed observations in this group was, in turn, associated with increased mortality, and the association between mortality and exposure to days of RN staffing below the mean was mediated by missed observations.

The ‘Chain of Prevention’ highlights that there is more to preventing deterioration than simply complying with an observation protocol. Although it seems clear that assistant staff can be trained to take vital signs observations, concerns have been raised about their ability to appreciate the significance of these signs and initiate appropriate actions. 140 Our data suggest that on many wards a high proportion of vital signs observations are undertaken by HCAs. Anecdotal reports of device sharing and imprecision in the coding of staff in the system make it likely that our overall estimate of 15% of observations taken by HCAs is an underestimate, but even without this consideration there are several wards where over 30% of observations are taken by HCAs. Although some published accounts suggest that most observations are now taken by HCAs,143 our findings could provide some reassurance: sensitivity to RN staffing levels indicates that observations in high-acuity patients are RN work, although there appears to be little difference in the proportion of high- and low-acuity observations undertaken by HCAs. Nonetheless, these findings, combined with the association between HCA HPPD and nutritional screening and the association between RN staffing and rates of failure to respond, create a picture that is consistent with completion of ‘routine’ assessments being sensitive to HCA staffing, whereas acute care is more sensitive to RN staffing levels.

It is clear, however, that the mechanisms through which missed observations can translate into adverse outcomes (and the interaction between the staff groups in these mechanisms) are potentially complex. The relationship between exposure to days of low HCA staffing and mortality was only partially mediated by missed observations, and the relationship between RN hours and mortality was not mediated. Clearly, other causal mechanisms are at play that are not directly mediated through vital signs observations. In simple terms, the effect of low RN staffing appears to be mediated by missed observations, but other mechanisms related to the overall level of nursing care available are not.

Although overall HCA staffing was not related to missed observations in high-acuity patients, the ward subgroup analysis and exploration of non-linear relationships suggest that the pattern may change at higher staffing levels. Significant interaction between RN and HCA staffing levels on medical wards provides a clear picture of HCA staff acting as a labour complement144 in that the effect of an increase in HCA HPPD is to increase the incremental effect of RN HPPD in reducing the rate of missed observations. However, evidence elsewhere of this complementary relationship is scant, both in the current study and in others. 19,74

The extent to which the work of unregistered assistants does indeed complement that of RNs is a crucial issue in the face of developments in the NHS, where the creation of a new cadre of assistant personnel, the nursing associate, is explicitly based on their presumed ability to complement the work of RNs. 145 Although these new roles are by no means directly comparable with HCAs, they highlight that the potential for support staff to complement or substitute for the work of RNs is somewhat hypothetical and the nature and consequences of reconfigurations in nursing work in patient surveillance are not well understood.

Missed care as an indicator of nurse staffing adequacy

A core aim of this project was to explore the potential to use measures of missed care, and vital signs observation compliance in particular, as a direct indicator of staffing adequacy. Although the results of the study do provide support for the role that missed care may play in mediating the effect of nurse staffing levels on outcomes, two conclusions are clear. First, missed observations are not the only mechanism. This is perhaps an obvious conclusion as the recording of vital signs observations is a very partial measure of the completeness of the ‘Chain of Prevention’. However, the second conclusion is that the rate of missed vital signs observations is not a good indicator of staffing adequacy. Given that 16% of all observations were recorded as missed, it seems clear that most missed observations are attributable to factors other than overall staffing levels, as estimated relationships suggest that it is unlikely that staff increases are capable of reducing the rate to anywhere near zero, particularly for high-acuity observations.

Partial representation of the quality of the care delivered does not preclude missed care from providing a valid indicator, as long as it represents an important aspect of care. Vital signs observations are clearly important, and this study has demonstrated the potential utility of routine data systems to provide records of missed care. Our findings have also established that missed observations may be associated with important patient outcomes. Therefore, ‘missed observations’ meet some criteria for a good indicator. 146–148 Nonetheless, the rate of missed observations falls some way short of providing an ‘ideal’ indicator, particularly if focusing on the adequacy of nurse staffing.

The observed associations with nurse staffing, although statistically significant, were modest. Furthermore, it is unclear what the ‘correct’ level of compliance should be. Although our stakeholder consultation suggested that near 100% compliance is desirable, it is unclear whether or not this would represent a meaningful and important improvement in quality. Indeed, it is unclear if all non-compliance with the protocol should be construed as a failure of quality because the impact on patient outcome and the scientific basis of the particular observation regime are unclear. Current observation protocols are largely based on custom and practice, or on expert opinion. 142 The potential to alter clinical outcomes depends on the ability to detect and recognise an acute change in a patient’s physiology. Therefore, more frequent vital signs measurement and more complete observation sets should increase the probability that deterioration will be detected early, and some published research results support this. For example, an Australian study149 reported reductions in unplanned ICU admissions and hospital deaths when the vital sign measurement frequency increased from a mean of 3.4 to 4.5 times per day. A study from the Netherlands150 found that vital signs measured three times per day resulted in better detection of physiological abnormalities than a regime of taking observations only when ‘clinically indicated’. Belgian researchers151,152 reported that the use of a standard observation protocol, with observation frequency increasing according to an early warning score based on the observations, led to an increased frequency of recording vital signs, reduced hospital length of stay and mortality in postoperative patients and fewer serious adverse events for patients discharged from ICU.

However, other studies make it clear that the ideal observation frequency remains uncertain. A protocolised ‘once a day’ early-warning score assessment may be sufficient to screen for major adverse events in hospital populations,153 and research from Denmark found that for low-risk patients 8-hourly observations were no better than 12-hourly observations for reducing clinical deterioration. 154 Although a recent review of monitoring techniques found that continuous patient monitoring allowed earlier detection of deterioration in general ward patients than ‘usual care’, there was insufficient evidence of any impact on patient outcomes. 155

Therefore, although there is some evidence that increased frequency of observations may improve detection and patient outcomes, a finding supported by the results of this study, compliance with a specific regime cannot be said to be evidence based and it is not clear that outcomes continue to improve as observation frequency increases. Although under-observation can delay the opportunity to detect patient deterioration and initiate remedial treatment, over-observation uses valuable nursing resources that may be better deployed to other essential aspects of patient care and risks disturbing patients unnecessarily, with adverse consequences including interruption of sleep. 156 Although these results demonstrate that some non-compliance appears to have adverse consequences, much of it may not, with nurses exercising clinical judgement to prioritise observations on those most in need, or to prioritise other patient needs. The results of this and other studies suggest that these judgements may not always be correct, but the shortfall between current levels of compliance and that required by the observation regime is considerable. Moving to 100% compliance with the current regime may not be the best investment of nursing time.

Although changes in compliance rates within a ward might be influenced by staffing levels and thus act as a ‘red flag’ (as envisaged by NICE12), it is unlikely that missed observations can be directly used to guide staffing decisions. We have demonstrated that other measures of missed care can now be derived from routine data; these have also shown a degree of sensitivity to nurse staffing and so might prove useful in monitoring quality, although the relationship of these measures to patient outcome has not been established.

Determining safe staffing levels

Studies showing associations between nurse staffing levels and the quality or outcomes of care are not new. This study provides important confirmation of such associations in the English NHS and provides improved confidence that the observed relationship is likely to be a causal one. However, the relationships observed in this study have not, in themselves, provided a basis to model required staffing on different wards at different times of day.

The evidence base for current methods of matching the size and skill mix of the nursing workforce is extremely limited, particularly in terms of demonstrating that staffing matched to the calculated requirements delivers improved patient outcomes. 157 Our findings are all based on staffing levels relative to the norm for individual wards (either by directly deriving variables relative to the norm for our mortality analyses or by including a random effect with missed care clustered within wards for missed care analyses). Ward staffing levels in the Trust were determined using the Safer Nursing Care Tool,158 the only evidence-based tool that has received early endorsement from NICE. 98 Although mean staffing levels deviated somewhat from what was planned, there was a strong correlation between actual and planned staffing levels.

Although the results of this study give an indication that the level of HCA staffing that had been determined by the tool might be close to optimal, based on the non-linear relationship observed with mortality (where risk increased as staffing levels moved from observed mean levels in either direction), the level of RN staffing cannot be said to be optimal when improvements in outcomes are observed, as RN staffing levels increase above the mean in a linear fashion. However, these planned staffing levels are unlikely to reflect daily variation in need. Although our findings give some limited support to the tool, it remains unclear whether staffing solutions based on daily (or more frequent) measures taken with the tool provide an effective basis for determining adequate staffing, or whether routinely collected data such as NEWSs might have a role.

Economic and labour market considerations

Increased staffing levels, by necessity, lead to increased staff costs. Based on the data presented in this report, increases in HCA staffing would be associated with increased cost, but with either neutral or negative effects on outcomes. However, we found that increases in RN staffing were associated with improved outcomes, and, in some circumstances, the additional costs were more than offset by the value of hospital bed-days saved.

Our economic modelling suggests that, had the Trust been able to maintain its average staffing and skill mix at the levels identified using the Safer Nursing Care Tool, this would be associated with improved outcomes and reduced length of stay. This would involve small increases in RN staffing and a similar decrease in HCA staffing levels. Although changes to RN staffing leading to improved outcomes were associated with increased staff costs under all scenarios, the value of the decreased stay offsets the cost of additional staff in this scenario. This essentially involved a small increase in skill mix, moving close to the 65% level recommended as a minimum by the RCN,125 with little, if any, change in total CHPPD.

Although this change does not result in any direct cost savings for the Trust, given that the bed capacity released is likely to be used, the value of the resource that is released for use by other patients cannot be discounted. Our economic models assumed no additional treatment costs were associated with the increased stays and we costed them simply as ‘excess bed-days’. Even under this modest assumption, the findings are consistent with a positive return on investment. However, the absence of patient- or person-centred outcomes or quality-of-life data makes a comparison with returns from other investments challenging.

Our study places an emphasis on RN staffing levels and skill mix as the key variables associated with patient safety and, potentially, cost-effective care. Studies from the USA have come to similar conclusions; for example, a strategy of raising the proportion of RN in the licensed nursing workforce to the 75th percentile, observed in a sample of 799 US hospitals (94% RN), was the only strategy modelled to be associated with a net decrease in hospital costs. Although this finding is not directly comparable with ours, because it considered skill mix only within the licensed nursing team, these findings provide no encouragement for strategies that seek to maintain care hours by introducing less qualified/lower-skilled workforce roles, or for benchmarking systems such as the NHS CHPPD,127 which implicitly treats all members of the ward team as potentially equivalent.

Although these findings imply that there may be benefit in increasing RN staffing levels, there is clearly a major shortage in supply of RNs in the UK. This seems likely to be exacerbated by demographic change, the consequences of public sector pay restraint, wider labour market changes and long-term mismatch between the supply of and demand for RNs. 159 The solutions to these long-term problems are not all within the reach of a single trust. Although national pay frameworks prevent employers competing for scarce supply of nurses on the basis of remuneration, a trust still has the opportunity to present itself as a more attractive employer based on non-monetary benefits and the working environment on offer. In this context, there has been a recent resurgence in interest in the concept of ‘Magnet Hospitals’, a term initially coined to describe those hospitals whose work environments enabled them to attract and retain professional nurses. 160 Other research,38 linking higher staffing levels to higher job satisfaction, lower staff burnout and intention to remain with the current employer, suggests the potential for a virtuous cycle whereby higher staffing levels reduce nursing turnover and sickness absence linked to burnout and fatigue. However, this has yet to be tested with a prospective study.