The Safer Nursing Care Tool as a guide to nurse staffing requirements on hospital wards: observational and modelling study

Article history

The research reported in this issue of the journal was funded by the HS&DR programme or one of its preceding programmes as project number 14/194/21. The contractual start date was in May 2016. The final report began editorial review in May 2019 and was accepted for publication in October 2019. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Peter Griffiths reports grants from the National Institute for Health Research (NIHR) during the conduct of the study [Health Services and Delivery Research (HSDR) 13/157/44, PR-ST-1115-10017 and HSDR 17/05/03)] and support from NIHR through a senior investigator award and as part of the NIHR Collaboration for Leadership in Applied Health Research (CLAHRC) Wessex. He is a member of the NHS Improvement safe staffing faculty steering group. The safe staffing faculty programme is intended to ensure that knowledge of the Safer Nursing Care Tool (SNCT), its development and its operational application is consistently applied across the NHS, using the evidence base underpinning the SNCT. This role attracts no remuneration. Jane E Ball reports grants from NIHR during the conduct of the study (PR-ST-1115-10017) and support from the NIHR CLAHRC Wessex. Rosemary Chable reports grants from NIHR during the conduct of the study (HSDR 13/157/44). Andrew Dimech reports that he was working on his clinical doctorate during the conduct of the study. Jeremy Jones reports grants from NIHR during the conduct of the study (II-LB-0814-20006, RP-PG-0610-10078 and HSDR 17/05/03). Thomas Monks reports grants from NIHR as part of NIHR CLAHRC Wessex during the conduct of the study.

Copyright statement

© Queen’s Printer and Controller of HMSO 2020. This work was produced by Griffiths et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2020 Queen’s Printer and Controller of HMSO

Overview

Approximately 310,000 whole-time equivalent (WTE) registered nurses were employed in NHS hospital trusts in England during July–September 2018, with an estimated 11.6% vacancy rate; that is, there were > 40,000 WTE registered nurse vacancies. 1 Multiple reviews of research have established that higher registered nurse staffing levels in hospitals are associated with better patient outcomes and care quality. Reported associations include lower risks of in-hospital mortality, shorter lengths of stay and fewer omissions of necessary nursing care. 2–5 Newer studies6–9 exploring patient-level longitudinal relationships and causal mechanisms provide increasing confidence that the observed relationships are, at least in part, causal. However, beyond providing an injunction to invest in ‘more’ staff, such studies rarely offer a direct indication of how many staff are required in a given setting. Given the size of the nursing workforce and the shortages in supply, the ability to properly determine an ‘optimal’ staffing level is an imperative from the perspective of both quality and efficiency of care.

The Francis Inquiry10 into the failings of the Mid Staffordshire General Hospital (Mid Staffordshire NHS Foundation Trust) identified low staffing as a significant contributor to ‘conditions of appalling care’. A key recommendation of this inquiry was the development of guidance for nurse staffing, including:

. . . evidence-based tools for establishing what each service is likely to require as a minimum in terms of staff numbers and skill mix.

Francis. 10

Contains public sector information licensed under the Open Government Licence v3.0

The use of tools can vary to support a range of operational decisions, for example how many staff to employ on a ward, which staff are needed on particular shifts and whether or not staffing is adequate, and for billing purposes. Despite the large number of nurse staffing tools available, the determination of the hours required to maintain ‘adequate’ or ‘quality’ staffing remains problematic. Commissioned to develop national safe staffing guidelines for the NHS in the UK following the Francis Inquiry, the NHS advisory body the National Institute for Health and Care Excellence (NICE) concluded that:

There is a lack of research that assesses the effectiveness of using defined approaches or toolkits to determine nursing staff requirements and skill mix.

© NICE 2014 Safe Staffing for Nursing in Adult Inpatient Wards in Acute Hospitals (Guidance). 11 Available from www.nice.org.uk/guidance/sg1. All rights reserved. Subject to Notice of rights. NICE guidance is prepared for the National Health Service in England. All NICE guidance is subject to regular review and may be updated or withdrawn. NICE accepts no responsibility for the use of its content in this product/publication

The evidence review undertaken by NICE was based on existing reviews and focused on the impact of staffing tools and methodologies on outcomes and quality,12 but other reviews of broader scope over many years have reached similar conclusions. 13–18

Despite the lack of evidence, soon after publishing its guidance in 2014, NICE endorsed the Safer Nursing Care Tool (SNCT) as a decision support tool for setting nurse staffing levels on adult inpatient wards to meet patient need, based on an acuity/dependency measure. 19–21 Already the most popular tool in use in the NHS, it is now used in the vast majority of acute NHS hospitals. 22 The present study explores the use of the SNCT and seeks to address a number of key questions about the tool, including (1) the extent to which the staffing requirements estimated using the tool provide staffing levels that accord with professional judgements about sufficient staffing to provide quality care, (2) the best approach to using the tool’s results to set the required staffing level and (3) the costs and likely consequences of staffing wards according to the levels prescribed by the tool, questions that, thus far, have been largely neglected in the literature about this or any other tool.

Nurse staffing levels and outcomes

Reviews of observational studies2–5,12,23,24 have consistently concluded that there is an association between higher registered nurse staffing levels and better outcomes for patients in acute hospitals. In 2007, a systematic review identified 96 studies and concluded from a meta-analysis of 28 that increased registered nurse staffing levels are associated with lower risks of in-hospital mortality (for intensive care, surgical and medical patients), shorter lengths of stay (for intensive care and surgical patients) and reductions in occurrences of various other adverse patient events. 4 The majority of the papers included in this 2007 meta-analysis were from the USA, but the association between nurse-to-patient ratios and in-hospital mortality has subsequently been confirmed in several other countries and regions, including in a large-scale cross-sectional study25 of more than 400 hospitals in many European countries.

Subsequent reviews have confirmed associations between nurse staffing and care quality across a range of outcomes and process measures, including omissions of essential nursing care, which has been identified as a key mechanism leading to adverse patient outcomes. 5 Because of the varying scope of material in the reviews, it is now hard to judge the full extent of the evidence, but an assumption that the size of the evidence base has more than doubled since Kane et al. ’s4 2007 review would probably be a highly conservative one. However, the evidence is, without exception, observational and, with few exceptions, cross-sectional, albeit based on large studies often involving hundreds of hospitals and millions of patients. 2

Overview of approaches to setting nurse staffing levels

Many approaches to setting nurse staffing levels have been described. Typologies to classify these approaches have been proposed by, for example, Hurst16 and Edwardson and Giovannetti. 15 We present the main types (using these proposed typologies with minor adjustments), along with examples, in Table 1. The number of nursing staff funded to work on a ward is called the nursing establishment. 11

Telford’s professional judgement method,38 which was first formally described in the UK in the 1970s and later documented by Hurst,16 shows how a shift-level staffing plan can be converted into a recommendation for the number of staff to employ. The method provides a framework for deliberation and consideration, but at its core the judgement of required staffing is based on the judgement of senior nurses, with no recourse to any direct calculation or formal use of an objective measure to determine need. 14 The calculations offered relate solely to the conversion of shift-level staffing plans to establishments or vice versa. In recent years this deliberative approach without formal measures appears to have been reflected in the US Veterans Administration staffing methodology. 39 Other approaches that could be characterised under this method include benchmarking approaches by which expert judgements are made about suitable comparators, and the staffing levels and achieved quality are compared between similar units.

Benchmarking exercises are often used to arrive at a nurse-to-patient ratio or equivalent (e.g. nursing hours per patient) to assign a minimum or fixed number of nursing staff per occupied bed. 16 Although characterised by Hurst as a distinct method, there is little initial formal assessment of patient requirements for nursing care, although consensus methods and expert judgement are often used when determining the staffing requirement for particular types of wards and selecting appropriate benchmarks.

Effectively, these approaches assume that all patients in similar wards have similar care needs, or that the average need is stable across patient groups for a given shift period, so that most of the time needs can be anticipated and met with a fixed roster. When demand is above this, it is expected that additional staffing requirements be determined. Although the benchmarking comparison with similar wards gives the appearance of objectivity, much depends on how the initial staffing levels were arrived at, and there is ample evidence that perceptions of staffing requirements are largely anchored to historical staffing levels. 40 Because some more recent approaches to monitoring workload extend this approach to take a wider view of activity, adding in admissions and discharge over and above the patient census, we will term these benchmarking and ratio approaches ‘patient volume’ approaches.

Other approaches attempt to account for variation in needs between patients, and these can be used instead of, or together with, minimum staffing levels. For example, patient classification systems (also referred to by Hurst as acuity/dependency systems) categorise patients according to their nursing care needs and recommend a staffing level for each category. 17 These categories may either be designed specially (e.g. classifications based on levels of acuity and/or dependency) or already be in use (e.g. diagnosis-related groups). 17 The SNCT is one such system, allocating patients to one of five acuity/dependency categories with a weighting (described as a ‘multiplier’) to indicate the required staff to employ associated with patients in each category. 19 On the other hand, the timed-task approach, rather than categorising patients, considers each new patient individually. The required staffing for a patient is calculated from time estimates for a list of the tasks and activities planned for that patient. 16 The commercial GRASP system, widely used in the USA, is one example. 15 In all of these approaches, the total staffing required is calculated by summing up the individual patient requirements, possibly with some overheads.

The approach to determining the required times for patient groups, specific tasks or broad areas of activity varies. The literature describes the use of both empirical observations and expert opinion to determine the average time associated with activities or patient groups. 41–43 In some cases, there is an explicit attempt to make workload/time allocations based on reaching some threshold of quality or optimum care. 44,45 Approaches to identifying the time required for work aside from directly caring for patients vary, but all approaches take this into account, often assigning a fixed percentage of time over and above that measured for patient contact.

Hurst16 also described regression-based approaches. These are based on models of the relationship between patient-, ward- and hospital-related variables, and the staffing level observed in ‘quality-assured’ wards. The regression coefficients, together with the values for each variable, are used to estimate the required staffing for other wards. Perhaps owing to the obvious complexity, there are relatively few examples, although Hoi et al. 46 provide one recent example, the workload intensity measurement system. In some respects, the regression-based models simply represent another approach to allocating time across a number of factors rather than directly observing time linked to specific activities. The RAFAELA system, widely used in the Nordic countries, although based on a relatively simple patient classification, also uses a regression-based approach to determine the staffing required to deliver an acceptable intensity of nursing work for a given set of patients in a given setting. 44,45,47

Edwardson and Giovannetti15 offer a slightly different typology, differentiating ‘prototype approaches’ from ‘indicator approaches’ and ‘task approaches’. Prototype and task approaches broadly resemble the patient classification and task approaches (respectively) previously described. Indicator approaches ultimately assign patients to categories but this is done based on ratings across a number of factors that are related to the time required to deliver patient care. These can include broad assessments of condition (e.g. ‘unstable’), states (e.g. ‘non-ambulatory’), specific activities (e.g. complex dressings) or needs (e.g. for emotional support or education). 15 The Oulu Patient Classification, part of the RAFAELA system, is one such example. Here patients are assigned to one of four classifications, representing different levels of care required, based on a weighted rating of care needs across six dimensions. 44 However, the inclusion of some specific activities in Edwardson and Giovannetti’s15 definition of indicator approaches makes it clear that the distinction from task-/activity-based systems is not an absolute one. Typically, however, task-based systems take many more elements into account: over 200 in some cases. 15

Evaluation of staffing methods/tools

One early review of nurse staffing methods, published in 1973,18 included a bibliography of over 1000 studies but found no evidence concerning the relative costs or effectiveness of different staffing methods. It concluded that there was no substantive basis for choosing any one system over another, little evidence for validity beyond ‘face’ validity and scant evidence even of reliability, summing up that ‘Although the intent of the methodologies is admirable, all are weak’ (p. 57). 18

Subsequent reviews have had to embrace an ever-growing body of research, and yet they reach remarkably similar conclusions. A review13 undertaken for the then Department of Health and Social Services in the UK in 1982 identified over 400 different systems for determining staffing requirements. Writing in 1994, Edwardson and Giovannetti15 noted the absence of published scientific evidence for a number of commercial systems, such as GRASP or Medicus, which were in widespread use in North America. They also noted that, although different systems tended to produce results that were highly correlated, they could, nonetheless, produce substantially different estimates of the required level of nursing staff for a given patient or unit. 15 Edwardson and Giovannetti’s15 analysis concluded by asking a number of questions, which remained unanswered by the evidence they reviewed:

• Do the results of workload measurement systems differ substantially from the staffing requirements judged by practising professional nurses?

• Does the implementation of a staffing methodology or tool lead to altered staffing levels or, conversely do historical staffing levels influence the assessment of need?

• Do workload measurement systems improve the quality of care?

• Do workload measurement systems result in more efficient use of nursing personnel?

More recently, in 2010, Fasoli and Haddock17 reviewed 63 sources (primary research, theoretical articles and reviews) and also concluded that there was insufficient evidence of the validity of many current systems for measuring nursing workload and staffing requirements. They described the field as dominated by descriptive reports of locally developed approaches, with limited or weak evidence of validity for even widely used and commercially supported systems. 15,17 They concluded that systems in use may not be sufficiently accurate for resource allocation or decision-making. 17,48

There is little basis for preferring any one tool or approach over another based on the available evidence. Professional judgement-based approaches, despite being open to accusations of subjectivity, cannot be readily dismissed, as there is no substantial evidence that moving from a judgement-based staffing model to one informed by a tool has improved any outcomes or made more efficient staffing allocations. 2,13,14,16–18,40 One of the most comprehensively researched systems determines the staffing requirement by titration against a subjective report of work intensity. 44,47 Uncertainty over the best method might matter little if different approaches gave similar results. Although direct comparisons are few and far between, it is, however, clear that different systems can give vastly different estimates of required staffing. 49–52 In one study,52 the five systems tested provided estimates that correlated highly. However, they offered a wide range of average staffing requirements for the same sample of 256 patients, with the lowest average being < 60% of the highest [6.65 vs. 11.18 hours per patient-day (HPPD)].

The reviews discussed here, as well as others,2,13–18,40,48,53 give a clear indication of the vast literature on nurse staffing methodologies, but the field appears to have made little progress over many years. Most research reports on the underlying development of tools or assesses their implementation, but little research validates the resulting tools, and the reviews indicate that none has evaluated the impact of implementation on outcomes for staff or patients.

The Safer Nursing Care Tool

Following the Francis Inquiry report and the development of NICE guidelines, the SNCT was the first tool endorsed by NICE for setting nurse staffing levels. Following this, NICE endorsed three other tools. 54 Of these, one is a rostering system incorporating an acuity/dependency tool based on the SNCT (Health Roster/SafeCare^®, Allocate Software, London, UK), one is a web-based tool that explicitly claims to use the SNCT criteria [iPAMS, formerly Albatross Financial Solutions; documentation found at https://docplayer.net/17204848-Ipams-integrated-patient-acuity-monitoring.html (accessed 2 November 2018); now IQVA UK/NHS Solutions] and one is a generic framework for reviewing establishments using benchmarking principles (Establishment Genie, Creative Lighthouse Ltd, Beaconsfield, UK).

The SNCT is a prototype tool. The documentation of the tool19 provides a framework for incorporating professional judgement and assessment of nursing-sensitive outcomes as part of a ‘triangulated approach’. However, the core of the tool is a method for determining the average patient need for a given ward based on assessing patients and allocating them to one of five acuity/dependency categories, each of which has a ‘multiplier’ corresponding to the typical workload involved in caring for a patient in that category. From these data, a ward establishment sufficient to provide staff to meet patient need is calculated, making provision for necessary ‘time out’, including study leave and expected sickness/absence rates.

The ubiquity of the SNCT is confirmed by surveys of English NHS acute hospital trusts. In 2010, as part of the RN4CAST study of the nursing workforce, we undertook a survey of a stratified random sample of 31 English acute hospital trusts. We found that a majority (59%) of these trusts used a formal acuity/dependency system to guide nurse staffing decisions. Most identified the tool used as the SNCT or its precursor (RN4CAST survey, 2010, unpublished data). Although the SNCT is designed to set nursing establishments based on periodic review, 36% of hospital trusts reported that they monitor acuity/dependency daily, making a responsive, flexible staffing policy potentially feasible. In 2017, we undertook a national survey of acute hospital trusts in England (91/148; 61% response rate) and found that > 80% were using the SNCT as part of their establishment-setting process. 22

Despite its endorsement and widespread use, there is no existing review of evidence about the SNCT.

Commissioning brief

This research was commissioned to address a number of key knowledge gaps identified by NICE and amplified in the commissioning brief 14/194 – ‘measuring nursing input, workload, activity and care’ from the National Institute for Health Research (NIHR) Health Services and Delivery Research programme (see the project web page at www.journalslibrary.nihr.ac.uk/programmes/hsdr/1419421/#/ for the full commissioning brief), specifically themes ii, ‘Reviewing and validating tools to measure nursing activity and workload’, and iv, ‘Making use of nurse staffing data in NHS organisations’.

Chapter summary

• Evidence that establishes associations between nurse staffing levels and important patient outcomes, including mortality, indicates the likely importance of getting nurse staffing levels right, but does little to indicate what the correct level should be.

• We identified a vast literature going back over many years, which described and purported to evaluate tools and systems for determining nurse staffing levels on hospital wards.

• Nurse staffing tools and systems are used to inform the employment of staff (setting ward establishments) and the deployment of staff (both in terms of staffing plans and deployment on the day) and as evaluating the successful delivery of the staffing plan, and for billing purposes.

• Although there are many distinct tools and systems, they tend to fall under one or more types of approach.

• The main types of approach are professional judgement approaches, patient volume approaches, prototype/patient classification approaches, approaches that are based on assigning times to tasks on a care plan, and indicator and regression approaches, where several factors are considered.

• Prototype/patient classification approaches, where an average time (or equivalent measure) is assigned to a patient ‘type’, are the most common and the most widely used.

• In the UK, the SNCT is an example of a prototype approach, assigning patients to one of five categories based on an assessment of their acuity and dependency on nursing care.

• The distinction between types of approach is not an absolute one and, for example, the SNCT relies heavily on professional judgement.

• Despite the vast literature, a series of reviews have drawn very limited conclusions about the tools and are unable to provide unequivocal support for the validity, costs or effects of any tool, or type of approach.

Recent literature on staffing methodologies

Review results

We found no new reviews and 36 primary sources published from 2008 onwards to consider (see Report Supplementary Material 1, Table 85, for fuller descriptions). We classified the sources according to the main purposes of the articles: description, comparison, development, operational research or evaluation. Some articles did not clearly sit in one or another category and were given a dual classification.

Discussion of recent research

Although recent years have seen a continued interest and a significant number of publications relating to systems and tools to determine nurse staffing requirements, the unanswered questions identified by Edwardson and Giovannetti15 in 1994 remain largely unanswered. The finding that outcomes improve as staffing levels rise above the ‘optimal’ level determined by the RAFAELA system80 is consistent with perceptions of adequate staffing being anchored by expectations set by current staffing levels, a phenomenon that has been widely noted and in turn is cited as a rationale for the use of formal systems. 38,39 It may well be that predetermined or historical staffing levels do alter the outcome of the workload measurement systems, despite the apparent objectivity.

Although there are perceptions of benefits, the effect of workload measurement systems on the quality of care remains unclear. In general, the results of a workload measurement system seem to correspond closely to professional judgement, although the magnitude and the significance of differences between the tool and professional judgement are hard to quantify in terms of implied staffing levels as they are generally not reported. However, different systems can give dramatically different results and so it is clear that there can be no single answer to the questions of whether or not workload measurement systems result in an improvement in the utilisation of nursing personnel.

The Safer Nursing Care Tool

In this section, we report specifically on research relating to the SNCT and its precursors. We considered all material that described or discussed the tool, although we have been selective in our use of purely descriptive accounts. We identified material by searching for the names of the tool and its precursors in databases (CINAHL, The Cochrane Library, MEDLINE, Scopus) and on the World Wide Web, and following paper references to find earlier documentation. We also enquired of the tool developers and members of our steering group. The searches were completed in November 2018. Our account of the tool’s development and history has also been informed by discussions with Keith Hurst and our advisory group member Ann Casey, two of the leading figures involved in developing and supporting the tool.

Literature review conclusions

The literature on staffing methodologies is vast and growing, suggesting continued interest. However, the conclusion of previous reviews that there is no substantial evidence on which to base staffing decisions or the selection of a decision support tool remains true. Despite the lack of evidence, it seems clear that an appetite for formal systems and tools exists. Although professional judgement remains the nearest to a gold standard, the desire to use a tool or another formal system to support, and indeed justify, such a judgement has remained a constant theme that can be traced back to Telford’s work in the 1970s in the UK,38 and no doubt further. The lack of evidence for existing tools seems to have propelled each generation (and, indeed, in some cases, each service) to develop a new tool. On the other hand, the widespread adoption of the SNCT in the UK might be indicative of a perceived validity, although NICE’s endorsement of it as an ‘evidence-based tool’ and the marketing efforts of e-roster companies that have embedded it may also explain its widespread uptake.

Although the time estimates for the SNCT appear to come from a large database, this is not well documented. However, in the face of rapidly changing care needs, it is not clear how long even such relatively extensive observation can be relied on, although the multipliers continue to be updated. Even if average nurse staffing requirements are accurately estimated, the costs and consequences of using the tool to base daily staffing on an average are unknown. Alternative approaches to setting required staffing levels using the tool’s measurement have not been explored.

Rather than continually invent new tools, it seems to be time to take a closer look at those already in use. Controlled trials may be problematic, but there are sufficient unanswered questions that much progress can be made before determining whether or not such a trial is strictly necessary. There is little evidence about many of the assumptions underlying the use of tools in general and, specifically for the UK, the SNCT. Questions about the extent to which the tool truly identifies a level of staffing sufficient to meet the needs of a ward of patients are unanswered, as are questions about the number of observations required to obtain an accurate baseline to estimate average need, or the costs and benefits of staffing according to the tool’s recommendations. The apparently simple assumption that staffing to meet average need is the optimal response to varying demand is also untested.

Chapter summary

• Reviews published up to 2014 concluded that there was insufficient evidence to establish the reliability, validity or fitness for purpose of any of the many tools that are described.

• Most studies were conducted on a small scale and reported local implementations.

• There is no evidence of the costs or effectiveness of using any tool.

• Our focused review of more recent literature confirmed this picture.

• Recent studies report the development of new tools but do little to establish ‘reliability’.

• There is some evidence that high workload as measured by some tools is associated with increased mortality.

• The published evidence does not, however, give any clear evidence that tools are identifying ‘safe’ or ‘optimal’ staffing levels.

• Professional judgement remains a key criterion against which staffing levels determined by tools are judged and, in some cases, forms the basis for setting the required staffing against the tool’s measure.

• We found evidence that there may be considerable variation underlying the measures determined using tools.

• We found evidence that setting staffing levels based on average demand could lead to understaffing for > 50% of the time.

• The clinical significance of such understaffing is unclear, but a ‘tolerance’ level of 15% is suggested in the literature.

• Our focused search for literature relating to the SNCT, a measure endorsed by NICE, confirmed that there is limited evidence about the tool.

• We found no published evidence on daily variation and the reliability of staffing estimates derived from the tool.

• We found some evidence that suggested that low staffing, relative to the tool’s recommended establishment, was associated with increased mortality.

• We found no evidence that nurses working on wards judged the staffing levels determined using the tool to be sufficient.

Setting

The study was undertaken using data from somatic acute adult care wards (excluding intensive care units) in one university hospital, two district general hospitals and one specialist cancer hospital trust based in London, south-east England and south-west England, with five hospital sites in total. The hospitals serve diverse populations, including those living in rural areas, deprived inner-city populations and specialist national referrals.

All four hospital trusts undertook regular periodic reviews of nurse staffing levels broadly in line with the approach recommended by NICE,11 with reviews undertaken two or three times per year. Two trusts had been using the SNCT as part of the process for some time and one had adopted it shortly before the study commenced, having not used a formal tool as part of the process previously. The other hospital trust had previously used the ‘patient requirements for nursing care’ tool, a multidimensional indicator-based system developed in Canada,101 but likewise had adopted the SNCT shortly before the study commenced. Three of the four trusts adopted a commercial system for daily reporting patient acuity dependency using the SNCT categories shortly before or during the study, with the fourth implementing a locally developed solution, initially with the primary intention of collecting data for the current study.

Over the course of a full year (2017), for each ward and for each day we sought to identify the staffing level actually deployed, the staffing level required as estimated using the SNCT, and nurses’ professional judgement about the adequacy of staffing.

Data sources

To address our aims, we need data on daily variation in staffing requirements as estimated by the SNCT, patient numbers, other ward factors that could affect workload, actual staffing, nurses’ judgements of the adequacy of that staffing and staff costs. The connections between the data requirements and the broad study aims (‘why needed’), as well as the data sources, are shown in Table 5.

For data field specifications for the hospital systems (patient administration systems, rosters and Bedview/SafeCare), see Appendix 1, Tables 28–30.

Variables

Piloting and data collection period

We met with a group of staff consisting of eight matrons/ward sisters/charge nurses and an IT systems expert at hospital trust A during July 2016 to discuss the proposed staffing adequacy questions and their implementation. We discussed the questions to ensure that interpretation was consistent among the group. In particular, we found a shared understanding of the concepts of ‘quality’ and ‘safety’, and tasks considered under ‘care left undone’ aligned with the goals of the study. Staff agreed that the questions were the right ones to ask to check professional judgement about whether or not staffing is right/enough. At this meeting, ‘specialing’ was identified as a large consumer of nursing resource, so we decided to include this aspect in our study. We also identified an issue with IT that affected how questions were displayed, and started to address how to deal with them at this meeting. We shared findings from this meeting with the other hospital trusts, including ideas about how to increase levels of completion.

A pilot study of the staffing adequacy questions was run in November 2016 on 22 wards at hospital trust A and 17 wards at hospital trust C. Following our analysis of response rates and correspondence between questions, some less informative/relevant questions were dropped, although these were still asked internally at some of the trusts. We also analysed the data from the pilot at trust C to identify patterns in levels of completeness. The pilot study showed that the data could be collected on busy wards.

Following the pilot study, the data sets were collected for the period 1 January 2017 until 31 December 2017, with the exception of wards at hospital trust B, which began routinely recording SNCT ratings in February 2017.

Data transfers, storage, cleaning and processing

Data extracts from hospital systems were provided by each hospital trust via a secure file transfer service. All patient data were de-identified by pseudonymisation at source. Data were stored on secure encrypted servers at the University of Southampton and access was restricted to the project team.

Our study involves linking multiple data sets. As wards had different names in the different data sets, we gave them unique identifiers, allowing us to link the data sets by ward and period (deterministic linkage). When linking data sets, we checked the number of rows before and after in case any rows failed to link correctly.

Where there were major changes, such as ward moves, changes to the patient population or bed numbers, data for that ward were split into parts corresponding to before and after the change. In the reliability and validity testing, we considered each part. In the simulation modelling, we took the first part and assumed that it was there throughout. For some wards, data were recorded separately for their sub-wards; for a description of how we dealt with this, see Appendix 1. We identified missing data; the procedure for dealing with missing data is explained in the relevant sections about variables (see Variables) and methods (see Overview of analytical methods).

Data cleaning, processing and statistical analyses were carried out in R statistical software version 3.5.0 (The R Foundation for Statistical Computing, Vienna, Austria) because of its flexibility in easily repeating analyses with different parameters. Data cleaning included checking the range of values for each relevant data field, cross-checking data fields and standardising the data fields across the hospital trusts. We checked for duplicate records and removed these where appropriate.

We removed days with no patients, but otherwise kept all ward occupancy data. We identified extreme values of actual staffing and estimated staffing requirements by looking at actual staffing per patient-day and weighted-average multipliers respectively. We used a widespread definition of outliers, namely points outside the mean plus or minus 3 standard deviations. Actual staffing per patient may be atypical if the ward is not functioning as normal (e.g. over the Christmas period). Estimated staffing requirements may be unusual if there is an error in the recording of SNCT ratings. Anomalies can also arise during periods when wards move or merge temporarily, as most staff apparently are attached to the original ward name and patients and some staff are attached to the current physical location. Outliers were removed from the descriptive analyses and further modelling. As days during which there were daylight saving time changes have 1 more or 1 fewer hour than the usual 24, we removed these days from all analyses.

For staffing adequacy questions with yes/no answers (coded 1/0) we removed invalid answers (not 1 or 0). We considered answers to the question ‘registered nurses needed’ potentially valid if at least 1 was entered. We considered answers to the question ‘nursing support workers needed’ as potentially valid if at least one staff member in total (‘registered nurses needed’ plus ‘nursing support workers needed’) was entered. On days when it was reported that there were not enough staff for quality and more staff had been requested than worked on the ward during the preceding 24 hours (7 a.m.–7 a.m.), we calculated the increases in registered nurses, nursing support workers and total staff requested over the number working.

Compared with the other hospital trusts, trust D had a very low average percentage of shifts with ‘enough staff for quality’ per ward. On closer examination of the responses, we found evidence of consistent reverse coding (using 0 to mean yes and 1 to mean no) at all trust D wards except two for this question. We identified this issue by cross-referencing against other questions and using logical rules. As there were two wards where reverse coding did not happen, we ruled out a software issue; rather, this appears to be a training issue. Having highlighted this issue part way through the study, we decided against retraining as this could have ended in a mix of correctly and reverse-coded responses. Instead, we switched the responses (1 to 0 and 0 to 1) in all of the wards where we identified reverse coding.

Overview of analytical methods

Simulation modelling

The ward data collected were used as the empirical basis for an analysis of staffing policies by simulation. Our research question is ‘How do the costs and percentage understaffed shifts compare between scenarios that set flexible (low), standard and high ward establishments, and how do other factors affect the results?’. We assess this using a scenario-based analysis.

We used a Monte Carlo agent-based simulation118,119 framework to model each ward in a hospital. The simulation was developed in the multimethod simulation software AnyLogic researcher edition version 8.3.2. (The AnyLogic Company, Quorn, UK; www.anylogic.com; accessed 15 November 2017). We designed this simulation model for comparing alternative ways of using the SNCT for setting the nursing establishment (total nursing staff employed in WTE) on inpatient wards in a hospital. A video demonstration is available at https://eprints.soton.ac.uk/430632/.

Our simulation model consists of a hospital with a user-specified number of wards. These wards are the ‘agents’, which move between being understaffed, adequately staffed or overstaffed (their ‘states’) each shift, as shown in Figure 1. To understand how generalisable our conclusions might be, we developed models based on data from each of the four hospitals, with the number of wards and other parameters derived from each hospital.

FIGURE 1.

Ward state chart.

In the simulation model for each ward and on each shift, we assume that staffing is planned according to a recommended level, based on the SNCT. We modelled a number of scenarios using the SNCT in different ways to determine this baseline level. In addition to the conventional approach to using the SNCT, whereby the staffing establishment is based on the mean staffing requirement, we considered setting baseline staffing at higher or lower levels. On any given shift, a mismatch between the planned staffing and the staffing required could occur because of variation in patient need or because the planned staff were not available (e.g. because of sickness/absence).

We describe the simulation model following the Strengthening the Reporting of Empirical Simulation Studies – Agent Based Simulation (STRESS-ABS) guidelines. 120 A summary is below, and for full technical details including the model logic and assumptions see Appendix 1. In particular, for simulation input parameters see Appendix 1, Tables 34 and 35, and for logic diagrams see Appendix 1, Figures 8–10.

Scope, assumptions and input parameters

We determined the feasibility of filling any staffing shortfalls first from re-deployment within the hospital (wards with excess staff), second from using bank staff and third from using agency staff. We defined triggers for the point at which more resource is needed and priority rules for which ward receives extra resources if there are insufficient resources in total. Here, we assumed that wards with the highest shortfall as a proportion of their total staffing requirement for that shift would be prioritised.

We used estimates from hospital trust B of the proportion of bank and agency shifts filled to model the likelihood of filling a shift from bank/agency (see Appendix 1, Tables 36 and 37); these were not available for the other hospital trusts. We also tested the impact of different assumptions about the availability of temporary staff. We assessed the implications of lower efficiency for temporary staff on wards by using varying assumptions about the relative efficiency of temporary staff (e.g. 1 agency staff hour = 0.75 permanent staff hours). We included time of the day and day of the week patterns for ward occupancy and for the bank and agency request fulfilment rates.

We included a factor to allow for unanticipated absences on planned shifts. For this, we assumed that each rostered registered nurse (or support worker) had a 3% (or 4%) chance of being absent with insufficient lead time to rearrange the roster. We also recognised that when assigning staff to shifts there will be some rounding to deal with the mismatch between whole bodies and the staffing required.

For each shift, we generated the number of patients in each acuity/dependency level. Because we had no data on individual patient trajectories, time periods (6-hour shifts) in the model were treated as independent of one another; for example, the number of patients and their acuity/dependency levels in one period do not affect the number and levels in the next. We assume that work done or not done in one period does not affect the staffing requirements in the next period. We also assume that requirements in different wards are independent of one another, that is that neither the numbers of patients nor the acuity/dependency levels of patients in different wards interact. We assume that the actual (rather than the planned-for) needs of patients in the same acuity/dependency level vary (see Appendix 1, Figures 11 and 12).

Running the model

The model was run to simulate data for a period of 1 year for each hospital and for each comparative analysis, because hospital finances are typically planned over 1 year. We ran the model 10 times (i.e. for 10 independent years) for each scenario and calculated 95% confidence intervals around the mean to assess the error around the estimate. We chose this number of replications because it gave 95% confidence intervals that were narrow enough for our purposes (for the base scenario, widths were < £0.25 cost per patient-day at each hospital trust and < 0.7% understaffed patient shifts at all trusts). We also calculated the ranges to assess the variability of results in different years. 121

Outputs

The key performance measures that we used to compare scenarios were the percentage of patient shifts that are understaffed and the cost per patient-day. We considered shifts to be ‘understaffed’ if the total effective staffing (i.e. accounting for the lower efficiencies of temporary and re-deployed staff) was < 85% of the estimated requirement according to the SNCT for the shift. This allowed a buffer of 15% either side of the estimated requirement within which staffing is considered adequate, as is also used in the RAFAELA tool. 47 We looked at other outputs for some sets of scenarios, where relevant (see Appendix 1, Table 38, for descriptions of all of these performance measures). The simulation model records the necessary outputs for each ward (agent) and over the hospital (system).

Verification and validation

Throughout the simulation model development process, we performed verification (checking that the model is implemented correctly in the simulation software) and validation (checking that we built an appropriate and accurate enough model). As recommended by Sargent,122 we documented the conceptual validation, computer model verification, operational validation and data validation checks that we carried out (see Appendix 1, Tables 39–43). Conceptual validation refers to assessing whether or not the conceptual model represents the real world accurately enough for the purpose of the study. Computer model verification involves checking whether or not the computer model is programmed correctly to match the conceptual model. Operational validation means whether or not the simulation model results are similar enough between runs to allow conclusions to be drawn. Data validation consists of checking whether or not the data used in the simulation model are accurate enough for the purpose. In conclusion, we are highly confident about the operational validity and computer model correctness and are relatively confident about the conceptual validity and data validity.

Sensitivity analyses

We performed multiway sensitivity analyses to test the impact of varying parameters, particularly those for which we do not have data, on the key performance measures (percentage of understaffed shifts and cost per patient-day). We used a 2^k factorial design and computed the average effect of each factor and the 95% confidence intervals to assess whether or not each effect is statistically significant. 123 We also tested for two-way interaction effects.

Scenarios

We investigated scenarios that differ in how they use the SNCT to set staffing establishments. The ‘standard’ approach calculates a staffing establishment based on the mean requirement over a data collection period. We compared the consequences and costs of using this approach to setting establishments with those of using an approach that relies more heavily on flexible staffing. This ‘low’ establishment is set to meet 80% of the mean requirement, with the intention that variable demand above this is met by internally re-deploying staff and/or by using temporary/flexible bank and agency staff. We contrasted this with a ‘high’ staffing approach, which sets staffing at the 90th percentile and hence staffs wards at a level that would be expected to match or exceed demand 90% of the time if that level of staff were deployed.

We explored the system complexity including running scenarios with different assumptions for temporary staff availability, and using data from different times of year to set the ward establishments (see Appendix 1, Table 44, for full descriptions of all scenarios. We ran all scenarios for all four hospital trusts separately, and ran some scenarios interacted with each other; for example, for each method for setting ward establishments, we ran all scenarios with different assumptions for the availability of temporary staff.

Expert and public involvement

Our multidisciplinary study team consists of experts in nursing research, operational research applied to health care, health economics and statistics, as well as senior nursing leads with responsibility for implementing staffing policies, from each of the four hospital trusts. Throughout the study, we held regular (fortnightly) internal team meetings and supplementary ad hoc meetings at which the team members providing the core analysis sought guidance from the relevant experts as appropriate. We also held regular meetings with the principal investigators at the hospital trusts, both face to face and via conference call, particularly in the set-up and data collection phases. A steering group of further experts who were independent of the study team oversaw the study. This group provided advice on the delivery of the project, including technical aspects and pertinent issues.

When developing our research proposal, we discussed appropriate patient/public involvement with Claire Ballinger [patient and public involvement lead for NIHR CLAHRC (Collaboration for Leadership in Applied Health Research and Care) Wessex] and Anya de Iongh (patient and public involvement champion for the ‘fundamental care in hospital’ theme of the CLAHRC). Following their guidance we sought no further direct patient/public involvement in prioritising or shaping the questions as these arose from the brief and need for technical assessment of the tool; instead, we focused on considering how the public could be involved in the proposed research and its governance and dissemination.

Based on this advice, we sought a lay member of our steering group with specific interest and expertise. To this end, we worked with Stephen Habgood, a lay member of the NICE safe staffing advisory committee for the development of guidelines in mental health, who agreed to participate in the project steering group. Stephen also has experience of staffing methodologies used in other sectors from his past work as a prison governor and is currently chairperson of a mental health charity. In addition, and guided by the advice from our patient and public involvement experts, we considered ward-based staff nurses as the potential end-users of our research in a way that is analogous to a patient receiving a treatment that might be recommended by an expert. Based on this, we have used multiple channels to connect with ward-based staff, including extensive use of social media and discussions with individual staff nurses at consultation events. These conversations have sensitised us to a number of issues in relation to our study, including several factors that we tested in the simulation modelling, such as requirements for nurses to have specific skills.

Ethics issues

Although results of the SNCT staffing tool may indicate staffing shortages and staff reports may also indicate concerns, hospital trusts already have established mechanisms for dealing with these issues. The University of Southampton Ethics Committee approved our study (reference 18809). The Health Research Authority approval number for our study is 190548. This study did not require NHS Research Ethics Committee approval because no data were collected directly from patients, only routine data were used and all patient data were de-identified at source.

Chapter summary

• In our study, we used daily SNCT ratings of patients from acute general wards in four hospitals to investigate the tool’s reliability and validity and to model the costs and consequences of using the tool in different ways.

• We explored how the estimated nursing establishments varied with different samples, exploring issues such as the sample size used (number of days), and the day of the week and the time of day at which ratings were collected.

• We explored whether deviations from the staffing level indicated by the SNCT were associated with nurses’ reports of sufficient staff for quality, omissions of care or missed breaks.

• We used the data to model different approaches to setting the establishment using the SNCT, comparing the ‘standard’ (mean-based) approach with a ‘high’ establishment (designed to meet assessed need 90% of the time) and a ‘flexible (low)’ establishment (designed to provide a minimum of core staff relying on flexible deployment to meet variable need).

• We explored the consequences of these different approaches for the frequency of wards and patients experiencing substantial understaffing.

• We explored the costs incurred by the hospitals in providing staff under these different approaches.

• We modelled the cost-effectiveness (in terms of cost per life ‘saved’) of alternative approaches to setting establishments using estimates of the effects of low staffing and exposure to high levels of temporary staff from a recent study undertaken in the NHS.

Staffing levels

Nurse-reported staffing adequacy

Other factors potentially affecting workload

The proportion of single rooms per ward varied both between and within hospital trusts. Hospital trust D had relatively few single rooms, on average 12% per ward (see Appendix 2, Table 58). In trust C the proportion of single rooms was high, with a mean of 58%, reflecting a number of wards that were exclusively single rooms. Average daily turnover varied between trusts, from 0.04 admissions and discharges per care hour at trust C to 0.08 admissions and discharges per care hour at trust A, but it was much more variable between wards within trusts. Between the lowest turnover ward in trust B and the highest in trust A, there was a 29-fold difference (0.01–0.29 admissions and discharges per care hour).

Staffing costs

We estimated the costs of registered nurse and nursing support worker staffing levels observed on each ward in the study hospital trusts. To do this, we used the observed hours provided per day by substantive, bank and agency staff at each Agenda for Change band and we report these as staff costs per patient-day. Summary estimates for registered nurses and nursing support workers combined, and for each group, are reported in Table 15 (for more details, see Report Supplementary Material 3, Table 94).

The mean ward cost in three of the hospital trusts is between about £140 and £150 per patient-day, while the fourth has a substantially higher mean cost (approximately double, at about £275 per patient-day). The relatively high cost for hospital trust C reflects the higher registered nurse staffing levels and skill mix.

There is substantial variation in staff cost per patient-day within wards and between wards in each of the hospital trusts. The mean cost per patient-day varies from £116 to £238, £112 to £203, £199 to £363 and £114 to £183 in hospital trusts A to D, respectively. The variation in cost arises from a combination of the underlying variability in staffing levels, skill mixes and band mixes shown in Table 8 (see Report Supplementary Material 3, Tables 87 and 88) and additional staffing costs arising from unsociable hours payments as well as from employing bank and agency staff.

Chapter summary

• Across the four hospital trusts, 81 wards participated, with > 85% of potentially eligible general wards participating over 1 year.

• We received SNCT ratings for 96% of possible ward-days, and on over 85% of ward-days we received answers to questions about staffing adequacy.

• In all, we obtained data linking SNCT ratings and staffing levels for 26,362 ward-days.

• Staffing levels and skill mix varied considerably within wards, between wards and between hospital trusts.

• There was a large contrast between total (both registered nurse and support worker) staffing in the three general hospital trusts (a mean of 6.5–7 care HPPD) and that in the specialist trust (a mean of 10.5 HPPD).

• Within hospital trusts, the range between the highest and lowest staffed wards was 60% of the mean.

• Skill mix (the percentage of nursing staff who are registered nurses) also varied between and within trusts, with mean skill mix in the general hospital trusts (49–56%) lower than that in the specialist trust (75%).

• All hospital trusts made significant use of temporary staff, with > 20% of all shifts worked by bank and agency staff in all trusts (range 21–24%).

• The use of agency staff was more variable between trusts (4–9% of all shifts worked).

• Required staffing estimated by the SNCT across all wards was, on average, 7.2 care HPPD, with calculated staffing slightly higher than the staffing we observed to be deployed in three general hospital trusts and considerably lower than staffing deployed in the specialist trust.

• Using the criterion of 15% below the requirement for that day to define ‘understaffing’, we estimated that 26% of all ward-days were understaffed.

• Nurses reported that they had enough staff to deliver quality care on 78% of shifts (according to morning responses).

• When nurses reported that they did not have enough staff for quality, the additional staffing required varied by hospital trust, averaging between 0.7 and 1.3 registered nurses, or between 0.8 and 1.6 total additional staff required.

• Mean hospital trust nurse staffing costs varied from £274 per patient-day in the specialist trust to between £141 and £150 per patient-day in the general hospital trusts.

Variation between different samples for setting ward establishments

The SNCT guidance recommends calculating the average staffing required by collecting data on patients’ acuities and dependencies across at least 20 weekdays, at least twice per year (20 days in January and 20 days in June).

Our data set includes assessments of SNCT ratings for patients on each ward in scope of each hospital trust on different days of the year. After removing missing data and outliers, the sample size per ward averaged 325 days. From this, we calculated the estimated staffing requirements (in WTEs) for each day (see Report Supplementary Material 3, Table 89). We then compared the recommended establishments calculated by averaging different data samples. We used confidence intervals to assess how well these averages approximated the true average requirement. We calculated the precision of the estimate by dividing the half-width of the resulting confidence interval by the mean value. 130

Approach 1: bootstrapping

We use bootstrapping here so that we can derive confidence intervals for the average staffing requirement for different sample sizes. The bootstrap is a well-known resampling procedure that is used to generate the distribution of a statistic of interest. Rather than continually resampling from the system where the data were originally collected, the original sample – our best estimate of the population – is resampled, with replacement, to generate a large number (e.g. 1000) of new bootstrap data sets. For each data set, the statistic of interest (e.g. the mean) is recalculated. The resulting distribution is the sampling distribution of the statistic and can be used for inference. 114,131

Figure 2 illustrates the effect of sample size on the recommended staffing for ward 1 at hospital trust D (an arbitrary choice). The points show the bias-corrected and accelerated 95% confidence intervals for the mean for each sample size. The gridlines are spaced 0.25 WTEs apart. On this ward, for sample sizes below 100 data points, the widths of the 95% confidence intervals for the mean are more than 1 WTE, and for the largest possible sample size from our data, the width of the 95% confidence interval for the mean is about 0.5 WTEs.

FIGURE 2.

Bias-corrected and accelerated 95% confidence intervals for the mean calculated staffing requirement in ward 1 at hospital trust D for different sample sizes.

Across all wards, the average width of the 95% confidence intervals for the means using the recommended minimum of 20 data points is almost 3 WTEs, with an average precision (half the width of the 95% confidence interval expressed as a % of the mean) of 4.1%, and, even with 160 data points, the average width is more than 1 WTE (Table 16). In other words, the recommended establishment using 20 data points is highly variable.

TABLE 16 - Average widths of 95% confidence intervals for the mean using different sample sizes

Half the width of the 95% confidence interval expressed as a % of the mean.

Sample size	Average confidence interval width (WTE)	Average precision (%)a	Number of wards
20	2.9	4.1	86
40	2.1	3.0	86
60	1.7	2.5	86
80	1.5	2.1	86
100	1.3	1.9	82
120	1.2	1.8	81
140	1.1	1.6	81
160	1.1	1.5	80
180	1.0	1.4	77
200	0.9	1.3	77
220	0.9	1.3	77
240	0.8	1.2	73
260	0.8	1.2	72
280	0.7	1.1	69
300	0.7	1.1	63
320	0.7	1.2	43
340	0.7	1.1	32
360	0.6	0.8	3

Although 69 out of 86 wards (80%) achieved a confidence interval of width 1 WTE or less with a sufficiently large sample size, for 20% of wards the available data were insufficient to estimate the establishment with that degree of precision. However, in most cases the required sample size was much greater than the recommended minimum of 20, and only two wards provided a confidence interval width of 1 WTE with this sample size.

How does the Safer Nursing Care Tool compare with professional judgement of staffing adequacy?

We assessed the extent to which apparent staffing shortfalls, measured by the difference between the deployed staffing and the daily requirement estimated using the SNCT (including specialing), were associated with the shift leader’s perception that staffing was adequate, using a number of different questions (asked at each census point). All analyses included random factors for ward and hospital trust. Multivariable analyses included control for other factors that may influence workload, including patient turnover, single rooms, ward type and day of the week.

Non-linear effects and interactions

We tested for non-linear relationships with apparent registered nurse and nursing support worker shortfalls, as would be expected if the SNCT provides a threshold for adequate staffing. We estimated models using effects that were significant in the main effects models, adding a variable for staffing shortfall squared for each staff group. Overall, we found that only one non-linear term was statistically significant and all non-linear effects were too small to be of practical importance.

For nursing care left undone, the non-linear term for registered nurse staffing was statistically significant (p = 0.021; see Appendix 3, Table 64). Although there is some tendency for the odds of reporting care left undone to increase more as staffing shortfalls increase above 0 (the threshold for adequate staffing defined by our SNCT base estimates), when shortfall is negative (i.e. when staffing exceeds the estimated requirements), increases in staffing are still associated with substantial reductions in reports of care left undone (Figure 3). Including the non-linear term did not improve model fits over the models with significant variables only (compare Tables 63 and 64 in Appendix 3), apart from for ‘care left undone’, where the Akaike information criterion was reduced by a very small amount (from 8380 to 8378).

FIGURE 3.

Change in odds of reporting care left undone with change in staffing shortfalls.

Next, we estimated models that included all statistically significant variables (including the non-linear term for registered nurse staffing shortfall for the nursing care left undone model) and the interactions between these variables. There were no significant interaction effects between apparent registered nurse shortfall and support worker shortfall, and both Akaike information criterion and Bayesian information criterion increased, indicating overall worse model fits (see Appendix 3, Table 65).

Shift-level analysis

We also investigated whether or not there is a relationship between the apparent shortfall and the ward leader’s perception of staffing adequacy in the previous 6-hour shift. As the SNCT recommends using professional judgement to apportion daily requirements into shifts, we used the average observed ward distributions of staff over the day as a proxy. Both registered nurse shortfall and nursing support worker shortfall during a shift are significantly associated with enough staff for quality, nursing care left undone and missed breaks as reported in the following shift.

Nurses were most likely to report that there were enough staff for quality on night shifts, followed by morning and then afternoon shifts (we did not have enough data to investigate evening shifts). Compared with night shifts, the odds of reporting enough staff for quality were 10% lower on morning shifts and 13% lower on afternoon shifts (see Appendix 3, Table 66). Similarly, nurses were least likely to report leaving care undone and missing breaks on night shifts, followed by afternoon and then morning shifts.

Although our day-level analysis found no evidence of important non-linear effects of staffing shortfall and no interaction between staff groups, in analysis at this level, a more complex picture emerges.

Including non-linear terms improves the fit of models (over the models with significant main effects only) as measured by the Akaike information criterion in all cases, and it improves the Bayesian information criterion for models predicting enough staff for quality and staff breaks missed (see Appendix 3, Tables 67 and 68). Both the registered nurse and nursing support worker shortfall squared terms are significant in most models (2/3 and 3/3 respectively) but their effects are in opposite directions.

Including interactions between shortfall and other predictors improves the model fits further for all three outcomes according to the Akaike information criterion (see Appendix 3, Table 69). The Bayesian information criterion is also improved for the model predicting staff breaks missed. There are several significant interaction effects in the shift models. In the model predicting enough staff for quality, the interaction between registered nurse and nursing support worker shortfall is statistically significant. To explore this interaction and the non-linear effects, we calculated the combined effects of the linear, non-linear and interaction terms for a range of plausible values of staffing shortfalls when all other variables in the model are held constant at mean or reference categories (Figure 4 and Appendix 3, Table 70).

FIGURE 4.

Combined effects of registered nurse and nursing support worker shortfall (change in odds of reporting ‘enough staff for quality’): registered nurse shortfall effect conditional on nursing support worker shortfall (shift-level analysis).

As deficits in registered nurse staffing reduce, the odds of reporting enough staff for quality are increased. The non-linear effect of registered nurse shortfall is very small and so over the plotted range the registered nurse shortfall effect is very close to being linear. The significant interaction effect appears to have marginal impact, with the plotted lines more or less parallel. For missed care and missed breaks, the relationship with registered nurse staffing appears to be linear, with no significant interaction effect. However, there are significant (but very small) non-linear effects for support worker staffing, which are difficult to interpret, suggesting benefit from both high and low staffing relative to assessed need but, in particular, benefits from staffing above the levels assessed by the SNCT.

The effect of shortfall on perceptions of staffing adequacy appears to vary to some extent by time of day. The interaction between nursing support worker shortfall and morning shifts has a statistically significant effect on reporting enough staff for quality. The interactions between registered nurse shortfall/nursing support worker shortfall and afternoon shifts have significant effects on reporting staff breaks missed and care left undone. Both of these interactions may play a part in explaining the complex non-linear relationships with support worker staffing shortfalls and it is possible that higher-order interactions exist. Given the relatively small observed effect sizes and the absence of clear theoretical propositions, we elected not to pursue this through modelling the current data as it seemed unlikely to yield insights related to our main aims and objectives.

Both registered nurse shortfall and nursing support worker shortfall have a larger effect on reporting staff breaks missed on surgical wards than on other wards. Registered nurse shortfall and nursing support worker shortfall lessen the impact of the proportion of single rooms on care left undone, and, in the case of nursing support worker shortfall, also on staff breaks missed; that is, if the shortfall is higher, the proportion of single rooms has less effect on perceived staffing adequacy.

Chapter summary

• The SNCT guidelines recommend a minimum sample size of 20 days’ worth of observations to estimate a ward’s required staffing establishment. On average across wards, the bootstrapped 95% confidence interval for this sample is almost 3 WTE staff members wide, with a precision of 4%.

• On average across wards, a sample of 80 days gives a confidence interval that is 1.5 WTEs wide, with a precision of 2%.

• We estimated that the sample required for a confidence interval width of 0.5 WTEs is over 900 days on average, although the required sample size varied considerably between wards.

• On average, the time of day when SNCT assessments were made caused very little difference to the calculated establishment (< 1%), although, for some wards, admission/discharge patterns meant that there were large differences.

• In general hospital trusts, there was little variation in the average staffing requirements between days of the week, although weekends had slightly lower staffing requirements. However, the magnitudes of differences were small (≤ 3.7%).

• On days when nurse staffing fell below the daily staffing requirement estimated using the SNCT, the odds of reporting that there were enough staff for quality were reduced by 12% for each care hour per patient-day shortfall. Correspondingly, the odds of reporting that staff breaks and nursing care were missed were increased.

• We found no evidence to indicate that the calculated staffing requirement represented a threshold, as staffing above this level was associated with increased chances of reporting that staffing was adequate (according to all three measures).

• The effects of registered nurse and support worker shortfall appeared to be of a similar magnitude and largely independent of one another.

• There is some evidence that non-linear effects and statistically significant interaction effects appear to result in little or no substantive changes to the overall results or conclusions.

• After controlling for any shortfall between required and deployed staffing, nurses on wards with higher proportions of single rooms were less likely to respond positively to the staffing adequacy questions, although the relationships were not statistically significant.

Interpretation

In this chapter we assessed the appropriateness of the recommended sample size (20 days) and timing (3 p.m.) of collecting data for determining ward establishments. We found that, on most wards, larger samples than the currently recommended minimum of 20 days would be required to obtain accurate estimates of the required staff to employ within 1 WTE. This level of precision may be unachievable, but, on average, a precision of ±1 staff member can be achieved with a sample of 40 days. Wards with more variable staffing requirements need larger samples to estimate establishments accurately. In most wards, the time of day (morning or evening) made little difference to the calculated establishment required. This suggests that there is little basis for the current recommendation of using observations at 3 p.m.

We investigated whether or not ward leaders were more likely to report that they had enough staff when they had the number estimated by the SNCT. After taking into account other factors that could affect this relationship, we found that when there are fewer staff than the tool suggests are needed, the chance of ward leaders reporting ‘enough staff for quality’ is less, while the chance of them reporting ‘staff breaks missed’ and ‘nursing care left undone’ is more. However, other factors not directly measured in the tool (ward type, single rooms, day of the week, time of day) may also independently affect reported staffing adequacy. When there were more staff than recommended by the tool, there were further improvements in all three reported measures and the relationships we saw appeared to be effectively linear. So, although the tool does provide a measure that is associated with professional judgement of staffing adequacy, giving some indication of ‘validity’, it does not include all factors that seem to influence workload. Furthermore, it does not identify a staffing level that is always perceived as sufficient, nor does it identify a threshold staffing level above which staffing is much more likely to be seen as sufficient.

Ward establishments

Unsurprisingly, ward establishments are highest when set using the ‘high’ staffing approach, followed by the ‘standard’ and finally the ‘flexible (low)’ method (Table 21). Establishments varied both between and within hospitals largely because of the size of wards. Hospital trust B has the highest ward establishments on average, and trust C has the lowest. The minimum ward nursing support worker levels do not always follow the general pattern (‘high’ then ‘standard’ then ‘flexible’) because of rounding whole people/shifts and the requirement for at least one registered nurse to be present on each shift.

Compared with the ‘standard’ (mean average) approach to setting the establishment, the ‘high’ approach to setting staffing (matching 90th percentile demand) results in an increased establishment of 10% on average across the four hospital trusts, although there is a greater increase at trust C. The ‘flexible (low)’ staffing approach, by definition, results in establishments that are approximately 20% lower, with differences arising as a result of rounding whole people/shifts.

Consequences of different approaches to setting establishments

Costs

At all hospital trusts, total costs increase as the ward establishments increase (Table 23). This is because of the higher costs of employing permanent staff. Although the temporary staff costs change in the opposite direction (increased costs of hiring temporary staff with lower establishments), the changes in permanent staff costs have a greater effect, in part because of limited availability of temporary staff, an issue that we explore further in Short-notice availability of temporary staff. The cost per patient-day is variable between wards within trusts. Staffing costs associated with ‘high’ staffing establishments were, on average, £11 per patient-day (8%) higher than with ‘standard’ establishments, while staffing costs associated with ‘flexible (low)’ establishments were, on average, £15 per patient-day (11%) lower than with ‘standard’ establishments. The relationship between understaffed shifts, establishment and costs is similar at all trusts: the higher the establishment, the higher the cost, but the fewer shifts classified as understaffed (Figure 5).

FIGURE 5.

Costs vs. understaffing for alternative methods for setting ward establishments at different hospital trusts.

Scenario analyses

Vacancies

Given the problems with high vacancy rates for nursing staff, we also investigated a scenario in which the ward establishments are set at the mean requirement across a baseline period (as for ‘standard’ staffing), but there are vacancies, set at 10%. This scenario leads to achieved ward establishments that are between ‘standard’ staffing and ‘flexible (low)’ staffing (see Appendix 4, Table 79).

Correspondingly, the average percentage of understaffed patient shifts and cost per patient-day also lie between the results for these two scenarios (Figure 6). However, when assuming high availability of temporary staff (90% rather than the empirical availability), for two of the hospital trusts (B and D), the ‘vacancies’ scenario costs the same as the ‘flexible (low)’ staffing scenario but with almost 10 percentage points fewer understaffed patient shifts (Figure 7). See Appendix 4, Table 80 for results for different availabilities of temporary staff.

FIGURE 6.

Costs vs. understaffing for ‘vacancies’ scenario vs. alternative methods for setting ward establishments at different hospital trusts.

FIGURE 7.

Costs vs. understaffing for alternative methods for setting ward establishments at different hospital trusts assuming high temporary staff availability (empirical bank and 90% agency availability).

Chapter summary

• Our simulation model used data from our observational study to model the consequences and costs of setting the establishment using the SNCT – ‘high’ establishments (set to match the level of demand observed on 90% of occasions) and ‘flexible (low)’ establishments (set to meet 80% of the average observed need) compared with the ‘standard’ approach whereby staffing is set to meet mean demand.

• In all cases, we assumed that hospital trusts would attempt to make up staffing shortfalls using internal re-deployment of staff between wards, or hire from a bank or agency.

• Compared with the ‘standard’ (mean average) approach to setting the establishment, the ‘high’ establishments were 10% higher, on average, across the four trusts.

• The ‘flexible (low)’ staffing approach results in establishments that are approximately 20% lower than standard (mean) based establishments.

• For our core models we used empirical data obtained from one hospital trust to estimate the availability of temporary staff.

• Understaffing is common in all simulations.

• Despite using temporary staff, the ‘flexible (low)’ establishments resulted in 63% of patient shifts falling ≤ 15% below the required staffing level on average across hospital trusts.

• The ‘standard’ establishments resulted in 31% of patient shifts with understaffing.

• The ‘high’ staffing establishments resulted in 19% of patient shifts with understaffing.

• Staffing that was > 15% above the requirement was less common, occurring on 15% of patient shifts when using ‘high’ establishments, 7% of patient shifts when using ‘standard’ establishments and < 2% of patient shifts when using ‘flexible (low)’ establishments.

• Increased staffing costs associated with the ‘high’ staffing establishments varied by hospital trust, but were, on average, £11 per patient-day higher (8%).

• Low staffing establishments were associated with daily savings of £15 per patient-day (11%).

• The differences in both understaffed shifts and costs between establishment scenarios reduce if greater availability of temporary staff is assumed.

• If it is assumed that bank staff can fulfil 50% of late requests and there is unlimited availability of agency staff, ‘high’ staffing establishments are associated with understaffing on 5.1% of patient shifts and an increased staff cost of £8 per patient–day (5%).

• Even with high availability of temporary staffing, ‘flexible (low)’ establishments are associated with high rates of understaffing because a threshold based on ‘whole people’ has to be reached to trigger a request for additional staff, actual patient need varies and we assume temporary staffing to be less efficient.

• Assuming unlimited agency availability, ‘flexible (low)’ establishments are associated with understaffing on 21.4% of patient shifts and a cost saving of only £3 per patient-day (2%).

• The use of internal re-deployment had a small (but mainly beneficial) effect on rates of understaffed patient shifts (< 2%) and daily costs (< £2 per patient-day).

• We found that adding a requirement for a nurse with a specific skill (in this case training to administer IV drugs) meant that if 60% of nurses were trained at each hospital trust, > 90% of patient shifts were properly staffed.

• Multiway sensitivity analysis indicated that results were only marginally affected by most of the assumptions made in the model.

• Assumptions for short-notice absence rates, whole-hospital versus restricted re-deployment pool, relative efficiency of temporary staff, and variation in need around each acuity/dependency multiplier made small differences to the rates of understaffed patient shifts (all < 1.5%) and negligible differences to costs (at most £0.20 per patient-day) on average across the four hospital trusts.

• Assumptions about the number of staff to request made quite large differences to the rate of understaffed patient shifts; if all requests were rounded up rather than rounding up/down to the nearest whole number, the rates of understaffing dropped by 14% in the ‘standard’ staffing scenario and the daily costs increased by £3.53, on average, across the four hospital trusts.

Interpretation

The tool’s guidelines suggest employing enough staff to meet demand on an average day. As hospitals have the potential to meet demand for nursing care with temporary staff when demand is higher than the establishment provides, and can re-deploy staff from potentially overstaffed wards to understaffed wards, we set out to compare how often understaffing might occur with different approaches to using the tool to set the establishment. We compared the standard approaches with a higher establishment (enough to meet demand on 90% of days) and with a lower establishment (to meet 80% of mean demand). We developed a computer program that simulates the staff required, staff available and staff reassigned to other wards, in addition to the hiring of extra temporary staff.

Employing enough staff to meet demand on an average day (the standard approach) led to about one-third of patient shifts being substantially understaffed in the simulation because there is limited availability of re-deployable and temporary staff, and they are less efficient than permanent staff. Mismatches also arise when demand is not a round number, as staff work specific shift lengths. Furthermore, the true demand is unknown at the start of the shift owing to different patient needs even within the same acuity/dependency level.

Employing fewer nursing staff than in the standard approach is cheaper, but substantial understaffing occurs on two-thirds of patient shifts because reassigned and temporary staff cannot make up shortfalls. The rate of understaffing remains high even when we assume that temporary staff are nearly always available to make up shortfalls, and in this situation cost savings become minimal.

Employing more nursing staff than in the standard approach is more expensive, but it substantially reduced the rate of understaffed shifts in the simulation. We look at the trade-off between the impacts of understaffing and costs in the next chapter.

Effects of low staffing

Average results of the costs across the three hospital trusts for the consequences and cost-effectiveness of different approaches to using the SNCT to set establishments under varying assumptions about temporary staffing are summarised in Table 26. For more detailed reports with ranges between the three hospital trusts, see Report Supplementary Material 5, Table 101.

Using our baseline assumptions about temporary staffing availability – that is, based on the empirically observed availability – staff costs are increased by 5.5% when using ‘high’ staffing establishments and reduced by 10.8% when using the ‘flexible (low)’ establishments.

‘High’ establishments are associated with a 1.2% reduction in bed-days used, and a relative reduction of 4.5% in the risk of death, equating to one life ‘saved’ for every 665 patients admitted to a hospital (number needed to treat) using this establishment. The additional staff cost per life saved is £20,423, with a net cost (after taking account of the value of bed-days saved) of £9506 per life saved.

By contrast, ‘flexible (low)’ establishments are associated with an 8.3% increase in the risk of death, equating to one additional death for every 361 patients admitted (number needed to harm) with a negative incremental cost-effectiveness ratio. This negative ratio can be interpreted as the cost per life saved associated with the ‘standard’ (SNCT mean-based) establishment when compared with the ‘flexible (low)’ establishment. Staff cost per life saved is £22,910, with a net cost per life saved of £13,967. In very simple terms, the cost-effectiveness of ‘high’ establishments relative to ‘standard’ is similar, indeed marginally better, than that of the ‘standard’ compared with ‘flexible (low)’ establishments.

If no temporary staff were used and there were no re-deployments, the differences between the scenarios based on the three approaches to setting establishments are maximised because differences simply reflect the establishments themselves. Compared with the ‘standard’ establishment, staff costs would be increased by 7.8% for ‘high’ establishments and reduced by 19.9% for the ‘flexible (low)’ establishments.

Under these assumptions, ‘high’ establishments were associated with a 1.4% reduction in bed-days used and a 4.2% reduction in deaths, which equates to one life ‘saved’ for every 762 patients admitted (NNT 762). Relative to the ‘standard’ establishment scenario, the staff cost per life saved is £32,129 with a net cost per life saved of £16,323. By contrast, ‘flexible (low)’ staffing establishments were associated with a 19.9% reduction in staff costs, a 2.4% increase in bed-days used and a 13.5% increase in deaths – one additional death for every 219 admissions (NNH). The negative cost-effectiveness values reflecting the relative value of moving from the ‘flexible (low)’ establishment to ‘standard’ are £23,874 per life saved (staff costs only) or £15,981 net.

The assumption of no flexible staffing whatsoever is, of course, completely unrealistic. However, it does serve as a useful base against which to compare the effect of varying the assumptions about temporary staff availability and internal hospital re-deployments because, as we saw earlier, it is clear that conclusions may be highly sensitive to these assumptions, particularly in relation to the availability of temporary staff. As assumptions about the availability of temporary staff become more ‘generous’, there is a general trend for differences in costs and consequences between the different establishments to reduce.

Under the ‘flexible (low)’ establishment scenario, both associated harms and cost savings are substantially decreased for any assumptions about temporary staffing when compared with the zero availability assumptions. Whereas staff costs are reduced by 19.9% relative to the ‘standard’ establishment when there are no temporary staff – reflecting the simple reduction in the establishments – when we assume empirical availability of temporary staff, cost savings relative to the ‘standard’ establishment are decreased to 10.8% (allowing for internal re-deployment). There is also a lesser impact on the death rate (a rise of 8.3% relative to the ‘standard’ establishment, vs. 13.5%) with the number needed to harm increased to 361 versus 219.

The relative cost-effectiveness of the ‘standard’ establishment compared with the ‘flexible (low)’ establishment is largely unchanged in the face of limited availability of temporary staff (empirical or 50% availability for both bank and agency). However, with high availability of temporary staff, the cost per life saved associated with the ‘standard’ [vs. ‘flexible (low)’] establishment is considerably lower, reducing to £10,347 per life saved if 100% availability of agency staff (with internal re-deployment) is assumed (£946 net of length of stay savings).

For the ‘high’ establishments, there is a clear trend for both lower relative costs and lower relative benefits as higher availability of temporary staff is assumed, mirroring changes for the ‘flexible (low)’ establishment scenario. However, the effect of increasing availability of temporary staffing is less marked because the original high establishment meets needs more often. Under the scenario with the most opportunity for hiring additional staff above the employed establishment (50% availability of bank, 100% availability of agency with internal re-deployment), staff costs of the ‘high’ staffing establishment relative to the ‘standard’ establishment are increased by 2.9% and the staff cost per life saved is estimated to be £16,464 (£5524 net of length of stay savings).

Allowing re-deployment of permanent staff between wards (before requesting temporary staff) consistently leads to lower estimates of cost per life saved associated with ‘high’ establishments. However, the differences are relatively small (< £1000) for all assumptions except no availability of temporary staff.

Temporary staffing effects

In our core models reported above, we assumed that deployed temporary staff were as effective as deployed permanent staff. However, there is some evidence that such staff may be less effective, and some studies have reported that higher levels of temporary staff are associated with increased mortality risk. In our recent study from which the coefficients for the effect of low staffing in the core economic models were derived, we observed that there was evidence of increased mortality when patients experienced very high levels of temporary staffing from either registered nurses or health-care support workers (1.5 HPPD or more). In determining the likely effect of different staffing policies, especially the ‘flexible (low)’ staffing policy, this is important to consider, as temporary staff will be deployed more often and in greater numbers. Therefore, we added the effect of high levels of temporary staffing to revise the mortality estimate used in the models reported above.

The pattern of results is largely unchanged but the relative advantages of higher establishments is much greater when greater availability of temporary staff is assumed (Table 27). For example, with 50% availability of temporary staff the ‘high’ staffing establishment is associated with a 4.4% reduction in deaths, compared with 4.0% when modelling low staffing effects only. With higher availability of temporary staff (50% bank and 90% agency), the ‘high’ staffing establishment is associated with a 4.1% reduction in deaths, compared with 3.2% when modelling low staffing effects only.

The relative harm of ‘flexible (low)’ establishment scenarios is increased to a much greater extent when considering adverse effects associated with high use of temporary staff. For 50% availability of temporary staff the increase in mortality associated with the ‘flexible (low)’ establishment is estimated as 8.1%, compared with 4.4% when modelling low staffing effects only.

Consequently, the estimates of cost-effectiveness become more favourable for higher establishments, especially when availability of temporary staff is high. For example, if 50% bank, 90% agency availability (with internal re-deployment) is assumed, the staff cost per life saved is £12,572 (net cost £3978) for the ‘high’ establishment scenario, compared with £17,454 (net cost £6146) when considering low staffing effects alone.

Costs and consequences of changing establishments at hospital trust level

To help understand what these results mean at a hospital trust level, we explored them in the context of each trust’s annual turnover. Trust A, which has an annual turnover of about £250M, would spend £1.7M (< 1% of turnover) more on nursing with ‘high’ establishments than with ‘standard’ establishments. We also predict that ‘high’ establishments would be associated with 67 fewer deaths per year than ‘standard’ establishments.

Hospital trust B, which has an annual turnover of > £500M, would spend £1.9M (< 0.5% of turnover) more on nursing with ‘high’ establishments than with ‘standard’ establishments. Here we predict that ‘high’ establishments would be associated with 126 fewer deaths per year than ‘standard’ establishments. In trust B, we found that in the 50% bank and 100% agency availability scenario, ‘flexible (low)’ establishments cost more (+£73,000) than ‘standard’ establishments, with 261 extra deaths predicted.

Finally, hospital trust D has an annual turnover of > £800M, and would spend < £1M (0.1% of turnover) more on ‘high’ than ‘standard’ establishments, with a corresponding 54 fewer deaths.

Sensitivity analyses

We focus reporting sensitivity analysis on the cost-effectiveness estimates derived from the low staffing (only) models, although estimates based on low staffing and high temporary staffing effects would in all cases show similar sensitivities.

Chapter summary

• We modelled the costs, effects and cost-effectiveness (in terms of cost per life saved) for the three general hospital trusts, considering both gross (staff only) and net (including any savings associated with reduced length of stay) costs.

• We used coefficients from a recent longitudinal observational study and considered the effects of low (below the mean) staffing, but also, as a secondary analysis, the potential adverse effects associated with high temporary staffing.

• Our core models assumed a limited availability of temporary staff (based on empirical data from one hospital trust), and internal re-deployment of staff within directorates.

• ‘High’ establishments are associated with a reduction of 1.2% in bed-days used, and a relative reduction of 4.5% in the risk of death compared with ‘standard’ establishments, which equates to one life ‘saved’ for every 665 patients admitted to a hospital (number needed to treat).

• The additional staff cost per life saved is £20,423, with a net cost (after taking account of the value of bed-days saved) of £9506 per life saved.

• If the potential adverse effects of temporary staffing are included, the number needed to treat is lower (n = 627), as is the cost per life saved (£19,390 staff costs and £8876 net costs).

• ‘Flexible (low)’ establishments are associated with an 8.3% increase in the risk of death, equating to one additional death for every 361 patients admitted (number needed to harm).

• The ‘standard’ (SNCT mean-based) establishment when compared with the ‘flexible (low)’ establishment gives a cost per life saved of £22,910 (net cost £13,967).

• As availability of temporary staffing increases, the cost per life saved of higher relative to lower staffing establishments reduces.

• As availability of temporary staffing increases, the impact of including an estimate of negative effects associated with temporary staffing also increases.

• With highest availability of temporary staff and including negative effects from temporary staffing, the staff cost per life saved for ‘high’ establishments compared with ‘standard’ establishments is low, at £11,305 (£3292 net).

• Sensitivity analysis shows that substantive conclusions are most sensitive to the estimates of the effect of low staffing on mortality.

• If higher effects on mortality are assumed, the cost-effectiveness estimates for higher staff establishments become much more favourable, especially when there is low availability of temporary staff.

• If lower effects of mortality are assumed, the cost-effectiveness for higher establishments becomes less favourable, although under all scenarios where temporary staff are available, net cost-effectiveness of ‘high’ establishments compared with ‘standard’ establishments is < £30,000 per life saved.

Interpretation

In this chapter, we assessed the cost-effectiveness of different approaches to setting ward nurse staffing establishments. We compared the standard approach (employing enough staff to meet demand on an average day) with lower and higher staffing levels. Previous studies have shown that understaffing and greater use of temporary staff increase the risk of in-hospital death and that lower staffing is associated with longer hospital stays. We used estimates for the sizes of these effects from a recent study undertaken in one of the participating hospital trusts. We considered both staffing costs and the costs of lengths of stay associated with changes in staffing levels. We calculated the trade-off between these costs and the lives saved.

Although more expensive, we found that employing more staff at the level needed to meet the demand observed on 90% of days could provide value for money compared with the standard approach. Under almost all assumptions, the additional cost of this higher establishment was < £30,000 per life saved, and under most assumptions the cost was much lower than this. Including the value of reductions in length of stay (bed-days saved) further improved the estimated cost-effectiveness, as did including the potential adverse effects of temporary staff on mortality rates and, importantly, increasing the availability of temporary staff. Under the circumstances where flexible staffing policies might seem most viable – where there is unlimited availability of temporary staff to meet any shortfall – we found that the higher establishment was still associated with fewer deaths, at an estimated cost per life saved (after taking into account savings from reduced length of stay) of £3292, than the standard establishment. At all hospital trusts, the investment in employing more staff would cost < 1% of their annual turnover.

On the other hand, using a smaller establishment and using more temporary staff to meet shortfalls would be associated with a substantial increase in patient deaths. Even assuming unlimited availability of temporary staff, and thus minimising staffing shortfalls under this scenario, we estimated that there would be up to one additional patient death for every 405 admissions, after considering the adverse effect of temporary staffing on patient outcomes. Where more realistic assumptions about temporary staffing were made, the increase in death rate was much higher. In all cases, the standard establishment appeared to represent a preferable, cost-effective alternative to this low ‘flexible’ establishment.

Summary of main results

In this observational study, we successfully gathered data about daily nurse staffing requirements using the SNCT and the staffing levels deployed from 81 wards across four NHS hospital trusts on a total of 25,362 ward-days. We collected data on staff perceptions of staffing adequacy on more than 23,000 ward-days.

Actual and required staffing levels varied considerably between hospital trusts, between wards within trusts and within wards. We found that on many wards the variation in required staffing level was skewed, such that mean demand, used as the basis for setting establishments using the SNCT, was below the median. This means that when a ward is staffed to meet mean demand, staffing is below the measured requirement more than 50% of the time. We estimated that, on average, 26% of ward-days were substantially understaffed (more than 15% under the requirement). In morning reports, shift leaders reported that they had enough staff to deliver quality care 78% of the time. In all of the hospital trusts we studied, there was a high level of temporary staffing (over 20%), although the majority of temporary staffing was by trust-employed staff through the local bank.

When the SNCT is used to estimate the required ward establishment, the currently recommended minimum sample of 20 weekdays gives a reasonable precision in percentage terms overall but, on average, the 95% confidence interval of the estimate is more than one WTE staff member either side of the mean. Only two wards achieved precision of less than a whole staff member (±0.5 staff member) with the recommended minimum sample, and for many this could not be achieved with the available sample (up to 1 year of data). On average, a precision of ±1 staff member was achieved with a sample of 40 days. Overall, day of week and time of day used to sample staffing requirements made little difference to the estimated establishments, although on some wards the effect was substantial.

Using daily measurement of the SNCT allowed us to estimate the staffing required for each day on each ward and to assess whether or not staffing shortfalls were associated with professional judgements of staffing adequacy. On days when staffing fell below the estimated requirement, odds of staff reporting that there were not enough staff to deliver quality care were reduced and there were increased reports of missed staff breaks and care. The relationships we observed were, essentially, linear, with no obvious threshold as staffing rose above the estimated requirement and no important interaction between registered nurse and support worker shortfall. There was evidence that having a high proportion of single rooms and being a surgical ward were associated with decreased perceptions of staffing adequacy. High patient turnover was also associated with decreased perceptions of staffing adequacy, although the relationships were not statistically significant.

When we simulated different approaches to using the SNCT to set establishments, we found that ‘high’ establishments – a level set to meet estimated demand on 90% of occasions – led to a much lower rate of shifts on which patients would be exposed to substantial understaffing. Conversely, a lower ‘flexible’ establishment – whereby temporary staff are relied on to respond to most variation in demand – results in a very high rate of understaffing. In large part this is due to the limited availability of temporary staff but it is due also to the unmeasured variation in patient need, the lower efficiency of temporary staff and the need to request additional staff in ‘quanta’ representing a reasonable working shift. Although the number of understaffed shifts associated with a ‘flexible (low)’ staffing establishment is greatly reduced if higher availability of temporary staff is assumed, it is always at a level substantially higher than that achieved using the standard approach to setting the establishment, which is, in turn, associated with more understaffing than the ‘high’ establishments, irrespective of the availability of temporary staff.

We modelled cost-effectiveness under a range of different assumptions about temporary staff availability and considering a number of different factors. ‘Flexible (low)’ establishments are associated with lower costs but a substantial increase in the risk of death. As greater availability of temporary staff is assumed, the increased risks are reduced but the cost savings are greatly diminished and the ‘standard’ establishment delivers improved outcomes at low cost per life saved. ‘High’ establishments, set to meet need most of the time, are associated with substantial decreases in bed utilisation (1.2%) and mortality risk (4.5%; number needed to treat, n = 665), with a staff cost per life saved of £20,423 and a net cost of £9506 (accounting for the cost of bed-days saved). Higher temporary staff availability and adding estimates for the effect of temporary staff on mortality make the cost-effectiveness estimates for ‘high’ establishments considerably more favourable.

Staffing requirements estimated by the Safer Nursing Care Tool

In common with other studies, we observed that there was greater variation between wards within hospital trusts than between hospital trusts. 133 Much of the variation between hospital trusts arose from the difference between the general hospital trusts and the specialist hospital trust. When including this specialist trust, the magnitude of variation within trusts was even greater. Most studies on nurse staffing have focused on hospital-level averages, which may not reflect important sources of variation. This has been observed previously in Belgian hospitals133 and is likely to be the case elsewhere, and it may explain why some cross-sectional studies of nurse staffing at the hospital level have not found the expected associations with patient outcomes. Much of the difference between wards is, however, entirely warranted by variation in patient need. Across all wards in the study, the estimated nursing establishment required ranged from 5.9 to 10.2 (in HPPD equivalents).

Both deployed staffing and estimated staffing requirements varied considerably between days on the same ward. Even in the hospital trust with the lowest variability, the daily staffing requirement had a mean within-ward standard deviation that was 6% of the trust mean.

In part because of this variability, when using the SNCT to estimate mean demand, larger samples than currently recommended would be required to obtain estimates of the required staff to employ within 1 WTE. This level of precision may be unachievable, but, on average, a precision of ±1 staff member can be achieved with a sample of 40 days.

• There is a limit to the degree of precision that can be achieved in practice.

• Wards with more variable staffing requirements need larger samples to estimate establishments.

• Although decisions still need to be made about base staffing levels, and the SNCT and NICE guidance recommend regular re-reassessment of the base establishment,11,19 this variability highlights the potential advantage of daily/shift-level measurement of nurse staffing requirements to ensure that deployed staffing matches variable need. For most wards, the time of day selected for the ratings used to estimate the establishment made little difference.

• For establishment-setting, the time of day selected to determine staffing requirements may make little difference on most wards.

• The specific current recommendation to undertake SNCT assessments at 3 p.m. is unwarranted.

• However, on some wards the timing of the assessment makes a large difference, and so the time chosen needs to be considered carefully.

• Similarly, variation in demand by day of the week makes little overall difference to the estimated establishments, but the samples chosen need to be mindful of patterns that might occur on particular wards.

However, intracluster correlation coefficients showed only moderate/good agreement between morning and evening measurements, and high variation in differences was observed on different wards. There were sometimes considerable differences in staffing required by time of day, irrespective of professionally determined within-day variation in staffing patterns. Events that cause a ward’s estimated staffing requirement to change are transfers, admissions, discharges and the worsening or improvement of a patient’s acuity/dependency rating. Near real-time data about the acuity/dependency of current patients could be beneficial, although the resources involved in additional data collection and input need to be measured against the potential benefits. Another finding of this study is that increased establishments to meet anticipated fluctuation in patient need may be cost-effective. Such strategies require fewer data than flexible strategies, which we found to be associated with increased harm and, often, little cost saving.

The standard SNCT takes no direct account of ‘specialing’, except in so far as this requirement is reflected in the staffing levels on the wards used to derive the multipliers. The inclusion of ‘specialing’ substantially increased the variability of requirements between days. The SNCT multipliers were developed from observations that may have included some specialing, although we have no way of knowing what proportion of observed patients were specials. Thus, the multipliers plan for the average level of observed specials at the time of the tool’s development, which is likely to be lower than it is today, as some patient groups requiring specialing, such as patients at risk of falls or those with severe mental health difficulties, are growing. 104

Understanding specialing requirements on wards is critical when planning nurse staffing levels

An approach to setting the establishment that assumed a symmetrical distribution of demand could be said to balance the risks of overstaffing with those of understaffing. Others have noted that variation in demand for nurse staffing on wards is not symmetrical around the mean. 72 We observed that the staffing requirement on many wards had a left-skewed distribution, meaning that mean staffing demand is likely to be exceeded more than 50% of the time. Thus, an establishment based on mean demand risks understaffing even before real-world challenges of matching staffing supply to demand, such as short-notice sickness absence and limited availability of temporary staff, are encountered. Furthermore, the risks associated with understaffing are potentially considerable, including the known risks of increased mortality2 and the increases in costs associated with adverse events and extended hospital stay. 134 On the other hand, ‘overstaffing’ may be associated with considerable ‘salvage value’ (to borrow a term associated with some operational research models). 72 Salvage value reflects the fact that staff who might appear to be excess to requirements still have value. This is in part because they could be deployed elsewhere, but, as we discuss below, ‘excess’ staff may also directly add value on their home ward, even when they are apparently surplus to requirements. This is because the measure used to determine the requirement does not identify an ‘optimum’ staffing level at which adding more staff does not improve care outcomes or there are substantial diminishing returns from adding more staff.

• Planning establishments to meet average demand makes understaffing more likely than overstaffing.

• The risks associated with understaffing are higher than those associated with overstaffing.

• The ‘salvage value’ of apparently excess staff is considerable.

How does actual staffing compare with the estimated requirement?

We estimated that, on average, 26% of ward-days were substantially understaffed, that is, more than 15% below the calculated requirement. We found that, on average, the three general hospital trusts had relatively small apparent shortages of nursing staff (≤ 0.5 HPPD). It is possible that we have not reliably identified all non-clinical shifts (e.g. management or education shifts) to remove them from the deployed staffing, meaning that the staff shortages may be larger than they appear. Furthermore, including specialing requirements more than doubles the percentage difference between the estimated requirement and the actual staffing at hospital trusts A and D. This again highlights that specialing is a large extra demand that hospital trusts appear to be struggling to resource, although some trusts rely on the support of volunteers and family members to help with this, which is not shown in our data. 104 Overall, this picture is consistent with trusts attempting to staff to the SNCT requirements and experiencing staffing shortfalls due to sickness, vacancies and limited availability of temporary staff.

By contrast, hospital trust C had a large apparent surplus of staff, which may result from a perception that SNCT staffing levels were not suitable in the specialist wards, the large proportion of single rooms and the problem of staffing smaller wards where the required registered nurse/support worker staffing may be less than a whole person. Modelling studies have suggested that smaller units require larger (relative) workforces. 135 We found that wards with a higher proportion of single rooms were more likely to report missed care, consistent with these wards having a higher staffing requirement that is not reflected in patient prototypes, for example because of increased walking distances and the challenges of undertaking routine visual checks on patients’ conditions. 84,85 Certain ward types, including those with more single rooms, have also been shown to have higher levels of indirect care, including handover times, which is consistent with single rooms reducing the opportunity for staff-to-staff communication. 85

• The SNCT multipliers may not properly reflect the staffing requirements of some units that are influenced by factors other than patient acuity/dependency.

• Smaller units and those with more single rooms may require more care HPPD and higher ‘multipliers’ to provide adequate staffing to reliably meet patient need.

Staffing ‘shortfall’ and nurses’ perceptions of staffing adequacy

The differences between actual staffing and estimated requirements, the ‘shortfalls’, are significantly associated with nurses’ perceptions of staffing adequacy. For example, according to our multivariable models, if the registered nurse staffing level was 1 hour per patient-day below the required level estimated using the SNCT, the odds of the shift leader reporting that there were enough staff for quality were reduced by 11%, the odds of reporting nursing care left undone were increased by 14% and the odds of reporting staff missing breaks were increased by 12%. The results were similar for nursing support workers.

• These results provide the first evidence, to our knowledge, that the SNCT assessments are related to nurses’ professional judgements of staffing requirements.

However, our models suggested that there are additional workloads not included in the SNCT multipliers, although they are, to some extent, recognised by the tool documentation as matters requiring professional judgement. 19 The effect of single rooms and, potentially, of other aspects of ward layout was noted above. In addition, nurses in charge of surgical wards were significantly more likely than those on medical/mixed wards to report nursing care left undone and missed breaks, and less likely to report enough staff for quality. This may reflect that nursing staff on surgical wards carry out different tasks from those on medical wards, such as changing dressings and taking out sutures from healed wounds, but it could also reflect different assumptions about what constitutes quality. Turnover per day was not significant in the models, although it is thought to have an effect on workload,6,8,9 and the non-significant effects we observed were consistent with higher turnover leading to increased workload.

After controlling for other variables, there was evidence that the day of the week affected perceptions of staffing adequacy. After controlling for the staffing shortfall, nurses were more likely to report that there were enough staff for quality and less likely to report missed care and missed breaks on Saturday than on Monday. Our determination of the required staffing assumed that a patient in any given acuity/dependency category required the same amount of care irrespective of the day of the week. However, rosters are often planned on the assumption that demand is lower at weekends, in part because some work that is required on weekdays is not required at weekends,6 including interactions with other departments that mainly function during the week, such as surgery and radiology. Many wards have lower staffing at weekends because of this. 6 However, although this evidence seems to lend support to the existing practice of tailoring staffing to such presumed variations in demand, we did not find a consistent pattern across days of the week, and Sundays were no more likely than Mondays to be seen as adequately staffed at any given level.

• Simple assumptions that wards require less staffing at weekends may be increasingly outdated in the face of shorter hospital stays, increased acuity and an increasing push toward 7-day working in the NHS.

We also modelled associations between deviations of actual staffing from estimated requirements and reported staffing adequacy at the shift level. Nurses were less likely to report that there were enough staff for quality on morning and afternoon shifts than at night. Similarly, they were more likely to report leaving care undone and missing breaks, especially on the morning shift. When modelling shifts, we did adjust the required staffing to reflect each ward’s typical (mean average) distribution of staff over the day. This finding is consistent with differences in workload between morning, afternoon and night shifts that are greater than planned for or, alternatively, with less variable demand at night. Overall lower staffing levels at night may also necessitate relative overstaffing as the ability to match demand exactly is limited by shift lengths and the quantum of staff that can be requested.

• The allocation of sufficient staff hours to night shifts may result in a relative shortage of staff during the day.

We explored these data for non-linear effects and interactions between the staff groups. If, for example, greater numbers of support workers enable registered nurses to work more effectively, we might see that the effect of a registered nursing shortfall was reduced when there were greater numbers of support workers. This complementarity is a key part of the rationale for adding support workers to the workforce, but, in common with a number of other studies,6,108,136,137 we see no substantial evidence of it occurring. In our models focusing on shift-level patterns, there was a small significant interaction between support worker shortfall and registered nurse shortfall, but when we plotted this it did not appear to have any substantial impact on the relationship with ‘enough staff for quality’. Similarly, although we saw some evidence of non-linear effects, these appeared to make little material difference. In particular, we did not observe that the chances of reporting that staffing was sufficient reduced only as shortfalls increased. Indeed, if anything, adding more staff above the required level estimated by the SNCT appeared to increase the chances of reporting enough staff for quality further. Results were similar for other measures of staffing adequacy and we saw no evidence of a threshold effect indicating reduced benefits from further staff increases.

For registered nurse staffing levels, this absence of a threshold has been observed previously. 9 The finding is consistent with a study showing that increases in staffing above the ‘optimal’ level using the RAFAELA tool were associated with further reductions in mortality. 80,81 Whereas SNCT multipliers are derived from wards that are assessed as meeting a quality threshold, many judgements of ‘quality’ are subjective and may be anchored to current expectations. 138 Equally, perceptions of ‘enough staff’ and ‘quality’ may be influenced by expectations, including historical staffing patterns, and so may not represent absolute judgements. We found that nurses reported enough staff for quality less often than they reported no missed care, which is consistent with a high quality threshold operating. However, the rates of missed care reported using our single-item measure were much lower than typically reported in studies that base overall missed care rates on responses across multiple items. 5 Therefore, this low rate of reporting may be an artefact based on the lesser ‘prompt’ for items to consider or it could arise from an expectation that certain items of care are no longer possible.

Although professional judgement may, in some respects, represent the ‘gold standard’ in judging staffing requirements, with no evidence that any tool provides a more accurate measure, both judgement and the tool are potentially subject to bias. 17 The SNCT acuity and dependency categories may provide a valid measure of varying demand without necessarily indicating the optimal level of staffing. The documentation of the tool as a whole recommends a ‘triangulated’ approach including professional judgements and outcomes. All are potentially subject to measurement error and subjectivity, but these findings do serve to emphasise the importance of not relying simply on a single measure.

• Although staffing requirements measured by the SNCT may correspond with professional judgement, there is no basis on which to judge that they are ‘optimal’.

• Benefits may continue to accrue at staffing levels above those recommended.

• Although professional judgement may be subject to anchoring bias, it remains an important part of any system for judging required staffing.

Setting establishments using the Safer Nursing Care Tool

We simulated three alternative staffing strategies for planning establishments using the SNCT: ‘high’ staffing corresponding to the 90th percentile of demand, ‘standard’ staffing corresponding to average demand and ‘flexible (low)’ staffing corresponding to 80% of average demand. Under our baseline assumption, with limited availability of temporary staff, understaffing (more than 15% below the measured requirement) was relatively common regardless of the approach. Understaffing occurred much less often with ‘high’ establishments (19%) than with ‘standard’ establishments (31%) and ‘flexible (low)’ establishments (63%).

Given the variable nature of demand within wards, there are times when wards are overstaffed. This happens much less than understaffing. Overstaffing by more than 15% was relatively rare in most scenarios. Even with ‘high’ establishments, it occurred less often than understaffing in half of the hospital trusts, with 15% overstaffing compared with 19% understaffing on average across the four trusts. With ‘flexible (low)’ establishments, overstaffing was virtually eliminated.

As we noted above, the potential ‘salvage’ value associated with excess staff is potentially high, so any apparent ‘overstaffing’ should not be assumed to be inefficient and, consequently, the elimination of ‘overstaffing’ should not be assumed to be beneficial. Given the evidence suggesting that benefits may accrue when staffing is above the measured requirements, the presence of these extra staff on the ward may add direct value to patient care. Another potential benefit from ‘overstaffing’ is the opportunity to cover staffing shortfalls on other wards without recourse to bank or agency staff, often referred to as ‘floating’ staff between wards. 139

Internal re-deployment of staff (‘floating’)

Staff ‘floating’ between wards has been shown to be unpopular and associated with staff anxiety, especially when staff move outside their specialty area. 139–141 Despite the potential contribution, overall in our models we found that, across all scenarios and hospital trusts, fewer than 1.5% of the total hours were worked by staff re-deployed from one ward to another. Opportunities to float staff between wards are constrained by staff quanta (because staff cannot effectively be moved partially or for very short periods of time) and are less frequent with lower staffing establishments when the benefits might be greater, because understaffing is common whereas overstaffing is rare. In our modelling we assumed that re-deployments were restricted to within directorates, although wider sharing may be possible. Extending the potential re-deployments to the whole hospital made very little difference (0.3%) to the rate of understaffed shifts and, in our economic models for the general hospitals, the inclusion of any internal re-deployment made only a modest contribution to reducing understaffing and costs. Re-deployments outside staff members’ specialties are also more likely to be associated with reduced efficiency and effectiveness of staff. 141

• Internal re-deployment (floating) of staff within directorates makes a modest contribution to reducing understaffing.

• Wider re-deployments across the hospital have very little impact on understaffing, and such assignments are known to be unpopular with staff.

Availability of temporary staff and costs

The use of temporary staff varied between scenarios, with lower establishments associated with higher use of temporary staff, as one would expect: ‘high’, ‘standard’ and ‘flexible (low)’ staffing scenarios had, on average, < 5%, 6% and 14% temporary staff, respectively.

In our core model, at all hospital trusts, the higher the establishment the higher the cost, but the fewer shifts are understaffed. This is because of the lower efficiencies of re-deployed/temporary staff and the limited availability of temporary (bank and agency) staff who are able to cover shortfalls according to our empirical data. If we model high availability of temporary staff, the costs of ‘flexible (low)’ staffing establishments become close to those of the ‘standard’ establishments while the risk of leaving shifts understaffed remains higher. In some trusts, when assuming largely unlimited availability of temporary staff, the costs of the ‘low’ establishments were higher than those of the standard establishments owing to the higher cost of temporary staff.

• The apparent cost savings of employing fewer staff are largely achieved by below-adequate staffing.

• When temporary staff availability is higher, making ‘flexible (low)’ staffing more feasible, cost savings are eroded while understaffing remains relatively high.

Beyond care hours and registered nurse skill mix

The SNCT provides a measure of overall demand for care but provides staffing estimates that are (in essence) equivalent to an overall measure of care HPPD. While this overall measure is now reported as a benchmarking measure in the NHS,142 there is little research that supports the use of a combined measure of registered nurse and support worker hours. This is because the contributions to patient safety and quality appear to be different, to be independent of one another9 or to have a complex interaction, suggesting that benefits from adding support workers arise only if there are sufficient registered nurses to supervise them. 6,143

Using the SNCT, decisions about skill mix are deferred to ‘professional judgement’ and are not subject to measurement, which might be seen as a significant limitation, although the evidence we reviewed suggests that this professional judgement might still be regarded as the ‘gold standard’ for judging staffing requirements. We used the empirically observed skill mix for each ward as the basis of our models, assuming that this reflected professional judgement of the relative skill mix required. In our previous study,6 we determined that this assumption is broadly correct, although difficulty recruiting registered nurses may lead to a slightly lower skill mix than planned. The associations between perceptions of staffing adequacy and registered nurse/support worker shortfalls were similar, which lends some support to the assumption that observed skill mix serves as a useful proxy for the ‘correct’ skill mix.

However, skill mix is, in reality, more complex than a simple issue of registered versus unregistered staff. Existing literature has rarely addressed this issue at all, although some literature from the USA has considered three broad groups (aides, licensed practical nurses and registered nurses) rather than two. 137 To begin an exploration of this complexity, in addition to modelling registered nurse skill mix in our core simulations, we performed further experiments in which we investigated the impact of wards requiring staff with particular skills in more detail. Based on discussion with our advisory group, we used the example of registered nurses who are trained in administering IV drugs.

We considered the presence of at least one IV-trained nurse as a requirement on all wards, as our advisory group noted the disruption involved when staff had to be obtained from other wards simply to administer IV drugs. We found that as the percentage of registered nurses with IV training increases, the number of shifts with no IV-trained nurse decreases non-linearly. At a hospital level, we determined that once 60% of registered nurses were trained, the rate of shifts that were understaffed according to this criterion was low (< 10% in all hospital trusts and < 2% in the general hospital trusts). Although IV training may rapidly become a ubiquitous skill for registered nurses, currently in the UK, newly qualified nurses will not have had opportunity to demonstrate competence in administering IV medication. This approach to understanding the likely demand for staff training to ensure that staff are available to meet patient needs when additional or specific training is required illustrates that simple approaches based on mean demand are unlikely to succeed.

• When taking a more ‘refined’ approach to skill mix, it is challenging to deliver specific skills to wards reliably unless a high proportion of staff are trained, even when the requirement is for a single member of staff with that skill.

• Increasing the complexity of ‘skill mix’ may prove difficult to manage efficiently.

Differences between hospital trusts and assumptions made

In general, understaffing in hospital trust A was the most sensitive to staffing policy (i.e. it had the greatest relative risk increase/decrease associated with alternative establishments) and this trust had lower levels of understaffing than the other hospital trusts when there was limited availability of temporary staff. This may be explained partly by differences in specialing requirements, which varied between trusts. Another reason for trust differences is the size of wards: wards at hospital trusts B and D tended to be larger than at those at trust A, which are larger than those at hospital trust C. Hospital trusts B and D had similar understaffing results across staffing scenarios. In general, across most scenarios, the results from hospital trust C, a specialist hospital with small wards and a high proportion of single rooms, differed substantially from those from the other three. The differences between the deployed staff and our estimated staffing requirements are also much greater in this trust.

Although the absolute and relative differences varied by hospital trust, the pattern of results was common across all and so the qualitative conclusions apply to all. In general, we report our results at the hospital level. However, managers may be more interested in staffing adequacy and costs evaluated at the site or directorate level. Individual wards can be affected more dramatically than others by policy changes, although, again, the patterns observed generally apply at the ward level.

• These results give general patterns that are likely to apply in other hospitals, but the extent to which any establishment delivers a high level of properly staffed shifts is variable between hospitals and between wards within hospitals.

• Several strands of evidence from our study suggest that the current multipliers, however used, may often fail to deliver enough staff in a specialist hospital trust with many small wards and single rooms.

Given the limited or lack of data available for some parameters, and the need to make some simplifying assumptions, we investigated the effects of parameter uncertainty on the results. We found that changing the values of most uncertain parameters did not have a practically significant effect on the hospital costs or understaffing results. The exception to this was the ‘rounding’ parameter. We assumed that requests for staff were rounded up or down to the nearest whole person, a practice endorsed by our advisory group and recommended by guidance in Australia (although this guidance has recently been amended in the state of Victoria). 144 If instead we had assumed that requests were always rounded up, there would have been an absolute reduction in understaffing of 14.3% patient shifts and an increase in costs of £3.53 per patient-day on average.

The current national sickness rates for nursing staff are 4.5%,145 so we assumed rates of unanticipated absence from short-notice sickness of 4% for nursing support workers and 3% for registered nurses. This approximates known differences in sickness absence between registered nurses and support workers. 146 Although short-notice sickness absence is an important reason that shifts end up being understaffed, even when establishments are high, a large relative (1% absolute) change in the assumed rate led to a small (< 1% absolute) change in the rate of understaffed patient shifts.

Nursing vacancies are recognised as a current problem in the NHS, so we ran some illustrative simulation scenarios considering a 10% vacancy rate. 1 This scenario had achieved ward establishments between ‘standard’ staffing and ‘flexible (low)’ staffing. Correspondingly, the average percentage of understaffed patient shifts and cost per patient-day also lie between the results for these two scenarios. However, differences between scenarios based on different baseline establishments followed similar patterns.

Economic modelling of using the Safer Nursing Care Tool in different ways to set staffing levels

In the economic modelling, we used the results of the simulation from the three general hospital trusts, together with estimates of the effects of the resulting staffing levels on mortality and length of stay,6,9 to estimate the staff and net cost per life saved for ‘high’ and ‘flexible (low)’ establishments relative to the ‘standard’ establishment.

We found that higher staffing establishments – ‘high’ versus ‘standard’ and ‘standard’ versus ‘flexible (low)’ – are associated with improved outcomes (lives saved, cost per life saved and number needed to treat) under all assumptions about temporary staff availability. When we assume higher availability of temporary staff, the difference in outcomes (costs, bed-days and deaths) between alternative staffing establishments is reduced, but the cost-effectiveness for higher establishments is more favourable.

Using our core assumption – with limited availability of temporary staff based on empirical observations – the ‘high’ establishment is associated with a reduction of 1.2% in bed utilisation and a reduction in mortality of 4.5% compared with the ‘standard’ establishment. This equates to one life saved for every 665 admissions. Staff costs per life saved are £20,423 and costs net of the value of bed-days saved are £9506 per life saved. If unlimited availability of temporary staff is assumed, these estimates are reduced to £16,464/£5524 (staff/net cost).

Although there are no studies with which we could make a direct comparison, these figures are broadly comparable with the estimated cost-effectiveness of implementing the nursing HPPD staffing method in Western Australia, an approach that sets ward establishments and nurse staffing deployments at a level that is higher than typically observed in the UK. Extrapolating life-years saved based on mean life expectancy, Twigg et al. 33 estimated a cost of AU$8907 per life-year gained. Assuming that the paper was reporting 2010 costs, converting to Great British pounds (GBP) using a 2010 exchange rate [mean AUD to USD exchange 0.92 and USD to GBP exchange 1.55 (source: www.macrotrends.net; accessed 10 February 2020)] and applying Consumer Price Index inflation gives £6462 GBP in 2017 prices (source: www.in2013dollars.com/UK-inflation; accessed 10 February 2020). Applying the average life expectancy and approximate discount used (18%) gives approximately £33,500 per life saved.

Unless very pessimistic assumptions about life expectancy or utility of life gained were made for our estimates, it seems likely that the cost-effectiveness estimates for the cost per life saved of ‘high’ establishments would lead to costs per quality-adjusted life-year that sit well below generally accepted thresholds for defining cost-effectiveness. 147 Indeed, ‘high’ establishments are potentially very cost-effective. 147

On the other hand, because the ‘flexible (low)’ establishment is associated with worse outcomes than the ‘standard’ establishment under all assumptions about temporary staffing, it is easy to reject it simply because it is not an effective strategy. However, it is worth noting that under circumstances where ‘flexible (low)’ staffing establishments come closest to providing an effective alternative – when temporary staff availability is high – cost-effectiveness estimates associated with the ‘standard’ establishments approach the level at which the ‘standard’ establishment would dominate economic decision-making, because the cost per life saved associated with it is so low.

The results are sensitive to effect estimates for mortality, but if we take £30,000 per life saved as a threshold and account for length of stay savings, ‘high’ establishments are potentially cost-effective for both lower and upper (95% confidence interval) limits for estimates under all assumptions tested about temporary staff availability (except the unrealistic zero temporary staff scenario). Using the mean estimate or upper limit, the cost-effectiveness estimates are much more favourable than this, even when focusing just on staff costs.

Under low staffing establishments, high availability of temporary staff is needed to cover shifts left unfilled by permanent staff to avoid understaffing. However, the higher the availability of temporary staff, the stronger the case for higher establishments in terms of cost-effectiveness.

The conclusions are not very sensitive to different cost assumptions, although assuming an increased cost per bed-day would lead to ‘high’ establishments being even more cost-effective. If agency staff cost more, the cost savings associated with lower establishments are reduced and potentially even eliminated (under assumptions of high bank and agency staff availability).

• ‘High’ staffing establishments are associated with costs per life saved that are likely to be assessed as cost-effective under most assumptions about temporary staffing, and the conclusion is not sensitive to most assumptions made.

• ‘Flexible (low)’ staffing establishments are associated with worse outcomes and diminished cost savings when sufficient temporary staff are available to make them a viable option.

• Investing in ‘high’ staffing establishments over ‘standard’ staffing establishments would cost < 1% of annual turnover at all hospital trusts studied.

Restrictions on decision-making are likely to arise because of budget constraints and, more significantly in the short term, limited supply of staff, particularly registered nurses. These findings, therefore, have implications as much for policy as for local practice and use of the SNCT. However, given the evidence that higher staffing levels and improved establishments associated with mandatory minimum staffing policies are associated with improved job satisfaction and retention,148–150 increases in establishment could still be associated with a cost-effective approach to managing the local pool of nursing staff if recruitment and retention of staff were improved as a result.

Other considerations

All hospital trusts involved in our study have moved towards routinely collecting SNCT ratings of patients at multiple time points each day. Although we have explored cost-effectiveness under realistic scenarios of limited availability of temporary staff, we have assumed that there would always be an attempt to make up staffing shortfalls. However, comparisons of our models with no availability of temporary staff with models assuming some availability provide support for a conclusion that this ‘tailored’ approach, rather than using a fixed daily staffing level, is cost-effective. However, this does not lend support to using these systems to set low base establishments that are flexed upwards as required. The results of this study emphasise the potential benefits of higher nursing establishments and show that using the SNCT to set establishments to meet predicted need 90% of the time, rather than to meet mean average need, could be cost-effective, substantially reducing the risk to patients that is associated with low staffing.

• Our findings can be used to support a flexible staffing policy, but one that is based on high, not low, establishments.

In our models, we have assumed that there will be an attempt to fill measured staffing shortfalls if such staffing is available. Because we have generally assumed that there are limits to the availability of temporary staff, it may be that our model provides a reasonable facsimile for a system in which professional judgement might also determine that, on occasion, existing staff are best placed to manage, and so some requests for additional staff are averted. The findings of this study do not establish the superiority of measurement over professional judgement.

The SNCT is generally ‘mute’ on the matter of skill mix. However, although these findings support setting higher establishments, other research makes it clear that benefits are unlikely to accrue if the higher establishments erode skill mix. Indeed, in our recent study6 we did not model the effects of increases in nursing support worker staffing because this strategy delivered worse outcomes at higher costs.

• Increasing establishments while decreasing skill mix is unlikely to be cost-effective.

In this study, we looked only at the SNCT. However, although the specific parameters that we estimated may not apply to other tools, our simulation models show the likely consequences of different approaches to using any measurement system, because needs vary around the measured means. These findings might be of particular relevance to US hospitals, where measurement systems are frequently used for real-time staffing deployment decisions. It is striking that, although much of the UK literature has been preoccupied with setting establishments (nurses to employ), and thus emphasises forward planning, the US literature seems to be much more focused on flexible staffing assignments using real-time or near real-time measurement. Our findings demonstrate that the success of any such approach requires that it operate with an appropriate baseline establishment; establishments are unlikely to be safe or cost-effective if they are set to meet only minimal demands.

Results in context

This study was undertaken at a time of shortages in the supply of registered nurses and increasing nurse vacancy rates in many NHS organisations. Although this may seem to render these results somewhat academic, the shortage of supply does not remove the need to quantify the demand properly. Over the lifetime of this project, the ‘crisis’ in nurse staffing across the UK emerged as a major policy focus, resulting in a number of initiatives to increase the supply of nurses and to improve staff retention. If these initiatives are, ultimately, to be successful, it is imperative that the conditions that nurses find themselves working in present them with a manageable workload that enables them to consistently deliver care of a high standard. Otherwise, efforts to improve supply may be thwarted by staff outflows in a competitive labour market. To use (and extend) an oft-quoted aphorism, it is hard to fill a leaky bucket, and, when planning to fix it, it is wise not to underestimate the size of the hole. A key challenge in achieving a stable workforce and improving retention is to ensure that we properly understand how many staff are actually needed to deliver quality care. Tools such as the SNCT have a role in this, although our study results point towards a baseline staffing requirement that may be higher than that identified by using the tool as currently recommended.

In common with other studies, our results indicate that registered nurse staffing levels that are higher than present levels and higher than those recommended by the SNCT might deliver additional benefits for patient outcomes, with a cost-effectiveness ratio that supports this approach. 6,33 In the face of ongoing budget constraints in the NHS, it may seem problematic to suggest additional investment in nursing. However, although it may be that policy-makers and the public choose to make other investments, it is important to understand the costs and consequences of options. Furthermore, although nurse vacancies are at a high level nationally, not all hospital trusts are equally affected, and the baseline staffing deployed varies considerably between trusts. In the absence of an understanding of how many nurses are required, it is hard to justify such variation. A simplistic model of economic efficiency focuses attention on hospitals with apparently high levels of nurse staffing, identifying them as potentially unproductive based on simple measures of HPPD. Our economic modelling results suggest that hospital trusts that have chosen to invest more heavily in nursing staff may be using their resources cost-effectively. Conversely, these results show that what might appear to be an efficient and effective response to nurse shortages – using low establishments and flexible deployment of staff – is unlikely to be a successful strategy. This strategy is likely to cause harm to patients, and, in the circumstances when it is most likely to be successful (which requires unfeasible assumptions about the availability of temporary staff), saves little, if any, money. In some cases, such a strategy costs more and increases harm.

In the face of staff shortages, it is tempting to consider alternative approaches for improving the effectiveness or efficiency of the current workforce. Although there is scant evidence for improved productivity arising from quality improvement initiatives, such as the Productive Ward programme,151,152 or structural changes, such as altering shift patterns,146 the importance of quality leadership and management in ensuring an effective and productive ward nursing team is largely beyond dispute. However, without dismissing the importance of such factors, they serve merely to remind that having enough staff is necessary but not sufficient. This is amply illustrated by one US study that showed little benefit from increasing staffing levels in hospitals that provided poor work environments. 153

Finally, this study has focused on the nursing workforce almost exclusively. The provision of health care is a multiprofessional endeavour. That we have focused on nursing does not mean that we underestimate the potential contribution of other professions to patient safety and quality of care. The current recognition of the importance of nurse staffing for patient safety in hospitals stands as a major correction to a historical tendency to overlook this, which was illustrated in recent times in the many pages of the Francis Report10 but certainly did not begin there. The evidence provided here relates to how best to plan the staffing of hospital wards by nurses. There may be potential to reconfigure ward establishments and to shift work between professional groups. However, simply including other staff on the nursing roster does not in itself lead to a redistribution of the work that is undertaken by nurses. Conversely, shifting resources from other groups towards nursing because of studies such as this one does not guarantee cost-effectiveness unless the costs and consequences of disinvestment in other services are also understood.

Limitations and future work

In this section, we identify some further research recommendations arising from the study limitations. These recommendations are summarised and prioritised in Box 1.

In our regression modelling, we were unable to identify clear ‘tipping points’ at which there was diminishing return from higher staffing levels and, thus, we could not identify ‘optimal’ levels. Therefore, our simulation is designed to achieve staffing levels that may still be below what is optimal. Our data from reports of numbers of nurses/support workers needed when staffing was low were too unreliable to fit similar models as a result of ambiguities around whether total staff needed or additional staff needed had been reported. However, repeating such surveys might help with identifying the ‘magic numbers’ of how many staff are required each shift, and modelling the relationship between this requirement and underlying factors (regression approach). It is possible that there is a range of acceptable staffing levels in a given situation (as we simulated using a buffer around acceptable staffing) and so ideas from fuzzy modelling and optimisation,154,155 which deal with ambiguity and vagueness, may be applicable.

In the simulation modelling, we assumed that requirements in different wards are independent of one another, that is that neither the numbers of patients nor the acuity/dependency levels of patients in different wards interact. This is unlikely to be upheld in practice. Although future work could relax this assumption, it introduces significant complexity to both the data collection and the modelling. As it seems most likely that periods of high demand in one ward would be associated with periods of high demand in others, the significance of internal re-deployment is likely to be overestimated by our current assumption. The impact on our results and conclusions is unlikely to be great because re-deploying staff between wards did not make substantial contributions to reducing low staffing and did not substantially alter cost-effectiveness estimates.

We did not take into account that there may be feedback loops whereby adverse effects of low staffing (increased risk of adverse events and longer hospital stays) could result in increased workloads for staff. It is difficult to determine whether or not such effects would make a measurable impact on overall results, but if results were influenced by such effects it would be in a manner consistent with our conclusions rather than serving to undermine them. Establishments already associated with high levels of understaffing would appear to be even more disadvantageous because of this positive feedback loop in which understaffing generates additional demand for staff.

The cost per patient-day is variable between wards because of different pay band mixes of the staff employed, which is accounted for in the modelling. These costs do not include temporary staff covering long-term/planned absences, and hence we observed higher costs in reality than in the simulation experiments. We did not have a breakdown of different amounts of ‘notice’ for sickness absence or enough data to estimate seasonality (clustering) of staff sickness absence and have not considered a reduction in availability of temporary staff due to their sickness.

We did not have data on the variability of workload within acuity/dependency levels. In our modelling, we assumed symmetric distributions and that each standard deviation was 10% of the respective multiplier, although it did not appear that our hospital-level conclusions were highly sensitive to this assumption. Future work could look at the impact of non-symmetric distributions, study the impacts of these distributions on ward results and collect data about the shape and variability of these distributions. This may also help confirm whether or not the categories 0, 1a, 1b, 2 and 3 are the best way of segmenting patients, and, if not, cluster analysis could be applied to generate categories that better group patients according to their nursing requirements.

In the process of deriving the SNCT multipliers from data on quality-assured wards, ready-for-action time was removed, but this means that recommended establishments are lower than those observed on the quality-assured wards. Nurses not only work on routine, plannable tasks, such as ward rounds, but also must react to emergent, urgent tasks, such as adverse reactions to infusions or emergency tasks involving responding to life-threatening incidents. 156 Simulation modelling or queuing theory could be used to investigate these dynamics, and how many staff are needed so that different priorities of tasks can be completed within appropriate time frames. However, as our work tends to point toward the overall benefits of higher establishments, this anomaly in the SNCT does not alter our conclusions, and the way in which the SNCT treats this ready-for-action time needs to be reviewed.

In this study, we limited our scope to setting ward establishments according to the SNCT and reacting to shortfalls each shift, that is, dealing with fluctuations in demand and unforeseen absences on the day. We did not investigate how to roster staff fairly157,158 or how to plan staffing levels to cover foreseen absences (e.g. leave, training and long-term sickness). There is interplay between different planning stages; the nearer to the start of the shift, the more information is known about the availability of nursing staff and the patients’ nursing requirements. Thus, further studies could focus on understanding the links between employment, deployment and rostering better, in particular how much tolerance needs to be built in between each planning stage (i.e. the ‘uplift’ for absences and flexibility to adjust rosters).

We found that specialing requirements are a major source of variability in demand. Future work could investigate specialing requirements in more detail, including the contribution of volunteers and relatives to specialing, whether or not this demand could be managed by a pool of staff at the hospital level (as fluctuations in demand are wide at the ward level) and perhaps a more detailed investigation into how much time specialing duties take (accounting for handovers and cohorts).

In our study, the judgement of staffing adequacy and ratings of the SNCT were undertaken by the same people or teams. Although it is difficult to picture a sensible way of routinely separating these two judgements to make them fully independent, the possibility that some degree of bias is induced cannot be discounted. However, the nature of the SNCT reports – numbers of patients in each category – is sufficiently distinct from the staffing adequacy questions that our findings are unlikely to be a simple product of common method bias.

We saw a degree of consistency in the results between the four participating hospital trusts but we cannot be sure that these results would generalise across all trusts. Certainly the quantitative estimates (rather than the qualitative relationships) do appear to differ by trust. Costs differ as a result of the bands of staff employed, and the ability of wards to match staffing to demand depends on factors such as ward size and specialing requirements. Therefore, the results may be different for hospital trusts with substantially different mixes of staff and ward types. Future research could look in more detail at the fit of the SNCT for particular types of wards. We have concentrated on hospital-trust-level results but our results do suggest that requirements may differ between medical and surgical wards, even when the measured staffing requirement appears to be the same based on the acuity/dependency rating. Our results also suggest that staffing requirements are much more variable on some wards than on others, which makes determining the correct establishment even more challenging.

We did not estimate directly the effect of low staffing on the sample of patients in the study. Although our effect estimates derived from a methodologically strong study that was undertaken in one of our participating hospital trusts, they were nonetheless derived from a single-site study. It is possible that the observational evidence used to estimate the outcomes association could be biased, although this coefficient is consistent with other estimates of effect, and most biasing mechanisms would tend to suggest an underestimation of the strength of association. 6 Cause and effect, although plausible and consistent with an increasingly large body of evidence, should not be assumed. Furthermore, we considered the impact of staffing levels on mortality only, but other aspects of care are likely to benefit from increased staffing, including patient experience,143 and staff outcomes such as sickness absence and turnover may also improve. Thus, decision-making based on mortality alone underestimates the likely benefit of increasing nurse staffing levels. Further observational studies are needed to extend this research to other trusts, and consider the influence of the availability of other staff groups (including other staff rostered on wards) and other outcomes. Different patient types may be affected to greater or lesser extents by understaffing, so this is another possible area to explore.

Finally, this was an observational study with modelling of scenarios. We cannot be sure that the simulated costs and consequences of different staffing establishments would hold in reality. To overcome the limitations of observational and simulation studies, experiments could be conducted whereby real ward establishments are systematically and purposefully changed. Although the challenges of undertaking such prospective intervention studies on sufficient scale are considerable, they are not insurmountable. Currently, the An Roinn Sláinte (Department of Health) in Ireland is backing an intervention study by supporting the increase of nurse staffing establishments to levels identified by a staffing system (www.trendcare.com.au/). The study will evaluate the effects on patient and staff outcomes. Likewise, the aforementioned study by Twigg et al. 33 was a prospective intervention study. However, neither of these studies is randomised, and undertaking such studies at a scale such as that achieved by the study we report here still seems unlikely. 159 While prospective and even randomised studies of different staffing strategies and methods for determining establishments are feasible, observational studies will retain a role, including in the search for a threshold that, in UK data at least, remains elusive.

Conclusions

The hospital trusts in this study have implemented the SNCT and have used it to guide establishment-setting. The staffing levels measured by the tool are associated with professional judgement of staffing adequacy and so the tool has some degree of validity, although we cannot show that the tool provides a measure that is superior to professional judgement. In modelling different approaches to using the data to set ward establishments, we have demonstrated that patient need can be better met by setting the ward establishment at a level that is above the mean. Other approaches risk patients being exposed to very frequent low staffing, requiring the use of bank and agency staff, who are not always available. Because of the consequences of low staffing, the higher establishments appear to be cost-effective when compared with lower ones. Our findings can be used to support flexible staffing policies guided by the SNCT, but such policies are cost-effective when based on establishments that are planned to meet patient need 90% of the time. A staffing policy that sets establishments based on mean demand is also cost-effective when compared with a staffing policy based on a ‘low’ core establishment (to which additional staff are added when required). This is in part because flexibility is constrained by the limited availability of bank and agency staff, but also because such a policy is likely to increase the unit costs of staff. Flexible staffing and low base establishments should not be equated, as low establishments are associated with increased patient harm even when temporary staff availability is high. Furthermore, when temporary staffing availability is high, cost savings are largely eliminated. Although the routine deployment of systems that allow real-time calculation of staffing requirements can help with ongoing monitoring of a ward’s staffing, for sufficient staffing to be delivered consistently, the priority remains correct determination of the necessary baseline. The best outcomes for patients are achieved at the highest establishments, provided that skill mix is maintained.

Steering group membership

We would like to thank all the members of our steering group:

• Professor Karen Spilsbury (chairperson), Chair of Nursing Research, University of York

• Professor Chris Bojke, Professor of Health Economics, University of Leeds

• Professor Sally Brailsford, Professor of Management Science, University of Southampton

• Ann Casey MBE, Clinical Workforce Lead NHS Improvement/Assistant Chief Nurse, University College Hospital

• Professor Jonathan Drennan, Professor of Professor of Nursing and Health Service Research, University College Cork

• Stephen Habgood, patient and public involvement representative

• Professor Alison Leary, Chair of Healthcare and Workforce Modelling, London South Bank University

• Mr Trevor Murrells, Statistician, King’s College London

• Ms Lorna Wilkinson, Director of Nursing, Salisbury NHS Foundation Trust.

Contributions of authors

Peter Griffiths (https://orcid.org/0000-0003-2439-2857) (Professor, Health Services Research) was principal investigator. He was responsible for the original conceptualisation of the study and study design, secured funding, oversaw acquisition of the data, data analysis (in particular economics), and interpreted the results and contributed to drafting the report.

Christina Saville (https://orcid.org/0000-0001-7718-5689) (Research Fellow, Operational Research) undertook descriptive and regression analyses, developed the simulation model and undertook simulation analyses, and drafted and revised the final report.

Jane E Ball (https://orcid.org/0000-0002-8655-2994) (Professor, Health Services Research) contributed to study design, acquisition of funding, interpretation of results and critical revision of the report.

Rosemary Chable (https://orcid.org/0000-0002-2598-0100) (Deputy Director of Nursing and Deputy Director of Education and Workforce, Nursing and Health Services Management) contributed to study design, acquisition of funding, acquisition of the data, interpretation of results and critical revision of the report.

Andrew Dimech (https://orcid.org/0000-0001-5212-1929) (Divisional Clinical Nurse Director, Nursing and Health Services Management) contributed to the original conceptualisation of the study and study design, secured funding, oversaw acquisition of the data, interpretation of results and critical revision of the report.

Jeremy Jones (https://orcid.org/0000-0002-2725-0937) (Principal Research Fellow, Health Economics) contributed to study design, acquisition of funding, oversaw acquisition of the data, data analysis (in particular processing), interpretation of results and critical revision of the report.

Yvonne Jeffrey (https://orcid.org/0000-0002-6001-7723) (Assistant Director of Nursing, Nursing and Health Services Management) contributed to study design, acquisition of funding, acquisition of the data, interpretation of results and critical revision of the report.

Natalie Pattison (https://orcid.org/0000-0002-6771-8733) (Clinical Professor, Nursing) contributed to study design, acquisition of funding, acquisition of the data, interpretation of results and critical revision of the report.

Alejandra Recio Saucedo (https://orcid.org/0000-0003-2823-4573) (Research Fellow, Health Services Evaluation) contributed to study design, acquisition of funding, interpretation of results and critical revision of the report.

Nicola Sinden (https://orcid.org/0000-0002-9497-6973) (Lead Nurse for Workforce, Nursing and Health Services Management) contributed to study design, acquisition of funding, acquisition of the data, interpretation of results and critical revision of the report.

Thomas Monks (https://orcid.org/0000-0003-2631-4481) (Principal Research Fellow, Operational Research and Data Science) contributed to study design, acquisition of funding, oversaw acquisition of the data, data analysis, interpretation of results and critical revision of the report.

Publications

Saville CE, Griffiths P, Ball JE, Monks, T. How many nurses do we need? A review and discussion of operational research techniques applied to nurse staffing. Int J Nurs Stud 2019;97:7–13.

Griffiths P, Saville C, Ball J, Jones J, Pattison N, Monks T. Nursing workload, nurse staffing methodologies and tools: a systematic scoping review and discussion. Int J Nurs Stud 2020;103:103487.

Data-sharing statement

The data set is available from the University of Southampton Institutional Research Repository (https://eprints.soton.ac.uk/435408/; accessed 25 February 2020) and does not contain individual-level or personal data. Further information, if required, can be obtained from the corresponding author.

Patient data

This work uses data provided by patients and collected by the NHS as part of their care and support. Using patient data is vital to improve health and care for everyone. There is huge potential to make better use of information from people’s patient records, to understand more about disease, develop new treatments, monitor safety, and plan NHS services. Patient data should be kept safe and secure, to protect everyone’s privacy, and it’s important that there are safeguards to make sure that it is stored and used responsibly. Everyone should be able to find out about how patient data are used. #datasaveslives You can find out more about the background to this citation here: https://understandingpatientdata.org.uk/data-citation.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health and Social Care.

Data specifications

Extraction of worked shifts and identification of worked hours

We derived the staffing variables (total/registered nurse/nursing support worker hours per day, skill mix and distribution of staff over the day) from roster systems. Data provided from roster systems were in the form of one row per shift and staff member. The data fields included duty date, actual shift start and end times, planned shift start and end times, staff type (substantive, bank or agency), ward and break duration, as well as the post title and band either recorded together across multiple fields or contained within longer shift type descriptions. For hospital trust B, the roster data were provided from separate systems for substantive and non-substantive (bank and agency) staff, with different data fields.

The following shift inclusion and exclusion criteria were applied. Posts that we classed as nursing roles and so included in the analysis were different types of registered nurses and nursing support workers, called by a variety of names (staff nurses, senior staff nurses, course nurses, junior ward sisters, ward sisters, charge nurses, advanced nurse practitioners, nurse specialists, community nurses, specialist nurse practitioners, nursing support workers, community health-care assistants, assistant practitioners, bank support workers, patient support workers, acute coronary syndrome nurses, advanced clinical specialist nurses and phlebotomists). Although maternity wards were excluded from the analysis, we included registered midwives and maternity care assistants who worked as bank/agency staff on other wards at hospital trust A. Staff in non-nursing roles, such as ward clerks, housekeepers, administrative/clerical staff and therapists, were excluded. Trainees, students and honorary staff, when recorded in the data, were also excluded because they do not count towards the ward establishment. Our analyses focus on worked shifts, so any cancelled shifts, rest shifts and unfilled bank/agency shifts were excluded.

Agenda for Change bands were used throughout the analysis for costings and to distinguish between registered nurses (at least band 5) and nursing support workers (below band 5). These bands were extracted from text in the relevant data fields for each hospital trust. Bands were not recorded for some advanced nurse practitioners, patient support workers and bank support workers at hospital trust A, where they were assumed to be band 7, 3 and 3, respectively.

Actual shift start and end times were usually provided; however, where the actual end times were missing, we assumed that these were the same as the planned end times. Hospital trust B did not have breaks recorded for bank and agency staff. Therefore, we assumed that small positive differences (0–1 hour inclusive) between planned and actual work time were due to breaks, but that otherwise differences were due to a shift being worked of a different length from that planned.

Recording of requirements for ‘specialing’

Specialing staffing requirements are in addition to the requirement generated from the SNCT, and hospital trusts have made adjustments to the tool or use parallel processes to deal with specials. The four hospital trusts recorded specials in different ways. Hospital trust A provided reports of requests for staff to special patients, as specialing was covered by bank/agency staff. From September 2017 onward, these contained shift names rather than times. To deal with this, we assumed that the most common shift times from the earlier data (night shifts from 7 p.m. to 7.30 a.m., late shifts from 1.30 p.m. to 7.30 p.m., early shifts from 7 a.m. to 1 p.m. and long day shifts from 7 a.m. to 7.30 p.m.). We converted these shift times into numbers of specials present in each ward at the start of each 6-hour shift. Hospital trusts B and C provided the numbers of specials recorded by the nurse in charge at census times. Hospital trust B also recorded cohort specials. Hospital trust D has defined extra categories, levels 1c and 1d, for patients requiring specialing by nursing support workers or registered nurses, respectively. This means that these patients are not included in the standard levels, as they were for the other hospital trusts. We assume that they are in level 1b.

Staffing variables (calculations)

We calculated the specials multiplier by substituting 24 hours per day in Equation 5 and rearranging to get the total required daily establishment in WTE for one specialed patient:

Correspondence between Safer Nursing Care Tool multipliers and daily care hours

Dealing with sub-wards

Some wards consist of several sub-wards that each separately recorded the staffing adequacy and SNCT level counts, but are rostered and/or recorded in patient administration systems as one ward.

We summed up the numbers of patients in each level across sub-wards to obtain the ward level counts, and discarded instances where data for at least one sub-ward were missing. As sub-wards may care for different types of patients, estimating level counts for the whole ward from partial data may not be reliable.

When calculating the completeness of the variables ‘enough staff for quality’, ‘nursing care left undone’ and ‘staff breaks missed’, we considered data for sub-wards missing if they were missing for all sub-wards. For these staffing adequacy questions with yes/no responses, we considered data valid if they were valid for at least one sub-ward. We combined the answers to the questions at sub-ward level into an answer at the ward level by using majority vote of valid answers. If there were equal numbers of sub-wards answered ‘yes’ and ‘no’, we took the negative answer (‘no’ for ‘enough staff for quality’, ‘yes’ for ‘nursing care left undone’ and ‘yes’ for ‘staff breaks missed’).

For the reported total staffing requirement variable (the sum of the answers to the questions ‘nursing support workers needed’ and ‘registered nurses needed’), we considered data for sub-wards missing if they were missing for any sub-ward. We combined the answers to the questions at sub-ward level into an answer at the ward level by summing up the requested registered nurses/nursing support workers across sub-wards. We used the normal validity criteria on the whole ward answer.

Staff costs

Simulation model technical details

Simulation model logic

An overview of the simulation logic is shown in Figure 8. Each stage is described in more detail in the following sections. The simulation was designed to work for any number of wards. The wards are ‘agents’ that interact with each other (within directorates) by lending/borrowing staff in an attempt to cover shortfalls of staffing in one ward with surplus in another. Outputs are updated at the end of each time step (day/shift) and aggregated at the end of each year.

FIGURE 8.

Simulation model overview diagram.

Before running the simulation

Before running the simulation for a particular hospital trust, information about its wards is imported from Microsoft Excel^® (Microsoft Corporation, Redmond, WA, USA) files to fill database template tables within the simulation. The three files contain occupancy data, acuity/dependency data and other ward information. The occupancy distributions consist of the number of patients, labelled with the ward identification numbers (a contiguous number for internal purposes and the number used for external reporting), day of the week and 6-hour-shift. The acuity/dependency distributions consist of the proportions of patients in each SNCT acuity/dependency level and the proportion of patients who require specialing, labelled with the ward identification numbers and observation number (used when sampling a row). The other ward information consists of the baseline staffing levels for each scenario (in WTE), the skill mix (in the morning, afternoon, evening and night, and over a 24-hour period), the distribution of staff over 24 hours (proportion of staff deployed in the morning/afternoon/evening/night), the directorate number, whether or not the ward is an acute admissions unit and some purely descriptive information (directorate, medical or surgical).

The probabilities of requests for temporary staff being fulfilled is imported from a Microsoft Excel file into another database template table. The template allows probabilities to differ between bank and agency, up to three staff types, weekend and weekday, and time period (morning, afternoon, evening and night, and over a 24-hour period).

Next, further parameter values that apply globally to the whole hospital trust (the system) are set. These parameters relate to:

1. nursing staff requirements – acuity/dependency multipliers, specials multiplier, standard deviation of multipliers, minimum number of registered nurses needed (constraint), demand for registered nurses with a particular skill

2. permanent staff – the baseline staffing level, the data sample used to calculate the staffing level, the staff types, absence chance for each staff type, absence length, proportion of registered nurses with a particular skill

3. re-deployed staff – re-deployment rules, priority sequences for providing and receiving re-deployed staff, efficiency of re-deployed staff, re-deployed staff shift length

4. temporary staff – rules for requesting temporary staff, efficiency of bank and agency staff, temporary staff shift length

5. display settings – the understaffing criterion to plot in charts, e.g. 15% or more under requirement

6. general settings – step length, round-down bound used when converting required hours to requested hours.

At start of a run

At the start of a run, variables tracking time (the step number and the shift number) are reset to zero. The wards, staff types and sharing groups are counted. Occupancy distributions are created from the occupancy data table. Data are placed in arrays, which are convenient structures for working with multidimensional data.

Then, the establishment (number of staff employed in WTE) is converted to the number of planned deployed hours per time step (6-hour shift or day), including applying the skill mix, rounding to whole people and dealing with minimum constraints, as follows. Note that the establishment does not need to be a whole number as staff may work part time, but the planned number of staff to deploy on each 6-hour shift should be a whole number. As in our other analyses, we use Equation 5 for converting the planned staffing in WTE to the planned total care hours per day (see Appendix 1).

The planned skill mix (proportion of staff who are registered nurses) and distribution of staff over the day in each ward is set as the average observed for that ward. There is a constraint that there must be at least one registered nurse present on each ward, so if the registered nurse hours is under 6, this is rounded up to 6. Otherwise, the registered nurse hours are rounded up or down to the nearest 6 hours. The remaining planned hours are assigned to nursing support workers, and again are rounded up or down to the nearest 6 hours.

For example, suppose that the planned nursing hours for a morning shift on a particular ward are 18, and the skill mix is 50%. This is equivalent to 9 hours of registered nurse time, which is rounded up to 12 hours. There are 6 hours left to cover, which are assigned to nursing support workers.

The planned deployed hours per day (sum over the four shifts) are converted back into WTE to enable calculation of the cost of employing this number of permanent staff.

Before time step

Before each time step, that is before the simulation switches to the next period (6-hour shift or day), the variables for this period are updated. These variables are the time step, the shift (one to four), the day type (weekday, Saturday or Sunday/bank holiday) and the planned staffing level for this shift. The deployment array (numbers of staff from each source and of each staff type deployed on each ward in what capacity) is reset at zero ready to be filled in at the next stages.

Before time step, in each ward

Next, the required staffing for this period is calculated for each ward in turn. For this, first the number of patients on the ward is sampled from the occupancy distribution for that ward, day of week and shift (morning, afternoon, evening or night).

Second, the acuity/dependency profile is sampled from the acuity/dependency data. This is done by selecting a random observation for that ward (we assumed that there were no day of week or time of day patterns). For each patient, the probability of being in each acuity/dependency category and the probability of requiring specialing are the corresponding observed proportions. The required staffing per patient (in WTE) is sampled based on the patient’s acuity/dependency category and specialing requirements. This is converted into the required staffing level for this period using the skill mix, distribution of staff over the day and minimum constraints (as for the planned staffing levels), but is not rounded.

On time step, in each ward

On the time step, that is, immediately when the period begins, the number of hours of staffing provided by permanent staff in this period is calculated for each ward. For this, the number of planned staff who are not unexpectedly absent (i.e. at short notice) is calculated. The chance of being unexpectedly absent can differ between staff types in the model. All of these staff are allocated to their home ward to start with. The (absolute) shortfall for each staff type is calculated as required minus allocated hours. Where applicable, the simulation checks which of the registered nurses working are IV trained (sampled probabilistically).

Similarly, the spare hours (hours that could be re-deployed to another ward) for each staff type is calculated. This is the allocated minus the required hours, rounded down to the nearest multiple of ‘re-deployed hours chunk’, as staff can be re-deployed only for fixed time periods.

On time step

Next, staff are re-deployed within sharing groups (directorates) to attempt to cover shortfalls for each staff type, as shown in Figure 9. The shortfall is rounded up or down (depending on the round-down bound) to the nearest multiple of ‘re-deployment chunks’. Requests for extra staff are triggered if the rounded shortfall for that staff type is more than zero, and if either the total shortfall or the staff type shortfall is more than the trigger (6 hours). To decide the priority of re-deploying extra staff to wards, lists of wards are sorted using the bubble sort algorithm,160 which works by comparing the shortfall as a proportion of the requirement (or spare hours as a proportion of requirement) for adjacent wards in the list and swapping them if they are in the wrong order.

FIGURE 9.

Summary process flow of staff re-deployment.

Following this, for the wards that are still requesting extra staff, for each staff type, first bank and then agency staff are requested, as shown in Figure 10. The hours requested are the shortfall rounded up or down (depending on the round-down bound) to the nearest multiple of ‘external work time’. The probability of a request for temporary staff being fulfilled depends on the staff source, the staff type, whether it is a weekday or a weekend and the time period.

FIGURE 10.

Process flow of hiring temporary staff.

Then, the effective staffing on each ward, given that re-deployed and temporary staff are less efficient than permanent staff, is calculated. The effective shortfall on each ward is calculated.

The staffing adequacy (understaffed, adequately staffed or overstaffed) on each ward is assessed according to a range of criteria (e.g. understaffing can be measured as ‘more than 15% under the requirement’, ‘more than one person short’). Depending which staffing adequacy criterion was chosen to be displayed, the state chart for each ward is updated (see Figure 1). The running totals of understaffed, adequately staffed and overstaffed time periods per ward are updated, as are the charts tracking their relative numbers. The array of the total number of staff deployed on wards from different sources and of different types is updated.

At the end of run (1 year)

At the end of each run, the total permanent staff employed (in WTE) and the number of agency and bank hours worked across the whole hospital are summed by staff type.

The total yearly costs are calculated. For this, first, the costs of employing staff and hiring temporary staff at the standard rates are calculated. Then, bonuses for unsocial hours for substantive, bank and agency staff are added.

The results of the run at both the ward and the hospital level are exported to Microsoft Excel.

Simulation input parameters

Variability within acuity/dependency levels

We have no information about the shape and variability of nursing requirement distributions within acuity/dependency levels. Even with relatively small variability within acuity/dependency levels (standard deviation set at 10% of the mean), and assuming that they follow symmetric distributions, the required staffing for patients in levels 1a, 1b and 2 would overlap substantially, as shown in Figure 11, and even more so for a standard deviation at 25% of the mean, as shown in Figure 12.

FIGURE 11.

Histogram of required HPPD if standard deviations within acuity/dependency levels were 10% of the means (multipliers).

FIGURE 12.

Histogram of required HPPD if standard deviations within acuity/dependency levels were 25% of the means (multipliers).

Staff availability

Table 36 shows the empirical percentages of all requested temporary shifts (both bank and agency) that were filled by bank staff at hospital trust B (i.e. assuming that bank staff were asked first). Morning shifts are those that start between 7 a.m. and 1 p.m., afternoon shifts start between 1 p.m. and 7 p.m. and evening/night shifts start between 7 p.m. and 7 a.m. Table 37 shows the percentage of empirical percentages of agency requests that were filled at hospital trust B. We did not have equivalent data for the other hospital trusts.

Simulation outputs

Simulation scenarios

Coefficients and baseline parameters used in cost-effectiveness models

Sample sizes for shift-level data