Safer and more efficient vital signs monitoring protocols to identify the deteriorating patients in the general hospital ward: an observational study

Implications for patient safety

There is now substantial evidence that inadequate monitoring is a major patient-safety issue, contributing to avoidable deaths and other significant adverse outcomes. 3,8,13 The Keogh review into 14 hospital trusts with high mortality rates noted that ‘a consistent theme throughout almost all of the organisations reviewed was the management of complex deteriorating patients and the monitoring of Early Warning Scores’. 9 The Keogh report is not unique in identifying mismanagement of physiological deterioration of patients in hospital. Numerous reports on patient safety and avoidable deaths have identified the failure to observe or respond to patient deterioration as a significant contributory factor. 1,8,9,21 This is often referred to as ‘failure to rescue’ and is thought to be sensitive to the quality of care provided, including levels of nurse staffing. 15–17,22,23 While progress in statistical modelling techniques and development of electronic medical records will likely lead to more sophisticated and accurate risk-prediction algorithms to detect deterioration from vital signs and other clinical data,24–26 they will not directly provide evidence on how frequently these measurements should be obtained.

Implications for staff resources

Whereas under-observation can delay the opportunity to detect patient deterioration and initiate remedial treatment,8,9 over-observation uses valuable nursing resources that could be better deployed to other essential aspects of patient care. Indeed, compliance with current monitoring protocols is often poor, with many observations missed or delayed. 11–14 This is particularly evident at night,12,27,28 when staffing is at its lowest. Results from our HS&DR ‘Missed Care’ study (ISRCTN: 17930973) show that compliance with vital signs observations is significantly (p < 0.05) associated with the level of available nursing staff. 29,30 Estimates of the time required to take observations are scarce, but a survey of 2917 registered nurses (RNs) across 46 acute hospitals in England showed that one-third had felt unable to undertake all necessary patient surveillance due to lack of time on their last shift. 15 However, even when staffing is sufficient, there are valid clinical reasons why nursing staff will deviate from protocols. Therefore, it is essential to have a monitoring protocol that is achievable on the ward and does not compromise other aspects of care.

With this in mind, and by adapting techniques from our previous work,15,31,32 part of this study is a prospective observation of recording vital signs across a range of wards. This will allow us to understand better the impact on staff workload and factors that might affect compliance with patient surveillance.

Research question

The review aims to address the question: ‘How often should vital signs be measured as part of routine monitoring of patients in hospital?’ Our search strategy for identifying and selection of studies is outlined below. The studies were divided into the following categories based on similarities in their design, main objectives and themes: primary research, review articles.

Identifying relevant studies

The initial literature search was conducted in November 2019 only including articles in English in the following databases: MEDLINE and EMBASE. The database search was expanded to remove language restrictions in April 2020. We checked for newly published relevant studies in April and October 2021 by screening papers citing studies already included in this review.

The search strategy was developed together with an academic librarian at Bodleian Health Care Libraries at the University of Oxford through a series of preliminary searches. Our search strategy included two key elements: (1) terms describing timing, frequency and intervals; and (2) terms related to routine monitoring of patients using EWSs or medical emergency team (MET) systems, as well additional search terms to exclude studies on non-adult patient populations as well as conference abstracts. We included primary research and relevant review articles in this scoping review because the latter further highlighted the existing evidence gap. The full list of resulting search terms is provided in Appendix 1.

Study selection

The initial screening for inclusion was based on the title and abstract of studies to eliminate studies that did not meet the minimum inclusion criteria. Two of the project team members (CK and IK) reviewed titles and abstracts generated by the original search against agreed inclusion and exclusion criteria. Any disagreements regarding inclusions were resolved through discussion, or, in the case of continued disagreement, by a third team member (OR). A full-text copy of each study was assessed against eligibility criteria by CK and IK to decide on final inclusion. Reference sections of selected studies were checked manually to identify additional relevant studies that were potentially missed in the database search. Relevant references as well as studies identified through other sources then underwent the same two-stage screening process to be included in this review.

Inclusion criteria: This review focuses on the monitoring of adult patients admitted to hospital wards. This review included publications in any language that reported on the speed of patient deterioration or the impact of patient monitoring on outcomes. Review articles addressing either of these questions were also included.

Exclusion criteria: Case studies, studies limited to outpatient care or care provision in community or rehab centres as well as studies on paediatric, pregnant or post-partum patients and patients directly admitted to ICU (without any monitoring on general wards) were excluded. Primary research studies were excluded if the article did not specify which frequency or protocol was used to monitor patients. Studies focusing on the escalation of care once deterioration was detected were also excluded, as were studies assessing compliance with an existing monitoring protocol. For primary studies that did not compare different monitoring regimes, studies were excluded if they did not present longitudinal vital sign data and hence did not provide information on the speed of deterioration which could potentially be used to guide the development of future monitoring protocols.

Charting data

Data from each article meeting the eligibility criteria were extracted separately by the two reviewers using a standardised data-extraction sheet. Collection included the following items: authors, year of publication, journal, type of study, study design and setting, patient population, vital signs and EWS monitored, monitoring protocols in intervention and control group (if applicable), methods and interventions, outcome measures, summary of results, key conclusions, recommendations, source. Identified studies were grouped into key themes based on similarities in their study design.

Search results

Initial searches yielded 515 studies (after removal of duplicates), of which 77 were selected for full-text review. Eleven papers met eligibility criteria and an additional eight were added through citation tracking and other sources before undergoing data extraction [see Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram, Figure 2]. The selected papers ranged from 2006 to 2020. The methodological quality of the studies was not formally assessed in line with the framework of scoping reviews. Meta-analysis was not feasible due to heterogeneity of study design, study populations and outcome measures.

FIGURE 2.

PRISMA flow diagram for the scoping review.

Of the 66 articles excluded during full-text assessment, 28 did not provide sufficient detail on the monitoring frequency used to collect data and a further 37 neither compared different monitoring protocols nor presented longitudinal vital sign or EWS data with high temporal resolution. We also excluded one study by Petersen et al. 52 that merely summarised, and hence duplicated, the results from an already included study53 as part of a published PhD thesis.

General characteristics of included studies

The majority of identified articles (68.4%) were published in 2016 or later. We identified two groups of articles:

• twelve primary research studies (see Appendix 2),41,45,53–62 of which one was a survey of hospital policies on the management of the deteriorating patient;55

• seven review articles (see Appendix 3)3,4,18,63–66 including one editorial4 and one opinion piece. 64

Key characteristics of primary research studies and review articles are detailed separately below.

Primary research

Of the 12 primary research studies identified (see Appendix 2):

• three were randomised controlled trials (RCTs),45,53,67 of which two were cluster/ward randomised;45,53

• seven were observational before-and-after studies or parallel group studies comparing different monitoring practice;56–62

• one was a retrospective observational study, assessing trends in EWS observations;54

• one was a qualitative review of hospital policies and guidelines on managing patient deterioration which is summarised separately below. 55

Articles reporting primary research originated from a variety of countries: three each from the USA, UK and Denmark, and one each from the Netherlands, Greece and Australia.

The majority of studies were carried out at a single institution: only one study involved two hospitals (one public and one private56). Study duration ranged from 8 weeks to 24 months, with six studies lasting between 3 and 7 months and four studies lasting 16 months or more.

The 11 quantitative studies involved a wide range of different study populations. One study focused on patients with acute exacerbated chronic obstructive pulmonary disease. 57 Five studies involved both medical and surgical patients41,53,57,61,62 and five involved different groups of surgical patients at different stages of their treatment journey. 45,56,58–60 The latter included one study of postoperative patients,56 one study of patients in a surgical trauma step-down unit,58 one with neurosurgical patients,59 one study of general elective patients45 and one study of elderly patients (≥ 65 years) with elective major abdominal cancer surgery. 60

Ten out of the 12 primary research studies examined interventions created by implementing or trialling new monitoring practice involving a change in monitoring frequency. Most commonly, studies compared intermittent and continuous monitoring practice (seven studies),41,45,57–61 followed by comparisons of different intermittent monitoring protocols (three studies). 53,56,62

Most studies involving continuous monitoring showed that continuous monitoring was able to identify more periods of deranged physiology;57–60 however, only one study investigated whether this had an impact on clinical outcomes. 60

Five studies investigated the effect of changes to monitoring regimes on adverse events or clinical outcomes such as length of stay (LOS), mortality, time to intervention of rapid response team (RRT) activation and ICU admission. 41,45,53,59,61 Evidence for the benefit of changing to continuous monitoring was varied. While some studies observed reduction in LOS either in hospital45 or in ICU61 and reduction of event rates including cardiac arrest, sepsis, mortality, ICU admission45,61 or RRT activation,59 others did not find significant differences in outcome rates such as mortality, cardiac arrest or transfer to high care. 41,59 One study (focused on the impact of different intermittent monitoring protocols in low-risk patients) found no benefit in increasing monitoring frequency to more than twice daily in this patient group. 53

Some studies highlighted technical problems and patient compliance with wearing continuous-monitoring devices as real issues. 41,45

One study showed that protocolised monitoring (three times daily) improved compliance over a clinical judgement model. 62

Two studies presented deterioration rates or patterns, providing indirect evidence for optimal monitoring frequency. 54,58 One study showed that 4-hourly monitoring of an EWS can provide an indication of a patient’s ICU LOS and mortality. 54

The paper by Freathy et al. 55 was a qualitative review of policies on the response to deterioration from 55 hospitals; 65% of hospitals used NEWS and all hospitals collected vital sign observations at a minimum frequency of two times a day, although observation protocols varied widely even when the same EWS was used. The study also revealed that policies often lacked specifics such as out-of-hour guidelines (missing in 40% of policies), and rarely specified the maximum permitted response time once deterioration was identified. It suggested that clear dissemination of service information to clinical staff is required and that hospitals need to review their policies and operating procedures, including guidance for whole 24-hour periods.

Included reviews

Seven of the articles we included were reviews, of which three were systematic18,65,66 and four were narrative reviews,3,4,63,64 reports from consensus conferences or opinion pieces (see Appendix 3). All but two3,65 were published in 2016 or later. Systematic reviews covered from 14 to 22 studies, while narrative reviews covered between 7 and 149 references.

One article pointed out that current monitoring practice is often based on tradition or clinical judgement. 64

Five articles discussed both continuous and intermittent monitoring,18 while one review focused on intermittent monitoring only63 and another considered vital signs measurements on ‘a routine basis’. 65

Four review articles concluded that there is some evidence that monitoring frequency has clinical relevance. 4,18,65,66 However, reviews presented varying evidence regarding the impact of continuous monitoring on clinical outcomes when compared to existing intermittent-monitoring practice. Cardona-Morell et al. 18 found no significant evidence for reduction in mortality with continuous versus intermittent monitoring but significant impact between different intermittent-monitoring practices. Sun et al. 66 concluded that the clinical evidence showed that continuous monitoring can reduce mortality, LOS, ICU admission and RRT activation, and Smith et al. 4 suggested that changes in monitoring practice can affect outcomes. Four review articles concluded that there is some evidence that monitoring frequency has clinical relevance. 4,18,65,66 However, reviews presented varying evidence regarding the impact of continuous monitoring on clinical outcomes when compared to existing intermittent-monitoring practice. Cardona-Morell et al. 18 found no significant evidence for reduction in mortality with continuous versus intermittent monitoring but significant impact between different intermittent-monitoring practices. Sun et al. 66 concluded that the clinical evidence showed that continuous monitoring can reduce mortality, LOS, ICU admission and RRT activation, and Smith et al. 4 found that changes in monitoring practice generally affect outcomes.

All selected review articles highlighted the lack of clinical evidence regarding optimum monitoring practice. They called for more research, particularly large clinical trials, into the clinical benefit of different monitoring frequencies, including the benefit of continuous monitoring.

One review pointed out that compliance with protocolised, intermittent-monitoring frequency is poor. There was no consensus on whether the introduction of protocolised monitoring (intermittent or continuous) improves adherence to monitoring schedules, with one article supporting this hypothesis18 and one suggesting evidence was inconclusive. 4

Lack of evidence on monitoring frequencies identified

The first paper to address the question of monitoring frequency was published by DeVita et al. 3 in 2010. The report from a consensus conference set out recommendations for identification and treatment of deteriorating patients. They highlighted that there was no evidence for the optimum monitoring frequency of patients in hospital but agreed that the assessment of vital signs should be made every 12 hours as a minimum. This review largely confirmed the lack of evidence regarding vital signs monitoring practices and timing of observations. A 2017 editorial by Smith et al. 4 stressed the need for research into best vital sign monitoring practice and required compliance with protocol. Storm-Versloot et al. 65 assessed the clinical relevance of routine vital signs observations and also indicated a lack of evidence. Other studies have also described the lack of consensus regarding necessary observation frequency but highlighted the importance of considering the nurse workload generated by monitoring demands. 74 This is also reflected in large variations in hospital policies guiding the observation of vital signs. 55

Protocolisation improves monitoring frequency

In an observational study of 804 surgical patients, Ludikhuize et al. 76 showed better detection of deteriorating patients with a regime for three sets of observations daily rather than nurse discretion. Their findings agree with several other studies showing that protocolised monitoring increased the number of observations taken and resulted in reduced adverse outcomes compared to monitoring based on clinical expertise or tradition. McGregor et al. 77 found standardised monitoring improved its reliability and was a step towards a patient-centred approach, while van Galen et al. 72 found the introduction of a protocolised EWS assessment was a good screen for major adverse events in the hospital population. In 2013, Hands et al. 12 found there was only partial adherence to the observation protocol with definite morning/evening observation peaks indicating observation rounds, although time to next observation (TTNO) decreased with increasing VitalPAC Early Warning Score (ViEWS), but not necessarily in line with the protocol. While these studies support the shift from monitoring guided by clinical expertise to protocolised monitoring, they were largely outside the scope of this review because they either did not provide enough information on the monitoring protocol used or did not investigate the impact of different intermittent monitoring schedules.

The evidence regarding the clinical benefit of an intermittent protocol is extremely scarce. This review identified only one observational study and one ward-level randomised non-blinded control trial53,62 that compared different monitoring regimes and investigated outcomes. Petersen et al. 53 showed that for low-risk patients observations made twice daily were as effective as three times per day in identifying clinical deterioration. However, they did not address the question of varying observation frequency based on patient risk. We did not find any patient-level RCTs investigating vital sign monitoring schedules.

Continuous versus intermittent monitoring

A number of recent studies have investigated devices to automate the observation of patient vital signs in an effort to gather evidence that supports widespread continuous monitoring of patients. Studies largely focus on the accuracy of these devices compared to manual observations by nursing staff,78 but some also investigated the comfort of wearable devices. 59,79–81 However, not all patients found the device acceptable and nearly a quarter of those in the study did not wear the monitoring device for the required time. 45 One study by Mestrom et al. 82 showed that automated systems can result in better compliance with protocols but results did not show an impact on outcomes in high-risk surgical patients.

The majority of studies identified in this review compared intermittent and continuous monitoring,41,57–61 exploring the potential benefits and drawbacks associated with continuous monitoring outside the ICU, and concluded that overall more research was deemed necessary. Watkinson et al. 41 found mandated electronic vital signs monitoring had no effect on adverse events or mortality in high-risk medical and surgical patients. A recent study by Areia et al. 79 identified benefits of continuous monitoring but suggested its use should be carefully considered given limitations such as excess noise, comfort, potential impact on patient mobility and direct nurse–patient contact should not be replaced.

Another potential issue associated with automated and/or continuous monitoring is the fact that, while evidence suggests that it improves vital sign documentation,82 it might reduce recognition and escalation as technology cannot replace clinical knowledge and acumen. 83 Introduction of continuous monitoring bears the risk of a reduction/limitation of nurse–patient interaction. Recognising the importance of nurse–patient interaction, which is often driven by vital sign observation, Klepstad et al. 179 recommended a maximum interval of 12 hours between observations, in line with DeVita et al. 3 and a suggestion by our stakeholders (see Chapter 7).

Our review showed that while continuous monitoring has the potential to play a major role at some point in the future, current evidence supporting the effectiveness of continuous monitoring is scarce41,45,59,66 and existing studies have identified issues with patient compliance45,59,61 and high alarm rates. 45,58,59,84,85

Compliance with existing protocols

While outside the scope of this review, we would like to highlight that any future evidence-based monitoring protocol will have to take into account that compliance with observation schedules will be limited. Hands et al. 12 found the vital signs monitoring protocol was only partially followed, although sicker patients were more likely to have overnight observations. Pimental et al. 86 reported that most deviations from protocol were attributable to various factors such as clinical judgement, the wish to avoid disturbing the patient at night and escalation might be more sensitive to nurse staffing levels than monitoring, which all need to be considered when developing new monitoring protocols. Similarly, Toth et al. 87 suggested avoiding waking low-risk patients at night might improve outcome (and patient satisfaction) as well as boost the efficiency of healthcare provision.

In summary, no evidence was found describing the best regime for intermittent monitoring, although for low-risk patients there was no support for assessing vital signs more than twice per day. There are trends towards the use of continuous monitoring and automation but benefits to support this outside of ICU are limited, while drawbacks include staff alarm fatigue and loss of nurse–patient contact. Therefore, further research into how different regimes for evidence-based intermittent vital signs monitoring is required.

Search strategy

We undertook a literature search up to 17 December 2019 to identify quantitative studies reporting the time spent by members of the nursing workforce (i.e. registered and licensed nurses, nursing assistants and equivalent roles – henceforth referred to as ‘nursing staff’) in undertaking vital signs observations, the length of time to take a set of observations or factors that influenced the time taken. The study methods were compliant with the PRISMA checklist. We searched CINAHL, MEDLINE, EMBASE and the Cochrane Library using the following search terms: vital signs; monitoring; surveillance; observation; recording; EWS; workload; time; and nursing (see Appendix 4). The search strategy was agreed by all authors and one author conducted the search.

Inclusion and exclusion criteria

We included studies that provided an evaluation of the time spent by members of the nursing workforce in gathering and recording any of the following vital signs (which are those included in NEWS). 20 These are heart (or pulse) rate; respiration rate (RR); body temperature; BP; level of consciousness; peripheral oxygen saturation (SpO₂), and the inspired gas (air or oxygen) at the time of SpO₂ measurement. 20 Because we anticipated that we would find limited evidence, we decided not to exclude studies that were not explicit in reporting which vital signs were being measured, provided that the focus appeared to be on these ‘standard’ observations. We focused on adult secondary and tertiary care ward settings, excluding studies exclusively in paediatric or maternity settings, as the necessary vital signs measurements are often different for these populations. We excluded qualitative-only studies, as our review question was quantitative in nature (i.e. time involved in vital signs activities). We retained studies that included other observations (e.g. patient weight, urine output) within the total times offered, as long as any or all of the components of NEWS were measured.

Data selection

One reviewer conducted the first screening of titles and abstracts for relevance. Two reviewers independently assessed the list of potentially relevant studies and identified studies for inclusion; any disagreements were resolved by discussion.

For quality appraisal, we focused on describing key aspects of the study likely to affect the validity of the results, including design, the methods of observation and recording, the vital signs observed and the setting and sample sizes, using a framework based on the Joanna Briggs Institute Critical Appraisal checklist for descriptive/case series. 104 Items that were not applicable to the main study question (such as confounding) were omitted from the checklist. The checklist comprises some items relating to risk of bias, and some concerning adequate reporting and statistical analysis, and poses questions to which possible answers are ‘no’, ‘yes’ and ‘unclear’. A response of ‘no’ or ‘unclear’ to any of the questions implies lower quality or else insufficient detail to judge the quality of a study. The checklist was completed by two reviewers, and one further reviewer resolved any disagreements. We did not exclude any studies based on their quality.

Data extraction

We extracted the following data from included studies: country; study design; sample size and setting; methods of vital signs measuring and recording; data collection; results; vital signs definition; mean (minutes); standard deviation (minutes).

Data analysis

Where authors reported only mean and 95% confidence intervals (CIs), we calculated the standard deviation as: (√n*(upper limit – lower limit)/t-value*2), where n is the sample size, upper limit and lower limit are those from confidence intervals. If the sample size is > 100, the 95% confidence interval is 3.92 standard errors wide. We initially considered conducting a meta-analysis, but the high heterogeneity between studies, in terms of sample sizes, settings and vital signs timing measurements, rendered this unfeasible.

Design of studies

Five publications were described as before-and-after studies,32,105–108 mostly evaluating the impact of introducing automatic electronic vital signs systems or continuous vital signs monitoring. Ten studies were classified as descriptive observational,56,109–116 and one study was a pilot study of a bedside clinical-information system. 117

Most (n = 11) used time-and-motion methodologies. 32,56,106,108,109,111,112,114–116,118 In eight studies, researchers collected data by directly observing nursing staff. 32,56,108,112,114–116,118 One study used data from a video recording of 48 continuous shifts. 109 Another plotted the number of steps that nurses took from the bedside to the computer when documenting vital signs in addition to measuring the time taken to complete and record vital signs observations using time-and-motion methodology. 106 In three studies, nurses were asked to estimate the time they had taken to complete vital signs observations. 107,110,111 In one of these studies, nurses noted the time taken to complete vital signs for each patient and calculated the total time spent on this activity. 107 In one time-sampling study, nurses were asked to report the activity they were engaged in when a personal digital assistant they were carrying vibrated at random times during the shift. 111 As indicated in Table 1, seven studies did not provide a clear description of how time to complete a set of vital signs was assessed; among these, three studies did not give any meaningful detail about how the time to complete a set of vital sign observations had been collected. 105,113,117 Among these, one study was available as abstract only. 113

Setting of studies

Coverage of settings ranged from 1 to 36 hospital wards of various types, namely: acute surgical/medical general wards, ICU, an emergency department, a cardiovascular unit, a trauma ward and a radiology unit. One study did not specify the type of hospital or which wards were included. 113

Methods of vital signs measurement

The 16 studies generally described taking vital signs observations as the time to measure/collect vital signs and the time to record/document vital signs. However, the specific set of vital signs chosen for measurement differed by study, which inevitably affected the overall time taken. Some included seven different physiological signs in a complete vital-sign set,32 while others included only four. 113 All studies reporting physiological signs measured temperature, heart rate (HR), RR and BP. 32,56,105,106,108,114–118 Some studies offered no specific description of the vital signs collected. 107,109,111,112 Also, studies did not always specify whether only complete sets of vital signs had been included for analysis. A number of studies included additional observational and assessment activities in the time taken to complete vital signs, such as completing fluid balance charts, checking an infusion pump and weighing the patient. 105,117 The measurement tools did not vary substantially across the years.

Methods of time recording

In general, the time involved in vital signs recording and documenting was reported in two different ways. A number of studies reported a mean time for taking a vital-sign set, mean time to record vital signs on charts or both. 32,56,105,107,108,110,113,114,116,118 Other studies reported the amount of time that nursing staff spent taking vital signs and/or recording them over a shift, per hour, per patient or over an amount of time (e.g. over 44.5 hours). 106,109,111,112,115,117 None attempted to disentangle the time taken to collect or document each vital sign (i.e. BP or oxygen saturations or RR) or EWS value separately.

Studies reporting mean times

Ten studies provided a total of 12 samples to estimate mean times for taking and/or recording vital signs (Table 2).

When studies investigated the time involved in measuring and documenting vital signs using pen-and-paper methods, mean times ranged from 3.58 minutes32 to 5.80 minutes. 56,116 When documentation was performed using electronic systems, mean times for measurement and documentation were lower, at 2.50 minutes in both studies. 32,105 We did not find any differences in mean times involved in vital signs measuring and documenting that could be attributed to different clinical settings, nor to different nursing personnel (i.e. RNs vs. nurse assistants). Differences in mean times appeared to be related to the combination of vital signs included within the recorded data set; the method(s) of vital signs measurement; the timing of entry into the patient record; the method of recording; the calculation of EWS values. In addition to mean times being variable between studies, it was clear that there was considerable variation in the time taken within some studies, as standard deviations were high.

All studies where reported times focused only on vital signs documentation involved continuous patient monitoring and focused on electronic systems of data transfer to the patient record. Electronic systems where vital signs were entered at the bedside seemed to be associated with reduced time. The mean times to document vital signs observations electronically at the bedside ranged from 0.90107 to 1.27 minutes,113 and the mean time for documenting vital signs outside the bed space was between 1.47 minutes113 and 2.02 minutes. 107 One study focused on the mean time difference between the time vital signs were taken and when the data were recorded in the patient’s record, and found that when staff were recording data on a vital signs monitor at the bedside and transferring them to a PC tablet the mean difference was 0.59 minutes; when vital signs observations were transcribed from handwritten notes to patient notes the mean difference was 1.24 minutes; and for handwritten observations to be transferred to a computer on wheels outside the bed space, the latency time was 9.15 minutes. 114

Studies which do not provide mean time estimates

Four studies reported the time involved in collecting and recording vital signs by hour or by nursing shift. 109,111,112,115 Hoi et al. found it took 144 minutes of total nursing time per day for a patient in the most highly acute and dependent category, where assistance with all care needs and multiple treatments were often required. 112 Adomat and Hicks showed that charting observations and record-keeping in two units ranged from a mean average of 5.44 to 10.78 minutes per hour in an HDU and from 10.66 to 17.43 min/hour in ICU. 109 Hendrich et al. showed that vital signs took up 7.2% of nursing time or 30.9 minutes in a 10-hour shift. 111 According to Yeung et al., the total time spent by each nurse performing vital signs observations was on average 12 minutes, albeit the unit of observation was not reported (i.e. per hour, per shift or per patient). 115

Erb and Coble reported vital signs documentation on a new automated system, where nurses record all vital signs at the bedside using a monitor that measures BP, pulse rate and temperature. 117 Vital signs data are stored on a computer at the nurse station unit, and the bedside unit and nurse station unit are connected directly. The authors compared this system to an older manual system, and found that it offered an overall mean time saving per nurse per shift of 11.86 minutes. 117 Fuller et al. reported that the time taken to document vital signs in a computer was 7 minutes per 10 patients,106 suggesting a mean time below that of any study reporting a per patient time.

Limitations

In appraising studies, we applied a checklist based on the Joanna Briggs Institute Critical Appraisal checklist for descriptive/case series,104 and all studies were of low quality. We highlighted key omissions of important details, for example which vital signs were included and how they were measured and recorded, or how nurses and/or patients were sampled. The results of the review illustrate the variety of factors that may influence the time estimates derived from the studies and demonstrate where information is missing. While we used a reproducible search strategy searching MEDLINE, CINAHL, EMBASE and the Cochrane Library, it is possible that we did not identify studies indexed elsewhere and not cited by the included studies. It seems unlikely that these exist in sufficient quantity to substantively change our conclusions.

Conclusions

There is currently insufficient robust evidence around the time nurses require to perform vital signs activities. To increase consistency and impact, we propose a framework for future studies to adopt measuring the time and workload involved in vital signs observations that includes (1) the methods of measurements, (2) the timing of entry of measured data into the charting system and (3) the approach to calculation of EWS values. This categorisation would be suitable for vital signs measured on an individual patient basis, or on the basis of vital signs ‘rounds’, where the mean time for each patient on the round could be calculated.

Recommendations for vital signs observations need to consider the workload involved and include consideration of the potential opportunity costs if observations are given higher priority at the expense of other aspects of nursing work. Vital signs observations are considered to be a fundamental aspect of nursing work and key to ensure early detection of patient deterioration. The lack of robust evidence means that those making clinical and managerial decisions about resource allocations, including workload planning around vital signs observations, must make these in the face of considerable uncertainty. Uncertainty means that the workload associated with changes to the frequency of observations associated with EWSs is unknown. At a system level, the costs from changes in policy such as the shift from NEWS to NEWS2, which increased the frequency of observations for some patient groups, are unquantifiable. On a ward level, the feasibility of implementing such changes and integrating them into existing work is uncertain. On another hand, workload reductions associated with the introduction of technology that facilitates continuous monitoring, which might reduce requirement for nurses to take vital signs observations, are equally uncertain. Measuring patients’ vital signs at times that are appropriate is key to avoiding patient deterioration and adverse outcomes. In the interest of patient safety, further research that uses vital signs and patient objective data aiming to define the optimal frequency of vital signs observations should be conducted.

Data collection

The data in the retrospective part of this study were vital signs measurements from routine nurse observations and their associated EWS along with subsequent outcomes for all adult patients.

In both study hospitals, patient vital signs data were taken from adult general medical and surgical wards where they were recorded electronically at the bedside. For OUH, the source system was SEND,132 whereas for PHU it was System C ‘CareFlow Vitals’133 for inpatient observations and Servelec Oceano134 for observations taken in the emergency department.

The monitoring systems demand that each observation set contains all parameters required to calculate the NEWS or its successor NEWS2. The structure of NEWS2 differs from NEWS by the inclusion of a specific oxygen-saturation scale for use in patients at risk of type II respiratory failure and the addition of ‘new confusion’ (C) to the AVPU scale. NEWS or NEWS2 values used in our analyses were those calculated and displayed by the monitoring software in accordance with the original guidance from the RCP. 1,20 For over 95% of patients, NEWS and NEWS2 are effectively identical, and we therefore did not distinguish between the two and used whichever was exported from the monitoring systems.

The EWS value also determines the time to the next vital signs observation (TTNO), that is when the patient should next be monitored. Once a patient’s vital sign observations have been entered by the hospital staff, the monitoring systems automatically display the EWS value and when the next set of observation should be taken, based on the hospital’s clinical escalation protocol.

With regard to the frequency of monitoring, slightly different escalation protocols were in operation at PHU during the study period (Table 6). The original version of NEWS was deployed at PHU in 2011 and had the schedule shown in the first column. It was updated in November 2016 to increase the frequency of observation for patients with a NEWS of 6. Finally, it was updated in February 2019 with the implementation of NEWS2.

In OUH, the SEND EWS135 was in use throughout the study period. Minimum observation frequencies were 720 minutes for patients with a SEND score of 0, 240 minutes for a score of 1 or 2 and 60 minutes for any higher score.

Portsmouth data extraction

In PHU, the extraction was performed by the Research Data Management Team. Extraction proceeded by identifying adult admissions from PAS for the period of interest for patients who had not registered with the NHS opt-out. Extraction of records from other sources was then constrained to link to this cohort of retained admissions. Real-world and source-system patient identifiers were removed and replaced with a patient study identifier.

There were two extractions. The first covered admissions from January 2014 to December 2018. A second extraction covered admissions from January 2019 to August 2020 (357,852 admissions and 3,511,486 observations).

Oxford data extraction

In OUH, data extraction was performed by the local Clinical Data Warehouse team in collaboration with two members of the study team (who hold honorary contracts with the Trust). Data for patients admitted and discharged between 2017 and 2020 were extracted. Patients registered with the NHS opt-out were first excluded, followed by admissions that met the study exclusion criteria. We extracted at total of 614,930 admissions from 259,047 unique patients from OUH.

Descriptive analysis and outcomes

Patient demographics, admission characteristics, observation frequency, average NEWS values and patient outcomes were compared across the three patient cohorts (development, internal validation and external validation). Descriptive statistics were calculated including counts, means (±SD), medians (IQR, Q1–Q3) and proportions. We further estimated outcome rates for each of the patient subgroups described above in both the PHU and OUH data sets.

We considered the following adverse events as outcomes in our analysis and modelling:

• DEATH: in-hospital death

• COMB: combined outcome of either in-hospital death or cardiac arrest or unexpected admission to a high-care ward, whichever comes first

• NEWS7: any of the events included in the combined outcome or a NEWS value greater than 6, and lastly

• NEWSINC: combined outcome or NEWS > 6 or a NEWS increase of 2 or more.

The numbers of admissions to each of six admission categories were calculated for every month within the study period. The six admission categories were emergency and elective admissions to medical, surgical and ‘other’ admission specialties, respectively.

Observation patterns

We calculated median NEWS, LOS and observation frequencies for each of the patient subgroups described above in both the PHU and OUH data sets.

For the analysis of observation patterns and protocol compliance, observations taken within 1 minute were considered duplicated and only the second was used.

Each vital signs set was allocated to an hour of the day, with the time for each set being labelled as the current hour at the moment of the timestamp, for example timestamps between 7:00 and 7:59 were labelled as 7:00. We calculated the number of vital signs sets collected each hour, as a proportion of the total number of vital signs sets collected in the 24-hour period, and stratified them by NEWS value grouped from ‘0’ to ‘9 or more’. We also calculated the number of vital signs sets collected each hour, expressed as a proportion of the total collected in the day, stratified by day of the week. We plotted the mean TTNO for each hour of the day, categorised by NEWS for each year in the study period.

We defined vital signs recordings as ‘late’ if they were in the latest 20% of TTNO for a given NEWS value and as ‘early’ if they were in the bottom 20% (Figure 7). The middle 60% were regarded as ‘on-time’. Stratifying compliance by NEWS allows assessment of the adherence with monitoring protocols independent of the increased frequency demanded for increases in severity of illness, that is, NEWS. The range of TTNO that was classified as ‘late’ and ‘early’ for each NEWS value is given in Table 7. We calculated the proportion of early, late and on-time observations for each month between February 2014 and June 2019 (January 2014 removed to avoid artefacts from data extraction). We further categorised the proportion of early, late and on-time observations by low (NEWS 0–2), medium (NEWS 3–5) and high (NEWS ≥ 6) severity of illness for each year within the study period.

FIGURE 7.

Distribution of observed time intervals between NEWS observations for three severity groups [low (NEWS 0–2), medium (3–5) and high risk (≥ 6)]. For each NEWS the earliest 20% of observation intervals were classified as ‘early’ and the latest 20% were classified as ‘late’.

Deterioration over time

To better understand changes in NEWS over the course of a patient’s hospital admission, we calculated the mean NEWS and average risk of adverse events (for outcome definition see Descriptive analysis and outcomes) for the first 10 days after hospital admission (day 0). We excluded patients without an observation in the last 24 hours of their stay who are presumed not to be monitored because they are on an end-of-life pathway. We then calculated a mean NEWS and mean NEWS component for individual vital sign for each 24-hour interval leading up to one of five outcomes: in-hospital death, combined outcome, NEWS7, NEWSINC and discharge alive.

Using the area under the receiver operating characteristics curve (AUROC), we evaluated the ability of individual vital signs to discriminate any of the adverse outcomes. We calculated AUROCs for the NEWS components (not vital sign measurements) of temperature, systolic BP, HR, breathing rate and oxygen saturation to account for non-linear relationships with outcome.

Workload associated with alternative monitoring regimes

Data from the retrospective and prospective observational studies were used to estimate the workload and staffing costs associated with alternative observation schedules.

Patient numbers, and their distribution across EWS categories, were used to model the expected number of vital signs observations to be undertaken under two candidate observation protocols. Combining the expected number of observations and the average time taken to complete an observation, we calculated the nursing time required for vital signs observations and the nursing cost per patient day under each alternative protocol. Expected numbers of vital signs observations were estimated by sampling patients on wards each day and calculating the daily number of observations associated with the vital signs observed in the protocol. To estimate the expected number of observations per day, we selected a random time (hour) each day and identified which patients were present, according to the patient administration system. For each time point, we extracted each patient’s most recent EWS and mapped this to the observation frequency (in minutes) in the protocol. We calculated the expected number of observations for each patient by dividing 1440 (the number of minutes in a day) by the expected observation frequency. Although an individual’s vital signs might change throughout the day (leading to a change in frequency), daily sampling for a ward meant that individual changes averaged out. We assessed the impact of different strategies for sampling within the day (e.g. at fixed hours of the day) and found that there was minimal impact of the daily estimates of numbers of observations.

For the base-case analysis, we considered the efficiency and patient risk of each protocol under an assumption that no scheduled vital signs observations were missed. Empirical evidence suggests this is not the case. 12,101,141

Cost associated with new monitoring regimes

The nurse time for undertaking vital signs observations was derived from our observational study. This included estimates of the time taken (5 minutes) per patient to complete an observation with an additional overhead to allow for preparation of equipment, hand-washing and documentation (see Chapter 4). Staffing costs were estimated by applying representative unit costs for substantive RN and healthcare support workers. 142 The cost estimates were weighted to reflect the proportion of vital signs observations undertaken by the respective grades of nurse staff, based on data from our observational study in which approximately 50% (355/715) of observations were conducted by RNs.

Incremental costs associated with different protocols were compared and used to determine the benefit [in terms of lives saved and quality-adjusted life-years (QALYs)] that would need to accrue from a particular regime for it to be judged as cost-effective, using a range of thresholds including average gross domestic product (GDP) per QALY143 (approximate £40,000) and a lower threshold for the NHS of £10,000 per QALY,144 reflecting opportunity costs associated with adopting new treatments in a resource-constrained system.

Cost-effectiveness analysis

We further evaluated the cost-effectiveness of each monitoring protocol with higher resource demand than the PHU reference protocol by calculating the cost of each life saved, based on estimates of the number and cost of QALYs associated with change in mortality.

To estimate the expected QALYs associated with changes in mortality we used the discounted and quality-adjusted life expectancy (DANQALE) tariffs reported by Meacock et al. 145 We took the weighted average tariff based on the age and sex profile of the patients who died in the development data set (patients admitted to PHU for whom the reference NEWS protocols apply) and assumed that each death averted would expect to gain associated QALYs. The estimated mean DANQALE for patients who died in the PHU development data set was 6.28 (95% empirical CI from 2000 bootstrap samples 6.04 to 6.54). As a sensitivity analysis we considered QALYs per life saved assuming 50% of the point estimate (3.1 QALYs per life saved) and at the point estimate + 50% (9.3 QALY per life saved).

Using these QALY per life gained estimates we calculated the proportion of deaths that would have to be prevented in order for any new regime to be cost-effective using three thresholds: £10,000 per QALY, £20,000 per QALY and £30,000 per QALY. This range spans recognised cost-effectiveness thresholds from the national per capita GDP143 (£29,147 in 2020) to the level considered by NICE as representing ‘exceptional value for money’ for new drugs (£10,000),144 with the mid estimate approximating to the threshold recommended for UK health policy purposes146 (£12,936 in 2008 approximates to £17,705 in 2021).

We determined a threshold for a ‘plausible’ impact on death which would therefore allow us to identify a regime as potentially cost-effective and thus worthy of further investigation. Large-scale studies estimate that between 3% and 6% of inpatient deaths are preventable. 147 In the UK, it has been estimated that approximately one-third of the 5.2% preventable deaths are attributable to failures in clinical monitoring. 92 This gives an absolute upper bound of plausible effect from improvements in monitoring of 2% of deaths averted. However, as alternative regimes are likely to give only a marginal improvement on the current NEWS-based regimes, this 2% threshold is unrealistic and so we set 1% as a threshold for considering any regime as potentially cost-effective (recognising that this is still a very large effect for such an intervention). Scenarios involving either equal or reduced costs are assumed potentially cost-effective and worth further consideration as they potentially improve outcomes at lower cost.

Summary of different data sets

After exclusions, the development data set comprised 174,290 hospital admissions to PHU from January 2014 to June 2017 aged > 16 years with at least one complete NEWS observation. The internal validation set included 98,109 admissions to PHU from July 2017 to June 2019, while the external validation set included 139,891 admissions to OUH over the same period between July 2017 and June 2019.

For both PHU data sets, approximately 53% of patients were female and the median age was 67 and 68 (development and internal validation data). Fifty-one per cent of patients admitted to OUH were female with a median age of 65. Just over 40% of admissions and observations were from patients over 80 years old.

The majority (71.2–75.3%) of admissions in all three data sets were emergency admissions and more than 61% were admitted to medical specialties. The proportion of admissions to surgical specialties varied from 29.7% at OUH to 35% in the PHU development data set. Around 1% of admissions were to ‘elderly medicine’. The proportion of patients who had a theatre visit was comparable for both validation sets (18.1% and 19.4%) but slightly higher in the development data (23%). While the exact reason for this drop remains unknown, it could be explained by the fact that an increasing proportion of surgeries were performed in a nearby private hospital (Spire Portsmouth Hospital). Only a small fraction of patients received a blood transfusion (< 0.5%) or chemotherapy (< 0.8%) during their stay, after exclusion of admissions who only had observations on the day they received treatment.

Median LOS and mortality rate were lower at Oxford than in either PHU data set. OUH LOS was 1.6 days with a mortality rate of 2.6%, whereas LOS of PHU was 2.4 and 2.7 days (development and internal validation data respectively) with a mortality rate of almost 4%. The data set from OUH included data from four different hospitals, one of which is an orthopaedic hospital, potentially explaining the lower LOS and mortality. Outcome rates for NEWS7 (combined outcome of first occurrence of death, unplanned intensive care admission, or cardiac arrest in hospital, or recorded NEWS of 7 or more) and NEWSINC (combined outcome of first occurrence of death, unplanned intensive care admission, or cardiac arrest in hospital, or recorded NEWS of 7 or more, or an increase in NEWS of 2 or more) are more consistent across data sets. Even though NEWS7 and NEWSINC are additive outcomes also including death, unplanned ICU admission and cardiac arrest, occurrences of high NEWS or NEWS increase are much more frequent, dominating these outcomes. NEWS7 and NEWSINC are therefore a better representation of deterioration, also capturing events when other outcomes (death, ICU admission, cardiac arrest) were avoided and were more consistent across patient populations.

Table 8 provides details on patient demographics and outcomes for development and validation data sets.

FIGURE 8.

Age distribution of admission to PHU in the development data set, split by sex.

Total admission numbers to PHU were fairly constant throughout the study period, with a slow increase in emergency admissions to medical specialties (Figure 9). In comparison, surgical admissions decreased slightly over time. Combined monthly admissions to the four data sets included in the OUH data set were about 50% higher than admissions to PHU (see Figure 9). The increase in admissions over the study period was also driven by an increase in emergency admissions to medical specialties – the blue dashed line in the figures.

FIGURE 9.

Time series of admissions per month to different specialties. Top: admissions to PHU between January 2014 and June 2019; bottom: admissions to OUH between January 2017 and June 2019.

There was a noticeable difference in the distribution of NEWS values in data sets collected at PHU and OUH (Figure 10). While in both data sets from PHU, used for model development and internal validation, NEWS 0 was the most common occurrence, NEWS 1 was most prominent in the OUH data set, indicating a clear difference in the patient population. These differences were consistent across all patient subgroups.

FIGURE 10.

Distribution of NEWS observations in the development, internal validation and external validation data sets, expressed as a percentage of the total number of observations in each data set.

For the majority of admissions in all data sets, LOS was shorter than 72 hours. At OUH (external validation data), more patients were discharged in the first 24 hours compared to PHU (Figure 11).

FIGURE 11.

Distribution of admission LOS in the development, internal validation and external validation data sets, expressed as a percentage of the total number in each data set.

Patient subgroups

In addition to the overall patient population, we also considered specific subgroups of patients in order to ascertain whether their characteristics might make them candidates for different monitoring regimes.

Outcome rates varied significantly between different patient subgroups. Table 9 shows outcome rates for the PHU development data set and key differences in outcome rates in either validation data set are discussed below. As expected, elective admissions had lower adverse outcome rates compared to emergency admissions across all outcome measures. For example, mortality was 0.3% in elective admissions while it was 5.3% for emergency admissions. Admissions without an observation in the last 24 hours had the highest outcome rates compared to any other patient groups (mortality: 34.9%). There are two possible reasons for missing observations at the end of an admission: (1) system failure, such as hardware or software malfunction, or (2) patients were put on an end-of-life pathway, when vital signs are not monitored. As system failure is a relatively rare event, admissions without observation in the last 24 hours were therefore assumed to be on an end-of-life pathway. This was supported by a 43% 30-day mortality rate in this patient group, indicating that these patients die either in hospital or in the community. Admissions to surgical specialties had lower outcome rates compared to admissions to medical specialties, with highest outcomes in the cohort admitted under specialty code ‘elderly medicine’ (mortality 13.1%). Patients receiving blood transfusion and chemotherapy had generally lower than average outcome rates, with mortality rates < 3.3%.

Outcome rates in the internal validation set (detailed data provided in Appendix 10) were comparable to the development data set. The most notable change was a reduction in mortality in admissions to elderly medicine, that is, admission with associated admission specialty code ‘430’ (from 13.1% to 9.2%) as well as in admissions of patients aged 80 or older, admitted to any specialty (9.7–8.3%).

In comparison, the external validation data set (detailed data provided in Appendix 10) had overall lower rates of mortality and combined outcome, but when including NEWS values of 7 or more, or considering an increase in NEWS by 2 or more points, rates were similar (DEATH, COMB, NEWS7 and NEWSINC respectively) (see Table 8). In particular, mortality was lower in emergency admissions (3.5%), admissions to medical specialties (3.6%) and ‘elderly medicine’ (4.0%), those receiving blood transfusions (0.8%) and admissions that did not include a visit to the operating theatre (3.0%). In contrast, mortality rates were higher in admissions receiving chemotherapy (3.8%).

Several patient subgroups (elective admissions, those younger than 30 years old, admissions to ‘elderly medicine’ and admissions receiving either blood transfusions or chemotherapy) had very low admission and/or outcome rates (see Table 9), limiting subsequent robust analysis. While these patients were included in model development (see Model development) as they contribute to the overall monitoring and hence nursing workload, findings from the descriptive analysis for these patient subgroups are omitted from detailed discussion. Full results for these patient subgroups have been provided in Appendix 10. Note that admissions to elderly medicine are included in the medical cohort.

Observation frequency in patient subgroups

In the PHU development data set, observation frequency, defined as median number of observations per day, varied across subgroups of patients even though median NEWS was comparable (Table 10). Patients without an observation in the last 24 hours of their stay had a higher median NEWS compared to average in the entire cohort in all three data sets, when considering their entire hospital admission period. That means, on average, these patients had a higher severity of illness, supporting the assumption that a proportion of these patients might have been on an end-of-life pathway. In the external validation data set, admissions to all medical specialties and elderly medicine also showed an elevated median NEWS (data not shown).

For each admission, we calculated the average number of observations per day (obs/day) and then calculated the median observation rate for each patient subgroup. In both PHU data sets, the average observation frequency was 4.1 obs/day; it was slightly higher (4.5 obs/day) in the OUH data set (data not shown). Lowest observation frequencies were found in patients without observation in the last 24 hours, with an average of 2.4 obs/day over the entire admission, reflecting the fact that patients on an end-of-life pathway are not monitored routinely. Other patient groups with reduced observation rates were those over 80, who make up almost 40% of admissions and observations (see Figure 8). Reduced observation rates are potentially driven by significantly longer average observation periods as well as a high proportion of patients on an end-of-life pathway. In contrast, the proportion of elderly patients in the OUH data set was smaller (around 30% of observations for patients over 80), with an observation rate of 4.3 obs/day and median LOS of 2 days, significantly closer to the average for the entire external validation cohort (data not shown).

In both PHU data sets, where observations of admissions with an operating-theatre visit could be split into pre- and postoperative periods, the observation frequency increased after surgery, probably due to increased monitoring in the immediate postoperative recovery period. After excluding observations taken in the first 48 hours after surgery, postoperative observation rates fell to preoperative levels (data not shown).

Variation in number of observations within 24-hour period

When vital signs sets were stratified by NEWS value and collection hour and expressed as a proportion of the total number of observations collected, the pattern of vital sign recording at PHU exhibited peaks of activity at 06:00, 11:00 and 16:00, and between 20:00 and 23:00 (Figure 12). However, peaks became less distinct from 2016 onwards, showing a slightly more even distribution of observations across a 24-hour period. This trend was sustained throughout the data collection period included in the validation data set and observed in all subgroups included in the detailed analysis (see Patient subgroups). There was no significant difference between weekdays and weekends.

FIGURE 12.

Recorded vital signs observation sets, stratified by NEWS value and collection hour and expressed as a proportion of the total number collected during the 24-hour period for each year included in the two PHU data sets. The development data set covers January 2014 to June 2017, the internal validation data set covers July 2017 to June 2019.

Observation patterns between the two hospitals were remarkably similar given the difference in observation schedules (see Data collection), with peak activity at OUH at 06:00, 10:00, 16:00 and 20:00 (Figure 13). Overall, a smaller proportion of observations were taken at night (23:00–04:00) at OUH, compared to PHU (up 1%).

FIGURE 13.

Recorded vital signs observation sets, stratified by NEWS value and collection hour and expressed as a proportion of the total number collected during the 24-hour period for each year in the OUH external validation data sets.

Similarly, we analysed daily patterns of the mean TTNO for each NEWS value. For data collected at PHU, this showed less variation across a 24-hour period from 2016 onwards, compared to data from 2014 to 2015 (Figure 14). From 2017 onwards, TTNO patterns show groupings that reflect the escalation protocols deployed in Vitalpac.

FIGURE 14.

Mean TTNO, stratified by NEWS value and collection hour for each year included in the two PHU data sets. The development data set covers January 2014 to June 2017, the internal validation data set covers July 2017 to June 2019.

Figure 15 shows slightly increased variability in TTNO over the course of a day at OUH compared to PHU, with a rapid increase in mean TTNO between 20:00 and 24:00. This is presumably because observations were not taken overnight for low- to medium-risk patients, regardless of NEWS value.

FIGURE 15.

Mean TTNO, stratified by NEWS value and collection hour for each year included in the OUH external validation data set.

Monitoring compliance

We classified the timing of observations as follows: the shortest 20% of observation intervals for each NEWS value were classified as ‘early’, the longest 20% of observation intervals were classified as ‘late’ with the remainder being labelled ‘on time’. For PHU data, there was a clear and sustained improvement in compliance from May 2015, with an increase in the proportion of ‘on time’ NEWS observations and simultaneous decrease of ‘late’ observations (Figure 16).

FIGURE 16.

Monthly time series of the proportion of observations that were taken on time, early or late for both PHU data sets, for model development and internal validation.

When observations are categorised by NEWS value, the proportion of ‘late’ observations was smaller for severely ill patients (NEWS of 6 or greater) than for low-risk patients (with a NEWS between 0 and 2) from 2016 onwards (Figure 17). In 2019, there was a noticeable decrease in the proportion of observations that were classified as ‘late’, from 17.9% in 2018 to 12.2% in 2019, for patients with a NEWS ≤ 6. A new monitoring protocol was introduced in February 2019 to accompany the roll-out of NEWS2. While the new monitoring protocol lowered the expected observation interval from 2 hours to 1 hour for NEWS 6, from 4 hours to 1 hour for NEWS 5, and from 1 hour to 30 minutes for NEWS ≥ 7, it also endorsed the use of customised observation schedules for certain groups of patients where authorised by an appropriate clinician and recorded in the patient notes. The two main uses of the customised scheduling were for elderly patients who were medically fit for discharge but awaiting a package of care, and some patients for whom it was normal to receive oxygen therapy. For both these groups the usual observation schedule was lengthened where approved.

FIGURE 17.

Proportion of ‘on time’, ‘early’ and ‘late’ observations categorised by risk groups – low (NEWS 0–2), medium (NEWS 3–5) and high risk (NEWS 6 or above) – for each year within the PHU data sets. The development data set covers the period from January 2014 to June 2017, the internal validation data set includes data from July 2017 to June 2019.

The proportion of ‘on time’ observations in the external validation data set collected at OUH was around 60% throughout the study period. From August 2018, there was a small decrease in observations classified as ‘late’ (data not shown). Note that OUH did not use NEWS during the study period and that the monitoring protocol included only three observation frequencies (12 hourly, 4 hourly and 1 hourly observations). In contrast to PHU, there was no difference in the distribution of ‘early’, ‘on time’ and ‘late’ observations with patients’ risk (Figure 18), presumably because of less variable observation frequencies dictated by the OUH monitoring protocol.

FIGURE 18.

Proportion of ‘on time’, ‘early’ and ‘late’ observations categorised by risk groups – low (NEWS 0–2), medium (NEWS 3–5) and high risk (NEWS 6 or above) – for each year within the OUH external validation data set.

Model calibration

Model calibration is a measure of the accuracy of risk estimates, that is, how they compare to observed risk. Details on the different metrics for judging model performance are provided in Model development and validation. Figure 25 shows model calibration in the internal validation data set, with relatively consistent performance for NEWS values up to and including 14. This cut-off was chosen to avoid bias in the results for very high NEWS values where observation numbers are small. In addition, patients with a NEWS of 15 or higher are likely to be so severely ill that they are admitted to ICU and/or continuously monitored, that is, intermittent monitoring regimes that are the focus of this investigation do not apply. For in-hospital death, calibration suggested a combination of slight overestimation, with intercepts ranging from −0.09 to −0.27, and slightly extreme risk prediction, with slopes ranging from 0.92 to 0.94 (Table 12). Model performance improved with increasing distance from NEWS observation, with similar results for the combined outcome. For outcome NEWS7 model performance was very consistent across the three time points at which calibration was assessed. The model had a slight tendency to underestimate risk for outcome NEWSINC.

FIGURE 25.

Model calibration for outcomes DEATH, COMB, NEWS7 and NEWSINC at 6, 12 and 24 hours after a NEWS observation for all NEWS ≤ 14 in the internal validation data sets, collected at PHU.

Model calibration for emergency admissions, admissions to medical specialties and admissions without theatre visit and patients aged 80 and older was broadly comparable to the entire internal validation cohort (Table 13). Considering all admissions with an observation in the last 24 hours, the intercept was ≥ 0.30 for all outcomes, demonstrating an expected substantial underestimation of average risk (i.e. calibration-in-the-large). Calibration slopes greater than 1 indicated that the model tended to produce risk estimates for DEATH and COMB that were too moderate. This was particularly noticeable in the surgical cohort.

Model performance decreased in the external validation set, consistently overestimating risk due to the lower outcome rates observed in this cohort (Figure 26, Table 14). Intercepts indicate that calibration-in-the-large is poorer compared to the internal validation, particularly for outcomes DEATH and COMB, which could be explained by the difference in outcome rates between the two sites. For the outcomes NEWS7 and NEWSINC, calibration coefficients < 1 suggest that risk estimates are also too extreme.

FIGURE 26.

Model calibration for outcomes DEATH, COMB, NEWS7 and NEWSINC at 6, 12 and 24 hours after a NEWS observation for all NEWS ≤ 14 in the external validation data sets, collected at OUH.

Model discrimination

Discrimination evaluates the ability of a model to distinguish between patients who have an outcome and those who do not. The ability of our survival models to discriminate outcome was very good for outcomes DEATH, COMB and NEWS7, with AUROCs > 0.890 (Table 15). For NEWSINC, AUROCs were significantly lower, with AUROCs ≤ 0.671 in the internal validation data and ≤ 0.599 in the external validation data. Unexpectedly, discrimination was slightly higher in OUH data than PHU data for outcomes DEATH and COMB, suggesting better model discrimination in the external data set. This was consistent with findings from previous studies. 137,148 Discrimination generally decreased the more time passed between NEWS observation and outcome.

Summary of discussion

• Patients continue to be flabbergasted that the interval timing of vital signs observations has no evidence base: ‘I’m surprised there isn’t the equivalent of a minimum service level’.

• There was an overwhelming response from patients that if it is necessary to disturb them at 2 a.m., they are OK with that: ‘Taking blood pressure is just as important as a good night’s sleep’, but they were also clear that observations should be done because they are needed.

• Patients reported they were largely unaware of time passing when in hospital.

• For patients, the knowledge that someone will check on them (even to just say ‘hi’) was just as important as actually having their vital signs taken/recorded.

• Staff also valued simple conversation and thought that this could be a trigger to taking new measures outside of a protocolised schedule ‘if a patient tells you they feel rough’.

• Both patients and staff mentioned that individuality was likely to be important to a patient’s need for monitoring (baseline pre-hospital values of low HR or high BP, etc.), as well as reason for admission, with different monitoring needs for post-surgery versus mental health patients for instance.

• ‘Individuality’ should include the physical location of patients – those patients in single rooms may need closer observation/monitoring than those in multiple-bay spaces where it became obvious that patients are (informally) part of the escalation practice: ‘each patient looks after the others’.

• It was acknowledged that although patients should each have their own schedule for monitoring, the reality tended to be that the timing of monitoring rounds is somewhat standardised and linked to system convenience rather than individual patient need (10 p.m., 2–3 a.m., 4–6 a.m., etc.).

• Patients confirmed that they associated more frequent care with higher levels of sickness and would ‘consider themselves lucky’ if they were being monitored every 12 hours compared to a patient in their bay on a 4-hourly monitoring schedule, although most patients couldn’t remember anyone having their vital signs taken anyway.

• Staff (more than patients) were concerned about missing deterioration (although this was linked to ‘missing the required observations’ and being investigated for not following guidelines), although some patients felt it was ‘better to be safe than sorry’.

• Patients could not remember ever being told why they were having their vital signs taken and would have liked more information at the time.

Many of these points were also raised by the second group (see below) so, for brevity, we will consider them together.

Summary of discussion

Many points raised were similar to those discussed with the Oxford (post-ICU) patient group.

From both groups it was clear that communication and common sense should prevail when considering when to take vital signs observations. Both groups also welcomed the contact time with the clinical staff and expressed the view that the observation-taking was more than simply checking physiological health status. The concept of ‘individualising’ the monitoring schedule to each patient was addressed in both conversations and in both groups’ patients were happy to be disturbed when resting if this was clinically necessary. The patients who had not been admitted to intensive care were more strongly vocal in being allowed to sleep rather than be woken for vital signs.

The key difference between the groups was that the post-ICU group (Oxford) was largely unaware of time passing, whereas the non-ICU group (Southampton) was much more aware of boredom and the need to punctuate the day with periodic moments.

All patients in both groups expressed surprise that the evidence for 2-hour/4-hour/12-hour observation frequency, etc. is limited.

Night-time observations: waking patients to carry out observations was felt to be inappropriate in most situations – although the group universally agreed that when this was likely to be absolutely necessary they would be happy to be woken up if they had been told in advance that they would be (‘If you tell me, I’m not going to mind’). It was also suggested that patients should be grouped by their observation needs as, although disturbances from personal observations were accepted, other people having their observations done was highly disturbing, and much less well tolerated, especially overnight.

Timing: there was unanimous agreement that observations should not be taken during ‘care’ times – especially mealtimes. It was acknowledged that a rigid adherence to a schedule was unlikely to be appropriate in the majority of cases. A 12-hourly schedule should not include routine observations set between 10 p.m. and 6 a.m. Most patients liked the predictability of observation times (‘It breaks the day up’; ‘Being in hospital is so boring’). All patients felt that staff should have good awareness of previous observations (‘you get woken up at the end of the night shift, then the day staff take them again’; ‘you can get a bit fed up about constantly having vitals done – can start to feel “oh no, not again!” …’). There was general agreement that better synchronisation between ward activities and observation rounds would benefit both staff and patients.

Individualised care: it was considered appropriate to monitor fewer individual vital signs for patients where deterioration is unlikely (orthopaedic patients with no additional comorbidities were proposed as a group where simple HR or BP might suffice) and there was broad support for patients admitted to hospital for different reasons to have different schedules of observations frequency and/or to utilise HCA resources more for patients who are largely ‘well’.

Self-directed care: there was widespread support for patients being able to take their own observations if they felt ill, although it was also acknowledged that some patients might lie about the results, or take observations too frequently. Some patients expressed that they would like to be consulted about the frequency of their observation schedule (‘I would like to say if I want to be disturbed or not’). Some patients wanted to know how they could take steps to improve their vital signs to speed up their recovery to be able to go home.

Circadian rhythm: a few patients suggested that EWS limits should be different depending on the time of day due to natural fluctuations in (for example) HR and BP.

Practicalities: a couple of patients mentioned the inconvenience of being connected to monitoring systems for periods of time and nursing staff being apparently unaware of the difficulties moving around when attached to monitoring equipment.

Technology: the group was aware of some of the capabilities of modern technology and volunteered information about camera monitoring (‘I would like to see this technology fast-tracked for night-time observations’) and simple continuous monitoring devices such as Fitbits. 167 It was queried if other factors (e.g. blood results) could be (or should be) part of the EWS.

Opportunistic conversation: quite a lot of the patients agreed that the observation time was a good opportunity to talk to clinical staff and suggested that patients would wait for the observations before asking questions about their care/treatment/plans to go home. Several patients were concerned that observation frequency could drop to very low levels and this would deprive patients of human contact and company during the day. They were keen to stress that, from their point of view, the observation rounds were more than simply a check on their physical health.

Escalation: almost all patients in the room were keen to be given feedback from the nurse/HCA about their vital signs (‘it’s nice to be told ‘you’re doing well’’) Most patients wanted some information about what would happen next and when the next set of observations would be taken, especially if the vital signs were ‘abnormal’ (‘I’d like to be told the course of action’). Some patients wanted more detail about their observations and asked for contextualisation (e.g. ‘is my BP high compared to a national trend, or high from a few hours ago?’).

Summary of early consultations with patients and lay contributors

• Patients understand that they will be monitored more frequently if they are more unwell. This was more obvious with the ICU group, which may be related to information they are given as they are monitored less in the ICU in preparation for ward transfer.

• Patients should be treated as individuals with personalised vital signs schedules (and thresholds).

• Patients should be involved in agreeing their frequency schedule if at all possible and it should be clear when the next set of scheduled observations will be.

• Contextualised feedback to patients following vital signs being taken is perhaps more important than staff realise.

• Patients like the security of regular monitoring and welcome the opportunity to have a conversation.

• Overnight monitoring is accepted but patients need to be alerted to the need for it at the start of the night.

• No patient on long frequency schedules (e.g. 8 hours or more between observation sets) should have routine observations taken overnight (e.g. between 10 p.m. and 6 a.m.).

• Patients support the use/development of monitoring systems that release nursing staff from ‘observation rounds’ (this includes new technology and HCAs taking observations as long as they are subsequently reviewed by qualified nurses).

• Post-ICU patients do not remember much from their time on the general wards, whereas the non-ICU patient group felt that the observation rounds broke the day up into manageable segments.

• Observations may be missed (without penalty) to accommodate mealtimes, although better synchronisation between ward activities and observation schedules should largely resolve inappropriate clashes.

• ‘Cohorting’ patients (putting beds in the same physical space) by their frequency of observation needs may prevent non-frequently monitored patients from unnecessary disturbance.

• Staff (from ICU-group meeting) expressed concern about missing required observations and being investigated for not following guidelines.

• Blood pressure monitoring is consistently reported as intrusive and uncomfortable.

Feedback for algorithm development and project outputs

• Advised physically grouping patients together according to observation frequency requirements.

• Consider the need for BP monitoring at every vital signs observation opportunity.

• Consider developing personalised observation frequencies.

• No observations between 10 p.m. and 6 a.m. for patients not requiring frequent observation.

• If overnight observations are required, tell patients when these will be taken.

• Always report back to the patient after vitals taken with information about results and what they mean (‘next steps’).

Summary of discussion

We used Twitter as a forum to collect views about vital signs observation practices from clinical professionals.

Over the course of an hour, five questions were posted:

• When do you think patients should be monitored more or less than NEWS/NEWS2 suggests?

• How do you prioritise patient monitoring when it is not possible to take observations as suggested?

• What do you tell patients (if anything) about how often they will be monitored and why?

• How can we strike a balance between patients’ well-being and other factors?

• What can be done to improve how observations are scheduled?

These questions were designed to complement insights from patients and carers, notes made by the work package three (WP3) team during their time on wards observing ward staff take patients’ vital signs (see Chapters 3 and 4: observational study methods and results), the original questions proposed in the funding application, and quantitative data available from work package two (WP2).

The full chat, including transcript and engagement stats, was exported from the WeCommunities website and subsequently analysed to discern common themes.

Several themes were apparent in responses, and to some extent these map to the patient and carer experiences outlined above. The ‘representativeness’ of patients and careers who are motivated to be part of wider hospital improvement groups was also discussed. This is a valid point that we need to be mindful of. At the same time, the project can only ever collect views from this type of patient and so this remains the best source for patient experiences that we can explore directly.

Two clear points from the patient and carer discussions were that overnight observations should be minimised and that patients should be included in the decision-making processes to agree how frequently a patient should be monitored. To some degree the TwitterChat group agreed with both these points but also expressed reservations about the extent to which this was possible.

Overnight observations were largely felt to be inappropriate [‘who decided that 2 a.m. and 6 a.m. were good times to wake a patient up?’; ‘it’s like ‘here’s some melatonin to help you sleep and get time oriented’ but also ‘we will be hitting you with that blood pressure cuff at 4 a.m.’’; ‘if medically stable (patient scoring 0) then reducing the frequency of observations, especially overnight, may be more beneficial for the patient holistically’] and this was balanced with a strong overarching belief that clinical expertise should guide decisions to observe patients outside of NEWS/NEWS2 defined protocols (‘I was always taught NEWS was to guide you but your own intuition overrules with monitoring’; ‘if your gut feels wrong, increase the monitoring’; ‘NEWS/NEWS2 does not replace the eyes and ears of the clinicians physically looking at the patient’).

Clinical expertise was referred to in responses to all questions and was a key factor when considering which patients were differently normal and who should therefore be observed ‘off-protocol’. These included patients being treated palliatively (‘replace NEWS with a symptoms chart to monitor for comfort’), those being investigated for possible stroke [‘escalate if blood pressure is high, even if not scoring (on the chart) so doctors can prescribe (anti)hypertensives’], or those with a sudden behavioural change (‘consider a patient who is usually a night owl, who is suddenly sleeping’) where holistic awareness of the patient including medication changes should guide interpretation of clinical signs. There was confusion about NEWS/NEWS2 protocol frequency due to limited evidence about observations generally (‘how do we provide information to our service users that variance from NEWS/NEWS2 was done with sound clinical judgement?’).

There was some discussion that electronic monitoring was reducing clinical skills (‘machines have taken away a lot of the touch and visual assessment; too focused on screens rather than the patient’) but at the same time the group acknowledged that with an electronic kit ‘anyone can take vital signs’ and that HCAs are asked to ‘continue routine observations on stable patients’ which allows more senior nurses ‘to focus attention on unstable patients’. This again picked up the topic of clinical expertise and ability/understanding to interpret observations, not just recording the measures, and the additional benefits that manual observations have over electronic that go beyond the (lack of) availability of kit and patients declining (‘we need to look, listen, and feel … to support decisions on frequency of observations’; ‘you can assess a patient just by looking at them’). Limited availability and accuracy of kit was a clear frustration with the participating group (‘you spend so long to find things’; ‘many a time the dynamap [electronic monitoring equipment] has given a wildly inaccurate reading’).

There was a small but consistent theme running through the questions that electronic monitoring is common outside of healthcare (‘every other industry and customer service has been doing it for 20 years, offering better leaner engagement and output at lower cost’) and that automated systems should run alongside manual systems to support efficiencies of care (‘I love a bit of technology but there is a process of learning and thinking when we put pen to paper which does not occur with electronic and automatic systems’).

Many participants in the group agreed that patients could be the best guide to observation frequency (‘we need to talk more to patients about observation so they can make informed choices’; ‘for many of them this isn’t their first rodeo and they know when they’re ‘not right’’). This view was somewhat juxtaposed against the rather limited ways in which patients seem to be included in decisions about their routine care where patients are rarely included in discussions about observation frequency, and really only told about frequency at all if on a high-frequency plan [‘I’m ashamed to say I only tend to do this if it’s very frequent’; ‘I put it in their care plan and explain that it will be BD (twice daily) in the first 72 hours of admission’; ‘I tend to tell patients how often I will be monitoring them and why when I’m doing frequent observations like hourly’]. A short side discussion on consent and engagement between patient and health professional suggests that patients are not often active participants in their care (‘I don’t think we do this enough and presume a patient consents’; ‘can’t say I’ve ever had this conversation unless the patient is on more frequent monitoring’; ‘sometimes nurses will explain why while they’re giving the obs but I think we get a bit complacent and assume everyone knows about observations or has heard it before’). There were mixed views on the observations round with some considering this ‘essential’, others advocating a patient-based approach, ‘chaotic at times but … not based on what suits us as nurses’.

Alongside clinical expertise, the importance of team communication was highlighted during the discussion, particularly with reference to when observation frequency is different from NEWS/NEWS2 protocol suggestion [‘rationale needs to be documented and communicated. There’s no point one person doing one thing with someone else doing another. Obs will become meaningless, misleading, and potentially put the patient at risk’; ‘ensure (off-protocol obs frequency) is documented clearly in notes and handed over’].

A small number of additional ‘vital signs’ were suggested to identify deterioration:

• glucose (especially for patients with diabetes)

• oxygen flow rate

• urine output (especially for cardiac patients)

• strength of heartbeat

• rhythm of heartbeat.

Feedback for algorithm development

• (Outside of scope for project) but it is clear that electronic monitoring is only as effective if wards have adequate kit.

• Consider recognition of clinical expertise of individual taking observations and adjust guidance on frequency/escalation accordingly.

• Highlight the importance of including patient in decision-making.

• Ascertain ‘normal for patient’ early in admission to allow adjustments to scoring and frequency.

• Require confirmation of ‘off-protocol’ frequency.

• Make evidence behind the algorithm/s clear to users to facilitate better discussion with patients and clinical team.

• Consider the need for BP at same frequency as other less intrusive observations, especially overnight.

• Consider alternative/additional vital signs for some groups of patients.

Summary of discussion

Work completed to date has recognised that vital signs observations (also referred to as ‘obs’) are conducted through two potentially conflicting patterns of work. Most of the clinical staff we have spoken to acknowledge that the NEWS escalation protocol creates an individualised schedule for each patient, dependent on their clinical response to the NEWS trigger thresholds at each time point. However, they also acknowledged that this can be difficult to deliver on a wide scale across a whole ward and the most common approach is to run ward rounds at specific times. This was observed by the WP3 team when they conducted their ward-based work.

We have also discovered that most clinical staff feel the NEWS system is not a ‘one-size-fits-all’ approach to monitoring and recognising physiological deterioration. Groups of patients who may not ‘fit’ the standardised practice include:

• Patients with diabetes as there is a general feeling that their glucose measures should be included in their observation sets.

• Renal and cardiac patients require fluid monitoring in addition to vital signs.

• Patients with COPD have differing oxygen requirements due to prolonged acclimatisation to living with low levels (this is partly addressed with NEWS2).

• Younger patients, and also the very old.

• Most people we have spoken to also want to see a difference drawn between surgical and medical patients.

Based on a prototype algorithm developed by the WP2 team, we presented first estimates of a risk-adjusted schedule of observation frequencies. Following technical adjustments to the algorithm, the specifics of both the current protocol in use at Portsmouth and alternative monitoring regimes changed. The underlying principles of identifying ‘acceptable risk thresholds’ remain valid and the stakeholder discussion highlighted clinical priorities that persist, with the final results presented in Chapter 6. Data presented to the group illustrated differences between workforce and health economic calculations between two schedules, or protocols for collecting patients’ vital signs.

The first protocol presented was based on the current EWS system in use at PHU. Data presented showed that patients admitted to hospital had an overall risk of death before their next observation of 0.41%; a risk of a combination of death, unplanned ICU admission, or cardiac arrest of 0.96%; a risk of combination of death, unplanned ICU admission, cardiac arrest, or an EWS above 7 of 4.69% and a risk of 6.1% of a combination of death, unplanned ICU admission, cardiac arrest, an EWS above 7, or their EWS increasing by two or more points.

Frequency of observations range from every 15 minutes (continuous) for patients with EWS 9 and above to every 6 hours for patients with scores 2 and below.

The assumed cost of this is £8756.70 per day when calculated with an equal division of workload between RNs and HCAs. This baseline cost also assumes no missed/late observations. We acknowledge that a significant proportion of observations are either completed late or missed altogether. 168 A comparison baseline cost of £11,140.90 was suggested for an observation schedule completed in full every 4 hours for all patients, regardless of their EWS.

We then presented an alternative scheduling system, determined by the WP2 algorithm, which was programmed to ‘optimise’ risk across all patients. This allows individual patients’ risk of the combined outcome of death, unplanned ICU admission, cardiac arrest, an EWS above 7, or their EWS increasing by 2 or more points to rise from 6.1% to 25.4% while maintaining the risk of death at 0.41% without increasing the existing workload.

Under the optimised approach the risk of a combination of death, unplanned ICU admission, or cardiac arrest reduces from 0.96% to 0.50%. The combination of death, unplanned ICU admission, cardiac arrest, or an EWS greater than 7 reduces from 4.69% to 2.58%.

As with the original data presented, this optimised schedule assumes no missed or late observations and that the workload is divided equally between RNs and HCAs. The optimised schedule indicates a reduced workload of 99.6 hours/day, compared with 243.2 hours/day for the baseline schedule. This suggests a cost reduction of £837,698 per annum.

The two protocols were discussed by the group in terms of how they differ, and how practical it would be to deliver the revised ‘optimised’ version in comparison to current practice. Some general points raised are summarised here:

• The group thought that 12-hourly (twice daily) observations should be a minimum for all patients.

• The group also thought that the threshold for increasing observation frequency should be an EWS > 3.

• The group concluded that patients with EWS > 6 could have their vital signs taken every 2 hours (although they recognised that the Royal College of Physicians recommendation is to repeat observation sets every hour for patients with NEWS/NEWS2 values of 5 and above).

• There was a strong feeling in the group that practical delivery of individualised patient scheduling might only be feasible with electronic systems.

• The general consensus in the group was that HCAs complete the majority of observations and therefore should contribute > 50% of the workload for the purposes of assessing costs. The group also thought they would be faster than RNs (although we discovered this isn’t the case from WP3 data which indicates ~4 minutes/set regardless of if completed by RN or HCA, or if on paper vs. electronic system).

Further discussion explored the proposed scheduling of the optimised protocol: specifically, how precise this should be and how ward staff could best remain aware of individual patient schedules.

• There was unanimous agreement that ‘no one thinks in 10-hour blocks’, proposing that (especially outside of electronic systems) timings should be rounded to the closest frequency matching common perceptions of divisions of time (e.g. 1, 2, 3, 4, 6 or 12 hours).

• However, the group agreed that showing 1.8 or 1.2 would allow ward teams to decide if ‘every hour’ or ‘every 2 hours’ was most appropriate on an individual basis. They felt this would be easily managed with automated systems that can alert staff when next observations are due. Traffic-light indicators at bed spaces were suggested as an extension to the electronic record system to alert ward teams of the urgency for observations.

• Visualisation of trends was suggested as a useful additional aid when considering individualised schedules.

• Clinical members of the group proposed that the ‘science should be separated from the social interaction’, suggesting a check-in approach with lower-scoring patients. This would give patients the contact they need as part of the observation schedule without the additional burden of having the observations taken. This approach was welcomed by the lay members (‘it’s nice to know someone has half an eye on you’).

• The group had a more focused discussion on the increased risk for patients whose EWS rises by > 2 between observation sets. Although at first glance this increased risk (6.1% to 25.4%) looked alarming, the group agreed that the absolute risk of any significant untoward event was likely to be low as the majority of these patients would be moving from scores of 0 or 1 to scores of 2 or 3. The group thought that the corresponding reduction in risk (4.69% to 2.58%) of patients’ EWS increasing to more than 7 was more important.

• The patients acknowledged they might be concerned if they noticed they were having fewer observations than other patients (‘this would worry me, but that’s not a rational response’). This could be mitigated through discussion with the ward team.

Feedback for algorithm development

• Clinical variation between some specific groups of patients is believed likely: recommendation for WP2 to assess differences/similarities where possible (acknowledging that not all data fields are available in sufficient numbers to derive fully working deterioration prediction).

• ‘Baseline’ workload and associated costs to be offered for 100% HCA and 100% RN and other divisions of workload in addition to the equal 50/50 split currently proposed.

• There was widespread support for the project providing an evidence base for intervals. To maximise impact (implementation in the real world), these intervals need to map roughly to generalised concepts of time on the hospital ward (continuous, every hour, every 2 hours, every 4 hours, every 6 hours, every 12 hours) but being clear about exact suggested intervals (e.g. 1.8/1.2 rather than round to 1 or 2 hours) will support individual decision-making.

• All patients should be reviewed at least twice each day.

• Patients scoring 1 or less on the NEWS/NEWS2 scale could have an additional check-in (without observation set being taken) each shift.

• Propose additional ‘vital signs’ for specific patient groups for NEWS3 development.

Stakeholder recommendations specific to WP2 output

• Minimum interval between two observation sets to be 12 hours.

• Adjust schedule for patients scoring 0 or 1 on the NEWS/NEWS2 scale to avoid overnight monitoring requirements between 10 p.m. and 6 a.m. (this is likely to mean bringing ‘scheduled’ observations forward to between 8 and 10 p.m.).

• Initial suggested intervals to be accurate to within 0.25 of an hour (15 minutes) but allow automated rounding to commonly understood intervals (hour, 2 hours, 4 hours, 6 hours, 12 hours) for simplicity when electronic systems are not in use.

• Add an interim check-in for patients scoring 1 or less on the NEWS/NEWS2 scale where an observation set isn’t required but patient has contact with care team.

• For health economics analysis, run costs associated with different division of workload between RNs and HCAs.

• Evaluate baseline and optimised protocols for the following patient groups if possible:

  • surgical patients

  • medical patients

  • patients with diabetes

  • renal patients

  • cardiac patients

  • patients with COPD

  • young patients (e.g. < 50 and 16–35 as potential subgroups)

  • very old patients (e.g. > 85).

Stakeholder recommendations for general project output

• propose additional ‘vital signs’ for specific patient groups for NEWS3 development (e.g. glucose, urine output, oxygen flow rate, strength and rhythm of heart beats)

• recommendation to include in all data recording systems a reminder to discuss observation frequency with patients especially if overnight monitoring is suggested

• recommendation for all electronic vital signs systems to include visual representation of trends

• recommendation to allow local clinical staff to ‘adjust for normal’ on an individual patient basis where this information is available.

Lay expert (retired nurse, currently volunteer at local hospital): approx. 1 hour

Conversation focused on the practicalities of delivering care to multiple patients with differing requirements and how individual clinical needs change over time: ‘you’ve got your fairly well and robust patients – not far from being discharged […] once/twice a day is fine. Then you go incrementally and end up with those who’ve come back from theatre […] every 15–30 minutes […] rigorous checks during their first day then increase duration between observations’. Views on physiological individuality were similar to the patients and stakeholders consulted earlier in the project in that ‘the 60–70-year-old who’s really ill will need more monitoring than the 25–35-year-old after their RTA [road traffic accident]’.

During the conversation the interviewee introduced the idea that practice change over time now means that nursing skills are more specialised, leading to less generalisation. Specifically ‘we used to have one nurse who did nothing else except observations on surgery day’ and that overnight care was overseen by a ‘fire and brimstone senior sister’. Concerns were raised that junior healthcare staff were no longer able to ‘spot ill’ patients without electronic monitoring.

The thought that patients should be grouped together according to their monitoring requirements and allowed to rest overnight (both points raised earlier in the project) was given provenance in that the interviewee described this historical practice from her own experiences ‘[we were] attuned to the fact that there were certain sections of the ward that needed rigorous observation’ and ‘we used to sneak up on people at 2 a.m. and check their respiration and pulse and only wake them up if they were ‘not right’’.

Clinical opinion (NHS consultant physician): 13 September 2021, approx. 1 hour and 45 minutes

This interview highlighted the importance of presenting supporting information behind electronic calculations. Much of the conversation centred on how to present the differences between current practice (NEWS/NEWS2) and the revised interval scheduling suggested by the algorithm work. Specifically, the interviewee wanted to know how many patients would come to harm in the various scenarios proposed and if there was evidence of a ‘gap in patients getting sicker and being captured later’, also summarised in terms of unknown uncertainty between observation sets (‘how do you know what happens to a patient if you don’t look at the patient?’). The current focus on economics as the main illustration of outcome was questioned as ‘money only follows after you have understood the amplitude of the difference in deaths < or other outcome measure as agreed >’.

The specific proposal offered by the ‘optimised algorithm’ that all patients with scores of 7 or higher should be monitored continuously was welcomed as a sensible approach. This would in their opinion ‘release observation time for nursing care’ and counterbalance the risks of missing or late observations at the more frequent rates. This assumption would appear to be consistent with the workload analysis presented alongside the risks of adverse outcome. The clinical view was that most people die with EWSs of 4–6, suggesting that more time for these patients might improve their outcomes.

The interviewee felt that evidence was needed for the decision to include ‘risk of increase in EWS by 2’ as part of the composite outcome (‘Why plus 2? Why not plus 3 or plus 4?’). They were also looking for evidence that other methods of analysis had been considered, such as trend recognition (‘Is there a difference between patients who run 5/6/7/6/8/7/8/6/ etc. and those who run 3/2/1/2/1/2/3/6?’) and simple < 4 hours or > 4 hours (‘can we run comparative analyses of outcomes for those seen under and over 4 hours after triggering?’). They also wanted clarification on how ‘every 15 minute’ monitoring had been costed for the economics analysis as it was not obvious from the information presented for review.

When thinking about presenting differences between schedules the following were suggested ‘to make it real’:

• number of deaths per 1000 people;

• number of people who currently go to ICU versus the number of people who might potentially go to ICU under alternative schedules.

These final thoughts were given to the algorithm development team to help clarify the reporting of the algorithm.

Summary of outcomes for different patient subgroups in the PHU internal validation data set

Summary of outcomes for different patient subgroups in the OUH external validation data set

Observation frequency in patient subgroups

Patient groups with lower-than-average observation rates were patients receiving blood transfusions or chemotherapy as well as patients admitted to geriatric specialties.

Highest median observation rates were found in the elective cohort, which could to some extent be explained by the shorter than average LOS of elective admissions and that for many patients who do not suffer an adverse outcome their observations are ‘front loaded’, that is, earlier in their stay.

Variation in number of observations within 24-hour period

Peaks of observation activity were slightly less pronounced in the elective cohort as well as in patients younger than 30 years old. Both groups have shorter than average LOS and by result observations were more likely taken independently from ward observation rounds. We observed more noise in the daily patterns of observations of geriatric patients due to the small size of the data set. Patients receiving blood transfusions and chemotherapy were not analysed separately because observations taken on the same day as treatment were excluded from the analysis, hence, skewing the results.

We observed slightly more variation across a 24-hour period in elective patients, with an overall lower mean TTNO (≤ 5 hours) but more distinct peaks compared to the entire PHU cohort at 06:00–07:00 and 20:00–23:00, where TTNO increased to 7 hours and 9 hours respectively. For small subgroups, that is, geriatric patients and patients ≤ 30 years old, patterns were too noisy to analyse, particularly in the PHU internal validation cohort (data not shown).