Implementing an artificial intelligence command centre in the NHS: a mixed-methods study

Electronic health record data

The quantitative work was based on the analysis of routinely collected healthcare record data de-identified and processed by the hospitals’ data teams and accessed via Connected Bradford – a regional integration of health care and other data available for research purposes. 38

We conducted a retrospective cohort population-based study using EHR data from the systems at the study and control sites. Participants of the study were patients who visited accident and emergency (A&E) and inpatient departments of BRI hospital and in-patient departments of Calderdale & Huddersfield NHS Foundation Trust (CHFT) between 1 January 2018 and 31 August 2021. We used complete sampling of EHRs within relevant periods. The duration of relevant periods was informed by the initial case description and unstructured observations in the qualitative work, which sensitised us to the information handled by the CC. De-identified UK NHS Digital Secondary Use Services (SUS) data were used. The SUS data are patient-level NHS England data made available to support national tariff policy and secondary analysis. Construction of the SUS data was conducted by Connected Bradford, located at the Bradford Institute for Health Research (BIHR) in Bradford. 39 The team uploaded the SUS data into a secure Trusted Research Environment on the Google Cloud Platform where relevant data were processed before final outputs were extracted. The Connected Bradford data service had worked with data from BRI for several years and had only recently gained similar access to data from CHFT; there were therefore occasions where more detailed data were available for the study site than for the control site.

Ethnographic data collection

The qualitative workstream conducted an ethnographic study comprising non-participant observations (including opportunistic interviews) with staff at the study site working in and around the CC and related organisational processes, document analysis and formal interviews with key stakeholders. Data collection took place between July 2021 and August 2022. Methods included ethnographic observations, document analysis and qualitative research interviews. In total 78 hours of observations were conducted involving 36 staff members that included 22 opportunistic conversations with CC staff and compilation of 100 pages of field notes between July 2021 and March 2022. A total of 19 interviews involving 16 members of key staff relevant to the initiative (15 at study site and 4 at control site) took place between May 2021 and August 2022 and ranged in length from 14 to 71 minutes, with a median length of 38 minutes (9.5 hours).

Following delays in gaining access to the study control site, interviews with staff in comparable roles to those conducted in the study site took place between 8 June 2022 and 11 August 2022. Four participants were interviewed in total, in senior management positions responsible for patient flow (two matrons and two senior clinical managers). In addition, we reviewed 64 documents relevant to the CC programme at the primary study site.

We recruited relevant staff at the CC and control site in key roles. NHS staff working in and around the CC were asked to take part in ethnographic observations. Sampling was in accordance with qualitative research practices to maximise variation in stakeholder perspectives. Potential participants were identified through clinical leads and early observations. Participants were told that they did not have to take part in the research if they wished and that they could withdraw up to the point that their data were anonymised (< 2 weeks following research interviews; < 1 week following survey).

In the following sections, we describe how these data were managed and used and the study design for each of the five substudies.

Approach

This substudy contributed to Objectives 1 (‘Impact Evaluation’) and 2 (‘Process Evaluation’) based on the hypothesis that operational efficiency, organisational processes and patient safety are affected by the flow of patients through the hospital. To study patient flow, we used process-mining methods41 to describe patients’ journeys through their hospital care. 42 Time-stamped data related to clinical events in individual patient journeys were extracted from the Connected Bradford data service and formatted to create event logs for patient cohorts of interest. An event log consists of a de-identified patient or hospital admission number, a date/time stamp showing the data and time an event occurred and an event or activity name. Typically, there are many events of potential interest within hospital systems and we worked iteratively with clinical domain experts to identify key events that ‘told the story’ of particular pathways of interest for patients at the two study sites.

Process mining

We constructed process models to represent patients’ log of clinical events (see Appendix 3) following a methodology our team had developed previously in dentistry,43 oncology,44 sepsis45 and primary care. 46 We evaluated these models by comparing their performance when constructed using various process-mining algorithms. The performance of the models was measured by:47

1. replay fitness, a measure of how many traces from the log can be reproduced in the process model, with penalties for skips and insertions; range 0–1

2. precision, a measure of how ‘lean’ the model is at representing traces from the log. Lower values indicate superfluous structure in the model; range 0–1

3. generalisation, a measure of generalisability as indicated by the redundancy of nodes in the model. The more redundant the nodes, the more variety of possible traces that can be represented; range 0–1.

We calculated conformance to a process model using the metrics defined in Fernandez-Llatas et al. 41 based on comparison between a normative model developed and agreed with domain experts and the actual events as found in the event log data. Conformance is measured as the proportion of moves-on-model and moves-on-log that ‘conform to’ the normative model against those that do not. One hundred per cent conformance suggests that all event logs fully comply with the normative model while 0% conformance suggests that the model is invalid in that none of the events follow the model. Our experience from previous work has been that few healthcare processes achieve 100% conformance due to the very individualised and personalised nature of patient care. We hypothesised that the CC should improve conformance to normative process models.

We identified process metrics of interest including process length and process conformance and calculated these for each month of the study to produce a time-series analysis to understand changes to process flow overtime. Patient flow as defined by the best-performing process model was described using the multilevel approach of Kurniati et al. ,48 which includes activity, trace and model measures. Where possible we conducted the same analysis between the study and control sites. Finally, we reviewed the results from our process flow investigation against the results from other parts of the study and reviewed these with clinical domain experts. The time-series results from our process mining are reported alongside the time-series data on Patient Safety (Substudy 3).

Approach

This substudy contributed to Objectives 1 (‘Impact Evaluation’) and 2 (‘Process Evaluation’) by directly evaluating the differences in patient safety outcomes before and after the implementation of the CC. The evaluation used longitudinal data analysis methods, for example including interrupted time-series analysis and latent growth modelling. We modelled trends in behaviour before, during and after the implementation of the CC, with consideration for the onset of the COVID-19 pandemic.

We approached the analysis in a responsive manner, revising the interruptions in response to discussions with the qualitative workstream as they identified key events from their discussions with the study site staff. We identified candidate variables of interest that included the Patient Safety Indicators from the Agency for Healthcare and Research Quality, for example pressure ulcer rate, in-hospital fall with hip fracture rate, postoperative sepsis rate. 49 We supplemented these with variables of interest identified by the qualitative research team and feedback from early PPIE workshops where patients identified measures that they considered important.

We assume that patient safety is influenced by the flow of patients and the quality of information (as encoded in EHRs) and therefore link data quality and patient flow metrics to those clinical outcomes logically related to patient safety.

In our protocol we had planned to handle unobserved confounders by comparison with the control site given that the control site was from the same geographical region, used the same EHR system, but did not have a CC. This was less successful than hoped as the pandemic response led to multiple changes in management style and approach at both study and control sites. Indeed, the control site reported that, part way through the pandemic, they had implemented a CC approach of their own using dashboard technology and insights from the study sight. Our longitudinal analysis considers those indicators that can be extracted from study and control sites that may provide insights into variations in data quality, patient flow and patient safety outcomes over time.

Metrics for patient safety, patient flow and data quality

We assume that patient safety is influenced by the flow of patients and the quality of information (as encoded in EHRs) and therefore link data quality and patient flow metrics to those clinical outcomes logically related to patient safety. Table 2 presents the initial list of proposed variables for analysis as published in our study protocol. 24

Data constraints

Our longitudinal analysis considers those indicators that can be extracted from study and control site that may provide insights into variations in data quality, patient flow and patient safety outcomes over time. Patient safety, patient flow and data quality are not directly measurable. Hence, indicator variables were used as proxy measures for these outcomes. We identified a ‘long list’ of candidate indicators based on stakeholder discussion and relevant literature and shortened this to a ‘short list’ of what was feasible given data availability.

Indicators for patient safety

Three indicators from the BRI hospital, the study site (mortality, re-admissions within 72 hours and postoperative sepsis), and two indicators (mortality and re-admissions within 72 hours) from the CHFT hospital, the control site, have been used. The proportions of mortality and re-admissions in hospital were calculated as the total deaths and re-admissions, respectively, divided by the total number of emergency admissions. Postoperative sepsis was calculated by dividing the weekly count of patients with sepsis diagnostic codes in their records by the total count of surgical operations conducted in that week. The surgical operation codes were extracted from the UK Health Security Agency list of operation codes published document. 50 Postoperative sepsis occurrences were identified using T814 ICD10 code. 51

Indicators for patient flow

Four indicators of patient flow (length of stay, clinician seen time, waiting time and average transition time) were used and were only available from BRI hospital, the study site. Inpatient length of stay in emergency admissions (defined as the duration between date and time of admission and discharge), ‘clinician seen time’ (the duration between A&E date and time of arrival and seen by a clinician) and A&E waiting time (the duration between A&E date and time of arrival and treatment) were used as indicators for weekly patient flow patterns throughout the study period. In addition, average times taken between A&E transitions (arrival, assessment, treatment, visit conclusion and check-out) were used as indicators for patient flow patterns during the same period.

These indicators were subsequently supplemented by the inclusion of patient flow metrics derived from process mining of the data (Substudy 2) to produce a multidimensional perspective on patient flow.

Indicators for data quality

Four indicators of data quality were available from the BRI hospital data. The proportion of missing dates of treatment and clinician’s assessment, and proportion of records showing valid transition (arrival → assessment → treatment → visit conclusion → check-out) and ‘left-shift’ (arrival ← assessment ← treatment ← visit conclusion ← check-out) of patients in A&E care were used.

Other variables for quantitative analysis

Dummy variables were created for each of the intervention components (‘broad patient flow programme’, ‘command centre tile roll-in’, ‘command centre goes live’ and ‘hospital-wide engagement and training’), COVID-19 pandemic and spikes of COVID-19 pandemic. The components of the intervention were given a value of ‘1’ starting from the date of its introduction until the introduction of the next component or phase, then a value of ‘0’ for the rest of the period. ‘COVID-19 pandemic’ was given a value of ‘0’ through February 2020 and a value of ‘1’, thereafter. A spike dummy variable was also added by setting ‘1’ for the COVID-19 spike periods based on the UK data52 and ‘0’ throughout.

The intervention and baseline phases were also modelled using five continuous time variables. At the date that phases started, a covariate was encoded with ‘1’ and increased by one unit for every time step for the duration of the phase; the covariate was encoded as ‘0’, otherwise. For example, the ‘Command Centre tile roll-in’ phase is defined between 1 May 2019 and 1 December 2019. Each week from 1 May 2019 was encoded 1, 2, 3, …, 29, 30 until the week commencing 29 November 2019, and encoded as 0, 0, 0, … from then on. In addition, seasonality was modelled by including dummy variables for the number of weeks in a year.

Quantitative analysis methods

Two types of models were explored: ‘simple tech’ and ‘complex’ models. For the ‘simple tech’ models, a three-phase, interrupted time-series model was used that consisted of a pre-intervention (first phase), ‘Command centre tile roll-in’ component (second phase) and ‘Command centre goes live’ component (third phase). For the ‘complex’ model, a five-phase, interrupted time-series model was used that consisted of pre-intervention (first phase), ‘onset broad patient flow programme’ component (second phase), ‘command centre tile roll-in’ component (third phase), ‘command centre goes live’ component (fourth phase) and ‘hospital-wide engagement and training’ component (fifth phase).

Interrupted times-series linear regression analysis was used to assess the impact of the Command Centre on the patient safety, patient flow and data quality. First, linear time-series models were fitted to the data. Tests for serial autocorrelation of residuals were conducted and all tests were statistically non-significant. Hence, regression models with autoregressive integrated moving average (ARIMA) errors were not sought. The Akaike information criterion53 and the Bayesian information criterion54 were used in selecting the models best fitting to the data.

To estimate the average transition time between different stages of A&E care, and to map the destinations of A&E patients, a number of process-mining techniques were used. 55

A five-phase interrupted time-series was used for the main analyses. To explore if the technology alone would have had an impact on outcomes, three-phase interrupted time-series models were used as sensitivity analyses. The ‘broad patient flow programme’ and ‘hospital-wide engagement and training’ were assumed as independent events of the Command Centre and adjusted as independent dummy variables in the sensitivity analyses models. Five per cent significance level and 95% confidence intervals (CIs) were adopted throughout. Analyses were implemented in R (Version 4.0.2).

Rationale for the ethnographic approach

With a complex system like the Bradford Command Centre implementation, it is important that the researchers develop an understanding of behaviour in context including the meaning of those behaviours and interactions within the immediate operating culture that has developed within the system. Ethnographic enquiry was selected in order to facilitate deep understanding of the technology in its broader social and organisational context, including human experience, engagement and interaction. 56,57 Ethnographic methods are widely used for understanding situated practices with technology. 58–62

Context for the ethnography

Bradford Royal Infirmary implemented their CC based on a centralised location that brings together operators with varying functional roles, each with access to their own real-time data feeds. The main purpose of bringing the roles together is that they have access to shared data visualisation that represents the real-time hospital state for the broader system that they are trying to control. The CC is connected to the broader system so that staff operating within can affect inputs into the system and respond to intelligence that is developed within the CC.

Participation and consent during fieldwork

All hospital staff involved in the organisation and/or delivery of patient care relative to the CC were eligible to participate in the study. Site access was made possible through a local collaborator and the Medical Director of the CC. Prior to data collection, the study was introduced to key staff relative to the CC through a series of face-to-face briefings across multiple shifts delivered by two of the authors, JB and CM. In addition, the Clinical Lead for the CC circulated information about the study to all staff working in the Centre. Written informed consent was obtained from staff observed in the Centre and those involved in early qualitative interviews. Later interviews were conducted online where the consent process was recorded verbally.

Ethnographic observations

Sampling

Sampling of observation periods was based on opportunistic access provided by key personnel and as agreed with CC Leads so as not to overburden staff. Observations took the form of approximately 4-hour windows with sampling of observations periods stratified in order to ensure representation of varied days of the week (including weekends), time of day and CC conditions (e.g. team handovers). As the study progressed, we used theoretical sampling for the ethnography to produce tracer issues and also for the interview participants to follow up emergent lines of enquiry. Initial observations were carried out by Carolyn McCrorie (CM) and Jonathan Benn (JB), and subsequently by CM, at different times of the day and on different days of the week. Researchers were situated within the CC room, usually towards the back of the room where there was available space (and to adhere to rules on social distancing required during the pandemic). As the study progressed, CM identified opportunistic moments within the staff workflow to carry out conversations with staff to check understanding and to explore what had been observed in more depth. CM and Josh Granger (JG) also observed the role of the CC in supporting operational planning meetings (silver tactical meetings) that took place at fixed times over the day.

Documentary analysis

We reviewed emerging hospital policies and guidance related to the CC (content reported in the original business case and CC set-up, reported impact on key performance indicators, activation programmes and training material for hospital-wide roll-out, iterations to the tiles and CC staff role descriptions) where practicable as an alternative to data collection involving staff to capture the ongoing implementation and monitoring process. A sampling framework to guide collection of documents (key documents and dates) was informed through earlier qualitative interviews with key members of staff and observations within and around the CC.

Qualitative research interviews

To complement our observational work, we conducted qualitative research interviews with key members of staff at multiple time points within the CC programme. Sampling was theoretically driven, based upon emerging insights from the observations and earlier interviews to include CC programme leads, key roles working in the CC, clinical leads in front-line areas interacting with the CC and organisational-level stakeholders representing operational strategy and clinical governance. We utilised the use cases (which emerged from observations within and around the CC) as a probe to compare operational planning, control and decision-making in specific priority areas, with and without the support of a centralised CC function, in order to enrich our understanding of how a CC operates within a health service context. An interview topic schedule with domains relevant to research aims was developed for use with programme leads and staff relative to hospital operational planning and patient flow and iterated throughout the study [including additional questions in light of feedback from patient and public (PPIE) representatives who attended our first patient-facing workshop and ongoing engagement with our PPIE co-applicant to ensure that the interviews covered topics relevant to patients]. Interviews were conducted by CM and JB either in person or online, audio-recorded, and professionally transcribed.

Data analysis

Data were anonymised and entered into the qualitative data analysis tool NVivo (Version 12.0) (QSR International, Warrington, UK) to facilitate data management. The analysis was an iterative and reflexive process and proceeded in parallel with data collection. Thematic analysis was undertaken on interview and ethnographic data, drawing upon concepts from Grounded Theory. 70,71

The coding frameworks were developed by the qualitative workstream research team (CM, JB, JG and RR) and included a framework for the interviews and a separate, complementary framework for the ethnographic data. Preliminary coding of sample ethnographic field notes was reviewed in research team meetings with feedback provided by RR and JB to the primary coder (CM). A process of open coding on the first four interview transcripts by CM and discussion of emerging concepts with the research team provided an initial inductive analysis framework. A second researcher (JG) independently coded the same four transcripts. Informal comparison of code patterns, using the constant comparative method for category and theme refinement, supported the development of conceptual code definitions and interpretations, which was discussed collectively in a series of researcher workshops (CM, JG and JB) and PPIE workshops (CM, JG and NS) and refined for consistency and focus by the workstream research team.

Six tracer issues were identified through frequency of referral to concepts during ethnographic observations, during interviews and in discussion within the research team. Theoretical sampling (analysis in parallel to sampling) informed subsequent data collection in order to build a comprehensive picture as to how intelligence is used to support operational planning, situational awareness and decision-making.

Data extracted from documents were analysed through an inductive process to capture key developments in design and functioning of the CC. The data were analysed in parallel to observation and interview data and informed lines of questioning for subsequent interviews (e.g. analyse the ‘official’ story vs. what happens in practice).

Through the coding and discussions regarding links and distinctions across transcripts, field notes and relevant documents and emerging themes mapped to the research aims, the later stages of our analysis followed the reflexive thematic analysis (RTA) process described by Braun and Clarke72,73 to produce a final inductive coding framework with a smaller number of selective codes.

Our PPIE co-applicant (NS) supported with qualitative data analysis for WS3, informed through reported best practice and discussion with colleagues within the CC on their recent experiences of related work. Involvement included a range of activities: (1) a training workshop on qualitative analysis (held in March 2022) based on an excerpt from one interview transcript collected eaconduct 40 interviews in totalrly in the evaluation Workshop 1; (2) a data analysis workshop (held in May 2022) based on the developing coding framework and exemplar quotes from interviews conducted at different phases of the process evaluation Workshop 2 and (3) an extended workshop (held in July 2022) to discuss data interpretation further from a lay perspective.

Data saturation

The value of data saturation as a guiding principle in RTA has been questioned;73 however, we did find the concept useful in guiding when we moved the focus of our enquiry between stages, for example from unstructured to structured observations and in terms of the main timepoints for the qualitative interviews.

The number of interviews and the scope for observation were significantly constrained by the restrictions imposed during the pandemic. We had planned to conduct 40 interviews in total, but the restrictions of the pandemic increased the time and complexity of organising and conducting interviews, leading to only 19 interviews in total. We would have like to conduct more than 15 interviews at the study site; specifically, we would have liked to do more follow-up interviews to discuss some of the findings, for example from the qualitative work. We would also have liked to have interviewed more than four people at the control site but even these four interviews had taken over 6 months to organise at the height of the pandemic. Limited access to the control site reduced our ability to investigate the comparisons between sites, which is disappointing as we surfaced interesting insights into how the control site had found different ways to meet the challenges of the pandemic but were unable to investigate these in detail.

We were allowed only one researcher to do ethnographic observation on site. They conducted 78 hours of observations at times and on days when we expected working patterns to vary as well as observing changing practice over time. This led to what we believe to be a comprehensive picture, but it would not be appropriate to claim data saturation given the complexity of the subject and its changing nature over time. Specifically, it would have been helpful to have more members of the study team involved in the observation to gain a range of perspectives, enable the team members to discuss the observations from their own direct observation and improve the depth of understanding across the research team.

Ethical issues

The study was approved by the NHS Health Research Authority (IRAS: 285933) on 1 April 2021. The Standards for Reporting Qualitative Research template was used to support complete, transparent reporting of qualitative research. 74

Rationale

With the increasing investment in, and implementation of, C2 centres in health care there is a need to develop a better understanding of the applicability of C2 principles to acute health care. Models of C2 were originally developed outside of health care, most notably in the military domain, and have long been the focus of research and development within these domains. C2 in health care, however, is a relatively new concept and as such there is a small but growing body of research in the area. Given this, there is a need to understand how acute health care can learn lessons from these mature safety-critical domains and how their C2 concepts and implementations can best be translated into the acute care environment. To understand this better, we planned and conducted a systematic scoping review supported by expert panel consultations to synthesise knowledge from across a range of non-acute healthcare domains to help inform the design and integration of C2 centres in hospital settings.

Review aims and objectives

The aim is to understand the principles of effective C2 as they have been developed in mature safety-critical industries and academic fields of enquiry, in order to synthesise and apply this knowledge in the development of hospital CCs in the acute healthcare setting. In doing so we scope and understand centralised C2 processes in safety-critical industries, including health care, and the key principles and contextual factors that may influence the applicability and transferability of C2 models and processes to a hospital setting. This review will help contextualise the findings from our qualitative and quantitative work.

Specific research objectives include the following:

• To identify key principles/theories of effective C2 based upon expertise developed in mature, non-hospital safety-critical industries, including models of C2 implementations or theories of effective C2 processes

• To describe and synthesise any available evidence for the efficacy of CC implementations in healthcare/high-risk industry

• To develop transferable principles and recommendations for health care.

Study design

This systematic scoping review was developed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for scoping reviews. 75 A scoping review methodology was deemed suitable as the research aimed to map the key concepts underpinning centralised C2, technologies, implementations and evaluations across a heterogeneous collection of peer-reviewed and grey literature. The synthesis of knowledge gained from the scoping review was guided by a series of interviews with members of an expert panel on C2 in safety-critical operations.

Search strategy

We conducted formal database searches for any peer-reviewed articles that discussed or evaluated centralised C2 and one or more of the identified domains (Table 3). We conducted an initial scoping review on 9 April 2021 using five databases (EMBASE, Global Health, HMIC, MEDLINE and Transport) which found 7310 records of which 147 were considered potentially relevant. Key papers were shared among team members and helped the researchers to prepare for and contextualise the work, notably in Substudy 4.

The search strategies were developed iteratively and comprised multiple search terms for centralised C2 and the domain respectively. The centralised C2 search terms were adapted from the Franklin et al. terms for CCs. 76 The domain search terms were developed with the research team, encompassing key terms within the domain identified through discussion, review of exemplar papers and review of preliminary search results for sensitivity and specificity. Four additional search strategies were used: to capture articles that more broadly review concepts of centralised C2, and to capture articles that cover any existing implementations of centralised C2 in a hospital or inpatient setting, to capture articles that cover centralised C2 and related concepts (e.g. high reliability or situational awareness) and to capture articles that cover centralised C2 and operations management.

The search strategy from the cross-industry review is reproduced in Appendix 2 and is summarised in a PRISMA diagram in Figure 2. Six databases were searched: EMBASE, PsycInfo, HMIC, MEDLINE, ProQuest and Web of Science. Informal searches were also conducted to obtain grey literature (e.g. white papers and guidelines) and other relevant articles outside of the searched databases. All searches were limited to English and published between 1 January 2000 and the day of the search in order to cover contemporary research and processes. The last search was conducted on 14 June 2022. Search outputs were downloaded to EndNote for management and screening. Supplementary information and references were also gathered from expert panel interviews in which subject-matter experts were interviewed about their understanding and experiences of centralised C2. The additional article/resource recommendations from the interviews were added to EndNote for data extraction.

FIGURE 2.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram for cross-industry review.

Inclusion and exclusion criteria

The target literature and subject-matter expertise for the scoping review was theoretical articles and commentary, empirical studies and evaluations of C2 implementations, domain-specific grey literature, and reported descriptive case studies across multiple safety-critical domains, including health care.

Two screening categories were defined in relation to the research aims and used to both categorise and screen the search results. These categories were defined as:

1. Reported cases and evaluations of operations CC implementations in safety-critical domains: any article that focuses on the design and/or implementation of a specific/named command/control centre (for operations in the target safety-critical domains) or operational control processes linked to a specific/named command/control centre in the target domain.

Excluded:

• Instances of reported control centres the purpose of which is not for control of ‘operational activity’ within the domain (e.g. cancer control centres, control centres for remote monitoring of patient data, poison control centre).

• Non-safety-critical operations (e.g. automated vehicle-fleet management, manufacturing supply networks or supply-chain control centres).

1. General literature, reviews, studies and theory about effective C2 in safety-critical domains: any article that focuses on C2 or related concepts as a topic within the target domains (either by referring to design, processes, related theory, research or reports of studies not linked to specific implementations).

Excluded:

• Articles with a focus on more general concepts not specific to C2 systems or processes (e.g. leadership, general communication, role assignment, preparedness or focus on cost-effectiveness).

• Reports on the development of methodology for studying or evaluating C2, without reporting evaluative criteria (i.e. don’t convey information about effective C2).

• Research that focuses on a non-safety-critical domain such as urban management or animal health emergencies or has a specific focus on team performance that does not directly relate to C2 (e.g. focus on a single role).

• The development, proposal or implementation of digital software/systems for information management or display/monitoring, or virtual CCs.

• The published protocol related to this research.

Study screening and selection

An initial phase of screening was conducted using EndNote and assessed the titles of each record found across the searches (n = 1963). Records were excluded if they were clearly not relevant to the study; the eligibility criteria were used to inform this decision. A sample of the initial body of records was reviewed and discussed among three researchers for clarity and understanding through which a high level of agreement was achieved. The remaining volume was screened by one researcher. The records marked for retention (n = 576) were exported to an Excel spreadsheet for further screening.

A second phase of screening assessed both title and abstract against the eligibility criteria. Two researchers independently reviewed a sample of 50 records. These decisions were then compared and discussed, resulting in an agreed understanding and interpretation of the eligibility criteria with no significant disagreements. One researcher conducted screening on the remaining volume; any queries were marked and discussed with the other researchers. During this process, records marked for retention (n = 273) were also classified for article type and their domain (e.g. military) (see Figure 2). Any records without abstracts present were carried forward to be reviewed for eligibility.

The remaining volume (n = 273) were then assessed for eligibility against the research aims. This led to the exclusion of articles that met the screening categories but were not relevant to the research aims (e.g. reviews of historical C2 in naval warfare). The eligible volume (n = 194) was separated and retained for data extraction.

Data extraction

A subset of exemplar articles was identified from the included literature set and subjected to full text analysis to draw out the main themes in response to the research objectives. This was done using principles from a thematic analysis methodology. 72 The remaining articles were analysed for relevant information that could be applied to the established themes.

Expert interviews

In order to support the findings from the scoping review literature, subject-matter experts were invited to informal interviews. The experts were identified through preliminary reviews of the literature conducted prior to the formal database searches and through dialogue with members of our steering group representing expertise in human factors and safety science. Invites were sent out to seven individuals of whom three agreed to interview. One additional interview was conducted where the expert was identified through snowball sampling. All interviews were conducted remotely via Microsoft Teams and recorded with the verbal consent of the expert. The recordings were transcribed and input into NVivo for analysis.

Data analysis and synthesis

Using the themes elicited through the data extraction method described above, the data were analysed to answer the research aims. More specifically this was done to highlight prevailing definitions and models of C2 and then discuss how C2 features and is researched in each domain. The articles classified as ‘screening category 2’ were used to discuss specific implementations of CCs and create a table of researched implementations featured in our data set. These discussions resulted in the further highlighting of three broad themes that featured across C2 in all domains. These themes were then discussed by drawing on a wide range of information in a cross-domain synthesis. The healthcare literature was analysed and discussed separately. The cross-domain synthesis and healthcare literature analysis were ultimately brought together in the final discussion to support recommendations.

Impact of the coronavirus-19 pandemic on the project

The CC had only been live at Bradford a number of weeks prior to the pandemic impacting on UK NHS hospitals. We therefore followed the emerging use of the CC as it supported Bradford’s response, including the addition of new ‘tiles’ for COVID management and the swift reconfiguration of services and hospital rooms. Our substudies on data quality, patient flow and patient safety were intended to provide quantification of the influence of the CC implementation but will be limited in their ability to distinguish such contributions from those motivated by the response to COVID-19. Our mixed-method approach and involvement from our international Study Steering Group has helped to define the context of this turbulent period and to describe the processes of change in the hospitals studied. Under the epistemic constraints of our pre-COVID, funder-approved protocol, we interpret our research through these contextual descriptions.

The pandemic also affected the study team. We were fortunate in being allowed to continue observations throughout the study but with social distancing and associated restrictions. Despite this, we were able to conduct the planned interviews at the study site, using a combination of face-to-face and online interviews. Access to the control site was, however, significantly delayed and when the research team did gain access late in the project timeline, this limited the extent of interviewees that could be recruited and we therefore prioritised key operational roles. Patient group meetings largely took place online.

The study sites, patients, the clinical professionals and all involved in the study went through an extremely challenging time. We consider ourselves fortunate that the majority of the study has been completed successfully and largely within planned timescales. We were unable to complete:

1. Exploring data-quality issues to the extent originally planned (Substudy 1). Data quality proved significantly more complex to measure. Our approach was to construct a ‘long list’ of candidate data-quality metrics and reduce this to a ‘short list’ of those that our clinical advisers felt were most appropriate and our data scientist confirmed would be achievable. Our original plan had been to investigate a wider range of data-quality issues through root-cause analysis based on on-site interviews with the informatics team and front-line staff. We were unable to get clearance for on-site access for more of our team and it was agreed that the people we needed to meet with had higher priorities.

2. Conducting the survey of CIOs attitudes to CCs. This survey would have provided important insights. During the pandemic, we were advised that CIOs would find our survey an unhelpful distraction from important work dealing with the pandemic. With some staff ill or isolating and additional challenges of getting access to hospitals and healthcare workers, we deferred this task until later in the project. The resource scheduled to design and implement the survey was successful in gaining a new role and left the project. We did not have sufficient time to recruit a new resource with the underspent budget before the project completed. We are proposing a follow-on project to disseminate the results, including to CIOs and hope to include this survey in that dissemination.

3. Disseminating the outputs of the research as widely as planned within the timeline of the original planned study. Our approach through the pandemic was to prioritise the gathering of information on the use of the CC. This was a challenging time for the research team, the study and control sites and the wider NHS and society and the work took longer and took more resource than originally planned as a result. Dissemination activity scheduled at the end of the project had to be condensed in order to complete the project on time.

What was achieved despite the coronavirus-19 pandemic

Despite the pandemic, we have produced a set of results which provides a unique insight into the potential use of CCs in the UK NHS that will help inform future investments in the approach. The results are presented in the following chapters in terms of patient and PPIE (see Chapter 3), a cross-industry review of the literature on C2 concepts supplemented by interviews with a panel of subject-matter experts (see Chapter 4), the results from extensive ethnographic observation at the study site and a process evaluation based upon staff interviews at both the study and control sites (see Chapter 5), the results of a quasi-experimental analysis of hospital data against control for the study period (see Chapter 6) and synthesis of findings to construct intervention logic to support future CC implementations and research evaluation, along with reflection on the study limitations including the implications of the changes detailed above (see Chapter 7). The project has a significant underspend which we are hoping to still use for dissemination and, if approved, this would include the planned CIO survey and our plans for PPIE communication.

Workshop 1: grounding workshop

Our first workshop in the PPIE workstream intended to:

1. gather lay perspectives on care co-ordination in hospitals

2. inform interview questions for hospital staff, as part of the qualitative workstream

3. contribute candidate measures of patient safety to the quantitative workstream.

The workshop output was summarised in an infographic (Figure 3) presenting the questions that should be asked of CC staff, and the measures of patient safety that should be analysed, in the qualitative and quantitative workstreams, respectively.

FIGURE 3.

Infographic produced by PPIE Workshop 1: grounding workshop.

Another outcome of the workshop was the lessons learnt, which we integrated into the planning and delivery of our second workshop:

1. Minimise jargon and abbreviations.

2. Explain and check understanding of technical terms.

3. Provide pre-meeting reading for participants that details the agenda for the day and summarises the content that will be discussed.

4. Provide pre-meeting checks of IT equipment (perhaps days earlier or on the morning of the workshop, to allow time for remedies).

Workshop 2: extending workshop

Our second workshop in the PPIE workstream intended to:

1. communicate the outputs of the research to the patients and public who are the ultimate beneficiaries of the work

2. continue the collaboration with patient and public by integrating their perspectives into interpretation of research findings

3. co-produce a communication plan.

Components for our draft communication plan were:

1. A list of messages – What messages needs to be communicated?

2. A list of stakeholders – Who needs to know about these messages?

3. A list of justifications – Why do these audiences need to know?

4. A list of media for distribution – How will we reach these audiences?

The scope of the second workshop was constrained to informing the communication plan rather than enacting it. The communication plan was summarised in an infographic (Figure 4).

FIGURE 4.

Infographic produced by PPIE Workshop 2: extending workshop.

Process models

Adapted from Lawson’s model, the Process Model is rooted in the idea that the CC seeks a desired state. 80 Data are obtained from the environment, processed and then compared with the desired state. Any discrepancy between desired state and current state leads to the CC making decisions about how to rectify this, actions are taken and communicated to their own forces. The issues with this model are that the desired state is defined and/or described by quantitative, discrete data. It is unclear how the model would cope if the actual state was highly uncertain or if changes in the environment led to consequences outside of the limits of the discrete state. ‘The model does indicate the central issue that command can be thought of as working towards some specified effect or intent but suffers, however, from its apparent reliance on a deterministic sequence of activities in response to discrete events.’77

Contextual models

Arising from Hollnagel’s Contextual Control approach, this model describes C2 as ‘a constructive operation where the operator actively decides which action to take according to the context of the situation together with his/her own level of competence’. 81

Different modes of control are offered under this model:

• Strategic Control – the ‘global view’, where the operator concentrates on long-term planning and higher-level goals.

• Tactical Control – characteristic of a pre-planned action, where the operator will use known rules and procedures to plan and carry out short-term actions.

• Opportunistic Control – characterised by a chance action taken due to time, constraints and again lack of knowledge or expertise and an abnormal environmental state.

• Scrambled Control – in response to a completely unpredictable situation where the operator has no control and must act in an unplanned manner, as a matter of urgency.

Time is a main function of C2 in this model: with more time available the operator perceives, they can gain more control of the task/situation. Stanton et al. found that transitions between control modes were consistent with this model. 82

Decision models

Based on the works of Rasmussen and Vicente the Decision Model comprises two parts of the decision ladder, the observation of the current system state after the detection of the need for action and the planning and execution of tasks and procedures to achieve a target system state. 83,84 The ladder contains two types of nodes, information-processing activities and knowledge states.

Shortcuts can be applied to the ladder. For example, a diagnosis of a current system state signals a procedure to execute. Stanton et al. note ‘The path in which the operator moves through the ladder is dependent on a number of factors including: their workload, experience and familiarity with the current task’. 77

Functional models

Smalley’s85 functional model of C2 identified 7 operational and decision-support functions and 10 information-processing activities (primary situation awareness, planning, information exchange, tactical situation reports, current situation awareness, directing plan of execution, system operation, system monitoring, system status and internal co-ordination and communications).

This provides a distinction between command (command and planning, navigation and piloting and tactical decision-making) and control (communications, system monitoring, system operation and operational co-ordination), which are separate but connected. These occur in two different ways: internal operation of the system (command and planning, communications and system monitoring) and interfacing with the external environment (navigation and piloting, operational co-ordination and system operation).

Stanton et al. conclude ‘in contrast to the other three models, Smalley’s model offers much higher fidelity for command and control’. 77

Command and control in defence

The concept of C2 is inherent in the operation of military organisations. Despite this there is no ‘authoritative formal UK or NATO definition for it’, which often results in a concept merged from the definitions of C2. 86 In a Joint Concept Note outlining, and titled, the Future of Command and Control, the British Ministry of Defence (MOD) offer a definition that describes C2 capability as ‘a dynamic and adaptive sociotechnical system configured to design and execute joint action’. 86

While the offered definition is comprehensive, the MOD also state that their approach to C2 has stemmed from industrial-era warfare and does not reflect the complex modern role of defence and military operations. In order to develop and progress, it is suggested that core thinking around C2 evolves with a view to increasing the agility of C2. Agile C2, or Edge C2, allows for an adaptive and responsive approach more relevant to the context of the operation or work at hand.

Edge C2 is suggested to be the most agile and advanced approach to C2 but is not the only approach. Other C2 approaches are named as conflicted C2, de-conflicted C2, co-ordinated C2 and collaborative C2. 86 It is important to note that these approaches are not mutually exclusive; they will interact across different organisations, and the different approaches will be appropriate for different situations. The ability to move between these as necessary in an informed and timely manner is dependent on the agility of the organisation and their ability to use de-centralised and adaptive C2 to improve synchronisation and maintain a drive towards unity of purpose.

In order to increase agility, the organisation requires the ability to collect and distribute data and transform them into intelligence for rapid decision-making across multiple domains and missions. 86 This highlights the need for sociotechnical integration in which people, structures, processes and technology are developed to be able to adapt to deliver a tailored system to meet the needs of a specific environment and context. Advances in technology will likely lead to the most significant changes in approach to C2. Development of a single information environment will enhance evidence-based decision-making at speed while allowing for humans and technology to be parts of a team. 86 Integration and use of interface technologies (i.e. command walls) will provide more natural interaction and improve situational awareness.

The introduction of technologies in this domain has already seen a shift in approach to operational control. Walker et al. suggest that there is a movement away from traditional C2 towards a network enabled capability (NEC) model. 87 In their study examining human performance under NEC and C2 systems, they present C2 as having a tight distribution of information in which ‘the commander is the only individual with an overall view of the situation’ and fire teams have a local view but otherwise work in isolation: ‘Everyone does not know everything’. 87 C2 also operates under a hierarchical structure in which individual roles can speak with the commander but not with each other and all decision-making rights rest with the commander, who could allocate autonomy, authority and discretion by defining and scripting the tasks of lower roles. Conversely, NEC has a broad distribution of information where fire teams receive regular situation updates from the system operator and can interact with their counterparts: ‘Everyone knows everything’. There is no communications hierarchy and collaborative working is encouraged and is facilitated by outcomes-based instructions and communications infrastructure.

Their findings suggest that utilising a traditional C2 structure resulted in accelerated task-completion times; however, this comes at a cost to other factors such as accuracy. While utilising a NEC structure was slower, there was a higher degree of accuracy and stability. The authors suggest that this trade-off is worth it when operating in a context with ‘inherent complexity, dynamism and asymmetry’ and allows the organisation to undertake more complex processes of optimisation; however, further research is warranted. 87

An alternative approach to integrating new technologies into a C2 model was reported by Hansberger and Barnette. 88 The United States Army Research Laboratory developed a model called ‘Command, Control and Communication: Techniques for the Reliable Assessment of Concept Execution’ (C3TRACE). This model represents ‘different organizational structures, individuals within those structures, the tasks and functions performed within a task, and the communication patterns between individuals in an organizational structure’. 88 While this model puts specific focus on communication in addition to C2, the key concepts remain the same as operator communication, workload, task times, situation awareness and decision-making were all reported variables. C3TRACE maintains a hierarchical structure but adopts a more flexible and adaptive workload-management process to facilitate information flow and reduce rates of dropped tasks. This model presents a more flexible and responsive form of C2 but does not go as far as the NEC model with regard to hierarchy of decision-making and instruction.

A common approach to understanding C2 systems is to utilise the Event Analysis for Systemic Teamwork (EAST) methodology. 89–94 EAST is described as ‘a macro-ergonomic method for extracting large scale systems level data on the emergent properties of C2 scenarios’. 79 The method does this by analysing the information flow and activity between three main network representations: Task Network, Knowledge Network and Social Network. 79 This provides insights into the participating agents and how they are arranged, how technology facilitates information flow, what tasks are enacted in response to which overarching goals, and the role of situational awareness. Situational awareness is one of the main focuses of EAST as it is defined as ‘one of the key emergent properties from command and control scenarios, and one of the major determinants of decision superiority’. 79 In Walker et al.’s research into applying EAST to emergent properties of C2 in a NEC scenario it was clear that situational awareness was dictated by the constraints placed on it by the facilitating communications technology and distributed among both social and technical agents within the system. 79 The social agents under this system had a degree of self-organisation provided by the distribution of situational awareness and cognition in tandem with the ability to adapt the technical agents to suit their needs and preferences. Eliciting these emergent properties of C2 provided insight into NEC C2 while also demonstrating the applicability and relevance of applying the EAST method to analysing C2 scenarios.

Command and control in transport

Systems of railway network control do not routinely use the language of C2 to describe the domain and work done. However, Farrington-Darby et al. ’s study elicited key features of the domain and there are obvious similarities with C2 systems. 95 For example, the study found that controllers manage a complex system with no direct interface and to do this they have to interpret the incoming data and form a mental model of the system. In C2 language, this would be referred to as situational awareness. At the time of the study, controllers were separated into distinct groups and located in control centres that varied in both physical set-up and make-up of the team present. There is some colocation of different roles, but this varies across different control centres, consequently a key part of the work is understanding the social distribution of both skill and knowledge across these teams. This requires not only expert knowledge of the processes and practices but also expert knowledge of the social relationships and responsibilities in the domain. Decision-making is distributed to each controller, who uses heuristics and guidance rules to aid their decision-making in order to achieve the high-level goals of the railway, for it to run safely and efficiently. It was also found that part of managing the railway involved responding and adapting to sudden increases to ‘full or overload situations’ resulting in a variable workload, some elements of which are safety-critical. 95 Operating successfully under these conditions requires resilience of the system and C2 agility.

The above findings were elicited using a naturalistic and ethnographically based field study. This approach allowed the researchers to observe the subtleties of the control room and understand the social elements of the work and the contextual nature of expertise in rail control centres. 95 The relevance of this social expertise was an unanticipated feature of the controller’s work that the researchers state is an example of the exploratory value of ethnographically derived methods.

Air traffic control (ATC) is another example of a transport domain where C2 systems are present. Walker et al. describe the C2 features of the work done by ATC as an example of distributed cognition in which the computational states (processes and knowledge used to achieve the goal) interact with the representational states (the environmental contributions to the total system), resulting in a highly complex and adaptive level of functioning. 96 As seen in other domains, situational awareness plays a key role within the system’s functioning. In this context, the controller forms a ‘picture’ of the airspace that arises from the interaction of several components. These components include designed parts of the system such as radar displays and communication channels and technology and incoming situational information such as a new aircraft entering the controlled airspace. The authors posit that applying distributed cognition to C2 scenarios requires a shift in approach to situational awareness from the traditional notions of individual situational awareness to the notion that situational awareness sits across the whole system; this includes the technical aspects of the sociotechnical system. 96 Additionally, it was found that the ATC domain has a diverging hierarchy that splits into groups further down the chain of command. This holds the benefits of a centralised hierarchical structure but allows for the flexibility and rapid response of localised decision-making which helps remove any delays that could occur from having to disseminate the relevant information throughout the hierarchy. 96

In the above research, the researchers adopted the EAST method seen used within the military domain. The paper describes using this method as going beyond ethnography. The strengths of an ethnographic approach are discussed (e.g. can be used to understand how information is used to support decision-making) alongside its drawbacks. Three overarching drawbacks are mentioned: the outputs from an ethnographic approach are often highly discursive and may not be easily reconciled with a non-social science audience, the requirement of being ‘imbued’ in the culture of the scenario can be incompatible with objectivity and validity in measurement, and their analyses are not always easily amenable to generalisation. 96 The researchers state that the EAST method can capture similar phenomena in a way that is appropriate and relevant to the designers of such systems by producing a systems-level description of the scenario. This method creates network diagrams that cover the social networks between agents, the dispersion of these agents, and a representation of the knowledge network in the domain that all include the non-human agents (i.e. technology) and the media that facilitate their communication. By doing this, the analysis has the following strengths: it avoids bias by focusing on objective and manifest phenomena, it is applicable to any domain and is comparable across and within domains, the outputs are graphical and easily interpreted, and it is underpinned by considerable detail that can be explored further. 96

In an article aimed at revealing potential areas for further improvement in reliability in the public transport domain, the dynamics and processes of operations control centres are discussed. 97 The authors focused their research on operations control centres in Germany and Singapore and their daily processes and procedures after the occurrence of an incident within the public transport network. Their findings were elicited through ethnographic methods: interviews, observations and documentary analysis.

One area highlighted for improvement by the research was communication. This includes both internal (between dispatchers and drivers) and external (with emergency services and passengers) communications. Internally it was recognised that incident situations can be challenging for drivers and dispatchers, and this negatively affects the level of communication between the two, which impacts the situational awareness within the system. It was suggested that additional training could be of benefit to the drivers and dispatchers and expanding the functionality of onboard computers to provide translations/subtitles of the conversations could help break down language barriers within the diverse workforce. 97 External communications are handled by dispatchers and in both instances (communication with emergency services and communication with passengers) could benefit from additional technology input to facilitate the sharing of information. Highlighting this area shows the importance of sharing information to aid situational awareness and how fundamental this is to the public transport operations system. It is also stated that the better the communication is, the better the dispositive measures are, which bolsters the resilience of the system. 97

The second area for improvement was the role/presence of automation in the domain. An increase in automated processes is suggested by the authors to both reduce the workload of the dispatchers and increase the quality and accessibility of relevant information. This would also be done by expanding the use of technology in the work done. The positives of better information-sharing and thus improved situational awareness are already apparent. The proposed automation would reduce the number of calls the dispatchers have to do in response to an incident, which gives them more time to concentrate on the dispositive measures needed and mitigate the negative effects on the service; this improves the resilience of the system. 97

The final area for improvement suggested by the authors was in the role of planning in the domain. It was recommended that the robustness of the timetables was evaluated. This has to do with the availability of services and the availability of replacement vehicles and staff in response to incidents that take staff or vehicles out of action. This availability is shown to have a large and significant impact on how negatively the incident can impact the service if no replacement is available. This recommendation is about the presence and quality of situational awareness in tandem with appropriate allocation of resources and how this can facilitate a timely response to a disruption in the service and maintain the resilience of the system in the highly dynamic task of incident management in the public transport domain. 97

Command and control in the power industry

Electricity power grid management is conducted from a central location, a CC, for real-time monitoring of grid conditions, assessing grid stability and maintaining system frequency. 98 Due to the physically dispersed nature of the power grid, situational awareness is essential for operations. This is achieved in this domain through technology that provides multiple streams of information such as spatial and geographical information, voltage levels, temporal information, functional information, substation information. Ready access to this information improves the operator’s capabilities and confidence to make timely and correct decisions.

Decision-making is informed by intelligent systems that perform model-based analytics which can quickly identify and recommend solutions to problems that may arise. The information from these analytics is readily available to a range of staff members (operators, engineers and managers) so that they can jointly assess a problem condition and together decide on the actions to take. This is a more collaborative decision-making process but still retains the traditional hierarchical structure we see under C2 models. Using model-based analytics further introduces the non-human agents seen in other forms of distributed situational awareness and distributed cognition above. These advances in technology adopted in a centralised CC provide grid operators with ‘wide-reaching visibility into the status of the grid, as well as the ability to predict and plan for potential problems’. 98

Salmon et al. studied this domain by utilising the already seen EAST method. 99 Their application of this methodology found support for distributed situational awareness between both human and non-human agents. It was found that by interacting with the non-human agents the operators could effectively and accurately update their individual situational awareness. There are notable differences in the form and level of individual situational awareness in this setting; however, these differences were observed to be complementary. For example, the CC operative held a very high-level overall picture of the scenario, whereas the field agent’s individual situational awareness was primarily focused upon the work they were engaged in at the time, working towards their own individual goals. This highlights the benefits of co-locating different roles and their respective pools of expert knowledge. Communication between centralised operators and field agents was found to be intrinsically structured by the knowledge that comprises the team’s distributed situational awareness, thus effectively coupling distributed systems. 99

The nuclear industry has been subjected to a vast amount of human factors and ergonomics research over the last 40 years and has benefited from this in areas such as the design and evaluation of processes, procedures and control rooms. It is arguably the source of safety culture that has highlighted the importance of management and organisational factors for the safety of nuclear power operations, and this has since been adopted by other safety-critical domains. While, like domains discussed above, the nuclear domain does not routinely use C2 language, it is clear that the key components of human factors as they pertain to safety in this domain have significant overlap with C2 terms and concepts.

A review on human factors engineering research into nuclear safety found that in research looking at groups and teams in nuclear operations, communication of the teams was a central focus and has seen increasing interest from 2010 onwards, looking at the communication structure in relation to operators’ activities with particular focus on overt behaviour and corresponding underlying cognitive processes. 100 This has been done by modelling team communication with a novel speech-act coding scheme to understand the difference between operator intention and enacted behaviour. This has included real-time feedback of individual actions to the wider team in order to enhance team situational awareness and increase team cohesion. Team situational awareness has also been adapted into computer-based systems with a view to improving team performance. Team resilience has seen a surge in focus in recent years, which has aimed to examine a team’s ability to adapt, co-ordinate and respond in abnormal operating conditions. This research has, however, been mostly conceptual in nature.

Research into the individuals involved in the operation of the nuclear plant has seen an increase in the areas of human error, human performance and situational awareness. Human-error research was prominent in the 80s and 90s, halted after 2000, but has seen a resurgence in the last decade. Cause classification methodologies have been used to try to identify contributing factors and provide explanations. Human performance research has highlighted the importance of experience and expert knowledge and their impact on executing tasks successfully. Situational awareness has seen greater focus since 2000 with the introduction of digital technologies spurring further interest in individual situational awareness. This has been recognised as an important factor when introducing increasingly advanced digital systems to aid operations. This importance and focus have prompted the creation of situational awareness reliability assessments to predict operators’ performance and reduce the chance of errors due to deficiencies in situational awareness. 100

Recently there has been research incorporating safety theory and methods from other domains into nuclear safety. For example, Teperi et al. have introduced a Safety II tool, a concept seen increasingly in health care. 101,102 This method highlights both successes and failures of operations in the nuclear industry to elicit a more accurate description of human-factor issues affecting operational events and provides learning related to resilience. The use of Safety II in the nuclear domain is novel and further research into its applicability and validity in this domain is needed.

Command and control in emergency services

While emergency services are closely linked to acute health care, they operate in different contexts and ways of working. Both situational awareness and distributed cognition have been researched in this domain using interviews, observations, documentary analysis and literature reviews. 103,104

Emergency medical dispatch, or ambulance control, is one such service. These services often operate regionally and independently, with size and structure varying depending on the location and area covered. Broadly, their work has two aspects: ‘call-taking, where calls for emergency medical assistance are received and prioritized, and controlling, where the most appropriate ambulance is dispatched to the emergency and ambulance resources are optimized in their areas of operations’. 104

Situational awareness in this context is twofold: operators maintain situational awareness of the control room (e.g. level of resources available, current allocation and location of ambulances), while also maintaining awareness of the wider world (e.g. other emergency services, local hospital capacity, major accidents). This awareness of the situation is critical to successful operation as ‘decisions are not made in isolation, but within the context of a dynamically changing situation’. 103 Senior operators utilise an ‘information hub’ strategy that allows them to be aware of changes in the areas under their control. Operators use situational awareness to form a mental model of the ambulance network and locations of allocated ambulances. It was reported that at times the operators did not use the mapping system they had to inform their decisions as they had an expert level of knowledge based on relevant static information (e.g. road network of their area) and could redirect ambulances based on this knowledge. Overall, their situational awareness has been described as a combination of this static information, dynamic information and a temporal component that they needed to keep track of. 103

The role of organisation structure has also been a focus within the emergency services domain. Ji and Ren discuss research into the application of C2 organisation theory to support emergency communication support organisation. 105 Broadly, the authors highlight support for the adoption of C2 systems. They state that the theory of C2 organisation can be applied to solve problems in the structure of organisations. Acknowledging the need for agility in response to dynamic mission environments, it is suggested that using C2 theory to establish the organisation structure and in parallel create organisational adaptation adjustment methods allows for a responsive and resilient communication support organisation. 105 However, it was found that using a three-stage method was too sensitive to a dynamic environment, resulting in wasted resources when utilising the adaptation methods. This highlighted the need to develop a more robust organisation structure that requires fewer adaptation methods in the face of dynamic situations. Further research is needed to develop the robust organisation structure but the authors state that the application of C2 organisation theory can help to ‘enhance the performance of emergency communications support command, to complete various tasks with high quality’. 105

As seen in other domains, C2 in emergency service operations has been studied using social network analysis. Houghton et al. used this method to elicit the differences in systems for fire services and the police. 106 Using Dekker’s command structure terminology, it was identified that the fire service adopted either distributed or centralised networks. 107,108 This was due to the need to have a central focus for conformation management but also allow for rapid response; this was more centralised in incidents where managing a mass of diverse information was required. 106 This demonstrates a need for agility in the C2 system of fire service response. In contrast, the police networks best matched the split network definition as the design arises from ‘procedures for eliciting well-defined information and clearly defined responses’. 106 The more procedural and legal nature of policing is likely responsible for this due to the need to maintain a log of the activities performed under police command. Distributed networks allow for more rapid response as agents work independently but the main advantages of using a centralised network are to manage information flow and co-ordinate response. The use of social network analysis has not only elicited this information but has also allowed the authors to represent this graphically for quick interpretation while also to reference both the emergent scenarios and the limited set of archetypes that are well defined. This method also allows for quantitative analysis provided by mathematical social network analysis techniques which provide subjective impressions from quantifiable statistics. Using this approach, even with relatively small networks and incidents for study, allowed the authors to identify issues relevant to C2 systems in general, such as ‘the balancing of speed of response against information quality and choosing between centralised and distributed command models’. 106

Command and control in health care

Although the literature on C2/CCs in health care is still at an early stage of development (especially in terms of robust research), there are already some emerging examples of C2 and CC use in the scholarly peer-reviewed literature. Most recently, C2 has been rapidly implemented in some settings in response to the COVID-19 pandemic. For example, Bruno and Petscavage-Thomas report on the introduction of a departmental CC for a radiology department in a US hospital in a response to change in ways of working during the peak of the pandemic. 109 Their CC included a well-defined hierarchy of roles and responsibilities that considered the need for some staff to work remotely due to the temporary social distancing requirements introduced at the time. It was ensured that the five staff members on-site covered all subspeciality areas needed and had a colocation of roles and expert knowledge. Additionally, the CC acted as a central communication hub and point of contact for the department. The centralisation of communication was aided by digital technologies, both in terms of ensuring the in-hospital communications came to this central point, but also so that staff working remotely had access to the relevant systems and information to continue working.

One example of this was the ability to hold daily meetings online both within the department and with clinical leads from other departments. This maintained situational awareness across the team and ensured that they could utilise all staff to distribute the workload and function effectively despite the novel remote-working requirements. Introducing a CC in response to the significantly abnormal operating environment (the pandemic) was deemed a success and responsible for the department being able to continue its clinical operations seamlessly. 109 In addition to aiding in the continuation of routine operations, using a CC also saw a sharp drop in safety incidents, ˂ 10% of prior levels, which the authors attribute to being able to communicate with other departments more readily (i.e. online), which led to more rapid identification and resolution of issues. The authors state that their ‘Command Center model enabled this success, which rested upon a robust IT infrastructure as well as clear, decisive leadership, with a high degree of autonomy afforded to each division, and open lines of communication at every level between all stakeholders’. 109

Successful use of C2 principles in response to the pandemic has been seen elsewhere, such as the reported use in Hutchings et al. , in which ‘the diverse and complex clinical situation that evolved at King’s College Hospital during the COVID-19 pandemic required an agile and dynamic C2 system with effective communication mechanisms’. 110

The use of C2 principles in acute health care has not been exclusively tied to the pandemic. Franklin et al. ’s scoping review into the use of hospital capacity CCs found that there were numerous anecdotal accounts of these centres being widely used in efforts to improve hospital patient flow and safety. 76 These centres are defined as involving the colocation of inter-disciplinary workgroups, the use of real-time data integrated from sources such as EHRs, and the management of two or more processes tied to patient flow. This demonstrates a centralisation of both situational awareness and decision-making, informed by the colocation of expert knowledge.

Table 5 summarises the specific CC implementations in health care we found with their target operations and implementation models. Additionally, the types of evidence offered in the related articles are noted. All the examples shown have been relatively recent adoptions showing a growth in use and interest of a centralised C2 approach in health care. Apart from one centre (the radiology command centre), all the centres have been introduced to manage operational functions (i.e. patient flow) rather than managing more localised medical functions and care. 109 It is also clear that technology has a significant role to play in command centres and may bolster their appeal as intelligent systems can provide enhanced situational awareness across the hospital. Another key feature of these implementations is the colocation of different teams and disciplines. As seen across other domains, this is a common feature of CCs and increases the ease of communication and decision-making, providing greater agility and resilience to the operations under the control of the centre.

Espoused aims of the programme

Based upon interviews with programme leads and programme documentation, the intention of the CC programme at the study site was to bring together real-time data from a multitude of source systems including the Cerner Millennium EHR system, process those data using advanced algorithms and display new intelligence on custom-built analytic applications (tiles). Visualisation of data that represent factors that impact care and patient throughput from a C2 and hospital oversight perspective would provide better understanding of current operational pressures than was currently available:

The main aims from a command and control and hospital oversight perspective were to give that understanding of what was happening from front to back door in a seamless way and the way that it was set-up originally was for the site team to have a visual aid for what was happening down in A&E, on the main hospital wards and also from a discharge point of view, so flow of patients move seamlessly from one area to the other.

Senior clinical leadership, 6/1

Remote access to the tiles was intended to support prioritisation of patient-related tasks at service or ward level, for example:

The ward link tile should be used for whatever that ward wants to call it, ward rounds, huddle, where you quickly flick through every patient and say, ‘right we’re going to see them first because they’re the sickest, but then actually they’re based on EDD, these ones are due over to go today so we’ll see them next, and then we’ll see all this lot in the middle’ and use that meaningfully.

Senior clinical leadership, 2/1

Design of the command centre technology: technical infrastructure and development

Key system components included a cross-system data feed that integrates data from all sources, a third-party data integration engine that converts data for display against the metrics and alerts displayed in the CC, visual display of data on tiles in the CC where each tile has a specific operational theme that can be filtered in different ways to support identification of issues and action and remote tile access across the hospital.

Development followed a human-centred design in the sense that each tile has a tile owner with expertise in the particular clinical areas that the tiles refer to and they worked closely with the developers to develop specifications. High-level ideas and priorities for development were informed at executive level:

The executives decided that they wanted a capacity snapshot, an ED status, the transfer tile, the three that were mandated from the executives at the time of purchase, the other ones were use cases brought forward by people in the organisation to say these are areas of concern where we think that the software would help us.

Senior clinical leadership, 1/1

… and through design sessions attended by the hospital management team:

All of the entire management team would’ve been invited. That kind of gave us the high-level ideas and the high-level priorities and then from there we got together all of the people involved in that topic to start co-design, what’s your real, what’s really your problem, not so much why doesn’t it work although I’m sure that came up, but what’s your problem and then what would help you and then how do we solve it.

Senior hospital leadership, 1/1

The CC system itself consisted of nine tiles consisting of real-time information and alerts (update approximately every 3 minutes) for patient care and intervention across the hospital site. The tiles are displayed on a Wall of Analytics in the CC room and can be accessed from desktops, mobiles and tablets linked to the hospital intranet. There are ongoing software development iterations to refine the tiles and their components. The main functions of the tiles are shown in Table 6.

Staff reported some deviation in function from the original intention regarding certain tiles. CC staff reported that the care progression tile was not used as intended as ‘what that’s displaying we get through so many other tiles anyway’ (Senior clinical leadership 2/1) and that issues with data quality impacted ability to make use of the tile: ‘Data quality issues means that it is not as useful as it could be’ (Senior clinical leadership 6/2).

Human work in the Command Centre

Capacity snapshot tile: informing patient flow planning and control

The primary goals of the tile are to: (1) provide real-time visibility of occupancy, available, clean and dirty beds as well as planned admissions, discharges and transfers and (2) highlight areas where demand (incoming transfers and admissions) exceeds capacity (number of available beds) to facilitate unblocking and problem-solving. The tile also pulls estimated and predicted discharge dates (EDDs, PDDs) from the Cerner Millennium EHR and uses this information in addition to other criteria (e.g. medically/functionally fit) to identify patients who are meant to be going home on a given day. If a patient appears on the tile but is not likely to be going home, it is an expectation for ward staff to update the information to ensure that CC and ward staff have an accurate picture of capacity and demand.

Example tile actions include:

1. Review wards with net negative beds to identify bottlenecks and wards that are struggling with capacity:

   • Prioritise incoming patients for those beds and consider alternative locations for the lower priority patients.

   • Prioritise tasks to complete ‘pending’ discharges to discharge more patients to create capacity.

   • Identify available capacity for a specific attribute (e.g. side room).

   • Create capacity as required for incoming patients through review of patients currently occupying those rooms for requirements.

One example of the way in which information reported on the tile is used to inform decision-making around movement of patients waiting for admission from the ED to hospital wards is shown below. CC staff view bed availability on the tiles and, where identified, will seek additional information as to appropriateness of placement (through screening patient notes):

I open the tile to see how many bed waits we’ve got. And I look to see if I’ve got any downstream beds that I can move patients from assessment areas into them. So I do all that on the tiles. I might think there’s a few there [on the wards], I wonder why when we’ve got patients waiting in A&E? So then I’ll go into the patients’ notes just to make sure there’s no reason why we haven’t used that particular bed.

Senior clinical leadership, 2/1

This may also involve seeking additional information at ward level as to any plans for available beds and prompts to create capacity for incoming patients: ‘I might ring the ward and say, “This bed’s available, do you have you any idea why it hasn’t been used?”’ (Senior clinical leadership, 2/1).

Patient transfers tile: responsibility for transfers across the whole hospital system

This tile helps with understanding where and how long patients have been waiting. It also highlights opportunities for patients to move based on downstream ward availability.

Example tile actions included:

• Flags when a patient has been assigned to a clean, available bed for 30 minutes or more as an indicator that the process of moving the patient has stalled. This prompts investigation into what barriers are preventing the move to be done.

• When multiple patients have been targeted to the same ward, the tile shows this and supports prioritisation to decide patient placement in those situations.

An example of the way in which the patient transfer tile has changed oversight and responsibility for transfers is shown below:

Having someone having oversight of every single patient transfer for the whole hospital, not just for medicine or surgery or orthopaedics but of the whole hospital that’s really an important part …. One of the benefits of having a command centre is all those metrics are pulled together in one place and … making sure that there’s someone responsible not just for that one patient on a ward but for the whole patients in the whole hospital is for me the difference the command centre makes.

Senior clinical leadership, 6/1

Deteriorating patient tile: using acuity data to manage risk

The deteriorating patient tile helps to identify and prioritise patients with deteriorating NEWS2 scores. In the CC, staff may use information on the tile to inform decision-making with regard to patient placement. Through filtering the tile at ward level, CC staff can determine areas with high-acuity patients and make decisions about ward capacity and staff allocation:

When [ward staff] say to us, ‘I genuinely can’t take this patient, acuity’s too high’, we can filter the tile to that ward and say, ‘well actually you’ve only got one patient with a high NEWS, are you talking about dependency rather than acuity’? And that will help us to determine do we need to send them an extra RN to support or can we send them a healthcare because it’s actually heavy personal cares that are more the issue.

Senior clinical leadership, 02/1

At ward level, information on the tile can inform the order that ward rounds will take to ensure that sickest patients are seen quickest. Operational leads reported that the tile was frequently used by the anaesthetic outreach team to filter for sickest patients and deterioration so that they can review those patients earlier (Senior CC and operational planning clinical leadership, 01/2 and 02/1).

Emergency department expediter tile

This tile provides information on the overall situation of the department (and department zones) including occupancy, escalation level (state of the ED based on locally agreed factors) and time to triage. The tile flags specific patient delays in care including breaches of wait-time targets and patients waiting to move to a different care setting (Document: training material). At an operational level, the tile was used to inform current pressures in the ED at strategic meetings across the day (Senior clinical leadership, 6/1;2).

Right patient, right place tile: detection and management of medical outliers

This tile displays inpatients who are outlying from a specialist area and aims to facilitate the movement of patients to appropriate locations. At ward level, medical teams can identify where the patients that they are responsible for reside so that they can be reviewed by the appropriate team daily (Senior hospital leadership and management, 06/1).

Senior Clinical Leadership staff check daily the number of outliers across the hospital and whether they have been reviewed (Senior clinical leadership, 2/1). Instances of non-review are then flagged up to the appropriate medical team (either via e-mail/WhatsApp or telephone) and raised at the operational meetings taking place across the day.

During early implementation, attempts to make use of the tile information at ward level exposed issues with data quality relating to patients being put under the wrong treatment function code and then showing up on the wrong consultants list. This meant that the tile was not fit for purpose in some areas:

As it turns out the treatment function codes for consultants is a very complex issue and patients are often put under the wrong treatment function code and they show up on someone else’s list and they’re not on their list for example ….

Senior clinical leadership, 1/2

The main data quality issues were addressed, and the tile was included in the programme for hospital-wide roll-out in 2022.

Integration of tile data to support operational oversight and planning

In February 2022, tile data were embedded within silver command/operational huddles that took place online at regular intervals during the day. Information on the tiles was used to support discussion and raise awareness around issues relating to flow: ‘The tiles are contributing to the group awareness of what’s going on to a different degree than you would have without the display system, so I think that adds a different level of understanding’ (Senior clinical leadership, 1/2).

Generation and use of intelligence

Information in the CC provided an up-to-date data feed that conveys current knowledge of hospital state for rapid decision-making and response. Intelligence was generated through tile activation and engagement with content, interaction with ward/department staff and discussion within teams.

Staff enactment of command and control

Informed by emerging themes from the cross-industry review, our analysis of the C2 work that staff do can be understood through three main concepts: monitoring, situational awareness and decision-making. In terms of monitoring, staff constantly observed and checked site positions in terms of patient flow and took actions based upon intelligence. CC staff maintained situational awareness through continuous monitoring and sharing of information coming into the CC, through formal and informal means, watching out for what is going on in the hospital environment. The action or process of making important decisions was visible through frequent discussion between clinical and leadership roles in the CC and responsive problem-solving. Examples of the way in which CC staff enacted C2 behaviour are shown in Table 11.

Context and culture in the Command Centre

The context and culture in the CC can be understood through artefacts staff engage with and intended purpose (environment), tempo of operations (speed and intensity of actions relative to the speed and intensity of unfolding events in the operational environment) and attitudes and values (culture of caring).

Variable data quality

Lag in some of the data presented in the CC and uncertainty regarding the current accuracy of information displayed in the tiles led to frequent activity to pursue up-to-date information and to manually check ‘real data’ sources. Staff working in the CC are almost constantly engaged with information-seeking and exchange through interaction with computers, telephones, staff coming into the CC, each other and conversations with staff in the ward environments. The majority of interactions observed concerned information regarding patient and bed figures; for example, ‘CSM1 on the phone when she asked “How many green patients have you got?” Ok, thanks’ (Field notes, 1) and ‘CSM4 was on phone with ED asking about figures’ (Field notes, 10). CC staff explained that the main purpose of telephone calls to the wards was to obtain up-to-date figures on bed state:

CSM4 stated to PCC1 ‘I’ll do Neonates and Covid shall I?’ To which they agreed. CSM4 then called around the wards to get their Covid figures. I asked why these were not available on the system in the centre to which they responded: ‘we need an up to date figure’.

Field notes, 1

Another example of the way in which data quality influenced intelligence and actions was observed in the silver command meetings:

Senior clinical leadership role stated that they reviewed the ward link tile and identified 24 patients with expired Expected Date of Discharge (EDD) and 62 patients with no EDD at all ‘so the information on Capacity Snapshot tile cannot be accurate’.

Meeting notes, 07/02

Tempo of operations and sense of pressure

Speed and intensity of actions relative to the speed and intensity of unfolding events in the operational environment.

In normal operations, pressure on CC staff was almost constant throughout the day and evening. Staff were always busy. As well as the routine work described earlier, unfolding events such as emerging pressures led to increases in intensity and speed of actions relative to the events. This often necessitated adapting ways of working:

On my arrival, a Senior clinical leader informed me: ‘It is already chaos. 105 people in ED following a big car crash. That’s why I am working at this (CSM) desk, to help get things moving quicker.

Field notes, 1

Emerging pressures for bed capacity gave rise to concern and tension for the team; for example, ‘During a silver command meeting discussion staff were discussing a situation where demand exceeded capacity, beyond that which was usual when CSM1-stated “I am nervous about the situation”’ (Meeting notes, 08/03). The researcher noted on these occasions that ‘the atmosphere was tense, air of panic’, for example (Meeting notes, 07/03).

Similar concerns were expressed in response to low staffing levels:

The member of staff stated to another ‘have you seen the RAG?’ sounded slightly panicked and concerned. I thought that this was in relation to the staffing levels RAG rating that had been circulated for the first time this morning. Levels indicated extreme pressures across the hospital.

Field notes, 15

During periods of reduced pressure, the atmosphere appeared relatively less tense, as seen in this example from field notes collected during an observation in the evening:

I noticed that there was relatively less tension/stress in the way in which the staff were working. Although there remained a constant workflow there was more interaction between staff unrelated to work in-between calls/data entry than I had noted previously (more relaxed, less formal on an evening? Related to relatively more beds available than I had observed on previous visits – see SitRep?).

Field notes, 10

Compassionate, personalised care for patients and staff

Staff working in the CC showed considerable compassion for patients and the staff working on the wards. Granular, personalised care for individual patients was apparent in many of their interactions; for example: ‘Patient needed to be moved but CC staff took into account the fact that the patient was diabetic and therefore needed to eat first’ and ‘Bless him, it’s his birthday [whilst referring to a patient due for admission]’ (Field notes, 6.2).

On occasions where CC staff interacted with ward staff in their departments, they provided support for staff experiencing difficult circumstances and shared their concern and response on negative outcomes:

The ED shift leader also discussed another patient of concern in the department. They mentioned that although vital signs were within acceptable limits that the patient did not look well at all (I got the impression that the situational update between staff was much more than numbers including tensions/concerns/atmosphere within the department that would not be captured within technical systems). When we returned to the Command Centre shortly after this, CSM4 checked up on the patient and informed me that the patient had been transferred to ICU and that there was likely to be a devastating outcome. The atmosphere at this time was flat as staff expressed sadness at the loss of a young patient.

Field notes, 10

Staff were observed to express gratitude, familiarity and warmth in their communication with ward staff: ‘CSM3 was on the phone and asked “can I have your bed please … thanks love, thank you … take care, bye”. Appeared to be asking for an update and expressing gratitude, familiarity, warmth in communication style’ (Field notes, 1).

Examples of the way in which CC staff were involved in personalised care interactions are shown in Table 12.

Responding to unanticipated events and escalating pressure

Table 13 shows the themes derived from analysis of CC staff interactions when responding to unanticipated events and escalating pressures. During the evaluation period, staff were operating within an environment that was unprecedented due to the pandemic. Often, during silver command meetings, staff commented on the pressures for example: ‘Trying to fast forward flow as this winter is like nothing we have ever seen before’ (Meeting notes, 08/02:2) and ‘Staff commented that the last 24 hours was the most challenging of their career’ (Meeting notes, 08/03:1).

Collaboration between staff members was often triggered by unexpected events; for example, ‘Sometimes CC staff will say out loud “well that’s strange …” and this triggers collaboration on clarifying unexpected data or identifying an investigative action’ (Field notes, 6.2).

On occasions where the information was not clear and/or the emerging issue could not be resolved immediately, CC staff would visit the ward/department to gather further information and support the situation:

Some discussion ensued about a need to deploy staff to the ED. PCC1 suggested moving a staff member (who was specialling a patient on a ward that was at risk from falls). CSM4 replied ‘We will see if that patient is settled before we think about moving that staff member again’. (I got the impression that they were looking out for the member of staff’s well-being whilst making this decision). CSM1 and PCC1 then agreed to go to the ward to look for themselves at the situation and then left the room.

Field notes, 10

Perceptions of the impact of the command centre on intended and unintended outcomes

In addition to observation of behaviour in and around the CC, during interviews with a range of staff connected to the CC programme, perceptions of the efficacy and impact of the CC in meeting its objectives were explored, along with potential impacts upon upstream operational management and organisational processes. Several themes emerged (Table 14) representing both intended and unintended consequences of CC implementation and operation.

Staff expressed views that the CC facilitated efficient bed and staff allocation, in-hospital transfers and planning for patient flow, not only easing flow but ensuring that patients were placed in the right destination initially to avoid the need for subsequent transfer. The perception of the CC was of a focal point for operational decision-making and problem-solving that extended beyond CC staff and operations managers. Front-line co-ordinators could contact or attend the CC in order to resolve complex operational issues that spanned functional areas and the view was expressed that the CC facilitated a system-wide perspective on these issues. This effect was enhanced through colocation of staff with different functional perspectives (namely clinical and operations management) in the same central location with access to system-wide data. The ethnography demonstrated substantial interactions between CC staff and multiple information systems as well as constant communication among the wider hospital services and wards. Characteristics that appear to support co-ordination and communication included the physical layout of the CC where site staff are located closer facilitating frequent informal communication and centralised access of diverse information (e.g. bed management software) for the site team to support co-ordination.

Respondent testimony suggested that the CC programme objectives to enhance the availability and use of real-time intelligence in hospital operations management had at least partially been met (see Table 14). The institutional response to the COVID pandemic was often cited as an example of effective use of real-time (or near real-time) data to monitor and respond to new cases of infection across both COVID and non-COVID wards. In broader areas of risk management, such as monitoring patient deterioration, views were expressed that the CC facilitated early detection and monitoring of at-risk patients, enhancing situational awareness. Finally, the implementation and subsequent operation of the CC had a secondary effect of enhancing data quality and awareness of data quality issues within the organisation.

Challenges in implementation and operation

In analysis of the interview data, we sought evidence of limitations of the CC system, both emergent and against espoused goals, along with narratives describing the inherent challenges of implementation and operation of this type of programme in an acute care setting.

An important unintended consequence of the CC implementation reported by both front-line and CC staff was a sense of being monitored among front-line units, sometimes leading to interventions (or fear of interventions) from the CC team that were perceived as unwelcome. This included sometimes challenging conversations about the importance of keeping electronic records up to date or acting on evidence of operational issues that were seen as being under local autonomy or ownership, rather than a centralised issue. It was suggested that front-line staff in some areas might delay updating records because they knew the CC team would be quick to respond with new allocations that would amount to increased local operational pressures. It was clear from subsequent interviews with both implementation and front-line staff that the reach of training on both electronic patient record (EPR) and on the intended function of the CC was limited. This may have influenced front-line staff engagement with the CC and the actions that they took, or not, to support functionality.

Data quality was a constant concern for staff working in and around the CC, and it was clear that there were limitations in how up to date and accurate (or complete) records were in the CC wall of analytics, often necessitating triangulation and verification from multiple sources and systems and discrepancy between what the CC wall of analytics displayed and what front-line staff reported. The added value of specific tiles was also questioned by front-line operations leads who believed that they already had access to real-time accurate data through alternative systems and that this had led to declined local uptake and use of the tile data. The design of the data integration pathways and systems additionally meant that certain conditions needed to be met by front-line units, in terms of updating electronic records and use of electronic systems, in order for displays and alerts to trigger properly in the CC. This necessitated some adjustments to human workflow at the data source. A summary of themes relating to perceived challenges in implementation is shown in Table 15 along with illustrative quotes.

Impact on patient safety

Table 22 presents a summary of the key indicators for patient safety based on the five phases of the intervention.

In BRI, when compared with the pre-intervention period, the weekly mortality decreased by 1.4% (95% CI 0.8% to 1.9%), 1.5% (95% CI 0.9% to 2.1%), 1.3% (95% CI 0.8% to 1.9%) and 2.5% (95% CI 1.7% to 3.4%) during the first (‘patient flow programme’), second (‘command centre tile roll-in’), third (‘command centre goes live’) and fourth (‘hospital-wide engagement resumption’) intervention periods, respectively. During the first, second and third intervention periods, the weekly per cent of re-admission within 72 hours also decreased by 2.7% (95% CI 1.7% to 3.8%), 2.5% (95% CI 1.4% to 3.6%), 2.0% (95% CI 1.0% to 3.0%) and 0.7% (95% CI 2.2% to 0.9%), respectively. The weekly postoperative sepsis data did not show a significant change during the study period.

In CHFT, compared to the baseline, the weekly mortality decreased by 2.0% (95% CI 3.1 to 0.9), 2.3% (95% CI 3.5 to 1.1), 1.3% (95% CI 2.4 to 0.2) and 3.1% (95% CI 4.8 to 1.4) for the respective intervention phases of BRI. However, except for the first intervention period, re-admissions within 72 hours showed a significant increase during the second (change = 2.6%, 95% CI 1.6 to 3.5), third (change = 3.6, 95% CI 2.7 to 4.5) and fourth (change = 2.2, 95% CI 0.8 to 3.5).

When BRI and CHFT are compared in regard to the indicator outcome changes during the study period, the weekly mortality significantly improved while the weekly re-admissions showed improvement in BRI hospital and otherwise in the CHFT.

Impact on patient flow

Table 23 presents a summary of results for patient flow. There was no significant difference in the weekly average of inpatient length of stay between the pre-intervention and the post-intervention periods. Table 24 shows the A&E waiting times and showed an increase of 62 minutes (95% CI 40 to 85 minutes) during the fourth (‘Hospital-wide engagement resumption’) intervention period when compared with the pre-intervention period. The first and second intervention periods also showed a decrease of 10 minutes (95% CI 4 to 16 minutes) and 9 minutes (95% CI 2 to 15 minutes) in the average A&E clinician seen time when compared with the pre-intervention period.

The transition time from arrival to assessment consistently improved during the intervention period; there were decreases of 0.9 minutes (95% CI 0.35 to 1.4), 3 minutes (95% CI 2.4 to 3.5), 9.7 minutes (95% CI 8.4 to 11.0) and 3.1 minutes (95% CI 2.7 to 3.5) during ‘patient flow programme’, ‘command centre tile roll-in’, ‘command centre goes live’ and ‘hospital-wide training programme’, respectively. However, the transition time from assessment, treatment and visit conclusion to the next respective A&E stage of care had increased significantly.

Impact on data quality

The data quality did not change significantly during the study period, except the weekly proportion of missing clinician seen dates significantly worsened during the ‘hospital-wide engagement resumption’ period when compared with the pre-intervention period (change = 17%, 95% CI 10.4% to 32.5%). Likewise, there was a significant increase in the weekly proportion of treatment dates missing during the ‘command centre tile goes live’ period when compared with the pre-intervention period (Table 25).

The proportion of arrivals and ‘concluded visits’ that progressed to the next ‘valid’ stage of A&E care (i.e. assessment and check-out, respectively) had largely improved during the intervention period. However, the proportions of those who were assessed or treated that progressed to the next ‘valid’ stage of A&E care (i.e. treatment and concluded visits, respectively) were consistently lower than the pre-intervention period (Table 26). However, the proportion of ‘reversal sequence of events’ or ‘left-shift’ (treatment to assessment and check-out to conclusion of consultation) worsened during the intervention period when compared with the pre-intervention period (Table 27).

Sensitivity analysis

Theoretical description and application of transferable command centre functions

In order to develop transferable intervention logic for hospital CC programmes, we sought to integrate theoretical perspectives on effective C2 from our cross-industry review with insights from our empirical observations and from the testimony of key informants interviewed during the course of the study.

In considering the lessons from synthesis of cross-industry expertise and perspectives for a theory of effective C2 in health care, the following functions of effective CCs were identified (Table 33), along with sources of evidence and rationale for their efficacy.

The findings from the cross-industry review suggest the importance of developing a mature framework of protocols for the operation of a CC, in addition to the installation of requisite technological functions and the integration of diverse organisational roles in the approach to staffing the CC. As a controlling system, the CC’s operating protocols should be defined relative to a commonly understood model of the ‘stable’ system state (relating to the system under control). Operational behaviour within the CC may then switch between varying modes in response to detected variations in system state. In high-risk industries, the advantages of this approach to control are that the modes of operation may be rehearsed through operational experience, training and simulation in order to establish routines that are geared towards, for example, efficient information-seeking, effective intensification and direction of the focus of monitoring activity, division of resources to explore future contingencies over varying temporal horizons, and other behavioural routines that represent preparedness, resilient responses and agility in constructing an effective model of uncertain/novel/unstable situations.

Critical to maintaining situational awareness and the capacity for timely operational intervention is instantaneous presentation of information concerning system state (i.e. in real time without lag), the completeness of information (or having the right information), the accuracy of the information displayed as a true representation of the system state, and, where any of the aforementioned may be questionable, knowledge or awareness of the likely areas of discrepancy and hence its potential impact on the capacity to make effective responses (system transparency). While the example of a CC implementation that we studied in this project integrated a broad spectrum of data and information linked to a complex system, there were scenarios in which data completeness, accuracy and lag were not optimised, resulting in the team’s resources being consumed in secondary activities such as data validation and workarounds.

Intervention logic for a digitally enabled hospital command centre

In order to support future development, implementation and evaluation of CC programmes in health care, we synthesised key themes from the qualitative analysis, contextualised theory from the cross-industry review on C2 and constructs from Implementation Science theory describing known implementation factors to produce an intervention logic model (Figure 8). Here we refined themes into named variables and factors that we could subsequently classify according to a higher-level scheme representing causal sequence and processes or mechanisms driving outcomes. The framework was informed by constructs from the Consolidated Framework for Implementation Research, including consideration of the inner setting for the intervention (e.g. culture, implementation climate, structural characteristics of the organisation), outer setting (e.g. external policies and patient needs), implementation processes (e.g. engaging, executing) and nature of the innovation. 127,128

FIGURE 8.

An intervention logic model for digitally enabled hospital CC implementations. GPs, general practitioners; EDD, expected date of discharge.

Implications of the logic model for evaluation of command centre implementations

In the structure of the model, ‘preconditions and enablers for the command centre’ and ‘organisational context’ refer to the factors identified within the study that enabled the CC programme to be initiated or that represented the impetus for the programme in the first place, including the requisite hospital data infrastructure, developer support, leadership vision and operating culture. We then represent factors concerning the programme design and implementation model within the framework, including both the key functional concepts (such as data integration, real-time visualisation and joint clinical–operational leadership) and development/implementation approach (e.g. iterative roll-out, user-centred design and change-management structures). Crucially, our ethnographic and qualitative investigations demonstrated the complex interplay between technical system design, the human and organisational dimensions of effective C2, and the implementation or change process undertaken to embed the system and promote uptake and engagement throughout the organisation.

The central category within the intervention logic model is concerned with organisational behaviour and more specifically the activities and processes that CC staff, programme leads and operations managers were engaged in during our ethnography. The daily business of CC operations included centralised, system-wide monitoring and operational oversight, including detection and mitigation of threats to the stable state of the system, in accordance with knowledge of C2 behaviours in organisational settings across a range of applications studied in our cross-industry review. The ethnography, however, additionally identified human activities and organisational behaviours in and around the CC that required a somewhat deeper immersion in the sociocultural context in order to understand their full significance and meaning. A considerable proportion of CC activity was undertaken to promote engagement across the front-line units of the organisation and develop or maintain relationships with key staff in front-line areas to promote effective two-way communication and co-ordination of activity. CC staff additionally made efforts to host visits to the CC from all areas of the organisation in an attempt to understand and resolve local operational issues and support staff in their remote operational roles. The CC therefore came to represent a focal point for escalation of what were often perceived to be hitherto intractable issues. It was additionally interesting to note that despite the high-level, centralised oversight afforded the CC team staff, and despite the high volume of individual patients represented in displays and metrics presented within the CC, the team showed particular diligence in providing a level of granular, personalised care for individual patients wherever the opportunity presented itself. This might include, for example, interrupting the flow of intrahospital transfer to delay an individual’s bed move until after a scheduled mealtime. Such behaviours demonstrate the value of including experienced front-line operational knowledge (in this case, nursing) in the CC team’s composition. It was suggested to the research team on more than one occasion by programme leads that the staffing composition and level of experience were critical if the team was to understand what the metrics displayed within the CC actually meant in operational terms on the ground.

Regarding evaluative outcomes from a CC programme of the type implemented in the study site, we have distinguished between intermediary (proximal) outcomes and long-term (distal or downstream) outcomes in our logic model. The rationale for this distinction is twofold. Firstly, the long-term outcomes tend to represent more pervasive, system-level parameters linked to quality, safety and efficiency of care within an acute trust and as such are subject to a broad range of influences (both within the control of the organisation and external to it, the pandemic being a prime example of the latter in this case). Secondly, specifying intermediary or proximal outcomes allows us to identify a range of factors that may be more directly influenced by the intervention itself drawing upon evidence from our investigations as to how organisational behaviour, decisions and intervention actions originating within the CC influence the broader system of care. Intermediary outcomes therefore represent, in many cases, the mechanisms by which the design and operation of the CC drive some proportion of variance in long-term outcomes. It should be noted, for example, that the CC operational focus was predominantly management of patient flow originating through emergency or non-elective admissions. We would not expect, therefore, to see effects of CC programme implementation or developments in certain service areas within the broader organisation, such as in maternity services. Similarly, high-level outcomes such as length of stay and wait times should be disaggregated to a level of granularity in which the impact upon specific service areas and waiting targets that are sensitive to this intervention can be evaluated.

An important finding from the study concerns the conception of a CC implementation which, based upon our experience in this study, should be regarded as a long-term programme of organisational development rather than a discrete technology deployment. In our logic model, a key process is ‘continuous review, development and integration of the CC within the broader system’. CCs require the capacity to evolve and procurement of off-the-shelf solutions with licensed support packages might actually hinder this process, by not being responsive enough to respond rapidly to emerging use cases, in comparison with ‘in-house’ developments. More specifically, each adopting organisation is likely to have specific user requirements and a specific data infrastructure profile that may make an off-the-shelf solution suboptimal for the needs of the organisation. Possessing ownership of the mechanisms for data integration (to facilitate rapid manipulation, troubleshooting and transparency in how metrics are compiled), our study suggests, is key to human confidence and trust in the system. Similarly, having uninhibited access to the organisations ‘raw’ data within an EHR would seem to be a prerequisite for organisations considering taking the step towards data-driven operational C2.

We observed certain instances where data lag and/or duplication of functions with parallel systems rendered data and flags within the CC obsolete and the team looking to alternative sources to maintain situational awareness. Effectively, the team’s mental model of system state tracked ahead of the presented data model of the system in the CC and drove interrogation of the presented data. The team essentially had developed ad hoc means of triangulating multiple data sources to arrive at a view in which they had confidence and in which the CC analytics was only one source of information. This was at odds with the system design rationale which was to create a real-time and accurate integrated source of information that would enhance decision-making by providing useful configurations of data with prioritisation flags to help guide attention and decision-making. Taking the CC system as a whole, there was a gap between what the underlying data model was delivering and what it might have contributed to support the human and organisational activity of operational C2; a gap that wasn’t quite closed, despite ongoing development efforts, by the end of our research involvement with the centre. This was in part due to local data-quality issues, but additionally inflexibility in systems that integrated and translated data streams and reliance on external providers to resolve suspected problems. In the design of the CC, where interplay between data presentation and human decision-making is concerned, consideration of system-level, in addition to tile-level, use cases and the human factors involved in interactions between the team and the CC system as a whole is implicated for future development.

Finally, a key finding from analysis of the many recorded narratives and field notes within our study was the emphasis placed upon the notion that effective digitally enabled hospital C2 was not just a technology-driven endeavour but a true sociotechnical system or intervention, in which the human and organisational work of C2 was of equal (perhaps even more) importance to the successful functioning of the system compared to the digital components. In practical terms, our intervention logic model therefore represents a sociotechnical framework for reasoning about the prerequisites, design elements, implementation factors, implicated organisational processes and proximal/distal outcomes for digital hospital CC implementations as long-term development programmes, based upon our experience in one such programme. Our observations and review of the programme at Bradford confirmed what the local developers and implementers knew at outset, that an undertaking of this nature was not simply a technology investment or informatics project, but a programme with considerable organisational change implications spanning all levels of the organisation and its front-line units.

As a complex organisational intervention, introducing centralised, digitally enabled hospital command systems may give rise to a range of intended and unintended consequences that affect human and clinical workflow across the system, in pursuit of data quality, as well as operational efficiency. The implementation of a system representing (even conceptually if not in actuality) a centralised system of close oversight, coupled with the capacity for reactive intervention in any area of the system, may challenge culturally engrained notions of clinical autonomy and self-regulation by front-line care teams. Again, this emphasises the importance of the right implementation model for such a system and crucially how its purpose and operation are presented to staff. The very language of military style ‘command and control’ may not be helpful here, as indicated by findings from our review, and there is considerable scope for additional work to understand how this type of initiative may be best adapted for the healthcare context through further robust evaluations of specific implementations and implementation models.

Impact on patient safety

In the BRI hospital, when the process of change and technology were considered as a ‘whole package’, only two of the three patient safety indicators showed improvement during the post-intervention period when compared with the pre-intervention period. In particular, the mortality decreased by 1.4%, 1.5%, 1.3% and 2.5% after implementing a ‘patient flow programme’, ‘command centre tile roll-in’, ‘command centre goes live’ and ‘hospital-wide engagement and training’ programmes, respectively, compared to the pre-intervention period. Likewise, the re-admission within 72 hours also decreased by 2.7%, 2.5% and 2.0% for the ‘patient flow programme’, ‘command centre tile roll-in’ and ‘command centre goes live’ periods, respectively, when compared with the pre-intervention period. However, the same degree of improvement in mortality was also observed in the control site during the same period; 2.0%, 2.3%, 1.3%, 3.1% for the respective intervention phases of the BRI hospital.

Impact on patient flow

The introduction of the CC did not have a significant effect on the three out of three of patient flow indicators. The length of stay only showed a non-significant decrease of 8.8 hours [standard error (SE): 4.5], 8.9 hours (SE: 4.9), 1.7 hours (SE: 4.4) and 0.55 hours (SE: 6.8). The waiting (time taken until patient treatment), clinician seen time (time until patient is seen by a clinician) and A&E transition time (time taken to progress from one stage of A&E care to the next stage of A&E care) also did not significantly improve during the study period.

Impact on data quality

The state of the data quality, measured as the proportion of missing dates, A&E visits progress to the next stage of care and ‘reversal sequence of events’, was worse when the CC was introduced, albeit non-significant. On average, the proportion of missing in treatment date (42.7%) and clinician date (23.4%) was substantial. Moreover, the missing proportions were worse during the post-intervention periods than the pre-intervention period. For example, the missing in A&E visitor’s treatment date was higher by 3%, 10.2%, 21.5% and 2.3% when a ‘patient flow programme’, ‘command centre tile roll-in’, ‘command centre goes live’ and ‘hospital-wide engagement resumption’ programmes were implemented, respectively. These suggest that the use of a CC may not have had any measurable impact on data quality.

Comparison with other studies

There is a paucity of studies that investigate the impact of multidepartment hospital-based CCs on patient safety, patient flow and data quality. A recent report from the Saudi National Health Command Centre indicated that the use of a ‘smart centre’ had led to mortality rate below 2% and reduction of intensive care unit (ICU) lengths of stay by 10% in emergency admissions. 21 While the mortality is in line with the findings from BRI emergency admission data the ICU length of stay reported by Alharbi and colleagues21 disagrees with the findings of this study. In fact, the average ICU lengths of stay (as calculated separately) for pre-intervention and post-intervention periods are 91 and 108 hours, respectively, which is an increase of 18.7% after implementation of the CC. One notable difference between the study and Alharbi et al. ’s21 is that the Saudi Arabian Command Centre was a national hub and the data used were of the first wave of the COVID-19 pandemic whereas the Bradford Command Centre was used only in the BRI hospital and more than 3 years’ worth of data were used for analyses.

In a report by Johns Hopkins Hospital, transfers of patients from other hospitals were improved by 46%, ambulance dispatch times reduced by 43 minutes and bed allocations of emergency admission patient reduced by 3.5 hours. 22,129 In addition, the CHI Franciscan Mission Control Centre also reported that the lost cases were reduced by 20% in the first 6 months of the year. 19 However, we do not have comparable data in the study and these two CCs were used as hubs for their respective groups of hospitals, unlike the Bradford Command Centre, which only serves the BRI hospital.

Objective 1 – evaluation of the impact of the command centre

The CC was observed operating effectively throughout the study period. Our ethnographic observations and interviews with 15 study site staff provide documentary evidence of successful use in a complex environment. The CC made a significant impact on the management of the hospital through the pandemic, including through the introduction of a COVID-19 ‘tile’ which was used to manage COVID-19-specific processes. Our results from ethnographic interviews and observation describe how the CC, and its multi-professional team of combined clinical/operational staff, worked with the new technology to change the way that the hospital operates. We identified unintended consequences that included front-line staff developing a sense of being monitored and interventions (or fear of interventions) from the CC team that were perceived as unwelcome. Linked to this were challenging conversations about the importance of keeping electronic records up to date or acting on evidence of operational issues that were seen as being under local autonomy or ownership. Data quality was a constant concern for staff working in and around the CC and there were limitations in how up to date and accurate (or complete) records were, often necessitating triangulation and verification from multiple sources and systems and showing discrepancy between the data in the systems and what front-line staff reported. Compromise, goodwill and a shared sense of purpose were necessary to ensure the CC was effective.

We were able to extract time-series data on patient safety, patient flow and data quality from operational systems by selecting representative indicators and plotting these over time. We were able to measure changes in these indicators over time and evaluate statistically the long-term impact of the CC on these indicators. We were not able to isolate improvements in these indicators that could directly be attributed to the introduction of the CC. Similarly, we were unable to isolate noticeable improvements in these indicators between study and control site. When the study site and control site were compared, we found improvements in mortality and reduced rates of re-admission at the study site but caution against drawing conclusions from this at a time when the pandemic was raging. Some indicators, notably data quality, worsened rather than improved. We reason that the pandemic had such a profound impact on all aspects of operation that it is not possible to separate out and measure the impact of the CC. Similarly, the later adoption of a CC approach by the control site means we cannot use it to draw strong comparisons.

Objective 2 – understanding the process of implementation of the command centre

Our project started after the CC had been implemented so our results rely on staff recall of the implementation. We identified five phases in the implementation: (1) pre-intervention, (2) a patient flow change programme, (3) CC tile roll-in, (4) CC go-live and (5) post-intervention engagement. Phase 2 was an organisational change, Phase 3 represents a soft-implementation period of training and familiarisation and Phase 4 represents the hard implementation of the new technology and new ways of working. Staff interviews suggest that the overall implementation approach was effective but that they found the implementation challenging and identified some need for more training and software improvements. The intention had been for a period of post-intervention engagement to support staff in getting used to new ways of working and to adapt procedures and technologies to optimise the new approach. This was disrupted by the pandemic, which started to impact on hospital operations only a few months after Phase 4 was complete. Staff recollections are therefore mixed between the pandemic response and the new technology, but there is strong evidence that staff worked well together to find ways of working that were consistent with the CC approach while solving immediate challenges.

Analysis of time-series data on patient safety, patient flow and data quality at different stages of implementation revealed patterns of change in response to the implementation, but these were confounded by the impact of the pandemic on the same outcome measures. When only the technology part of CC was assumed as the intervention, there was no significant difference between the pre- and post-intervention periods in the patient safety and patient flow indicators. The data quality had largely worsened in the post-implementation phase, and we attribute this to the impact of the pandemic. Qualitative results show that the CC has had a long bedding-in process and that this is expected to be a long process as the hospital and its staff adapt to new ways of working. Our qualitative results suggest that major improvements in patient flow, patient safety and data quality have yet to be achieved.

Objective 3 – contextualising the findings using cross-sector and cross-industry perspectives

Results from the literature review found a strong body of research to support the adoption of a CC approach as part of a successful and resilient organisation. CCs are described as supporting situational awareness, decision-making, team structure and workload with the main aim of successfully delivering safety-critical operations reliably over time and in the face of dynamic risks and variations in the operating environment and system conditions. Digital technologies need to be tailored to the work done in the respective domain and should contribute to system resilience. Most articles attribute performance improvements to the physical and functional features of the centres themselves; this often includes the use of technology to generate and display real-time and/or predictive data in the centres. The implementation process usually affects process and policy changes in the organisation, including introducing new ways of working and workload distribution, adding new roles and altering the existing hierarchy of decision-making and responsibility. The literature advises caution in attributing improvements to the physical and functional aspects of the CCs versus the process and policy changes within the organisation that often arise out of the implementation process.

There is emerging evidence that a CC approach can be adopted in acute health care. Effective implementations are characterised by a strong sense of shared situational awareness within a team, with a shared focus on specific focal points for intelligence and intensification of this focus as the threat level increases. System resilience is maintained in these implementations through simultaneous responsive and anticipatory strategies with variable resource allocation for both proactive planning for expected deviations and events with varying timescales. One paper advised caution in using the term ‘command and control’, as it may overly restrict the exploring of new ideas and new approaches seen as important to meeting the specific needs of health care, hospitals and staff given the strong culture of autonomy on the clinical front line.

Objective 4 – synthesis of research findings to inform future investment and practice

The results of our qualitative investigation correspond to themes identified in the cross-industry review. Tensions between the CC and local decision-making within hospital departments were evident throughout the interviews and observations and closely reflect existing literature from other industries. Digital technology should make operational activities transparent for all, but the literature emphasises the importance of systems that are closely tailored to needs and information that are reliable. We identified two major challenges: (1) a strong culture of local autonomy in operational, as well as clinical, decision-making and (2) concerns that data within systems may not be sufficiently accurate or up to date to reflect the reality on the ground. Both challenges are widely recognised in the healthcare literature but are less dominant in other industries with stronger traditions of C2 and well-established, centralised systems. Our quantitative work reflected the data challenge – it was time-consuming and difficult to extract operational data and compare performance metrics and our approach is relatively novel in the literature and for the study site.

The implementation of the CC at Bradford has been a success in changing the operational approach. We found evidence that the approach to implementation was broadly successful but that benefits take time, and significant additional innovation to realise. We consolidate our learning as an intervention logic model that can be used by other hospitals planning an implementation of a digitally enabled hospital CC. This study was limited by the pandemic and further work would be needed to make stronger recommendations for investment and policy.

Continued study of the Bradford Command Centre

As the first of type in the UK NHS, the Bradford Command Centre should be a national reference point for other NHS hospitals considering similar approaches. Our report contains many details that should be of interest to the stakeholders involved in similar projects and the management team at Bradford have been open to showcasing their approach to colleagues. Continued study of the evolving adoption of the CC technology and working practices at Bradford should provide a longitudinal perspective that goes beyond the decision to adopt a CC by generating learning about how to sustain success in the long term.

We recommend:

1. Further dissemination of the study findings through a wider range of academic and non-academic channels. Specifically, the lessons learnt from the study should reach policy-makers, clinical and management decision-makers, NHS staff, patients and the general public via informative videos, infographics, presentations and workshops at NHS conferences, CIO and associated networks, invited talks and social media. We will work with NIHR and others to actively disseminate this impact.

2. A follow-up study at repeated intervals of approximately 2 years should be able to track the evolution of the Bradford Command Centre and ‘tell its story’ with lessons learnt to inform other adopters. This should be achievable with a ‘light touch’ by repeating our mixed-method study design combining ethnographic observation within the CC and follow-up interviews with previous study participants and others in relevant roles. Having built the infrastructure for our quantitative analysis using routine data from the Connected Bradford Data Service repeating the data analysis involves relatively little additional cost. There is potential for Bradford, as the first of its type in the UK, to continue to help inform the NHS for many years to come.

Implementation of command centres in National Health Service hospitals

In common with many organisations, the NHS has a tendency to look for efficiency savings to fund capital investment and widespread organisational change while retrospective evaluation tends to focus on the effectiveness that was achieved. This case study shows that effectiveness can be achieved.

We recommend:

1. Command centres are a viable approach to improving hospital management that should be considered for future investment and practice.

2. The technology prerequisite for a successful CC is that it must be built on a foundation of reliable, modern hospital-wide information systems. Poor systems integration, poor usage and above all poor data quality will undermine implementation if not addressed. A new CC system is unlikely to be right first time so software changes should be anticipated and budgeted for.

3. The organisational prerequisite for a successful CC is that the strategic leadership team are prepared to sustain support for an investment in organisational change as well as the additional technology and estates cost. Our case study shows that higher visibility of patient flow and centralisation of decision-making can be uncomfortable for staff, local management and potentially patients too. At Bradford, incremental, responsible and responsive change backed by enthusiastic leadership has proved successful and is an approach that is likely to be effective elsewhere.

Further research on the potential for command centres to transform National Health Service operational management

This study was not able to provide definitive evidence to demonstrate major benefits for widespread adoption of CCs. It does, however, provide a base for others to do so. Our cross-industry review demonstrates solid benefits to operational management that CCs have provided to other industries where either safety is a major concern or where flow needs to be efficiently managed. NHS hospitals require both safety and efficient flow. They are, however, highly complex relative to other industries that have implemented CCs so it is reasonable to conclude that it will take time and considerable study to find the best way that CC approaches can be adopted before their full potential can be realised.

We recommend:

1. Further studies will be needed to prove a more extensive assessment of the critical success factors and benefits case for NHS CCs. Such studies would do well to follow a mixed-methods approach rather than relying on just quantitative data (we have found it hard to attribute longitudinal change to specific interventions) or qualitative data (we have found this rich in anecdotal evidence and learning but insufficient to be conclusive). Specifically, a benefits-management approach could be embedded within new CC implementations and linked directly to the research methods employed in the study following an action-research philosophy.

2. Our work developing a long list and then a short list of indicators for patient safety, patient flow and data-quality merits further development. In our limitations we noted that a wider list of indicators would have been more effective and access to richer data would be a benefit. Data analytics using routine EHR data is becoming more firmly established with new machine learning and AI techniques able to identify, learn from and potentially act autonomously on changes in trends in time-series data. Process-mining proved useful in understanding, mapping and measuring flow and has the potential to be used for the evaluation of CC interventions and, in time, can be built into more advanced CC systems as process-aware intelligence. Standardisation on indicators between hospitals and across CCs would allow comparison across more hospitals and support the evaluation of other implementations.

3. An NHS CC implementors network would allow researchers and practitioners who are interested in understanding the challenges and potential of CCs to share learning, research methods and critical success factors. An approach that embeds research about CCs with NHS activity implementing them is likely to support rapid adoption and transformation of the NHS more dynamically than long studies that develop an evidence base before action.

Study participants

First and foremost, we thank the NHS staff working in and around the Command Centre at Bradford Teaching Hospitals NHS Foundation Trust and in key operational roles across both the study site and the control site, for giving up their valuable time to be interviewed by the research team and for supporting our ethnographic work on site during what was an unprecedented period of demand and challenge for the service. We hope that the insights generated within the research will help to replicate their commitment and innovation in future service developments elsewhere.

Study Steering Group

We thank members of the Study Steering Group who have provided invaluable insight, guidance and motivation throughout the project: Iain Buchan (Chair), Hilary Thompson (PPI representative), Paul Charnley, Sarah Cindy Fedell, Hamish Fraser, Charles Koo, Farah Magrabi, Sue Mason, Mark Sujan, Sean White, Patrick Waterson. The Workstream 3 team in particular would like to thank Hilary Thompson for her help in securing access to the control site and Patrick Waterson and Mark Sujan for generously making their expertise in human factors and safety science available to the cross-industry review work.

Patient and public involvement and engagement

We thank members of the Yorkshire Quality and Safety Research group patient panel and participants of our patient and public workshops for providing patient and public perspectives on the design, interpretation and communication of the study.