Clinical effectiveness and cost-effectiveness of emergency surgery for adult emergency hospital admissions with common acute gastrointestinal conditions: the ESORT study

Article history

The research reported in this issue of the journal was funded by the HSDR programme or one of its preceding programmes as project number 18/02/25. The contractual start date was in October 2019. The final report began editorial review in November 2021 and was accepted for publication in May 2022. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HSDR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Copyright statement

Copyright © 2023 Grieve et al. This work was produced by Grieve et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaption in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.

2023 Grieve et al.

Background and rationale

Emergency surgery (ES) poses a considerable global burden to health systems and is associated with high morbidity, mortality and resource utilisation. 1 ES accounts for approximately 750,000 admissions per year in England alone,2 with surgical procedures representing approximately 10% of the annual NHS budget. 3 Following the COVID-19 pandemic, in England, NHS waiting lists are projected to reach 13 million by 2025, with implications for hospitals’ capacity for elective surgery and ES. 4 For common acute conditions that present as emergency admissions, an important clinical decision is which patients should receive ES and which patients should receive non-emergency surgery (NES) strategies, which can include medical management, non-surgical procedures (e.g. radiological-guided drainage of abscess) or surgery deferred to the elective (planned) setting.

In England, although there are approximately 4000 NHS consultant general surgeons who spend, on average, 50% of their time on emergency general surgery admissions, there is insufficient capacity to provide ES 24 hours a day, 7 days a week. 2 In 2016, there were 697,314 emergency general surgical admissions to NHS trusts in England, of which 305,507 (43.8%) did not receive an operative procedure. 5 The Getting it Right First Time report2 for emergency general surgery found wide variation across NHS trusts in care quality and outcomes after ES, which may reflect local logistical and resource constraints, but also clinical uncertainty. 5

Emergency surgery rates for patients with acute gastrointestinal conditions have declined over the last 20 years,6 and protocols for NES strategies have been developed and implemented as part of randomised controlled trials (RCTs). 7–12 However, relatively few RCTs in the emergency setting have compared ES and NES strategies for patients with acute conditions and, in general, these trials have included an insufficient number and range of patients to inform routine service provision. For patients with uncomplicated acute appendicitis, several RCTs and ensuing meta-analyses have compared ES with NES strategies, but the evidence is equivocal. 7,8,13 For patients with acute cholecystitis, published RCTs have reported reduced complications following ‘early’ and ‘delayed’ surgery, but have not considered non-operative strategies. For patients with acute diverticular disease, NES strategies are well developed, but published RCTs have considered different forms of ES, rather than comparing ES with NES strategies. 14,15 For other acute conditions presenting as emergency admissions, such as abdominal wall hernia and intestinal obstruction, no published RCTs of ES versus NES strategies exist, amid ethical concerns about randomisation. 16

In observational studies comparing ES with NES strategies, the major concern is confounding by indication (i.e. patients who receive ES may be sicker). 17 Traditional risk adjustment methods are unable to fully allow for prognostic differences between the patients receiving NES strategies because information, for example from radiological investigation, is not available within routine data sources. As these unmeasured variables may predict both ES receipt and outcome, these studies are liable to provide biased estimates of the effectiveness of ES. 17

The appeal of an instrumental variable (IV) design is that it can provide accurate estimates of treatment effectiveness even when there are unmeasured differences between the comparison group. 18,19 Keele et al. 20 developed an IV design to address confounding when evaluating ES in the USA using claims data. Keele et al. ’s study20 used the surgeon’s ES rate across preceding emergency admissions as its IV, and reported that this variable appeared to meet the criteria required to be a valid IV. Keele et al. 20 found that for some conditions, including diverticular disease, ES led to higher 30-day mortality than non-operative care; however, these results may not apply to the NHS in the UK, where thresholds for ES may be different. In addition, although the provider’s preference for ES appeared to be a valid IV in the context of US claims data, it would require further assessment before it could be used in assessing the effectiveness of ES from routine data in the UK.

A further important gap in the evidence required to inform ES provision is that few previous studies have evaluated the cost-effectiveness of ES compared with NES strategies for these common acute conditions. For patients with appendicitis, previous cost-effectiveness analysis (CEA), like the precedent comparative effectiveness literature, provide equivocal results. Some studies have reported that NES strategies result in improved outcomes and lower costs than ES. 21–23 By contrast, other studies report that ES leads to improved outcomes, at either additional8 or reduced24 average costs. For patients with acute cholecystitis, studies have provided conflicting evidence, with some studies25,26 reporting that ES is more cost-effective than delayed surgery and other studies27 reporting that outcomes are better and costs are lower if surgery is delayed. None of these studies have considered NES strategies other than delayed surgery and, similarly, for patients with other common acute conditions, including acute diverticular disease, abdominal wall hernia and intestinal obstruction, there is little available evidence on the cost-effectiveness of ES compared with NES strategies.

A final concern is that previous studies evaluating ES strategies for patients with common acute conditions presenting as emergency admissions have failed to include patients from sufficiently broad populations to report relative clinical effectiveness and cost-effectiveness across population subgroups. 7–17,21–27 Reports by the Royal College of Surgeons (RCS) of England (London, UK) have generally found lower levels of ES for patients aged > 75 years than for patients aged 65–74 years, notably for patients with acute cholelithiasis. 28,29 The RCS guidelines have emphasised that research is required that simultaneously considers which factors, beyond age, may modify the relative clinical effectiveness and cost-effectiveness of ES. Such studies should consider factors such as the patients’ frailty levels at emergency admission and whether or not patients have multimorbidity. Although such factors may be related to age, the factors may still modify the clinical effectiveness and cost-effectiveness of ES compared with NES strategies after allowing for biological age.

Aims and objectives

The aim of the Emergency Surgery OR noT (ESORT) study was to evaluate the clinical effectiveness and cost-effectiveness of ES compared with NES strategies for patients with common acute conditions presenting as emergency admissions to NHS hospitals in England. The specific objectives were to evaluate the:

• relative effectiveness of ES compared with NES strategies for five common acute conditions presenting as emergency admissions

• relative cost-effectiveness of ES compared with NES strategies for five common acute conditions presenting as emergency admissions

• clinical effectiveness and cost-effectiveness of ES compared with NES strategies for specific patient subgroups.

Changes to the research proposed

Public and patient involvement

The PPI work in the ESORT study sought to develop a clearer understanding of what is important for patients, their families and the public in general when someone arrives at a hospital in an emergency. Patient perspectives were embedded into the ESORT study from the very start, with close involvement from two patient advocates in the application to National Institute for Health and Care Research (NIHR), PPI panels held at the design and translation stages of the project, and review by the North Thames Applied Research Collaboration Research Advisory Panel at key stages.

We followed the GRIPP2 (Guidance for Reporting Involvement of Patients and the Public 2) methodology in reporting PPI processes, and the table is available as Report Supplementary Material 1 and on our study website. 30

Report overview

This report details the three interlinked components of the study. Chapter 2 explains how the patient populations and the ES and NES strategies were defined from the consensus of the clinical panel, and then from the HES data. Chapter 2 also highlights the wide variation in ES rates across NHS acute trust hospitals for common acute conditions presenting as emergency hospital admissions. This unexplained variation underlies the IV design, which is used to address confounding in assessing relative effectiveness (see Chapter 3) and cost-effectiveness (see Chapter 4) of ES. In Chapter 3, we provide a summary of the methods for assessing clinical effectiveness, and summarise the results both overall (objective 1), and according to prespecified subgroups (objective 3). Similarly, in Chapter 4, we summarise the main aspects of the CEA, and provide the results overall (objective 2) and according to subgroups (objective 3). Each of the three substantive chapters are also supported by additional material that is provided in the appendices, report supplementary material and on the ESORT study website. 30 In Chapter 5, we discuss the overall findings from the study, including findings from the PPI translation workshops, and provide recommendations for clinical practice and for further research.

Introduction

The design of the ESORT study was based on the principles of a ‘target trial’, that is, an observational study that seeks to emulate a (hypothetical) pragmatic trial. 31,32 Three key issues with applying a target trial framework to routine observational data are: (1) defining the target population according to inclusion and exclusion criteria that would be applied in a hypothetical trial (see Study eligibility), (2) defining the intervention and comparator strategies (see Clinical panel criteria) and (3) defining the time when the eligibility criteria are met and a decision on treatment strategy made, or time zero [see Definition of day (time) zero]. The fourth and fifth key issues (i.e. those of defining the outcomes of interest and of the approach to handle unmeasured confounding in the absence of randomisation) are addressed in Chapter 3 (see Chapter 3, Outcomes, Instrumental variable: the tendency to operate, Patient-level covariates and Statistical analysis). Conceptualising the observational study in this way encourages researchers to make explicit decisions about the study design and the assumptions required to identify causal treatment effects.

The ESORT study used routine HES data for England to define cohorts of emergency admissions for five common acute gastrointestinal conditions: (1) acute appendicitis, (2) cholelithiasis, (3) diverticular disease, (4) abdominal wall hernia and (5) intestinal obstruction. This chapter describes (1) the criteria for defining the cohorts, (2) the criteria for identifying patients who had ES, (3) the patient characteristics of the five cohorts and (4) variation in ES rates across hospitals.

Methods

Results

Variation in emergency surgery rates between hospitals

Variation in rates of ES between the 175 hospitals contributing patients to the cohorts is shown in Figure 1. Greatest variation between hospitals was found in cholelithiasis, with a median rate of 18.4% [interquartile range (IQR) 11.4–28.1%; minimum, 2.3%; maximum, 66.4%]. A more moderate variation was found for hernia (median, 59.8%; IQR 54.2–65.8%; minimum, 30.8%; maximum, 79.2%) and appendicitis (median, 93.0%; IQR 91.0–94.5%; minimum, 67.5%; maximum, 98.6%). The least variation was observed with intestinal obstruction (median, 30.0%; IQR 27.0–33.2%; minimum, 20.4%; maximum, 51.4%) and diverticular disease (median, 11.2%; IQR 9.1–13.3%; minimum, 3.5%; maximum, 21.0%).

FIGURE 1.

Variation in rates of ES in emergency admissions to 175 NHS acute general hospitals in England between April 2010 and December 2019. (a) Appendicitis; (b) cholelithiasis; (c) diverticular disease; (d) hernia; and (e) intestinal obstruction.

Further analysis examining case mix-adjusted rates in ES at the level of the NHS trust has been reported elsewhere. 36

Discussion

This chapter describes the criteria used to define cohorts for the five acute conditions and the methods used to determine which patients received ES and which patients received NES strategies. The characteristics of each cohort are described and the variation in rates of ES between hospitals is reported. The process drew on the expertise of a clinical panel to identify emergency general surgical populations and to define ES following ‘target trial’ principles, using routine observational data. 31,32

This research extends previous studies28,29,37 that have found lower rates of ES for specific subgroups of older patients presenting with acute conditions within the UK. Reports from the RCS of England28,29 have generally found lower levels of ES for patients aged over ≥ 75 years than for patients aged 65–74 years, notably for patients with acute cholelithiasis. The reports28,29 discourage ES rationing by biological age and have called for further research on ES and age to also consider comorbidities and frailty. 28,29 The results from the ESORT study show that ES generally decreases with age, a finding that remains after adjusting for a wider range of case mix measures, including frailty. 35 For patients with acute cholelithiasis and appendicitis, the age gradient is especially steep and goes across the age distribution.

For patients presenting with hernia, previous studies reported higher ES rates for patients aged > 75, than for patients aged 65–74 years. 28,29 Findings from the ESORT study indicated that this was not the case for patients in the oldest age groups (i.e. aged ≥ 85 years), a finding that remained after allowing for differences in other patient characteristics. 36

Large variation in the rates of ES was found for patients presenting with acute cholelithiasis, which suggests that, in some hospitals, National Institute for Health and Care Excellence (NICE) guidelines,38 which recommend laparoscopic cholecystectomy within 7 days of diagnosis, are not being followed. These guidelines are informed by evidence from a meta-analysis that reported improved outcomes for ES compared with delayed cholecystectomy for patients with biliary colic, acute cholecystitis or gallstone pancreatitis. 39 Despite these recommendations, related research37 has also reported large levels of unexplained variation across NHS trusts in ES over a 2-month time period.

Previous research37 on trust-level variation for patients with benign gallbladder diseases reported higher rates of ES in centres with a specialist hepatobiliary centre available, which may reflect better availability of operating theatre space, clearer understanding of the evidence comparing emergency and delayed cholecystectomy, or the enthusiasm to deliver an emergency cholecystectomy service. This previous study37 also found that other trust-level factors, such as the availability of ES operating lists specific to the condition and the number of consultants with expertise in the specific forms of ES, were not associated with ES rates for patients with benign gall bladder diseases. 37 Surgeon-led quality improvement initiatives, such as the Cholecystectomy Quality Improvement Collaborative, may also explain higher uptake of ES in some hospitals. 40

More moderate variation between hospitals for acute appendicitis and abdominal wall hernia may reflect (1) the lack of evidence about which patients benefit from ES versus NES for these conditions, (2) that there are less well-defined care pathways and (3) a lack of clinical guidelines in the UK to inform the choice of whether or not the patient has ES. 41,42 It is also important to recognise that, over the ESORT study time period, patients with an abdominal wall hernia presenting as emergency admissions were not managed by a distinct surgical subspecialty, which may have hindered attempts to standardise practice. 42 In addition, different local policies on restricting elective hernia surgery may have affected emergency provision. 43,44 For patients with uncomplicated acute appendicitis, the emerging evidence for antibiotics as an alternative to ES may explain variability. 7,13

Lower variation in rates of ES for patients with intestinal obstruction and diverticular disease may reflect increased standardisation in the clinical management of the conditions over the ESORT study’s time period, the emergence of evidence from RCTs14,45,46 and the development of clinical pathways. For diverticular disease, there is consensus in the UK about the surgical specialty (i.e. colorectal surgery) that manages patients. For patients with acute diverticular disease, ES has declined over time,47 and the low ES rate reflects current NICE recommendations that encourage NES strategies and the lack of high-quality evidence on the effectiveness of ES for patients with acute diverticular disease. 48

The variation that was identified in rates of ES between hospitals provides the basis for the IV (i.e. a hospital’s TTO) that will be used in evaluating the clinical effectiveness (see Chapter 3) and cost-effectiveness (see Chapter 4) of ES in comparison with alternative NES treatment strategies.

Introduction

For common acute conditions that present as emergency admissions, an important clinical decision is which patients should receive ES and which patients should receive NES strategies. Here, NES strategies include medical management, non-surgical procedures (e.g. radiological-guided drainage of abscess) and surgery deferred to the elective (planned) setting. ES rates for patients with acute conditions have declined over the last 20 years. 6 Protocols for NES strategies have been implemented as part of RCTs, and for some conditions, such as acute appendicitis, there is some evidence of improved outcomes when compared with ES strategies. 7,13 For other conditions, such as acute cholelithiasis, RCTs have found that NES strategies may have unintended consequences, with patients having recurrent symptoms and delayed surgery, therefore, leading to further pressure on surgical waiting lists. 9 However, to the best of our knowledge, none of these RCTs have included routine emergency admissions to hospitals. Similarly, for some acute conditions, no RCTs, to the best of our knowledge, have been conducted to compare ES with NES strategies. 7,9,13–15,49

This lack of evidence has resulted in clinical uncertainty about the benefits and harms of ES for patients with acute conditions. Consequently, there is wide variation in clinical practice2 and in rates of ES across NHS trusts in England for emergency admissions with common acute conditions, with patients of similar prognosis more likely to receive ES in some hospitals, and NES strategies in others. 36

The aim of this chapter is to assess the relative clinical effectiveness of ES compared with NES strategies for the following five common acute conditions: (1) acute appendicitis, (2) cholelithiasis, (3) diverticular disease, (4) abdominal wall hernia and (5) intestinal obstruction. The analysis exploits the variation in ES rates across NHS hospitals in England (see Chapter 2), and reports relative clinical effectiveness overall and according to prespecified subgroups, in particular age, sex, number of comorbidities and level of frailty. The analyses use data on cohorts of emergency admissions to 175 NHS acute hospitals for the five common acute conditions (see Chapter 2). The study protocol and statistical analysis plan were developed following the principles of emulating a target trial (see Report Supplementary Material 4). 30

The design and proposed analysis of the ESORT study were informed by a Patient and Public Advisory Group during two online workshops held in July 2020. 30 The workshop participants were asked to consider outcome measures for patients following emergency admission to hospital for acute conditions. The group agreed that an appropriate measure would capture mortality and the number of days in hospital.

Methods

Results

Subgroup results

Subgroup results for the relative effectiveness of ES compared with NES treatment strategies in the five conditions are reported for the number of DAOH and its two components (i.e. 90-day mortality and LOS) in Figures 2–16. Outcomes are reported according to age, sex, level of frailty, number of comorbidities, subdiagnosis and year of admission.

FIGURE 2.

Mean differences in number of DAOH between ES and NES treatment strategies for appendicitis subgroups.

FIGURE 3.

Mean differences in 90-day mortality between ES and NES treatment strategies for appendicitis subgroups.

FIGURE 4.

Mean differences in days in hospital in the first 90 days between ES and NES treatment strategies for appendicitis subgroups.

FIGURE 5.

Mean differences in number of DAOH between ES and NES treatment strategies for cholelithiasis subgroups.

FIGURE 6.

Mean differences in 90-day mortality between ES and NES treatment strategies for cholelithiasis subgroups.

FIGURE 7.

Mean differences in days in hospital in the first 90 days between ES and NES treatment strategies for cholelithiasis subgroups.

FIGURE 8.

Mean differences in number of DAOH between ES and NES treatment strategies for diverticular disease subgroups.

FIGURE 9.

Mean differences in 90-day mortality between ES and NES treatment strategies for diverticular disease subgroups.

FIGURE 10.

Mean differences in days in hospital in the first 90 days between ES and NES treatment strategies for diverticular disease subgroups.

FIGURE 11.

Mean differences in number of DAOH between ES and NES treatment strategies for hernia subgroups.

FIGURE 12.

Mean differences in 90-day mortality between ES and NES treatment strategies for hernia subgroups.

FIGURE 13.

Mean differences in days in hospital in the first 90 days between ES and NES treatment strategies for hernia subgroups.

FIGURE 14.

Mean differences in number of DAOH between ES and NES treatment strategies for intestinal obstruction subgroups.

FIGURE 15.

Mean differences in 90-day mortality between ES and NES treatment strategies for intestinal obstruction subgroups.

FIGURE 16.

Mean differences in days in hospital in the first 90 days between ES and NES treatment strategies for intestinal obstruction subgroups.

The relative effectiveness of ES compared with NES strategies was modified by age group, with ES less effective for some subgroups of older patients. For patients aged 80–84 years and ≥ 85 years, the difference in the mean number of DAOH following ES and NES strategies were, respectively, as follows: –11.81 (95% CI –16.50 to –7.12) days and –0.58 (95% CI –9.12 to 7.95) days for appendicitis (Figure 2), 0.76 (95% CI –2.04 to 3.57) days and –4.31 (95% CI –9.76 to 1.14) days for cholelithiasis (see Figure 5),–12.49 (95% CI –35.21 to 10.23) days and –23.98 (95% CI –53.65 to 5.68) days diverticular disease (see Figure 8), –3.34 (95% CI –7.89 to 1.21) days and –4.78 (95% CI –9.88 to 0.31) days for a hernia (see Figure 11) and–24.7 (95% CI –39.82 to –9.58) days and –8.64 (95% CI –19.35 to 2.07) days for an intestinal obstruction (see Figure 14). Conversely, ES was more effective than NES strategies in some younger age groups for cholelithiasis, hernia and intestinal obstruction. In four conditions, the comparatively lower number of DAOH with ES strategies in those aged ≥ 85 years was, in part, due to higher 90-day mortality. The exception was for the hernia cohort, for which the lower 90-day mortality with ES (see Figure 12) was offset by a longer LOS (see Figure 13).

There was some evidence that the mean number of DAOH was smaller by –1.46 (95% CI –2.76 to –0.16) days with ES, compared with NES strategies, for females with appendicitis (see Figure 2), whereas the mean number of DAOH was larger by 0.77 (95% CI 0.11 to 1.43) days with ES, compared with NES strategies, in females with cholelithiasis (see Figure 5) and larger by 9.69 (95% CI 4.19 to 15.20) days with ES, compared with NES strategies, in females with intestinal obstruction (see Figure 14). Similar differences were not observed in men with these conditions.

The largest differences in number of DAOH were observed in patients categorised as being severely frail according to the SCARF index. In all five conditions, the number of DAOH at 90 days was smaller following ES, compared with NES strategies, with mean differences of –21.0 (95% CI –27.4 to –14.6) days for appendicitis,–5.72 (95% CI –11.3 to –0.2) days for cholelithiasis,–38.9 (95% CI –63.3 to –14.6) days for diverticular disease, –19.5 (95% CI –26.6 to –12.3) days for a hernia and–34.5 (95% CI –46.7 to –22.4) days for an intestinal obstruction. For four conditions, the smaller number of DAOH was attributable to both an increase in 90-day mortality following ES and longer LOS, compared with NES strategies. The exception was with hernia cohort, where there was increased LOS but no difference in mortality.

A smaller number of DAOH with ES treatment strategies was also found for patients categorised as having ‘moderate frailty’ in appendicitis, hernia and intestinal obstruction cohorts, and also in patients with ‘mild frailty’ in the appendicitis cohort. For patients who were categorised as ‘fit’, the mean difference in number of DAOH tended to favour ES for diverticular disease (5.35 days, 95% CI –2.56 to 13.28 days), hernia (2.26 days, 95% CI 0.37 to 4.15 days) and intestinal obstruction (18.2 days, 95% CI 14.8 to 22.47 days), and was similar or smaller for appendicitis (–0.18 days, 95% CI –1.56 to 1.20 days) and cholelithiasis (0.93 days, 95% CI 0.48 to 1.39 days).

The relationship between comorbidity and differences in number of DAOH following ES or NES strategies did not follow the same pattern as observed for frailty. For patients with three or more comorbidities, ES strategies led to a smaller number of DAOH at 90 days for patients with acute appendicitis (mean difference –12.55 days, 95% CI –23.61 to –1.49 days), a larger number of DAOH at 90 days for patients with an intestinal obstruction (mean difference 26.37 days, 95% CI 8.71 to 44.02 days) and similar average number of DAOH at 90 days between the comparison groups for the other three conditions.

Diagnostic subgroups of patients for whom ES strategies indicated smaller numbers of DAOH were patients with diverticular disease with perforation and abscess (–11.55 days, 95% CI –19.03 to –4.06 days), and patients with an intestinal obstruction with a diagnosis of gallstone ileus (–14.15 days, 95% CI –19.08 to –9.22 days). Conversely, patients with a colorectal cancer diagnosis were found to have a larger number of DAOH following ES (see Figure 14). For four out of the five conditions, for each year of admission, the estimates of the effectiveness of ES compared with NES were consistent with the overall estimates. The exception was for appendicitis (see Figure 2), for which the mean number of DAOH was larger for NES strategies for the last 3 years of admission (see Discussion).

Sensitivity analyses

The overall effects of the ES compared with NES strategies were similar when alternative standpoints were taken to the analysis (Figure 17). Specifically, when alternative, more stringent criteria for the definitions of ES were used, the mean differences in the number of DAOH remained below 4 days for each of the conditions. The overall results were also robust to the use of alternative definitions of DAOH and of the ‘quality’ of acute care, and the exclusion of observations from hospitals with lower volumes of emergency admissions. The regression analysis found that ES was associated with an average reduction in number of DAOH for patients with diverticular disease, but did not take account of unobserved confounding.

FIGURE 17.

Sensitivity analysis for DAOH. (a) Appendicitis; (b) cholelithiasis; (c) diverticular disease; (d) hernia; and (e) intestinal obstruction.

Discussion

For patients presenting as emergency admissions to hospital with acute appendicitis, cholelithiasis, diverticular disease, an abdominal wall hernia or an intestinal obstruction, the analysis of clinical effectiveness found that, overall, the average number of DAOH at 90 days was similar following ES and NES strategies. There were differences in the relative effectiveness of ES compared with NES strategies according to patients’ levels of frailty, age and number of comorbidities. For patients with severe frailty, according to the SCARF index, the average numbers of DAOH were smaller following ES compared with NES strategies for all five conditions. These findings have implications for clinical practice, given the importance of providing ES and NES strategies that will benefit the individual patient.

This analysis adds to the previously limited evidence7,9,13–15,49 on the clinical effectiveness of ES compared with NES strategies, such as delayed surgery, interventional radiology or medical management. Recent trial evidence has reported equivocal results for selected patients with uncomplicated acute appendicitis, with a high proportion of patients in the NES group requiring subsequent surgery. 7,8,13 The COVID-Harem study21 exploited the increased rates of NES strategies following the first wave of COVID-19 infections, and reported that NES strategies led to short-term cost savings. Each of these previous studies has helped developed clinical protocols for NES strategies for patients presenting as emergency admissions with acute appendicitis. The general roll-out of these protocols may be one reason for why the ESORT study observed a relative improvement in the mean number of DAOH for NES compared with ES for the last 3 years of emergency admissions (i.e. 2017/18, 2018/19 and 2019/20). For patients presenting with acute cholelithiasis,38 NICE guidelines, informed by meta-analyses,39 recommend laparoscopic cholecystectomy within 7 days of diagnosis, but ES rates vary across NHS hospitals. 36,37 For patients with acute diverticular disease, published RCTs14,15 have not recruited sufficient numbers of patients to evaluate ES and NES strategies. For patients with an abdominal wall hernia or an intestinal obstruction presenting as emergency admissions, there are, to the best of our knowledge, no published RCTs comparing ES and NES strategies. Hence, the finding that ES and NES strategies led to similar average numbers DAOH at 90 days across unselected patients routinely presenting as emergency admissions adds to limited knowledge for common acute gastrointestinal conditions.

National guidelines have discouraged rationing ES by age alone, and recommends further research on the relative effectiveness of ES for subgroups according to frailty and comorbidities. 28,29 In the ESORT study cohorts, for whom up to 6.5% of patients receiving ES were classified as severely frail, we found that ES was less effective than NES strategies for severely frail patients. The reductions in mean numbers of DAOH following ES were partly driven by increased all-cause mortality at 90 days (except for the hernia cohort), and partly driven by the increased LOS. RCTs of ES strategies have either excluded frail patients or not considered if frailty modifies the relative effectiveness of ES and NES strategies. 7,9,13–15,49 The current study emphasises the importance of considering frailty alongside other factors, such as age, number of comorbidities and diagnosis for patients, when deciding whether or not an individual patient should have ES.

This analysis has several strengths and limitations. The study included a sufficient number and range of patients with these common acute conditions to provide precise estimates of the clinical effectiveness of ES compared with NES strategies as provided in routine practice. The study used a previously developed IV method to address confounding, and provides comparative effectiveness estimates that apply to both the overall populations and subpopulations of interest. The limitations of this paper are that detailed information from imaging or diagnostic procedures was unavailable, and so the definitions of subgroups were broad. In addition, the categorisation of ES and NES strategies assumes accurate coding of OPCS-4 procedures and episode dates. Although it is conceivable that there were coding differences across NHS trusts or over time, it is unlikely that this led to substantive differences in the estimates of relative effectiveness. The primary outcome measure (i.e. DAOH) did not consider other dimensions of health, such as health-related quality of life (HRQoL), as this information was not available within the HES data. The definition of NES strategies was limited to OPCS-4 procedure codes and, therefore, could not capture other aspects of clinical management, for example the type and duration of antibiotic therapy. The ability of the IV approach to deal with unmeasured confounding may be undermined if the requisite assumptions do not hold. The finding that the IV balanced important case mix measures, such as frailty and comorbidity, provides assurance that it would also ensure that indicators of disease severity that are not available in HES data were also similarly balanced between the ES and NES groups.

In summary, this analysis found that ES and NES strategies for patients presenting as emergency hospital admissions with common acute conditions led to similar average numbers of DAOH at 90 days. For patients with severe levels of frailty, ES strategies led to worse outcomes than NES strategies for each of the five acute conditions.

Introduction

The second and third objectives of the ESORT study required assessment of the cost-effectiveness of ES compared with NES strategies for patients with common acute gastrointestinal conditions presenting as emergency hospital admissions. The second and third objectives emphasised the importance of providing evidence on relative costs and cost-effectiveness of ES, recognising the opportunity costs of providing ES and that, for some patients, NES strategies may prove more cost-effective. Although CEA methods are relatively well established for evaluating health technologies, the evaluation of health service interventions in general, and ES in particular, raise particular challenges for recommended CEA approaches. 63,64

The ESORT study, in common with previous CEAs of ES for cholelithiasis or appendicitis,24,25 did not have suitable evidence from RCTs to inform the assessment of relative effectiveness within the CEA. For patients with acute appendicitis, it was unclear if the published RCT evidence7,8,65–67 applied directly to provision of ES versus NES for NHS funded emergency admissions in England. In particular, these RCTs did not recruit from hospitals in England, nor did they include the full range of patients who would present as routine hospital admissions. For the other four acute conditions, there was, to the best of our knowledge, no RCTs that have compared ES with NES strategies for patients presenting as emergency hospital admissions. The CEA for the ESORT study, therefore, used the same IV design as for the evaluation of clinical effectiveness (see Chapter 3) to address concerns about confounding when evaluating the cost-effectiveness of ES compared with NES strategies. 55,68–70 The local instrumental variable (LIV) approach taken also allowed the study to meet the third objective in reporting cost-effectiveness results according to patient subgroup. 55,71

A further challenge that arose in assessing the relative cost-effectiveness of ES from routine (HES) data was that requisite information about HRQoL was not available. Although previous CEAs of alternative forms of surgery in the elective (planned) setting have used HRQoL measures linked to HES data,72 such linked data were not available in the emergency setting. Hence, literature reviews were undertaken for each of the five conditions to identify the most appropriate HRQoL estimates, acknowledging the challenges of HRQoL assessment for patients presenting as emergency hospital admissions. 73 We now describe the CEA methods used in the ESORT study.

Methods

Results

Subgroup results for the cost-effectiveness analysis

Figures 18–22 reported underlying heterogeneity in the INB estimates according to subgroup. Three results are worth emphasising (for a summary, see Table 20). First, the relationship between age group and the mean INB estimates differed by condition. Although ES was more cost-effective for younger patients with cholelithiasis or a hernia, for patients with diverticular disease the estimated INBs were higher for patients aged ≥ 55 years. For patients with acute appendicitis or an intestinal obstruction, there is not a clear pattern in the INB estimates across age groups. Second, the relative cost-effectiveness of ES compared with NES strategies differed according to the number of comorbidities. For example, on average, for patients with two or more comorbidities, ES was the more cost-effective strategy for patients with diverticular disease or an intestinal obstruction, whereas for patients with appendicitis, cholelithiasis or a hernia, NES was the more cost-effective strategy. However, the CIs for the INBs for both sets of estimates included zero. Third, the cost-effectiveness of ES differed by frailty subgroups. For the appendicitis, cholelithiasis, hernia and intestinal obstruction cohorts, there was strong evidence that NES strategies were cost-effective among patients with severe frailty, with the respective mean INBs for the subgroups being –£18,700 (95% CI –£23,900 to –£13,600), –£7700 (95% CI –£13,000 to –£2370), –£16,600 (95% CI –£21,100 to –£12,000) and –£19,300 (95% CI –£25,600 to –£13,000). The results for diverticular disease suggested that NES strategies were also cost-effective for this subgroup, albeit with greater uncertainty around the point estimate (–£9230, –£24,300 to £5860). NES strategies were more cost-effective than ES among patients with moderate frailty, with a mean INB estimates of –£5750 (95% CI –£7810 to –£3690) for appendicitis, –£5630 (95% CI –£8150 to –£3110) for a hernia and–£9320 (95% CI –£14,900 to –£3740) for an intestinal obstruction. For patients who were otherwise ‘fit’ at baseline, ES was more cost-effective, with mean INB estimates of £718 (95% CI £294 to £1140) for cholelithiasis, £5180 (95% CI £684 to £9680) for diverticular disease, £2041 (95% CI £996 to £3090) for a hernia and £7850 (95% CI £5020 to £10,700) for an intestinal obstruction.

FIGURE 18.

Estimated INBs (£) of ES vs. NES strategies for appendicitis subgroups.

FIGURE 19.

Estimated INBs (£) of ES vs. NES strategies for cholelithiasis subgroups.

FIGURE 20.

Estimated INBs (£) of ES vs. NES strategies for diverticular disease subgroups.

FIGURE 21.

Estimated INBs (£) of ES vs. NES strategies for hernia subgroups.

FIGURE 22.

Estimated INBs (£) of ES vs. NES strategies for intestinal obstruction subgroups.

Figures 23–27 report the individual-level estimates of incremental costs and QALYs for the five conditions. Here, for illustration, the results are stratified by frailty level. For patients with severe frailty, the proportions for whom ES is estimated to be cost-effective are 0.0657% (appendicitis), 0.443% (cholelithiasis), 46.9% (diverticular disease), 0.00% (hernia) and 7.52% (intestinal obstruction). For patients who were fit, the corresponding proportions are 59.0% (appendicitis), 92.5% (cholelithiasis), 87.1% (diverticular disease), 82.0% (hernia) and 61.6% (intestinal obstruction).

FIGURE 23.

Cost-effectiveness plane of person-centred treatment effects for costs (£) and QALYs for appendicitis.

FIGURE 24.

Cost-effectiveness plane of person-centred treatment effects for costs (£) and QALYs for cholelithiasis.

FIGURE 25.

Cost-effectiveness plane of person-centred treatment effects for costs (£) and QALYs for diverticular disease.

FIGURE 26.

Cost-effectiveness plane of person-centred treatment effects for costs (£) and QALYs for hernia.

FIGURE 27.

Cost-effectiveness plane of person-centred treatment effects for costs (£) and QALYs for intestinal obstruction.

Discussion

This CEA finds that for patients presenting as emergency admissions with the five acute conditions, neither ES nor NES strategies were more cost-effective after applying an IV approach to address confounding. For all five conditions, the mean INBs were surrounded by 95% CIs that included zero. For patients with an abdominal wall hernia, ES led to a small increase in mean QALYs and in mean costs at 1 year. For the other four conditions, the mean costs and mean QALYs were similar for ES and NES strategies. The finding that any overall difference between the comparison groups in average cost was moderate and uncertain reflected that, although the mean index admission costs were higher for the ES groups than for the NES groups, the proportions of patients who were re-admitted to hospital before 1 year were higher in the NES groups than in the ES groups (see Table 16).

Beneath these overall findings, the CEA identified differences in relative cost-effectiveness according to patient subgroup. For patients with severe frailty, the mean costs were higher (see Appendix 6) and the average QALYs lower (see Appendix 6) and, hence, the estimated INBs were negative following ES, compared with NES strategies (Table 20). Conversely, for subgroups of patients who were ‘fit’ or without comorbidities, the ES strategies tended to be more cost-effective (see Table 20). The subgroup findings reflected that the higher risks of all-cause mortality were in the ES group for patients with severe frailty and more comorbidities (apart from for patients with diverticular disease). Conversely, ES strategies were generally more cost-effective for patients with lower baseline risks of death, such as patients who were ‘fit’ and without comorbidities.

This paper contributes to the limited previous literature7,8,24,25,65–67 evaluating the cost-effectiveness of ES for these common acute gastrointestinal conditions. For patients with acute appendicitis, some previous studies8,21 have suggested that NES strategies can reduce average costs. The studies8,21 reported that around 20% of patients in the NES arm had appendicectomies by 90 days21 and 1 year,8,23 but that the additional costs of the NES strategy were insufficient to offer the additional costs of ES within the index admission. The ESORT study results differed in that almost all of the patients in the ES group would have had appendicectomy by laparoscopic, rather than by open repair, and that the broad inclusion criteria would have included patients with complicated as well as uncomplicated appendicitis and patients with suspected appendicitis who did not have clinically or radiologically confirmed diagnoses. In the ESORT study, the NES group had rates of re-admission and reintervention that led to sufficiently higher costs to offset the additional index admission costs of the ES strategy (see Table 16). The differences between the ESORT study and these previous cost results8,21 may reflect the differences in the study designs and that, unlike the 90-day results for the COVID-Harem study,21 the ESORT study follow-up extended to 1 year (base case) and 5 years (sensitivity analyses) and, therefore, included higher (adjusted) rates of re-admission and reintervention in the NES group, than in the ES group, beyond 90 days.

For patients with acute cholelithiasis, the ESORT study differed from CEA based on decision models that found that ‘early’ versus ‘delayed’ cholecystectomy was less costly and more effective25,93 and, therefore, more cost-effective. 26 Our results also differed from a trial-based CEA, which found that delayed cholecystectomy was more cost-effective than early cholecystectomy. 27 The distinguishing features of the ESORT study are that it had broader eligibility criteria than the RCT or cohort studies underlying the previous CEA and it considered re-admissions and reinterventions for up to 1 year (rather than 30 days). In addition, unlike the previous cohort studies, the ESORT study used an advanced quantitative approach (i.e. IV estimation) to balance confounding factors between the groups. A fundamental difference between the ESORT study and the preceding CEA was that previous studies defined the comparator group as patients having ‘delayed surgery’ who may have been differed in prognosis from patients who had ‘early surgery’. By contrast, in the ESORT study, the ES and NES groups were defined at the same point in time (when they were seen by a surgeon), and the IV approach aimed to balance the patient characteristics at that time point. A key reason why the ESORT study found that average costs were similar between the comparison groups at 1 year is that, although the NES strategy arm had higher rates of re-admission and (re)intervention after the index admission, the differences in adjusted rates [53.4 vs. 83.3 for re-admission, 19.2 vs. 63.2 for (re)intervention] were relatively small.

For patients with acute diverticular disease or an abdominal wall hernia, a previous CEA of surgery compared with watchful waiting has been undertaken in the elective setting;94,95 however, there is no precedent CEA of direct relevance to compare with the ESORT study. For patients with an intestinal obstruction, the ESORT study included a broad range of patients, including patients admitted with cancer diagnoses and patients with adhesions from previous surgery. Previous CEAs have found NES (i.e. stenting) to be cost-effective for patients with an incurable large bowel obstruction85 and ES to be cost-effective for patients with an adhesive small bowel obstruction. 96

The ESORT study has several strengths. First, the CEA extended the previously validated IV approach,20 used in the assessment of clinical effectiveness (see Chapter 3), to the CEA context. The study exemplified how an IV approach can be applied to electronic health record data to address confounding and heterogeneity in providing cost-effectiveness estimates overall, and according to patient subgroups. The ESORT study, therefore, addresses a gap in the applied methods literature. (Note that a review97 of CEA, which used observational data, highlighted that < 5% of studies used IV approaches.) In addition, the LIV approach fully recognises heterogeneity, but has not previously been applied to CEA. In extending the LIV approach to the CEA context, it was necessary to allow for individual-level correlation between costs and outcomes. It was also important to apply appropriate costs models for handling the highly right-skewed nature of the cost data, notably for conditions such as diverticular disease and intestinal obstruction, which included patients with extremely high 1-year costs (i.e. costs in excess of £100,000). The LIV approach provided cost-effectiveness estimates according to the subgroups of interest. For the appendicitis, cholelithiasis and hernia cohorts, the LIV approach (base case) provided similar cost-effectiveness estimates to estimates from regression approaches that assumed all confounding factors were observed (i.e. SA6) (see Table 18). For patients with acute diverticular disease or an intestinal obstruction, the regression analysis provided quite different results to the LIV method. The most plausible explanation for the difference is that for acute diverticular disease and intestinal obstruction the choice of ES or NES strategy was informed by baseline prognostic factors that are not available within the HES data, for example the patient’s disease stage. Hence, it seems plausible that patients with these acute conditions who were of worse prognosis according to unobserved baseline measures were more likely to receive ES than NES. Although the IV approach was able to ensure that these factors were balanced between the comparison groups, the regression analysis was subject to residual confounding in favour of the NES strategies group.

Second, the application of IV methods to CEA raised the danger of imprecise estimates. Although the overall CEA estimates were, indeed, uncertain, the large representative samples of patients enabled the study to report sufficiently precise subgroup results. The ESORT study, therefore, demonstrates the value of applying IV approaches to CEA, with concerns addressed at the study design stage by undertaking sample size calculations using pilot data as part of prespecified analysis plans (see statistical analysis plan on project webpage30) and by including sufficient hospitals (n = 175) and time periods (10 years) to provide estimates with sufficient precision at subgroup level.

Third, the HES data, although having common features of electronic health record data (notably the potential for confounding and heterogeneity), were of generally high quality, with baseline covariates, all-cause mortality and resource use data available for ≈95% of patients. Economic evaluations that use routine data face inevitable challenges, above and beyond those that face comparative effectiveness studies. CEAs are required to offer a comprehensive description of the comparison groups. For both groups, the information available on defining the strategies was limited to that available from the OPCS-4 procedure codes. The information available from OPCS-4 procedure codes was sufficient to categorise patients into the ES and NES strategy groups, and to define use of specific operative procedures in both comparison groups, but OPCS-4 procedure codes did not provide full information about non-operative strategies, for example use of antibiotics for patients with uncomplicated acute appendicitis. It is possible that the HES data were subject to coding errors that were then incorporated into the estimates of cost and cost-effectiveness, although previous research found that costs estimated from HES data were very similar to those derived from medical records. 98

The base-case analysis took a 1-year time horizon. The 1-year time horizon was judged to be sufficient to capture important differences between the comparator groups, such as differences in all-cause mortality or re-admissions. The advantage of this time horizon is that it enabled all patients to be included in the analysis without censoring or making assumptions about future costs and outcomes beyond the observed data. The sensitivity analyses found that the overall results were similar if the time horizon was extended to 5 years. For patients with appendicitis or diverticular disease, the SA suggested that the point estimates for the INB were somewhat different with the 5- year time horizon compared with 1-year time horizon (see Table 18), but this SA should be interpreted with caution. The 5-year results were highly uncertain, with 95% CIs that overlaped the base-case estimates, and reflected the much-reduced sample sizes. The extended time horizon raised additional concerns for the IV approach, in that the underlying assumption that the hospital’s TTO had no direct effect on the end points, beyond that through the comparison groups, was less tenable over the longer time horizon.

The CEA did not take the recommended perspective of agencies such as NICE, which would have required consideration of all costs from the health and personal social services perspective. The ESORT study used resource data from the HES inpatient database, and only the ensuing inpatient costs were included in the analysis. The inclusion of this narrow range of costs raises the concern that for some patient groups the total costs per patient were underestimated, for example those patients who had severe levels of frailty might have had high use of NHS-funded social care. However, the rationale for only including inpatient costs was that differences between ES and NES strategies in inpatient costs were anticipated to be the main driver of the incremental costs. Although, the ESORT study could have also used linked data on HES outpatient data, these data are of poor quality and are likely to be inaccurate. Similarly, although information could have been used on patients’ discharge destination (e.g. care home), patients’ discharge information was judged as being unreliable and its use would have required further untestable assumptions. An alternative approach would have been to survey samples of patients and their carers, and ask them about their use of health and personal social services outside the acute hospital. However, this approach would have added to the study’s burden and duration, and is prone to problems of missing data and recall bias. Previous CEAs of ES compared with NES strategies for patients with acute appendicitis,21 and of alternative forms of ES for patients with ruptured aortic aneurisms,77 reported that the main drivers of relative costs of the alternative strategies are inpatient costs.

Finally, in common with any approach to address confounding, the implementation of the LIV methods made assumptions; in particular, that the relationships of the covariates and the IV with both the treatment receipt and the outcomes were correctly specified. Here, more flexible data-adaptive approaches may be helpful, although these approaches have not yet been extended to this context. A further consideration is that the subgroup analyses presented here represent the average estimated effect for individuals within the group, rather than the causal effect of group membership per se. Although the subgroups used here were prespecified within a statistical analysis plan, in other contexts spurious subgroup effects may be obtained by ‘p-hacking’. 99

Overview

The ESORT study evaluated the clinical effectiveness and the cost-effectiveness of ES compared with NES strategies [e.g. medical management, non-surgical procedures (e.g. radiological-guided drainage of abscess), surgery deferred to the elective (planned) setting]. The ESORT study considered patients presenting as emergency admissions to hospital with suspected acute appendicitis, cholelithiasis, diverticular disease, an adonomial wall hernia or an intestinal obstruction. The study used HES data to define the study cohorts and the comparator strategies. The ESORT study also used the same HES data linked to ONS mortality to derive the primary clinical outcome (i.e. DAOH at 90 days) and to calculate QALYs and hospitalisation costs at 1 year. The ESORT study reported the clinical effectiveness and cost-effectiveness of ES compared with NES with an IV approach that addressed confounding and recognised heterogeneity in the relative effectiveness and cost-effectiveness of ES according to patient subgroups.

The main findings were that, overall, the average numbers of DAOH at 90 days were similar following ES and NES strategies for each of the five cohorts of patients with acute gastrointestinal conditions requiring emergency admission. The CEA did not provide strong evidence that either strategy (i.e. ES or NES strategies) was more cost-effective overall across each of the five cohorts of patients.

The ESORT study found that for each of the five conditions the relative clinical effectiveness and cost-effectiveness of ES compared with NES strategies differed by patient subgroup, notably according to number of comorbidities (using the Charlson Comorbidity Index) and frailty level (using the SCARF index). For patients with acute appendicitis or and abdominal wall hernia who had two or more comorbidities, NES strategies led to increased average number of DAOH at 90 days, and was cost-effective for these two conditions and for patients with cholelithiasis. For patients with acute diverticular disease or an intestinal obstruction who had two or more comorbidities, on average, ES was cost-effective compared with NES strategies.

For patients with severe frailty at baseline, the NES strategies led to an increase in mean number of DAOH (for all conditions), a reduction in 90-day mortality (for all conditions apart from hernia), a reduction in LOS (for all conditions) and were cost-effective compared with the ES strategies. For patients who were ‘fit’ at presentation, on average, ES strategies were relatively cost-effective compared with NES strategies.

Public and patient involvement

The views of patients who had direct experience of admission to a hospital in an emergency informed the design of the study, the presentation of the findings and dissemination proposals. A key element of this was that, during the two ‘design workshops’, 14 PPI panellists endorsed two of the outcome measures: (1) mortality and (2) DAOH. As part of the two ‘translation workshops’, 12 PPI panellists carefully considered the Plain English summary and the ways in which the research should be presented, particularly in relation to the key findings regarding frailty (see Report Supplementary Material 5).

Our approach to ‘patient and public’ in the context of recruitment was to interpret it widely, and we invited patients, family members and those interested in supporting research to our PPI panels. This broad approach meant that we had a breadth of perspectives among our PPI panellists and advisors and, therefore, PPI voices were not simply individuals who had had particular positive or negative experiences of either ES or NES.

The COVID-19 pandemic necessitated flexibility in our approach to PPI, with the main change being that all panels were held virtually over Zoom. Briefing materials were provided in advance, mostly accessed on the Sway platform. 30 Many panellists commented favourably on this approach and praised the briefing materials.

Panellists also commented that the virtual format improved accessibility, as many panellists would have not been able to participate in an in-person meeting in London because of physical disabilities or caring commitments precluding their ability to travel. The benefits of the virtual format, coupled with close attention to ensure that any accessibility issues (e.g. sight impediments) were addressed, meant that we were able to attract a diverse range of individuals to the PPI panels. The diversity in age, sex and experience also helped reduce the potential for ‘group-think’ in our PPI panels and meant that panellists freely expressed different views and perspectives in their groups.

Findings in the context of related research

The ESORT study extends previous literature2,5 that has raised concern about variations in ES rates. The ESORT study identified unexplained variations in ES rates across NHS acute hospitals for each of the five conditions, which remained after adjusting for differences in prognostic measures, such as age, sex, number of comorbidities and frailty level. 36 This variation was observed within clinically relevant definitions of the target populations of interest. The ESORT study design was informed by the concept of the target trial,31,32 and used the consensus of a clinical panel to define population eligibility criteria and the range and timing of surgical procedures that constituted ES.

The ESORT study exploited this variation in ES rates across hospitals, within an IV design,18 and compared patient outcomes from hospitals that differed in their TTO20 to report the relative effectiveness of ES compared with NES strategies. The ESORT study, therefore, adds to the limited evidence on the relative clinical effectiveness7,8,13,14,21,23,66,67 and cost-effectiveness of ES for common acute conditions presenting as emergency admissions. There are relatively few RCTs that have compared ES with NES strategies in the emergency setting7,8,13,23 and, to the best of our knowledge, none that have considered relative clinical effectiveness and cost-effectiveness according to patient subgroups.

Previous RCTs7,8 of ES compared with NES strategies for patients with acute appendicitis have reported mixed results, with some RCTs reporting better outcomes following ES8 and others reporting NES strategies as more effective. 7 None of the RCTs included the broad range of patients, including patients with complications, who present with suspected acute appendicitis to NHS hospitals in England, and the CODA (Comparison of Outcomes of Antibiotic Drugs and Appendectomy) and APPAC (Appendicitis Acuta) trials considered open rather than laparoscopic appendicectomy,7,23 which is the common surgical technique for appendicectomy in the NHS. It is, therefore, challenging to interpret the results of these previous studies for NHS decision-making.

The COVID-Harem study was an observational study that considered antibiotics as a first-line alternative to appendicectomy, which for about two-thirds of patients was by the laparoscopic route. 21 The COVID-Harem study had similarly broad inclusion criteria to the ESORT study, and included patients following the initial COVID-19 lockdown in the UK on 23 March 2020. At the time the COVID-Harem study was recruiting patients, there were concerns about viral transmission of COVID-19 via aerosolisation during laparoscopy, which led professional societies to recommend management with antibiotics (NES strategy) rather than ES for patients with uncomplicated acute appendicitis. 100 The COVID-Harem study found that the NES strategy led to fewer days in hospital and lower average costs, compared with ES, at 90 days. 101,102

The differences between the findings of the ESORT and COVID-Harem studies may reflect the following. First, the COVID-Harem study used information from imaging to confirm diagnosis of acute appendicitis; however, such information was not available in the ESORT study and so the ‘acute appendicitis cohort’ included patients with ‘suspected’ rather than ‘confirmed’ appendicitis. In the COVID-Harem study, around 3% of patients had negative appendicectomies,21 which is lower than rates reported by the RIFT (Right Iliac Fossa Treatment) study for routine clinical practice in the UK,103 prior to the onset of the COVID-19 pandemic. The RIFT study reported negative appendicectomy rates of 28% for women and 12% for men, which are more representative of routine practice and likely to apply to the ESORT study, which included broad populations of patients with suspected acute appendicitis. The ESORT study found that for men ES was the most cost-effective strategy, whereas for women NES strategies were more cost-effective than ES, which may reflect the higher negative appendicectomy rate in routine practice. A second important difference between the ESORT and COVID-Harem studies is the duration of follow-up. The CEA for the ESORT study reported that the proportions of patients who had re-admissions for the NES and ES strategy groups were 53% and 27%, respectively, with 16% and 2.3%, respectively, having further operative procedures over the 1-year follow-up period. By contrast, the proportion of patients in the COVID-Harem study who had re-admissions following NES and ES were 37% and 20, respectively; however, this was over the shorter follow-up period of 90 days. The COVID-Harem study will report further results at 12 months. The APPAC trial66,67 assessed 5- and 7-year outcomes and resource use for antibiotics compared with open appendicectomy for uncomplicated acute appendicitis. The APPAC trial found that the median total days in hospital at 5 years and HRQoL at 7 years were similar between the groups. 66,67

For patients with acute diverticular disease, RCTs have not contrasted ES versus NES strategies, but rather different forms of ES, such as lavage versus resection. 14,15 There are no previous observational studies of ES versus NES strategies for patients with acute diverticular disease, although the ongoing DAMASCUS study plans to contrast outcomes for patents following ES versus NES strategies. 104

For patients with acute cholecystitis, which was the most common subcondition (≈35%) within the more general diagnosis of ‘cholelithiasis’ used in the ESORT study cohort, meta-analyses of published RCTs indicate that ES leads to improved outcomes,9,39 and that it is relatively cost-effective. 25,93 However, these studies compared ES with later surgery9,25,39,93 and did not include a ‘no operative procedure’, which is within the definition of NES used in the ESORT study. Hence, the finding from the ESORT study, that is for patients with ‘acute cholelithiasis’ outcomes and costs were similar following ES and NES, is not directly comparable to previous studies.

For patients with an abdominal wall hernia, the proportions of patients in the ESORT study receiving ES and NES strategies were more similar (ES, 58.8%; NES strategies, 41.2%) than for the other four conditions. This similarity may indicate greater equipoise in the choice of ES compared with NES strategies. In addition, for patients with an abdominal wall hernia, any differences between the comparison groups in the prognosis of patients at baseline was small. In the elective setting, RCTs have been carried out for patients with a hernia, but these RCTs have mainly considered the clinical effectiveness and cost-effectiveness of laparoscopic surgery compared with open repair,86 and included patients with only minimally symptomatic disease. 105,106 Previous observational studies have suggested that watchful waiting for patients with ventral hernias may be safe107 and other studies have suggested that repair at diagnosis is cost-effective. 108 However, these observational studies all face concerns about confounding by indication. 109 The importance of a RCT to address the evidence gaps concerning ES and NES strategies for patients with acute hernia is also highlighted by an ongoing observational study that is assessing patient outcomes, including complications following ES and NES strategies. 42 The RCT42 is anticipated to provide complementary evidence to the ESORT study (see Implications for further research).

The ESORT study found particularly large heterogeneity in the estimates of the relative clinical effectiveness and cost-effectiveness for patients with intestinal obstruction according to patients’ baseline prognosis. This level of heterogeneity reflected differences in the underlying reasons for emergency admission within the two broad categories of obstruction: (1) following intestinal adhesions from previous surgery (small bowel obstruction) and (2) from cancer (large bowel obstruction). Published RCTs85,110 have suggested that, for patients with intestinal obstruction due to left colon or rectal cancer, forms of NES, such as endoscopic stenting, could lead to similar outcomes110 and reduced costs,85 compared with ES, but these findings are from RCTs85,110 that recruited small samples of patients with short periods of follow-up. For patients with a small bowel obstruction, some studies have suggested that ES may be more clinically effective and cost-effective than NES,96 whereas other studies report that outcomes and costs were similar following ES and watchful waiting. 111,112 Although this evidence, like the ESORT study, was based on administrative hospital data, the study designs did not address concerns about confounding by indication. 96,111,112

Subgroups

The ESORT study reported that for some conditions the relative clinical effectiveness and cost-effectiveness of ES compared with NES strategies differed according to the patients’ chronological age, number of comorbidities assessed (according to the Charlson Comorbidity Index score) and frailty level (assessed using the SCARF index). The ESORT study found that the relationship between age group and cost-effectiveness was somewhat different for diverticular disease compared with the other four conditions. For patients with diverticular disease, ES was more cost-effective for patients aged 55–80 years and also, with greater uncertainty, for patients aged > 80 years. However, for younger patients (aged ≤ 55 years) with diverticular disease there was no strong evidence that either strategy was more cost-effective. For the other four conditions, NES strategies were generally more clinically effective and cost-effective for older patients, although the results did not support a particular age ‘cut-off’ point. For example, for patients with acute appendicitis, the age at which there was reasonably strong evidence that NES strategies were relatively cost-effective was around 70 years old; however, for patients with an abdominal wall hernia, the analogous age at which NES strategies were relatively cost-effective was ≥ 85 years.

The ESORT study found that for patients with appendicitis, cholelithiasis or a hernia, and without comorbidities, ES was the more cost-effective strategy, although the results were somewhat uncertain, with the 95% CIs around the INBs including zero. For patients with diverticular disease or an intestinal obstruction, ES was more clinically effective and cost-effective, on average, for patients with two or three comorbidities, but, again, with 95% CIs around the mean INBs including zero.

The most consistent subgroup effects across conditions were found for the level of frailty. ES strategies were generally favoured in patients classified as ‘fit’ and NES strategies were generally favoured in patients classified as ‘severely frail’. The ESORT study found that for patients who had severe levels of frailty at emergency hospital admission, the NES strategy led to an increase in average number of DAOH at 90 days and was cost-effective compared with ES strategies. The proportion of patients with severe frailty ranged from 0.5% (in the appendicitis cohort) to 6.5% (in the intestinal obstruction cohort). The ESORT study found that for patients with moderate levels of frailty, NES strategies were also cost-effective for patients with appendicitis, an abdominal wall hernia or an intestinal obstruction. Conversely, for patients who were categorised as ‘fit’, for the cholelithiasis, hernia and intestinal obstruction cohorts ES strategies were more effective and were more cost-effective for all conditions (although with some uncertainty for the appendicitis cohort). The proportion of patients classified as ‘fit’ ranged from 45% in the intestinal obstruction cohort to 83% in the appendicitis cohort.

The concept of frailty is distinct from biological age and comorbidity, and has been described as a ‘dynamic and heterogeneous manifestation of age-related decline in physiological reserve and increased vulnerability to stressors’. 35,113,114 The ESORT study assessed frailty with the SCARF index, which uses ICD-10 codes to define deficits according to functional impairment, geriatric syndromes, problems with nutrition, cognition and mood, and medical comorbidities, with severe frailty defined as the presence of six or more deficits. 35 The SCARF index can provide a frailty assessment solely using routine data sources, such as HES. Previous research in a quite different context (breast cancer) found that frailty assessed by the SCARF index was independently associated with rates of surgery and outcomes, after adjustment for other baseline characteristics, including age and number of comorbidities. 35

The construct behind the SCARF index is similar to the Clinical Frailty Score115 and the Electronic Frailty Index,116 which have previously been used to assess frailty prior to ES, and found to be independently associated with greater risks of postoperative mortality and morbidity in reports by NELA and as part of the Emergency Laparotomy Study. 59,117 The ESORT study extends these findings by including conditions that do not require emergency laparotomy, and by considering frailty level as a potential effect modifier in assessing the relative effectiveness of ES compared with NES strategies. The finding that the relative clinical effectiveness and cost-effectiveness of ES is modified by frailty level, after adjusting age and number of comorbidities, supports the inclusion of frailty assessments as part of routine preoperative assessments prior to either ES or NES strategies. A RCS report on the ‘high-risk general surgical patient’118 states that all patients over aged ≥ 65 years should have frailty assessed, but this does not happen for all relevant emergency admissions. 59,117 The ESORT study, therefore, provides evidence to support recent guidelines that have emphasised the importance of frailty assessment as part of preoperative assessment. 119

The findings from the ESORT study also support previous recommendations that the choice of ES or NES strategies should not be according to chronological age,28,29 but should recognise the importance of other factors, such as comorbidity and frailty, in advising patients and their carers of the relative benefits and risks of ES and NES strategies. NES strategies include later surgery, which offers the opportunity to stabilise the patient’s condition, as well as potential to improve outcomes and avoid subsequent re-admissions (see Implications for further research).

Strengths

The ESORT study addresses an important question about the provision of ES, by considering five different common acute conditions that often present as emergency hospital admissions. This study uses a large number of nationally representing data, comprising a total of 887,495 emergency hospital admissions to 175 hospitals over a 10-year period. Previous evaluations of the clinical effectiveness or cost-effectiveness of ES compared with NES strategies for patients with acute conditions have included a narrow range of patients and small samples of patients or hospitals. 7,8,13,21,23,66 By using routine data from all NHS acute hospitals, the ESORT study was able to include large representative populations of patients, including subgroups such as patients with severe levels of frailty, older patients or patients with long-term conditions, who have been excluded from previous studies. 36 The ESORT study reported estimates of relative clinical effectiveness and cost-effectiveness for prespecified subgroups, with sufficient precision to provide useful estimates for informing clinical practice.

The study exemplifies how an advanced IV approach can be applied to routine data. Here, conventional risk adjustment approaches are problematic, as they rely on the assumption that all the potential confounders have been observed. This assumption is implausible in this context, as the choice of ES or NES strategies reflects the patient’s prognosis according to measures not available within the HES data, such as the patient’s disease state. Hence, estimates from conventional risk adjustment approaches using standard regression analysis, such as those considered in the sensitivity analyses, will be prone to bias. In particular, for patients with acute diverticular disease or an intestinal obstruction, we found that patients who had ES rather than NES strategies were of a more severe case mix, according to both measured characteristics (e.g. age, frailty, number of comorbidities) and – it is likely – unmeasured factors (e.g. size of abscess). Hence, the finding from the regression analysis, that is that ES led to increases in mortality for patients with diverticular disease, is likely to reflect confounding by indication. By contrast, the IV approach that avoids assuming that all measured confounders have been observed found that, overall, clinical outcomes and costs were similar for the ES and NES strategies, and that the estimated INBs were close to zero, with substantial uncertainty surrounding the estimates. Although the IV approach also made assumptions, we found that, given the large sample size and the high levels of variation in ES rates across hospitals, the IV was sufficiently strong and balanced the observed covariates between the comparison groups. This research, therefore, extends previous methodological research that has reported that a preference-based IV can be developed from routine data and pass stringent assessments of IV validity. 20 Hence, although such IV approaches are unusual within the economic evaluation or health services literature, the ESORT study suggests that they may have wider applicability in evaluations that use routine data, which are prone to the major problem of confounding by indication. 120

Limitations

The ESORT study’s inclusion criteria and definition of baseline, subgroup and resource use variables were reliant on the coding of diagnosis and procedures available from the HES data. One concern this raises is that for those patients without diagnostic or imaging information available, the diagnostic categorisation was imprecise. For example, the ‘acute appendicitis cohort’ would have included patients with ‘suspected appendicitis’ who did not have appendicitis. The ESORT study, as with any observational study that attempts to estimate relative clinical effectiveness and cost-effectiveness, was required to make assumptions. The ESORT study design took an IV approach that avoided making the unrealistic assumption that all the potential confounding variables were observed. Instead, the IV approach assumed that the hospitals’ TTO did not have a direct effect on the outcome. Although this assumption could not be verified from the data, the study provided evidence to support the underlying assumptions, and found that the study conclusions were robust when alternative standpoints were taken, for example to the different variables used as proxies for the quality of acute care within the hospital.

The ESORT study measured inpatient costs from the perspective of the NHS acute hospital, and so costs of other aspects of health and personal social services were excluded. The choice of costing perspective was driven by the availability of the resource use data that were available from the HES data requested, which were limited to inpatient admissions to NHS trust hospitals. The exclusion of broader costs, for example of NHS-funded care outside the acute hospital, could have led to the underestimation of the incremental costs of the NES strategy for some subgroups, such as patients with severe levels of frailty. These subgroups may have incurred these costs following discharge from the acute hospital.

The ESORT study was able to provide recommendations according to patient subgroup definitions available in routine data. The study did not have information pertaining to disease stage, physiology or underlying health conditions, which are generally available to the clinical team, patient and carers when deciding whether ES or NES strategies are more appropriate for the individual patient. In addition, the analysis of the ESORT study does not consider relative clinical effectiveness and cost-effectiveness according to combinations of subgroup variables.

Finally, although the ESORT study was able to define ES according to OPCS-4 procedure codes and clinically relevant time windows, information was not fully available on specific NES strategies, for example whether or not the patient had antibiotics. Further research would be useful to define protocols for NES strategies, especially for those conditions, such as abdominal wall hernia, for which previous RCTs have not been undertaken (see Implications for further research).

Conclusions and implications for provision of emergency surgery and non-emergency surgery strategies

• The ESORT study finds that, overall, for patients presenting as emergency hospital admissions with common acute conditions, ES and NES strategies led to similar average numbers of DAOH at 90 days, and neither strategy was more cost-effective. The initial additional costs of ES procedures were generally offset by higher re-admission and later operative costs following the NES strategies, over a 1-year period.

• The study found differences in the relative clinical effectiveness and cost-effectiveness of ES compared with NES strategies according to patients’ chronological age, number of comorbidities and frailty levels (see Table 20).

• For patients with appendicitis, cholelithiasis, a hernia or an intestinal obstruction, NES strategies were generally more clinically effective and cost-effective for older patients, although the results did not support a general age ‘cut-off’ point.

• The ESORT study found that relative cost-effectiveness differed according to the number of comorbidities. NES strategies were more cost-effective, on average, for those patients with acute appendicitis, cholelithiasis or an abdominal wall hernia, who had two or more comorbidities. For patients with acute appendicitis, cholelithiasis or a hernia, ES was, on average, more cost-effective for those patients without comorbidities. For patients with acute diverticular disease or an intestinal obstruction, ES was more clinically effective and cost-effective for patients with two or more comorbidities.

• For patients with severe frailty at emergency admission, NES strategies led to better outcomes at 90 days and 1 year, and were relatively cost-effective for all five conditions. ES strategies were generally, on average, more cost-effective for patients who were ‘fit’ at presentation.

• The ESORT study results emphasise the importance of considering NES as well as ES strategies for patients presenting as emergency hospital admissions with common acute conditions. Overall, neither ES nor NES strategies were more cost-effective. For some patient subgroups, such as patients with severe frailty, patients with certain acute conditions with two or more comorbidities and some older age groups, NES is more cost-effective. For other patient subgroups, including patients who were fit at presentation, ES was estimated to improve outcomes and to be cost-effective.

• The evidence from the ESORT study on the relative clinical effectiveness and cost-effectiveness of ES compared with NES strategies can complement that from ongoing cohort studies, and inform future updates to NICE guidelines for emergency and acute care, including guidelines for cholelithiasis and diverticular disease. 38,48

• The ESORT study supports ongoing RCS initiatives to encourage frailty assessment at presentation as part of comprehensive geriatric assessments, which can help in deciding the best care pathway for patients with acute conditions presenting as emergency admissions.

• The ESORT study provides additional information, alongside that from previous and ongoing research, to inform patients and carers about the potential benefits and risks of NES, as well as ES strategies, and that these may differ according to factors beyond chronological age, including frailty level and number of comorbidities.

Implications for further research

• The ESORT study highlights the importance of the appropriate choice of ES or NES strategies for patients with comorbidities, including patients with multiple long-term conditions. For these patients, it is especially challenging to assess the balance of benefits, risks and costs between prompt intervention with ES and ‘optimisation’, which could include reversing aspects of frailty as part of NES strategies (e.g. later surgery). 121 Although HES data have been used previously to identify multimorbidity ‘clusters’ in all elective and emergency hospital admissions,122 clusters have not been identified for emergency admissions for acute conditions, and this is an important gap in evidence, as patients with long-term conditions that tend to be ‘quicker to optimise’ (e.g. atrial fibrillation) may benefit more from ES than from NES strategies. By contrast, patients with underlying conditions that are ‘slower to optimise’ (e.g. kidney disease) or ‘unlikely to optimise’, (e.g. late-stage dementia) may benefit more from planned surgery or medical management, as this can allow the patient’s underlying condition to be stabilised. Further research to identify clusters of patients with acute conditions who present for ES or NES could, therefore, be useful to inform how best to ‘personalise’ the choice of strategy. This proposed research aligns with NIHR strategy ,which emphasises that research on multiple long-term conditions (multimorbidity) is of high priority. 123

• These findings suggest that RCTs for ES compared with NES strategies for patients with acute gastrointestinal conditions now warrants consideration. In particular, for patients with an abdominal wall hernia, the ESORT study found similar rates of ES and NES strategies, and that ES led to a small increase in average QALYs at a small additional cost. To the best of our knowledge, there is no previous RCT evidence for patients presenting with acute hernia in the emergency setting, and care pathways are not clearly defined. The ESORT study findings may, therefore, help motivate a future RCT comparing ES with NES strategies for patients with an abdominal wall hernia. This RCT could investigate the relative outcomes, costs and cost-effectiveness of ES compared with NES strategies, with eligibility criteria informed by the subgroup results of the ESORT study. The RCT could also draw on related research, which examines complications for patients following acute symptomatic hernia. 42

• Further research is required on patient-reported outcomes following ES and NES strategies. For conditions such as acute diverticulitis, previous studies have measured HRQoL for only small samples of patients,15 and this has not been used in comparing ES with NES strategies.

• Following the COVID-19 pandemic, NHS waiting lists in England are projected to reach 13 million by 2025,4 with implications for hospitals’ capacity for elective surgery and ES. The ESORT study approach of combining large-scale routine data with advanced analytical approaches to address confounding and heterogeneity could be extended to other acute conditions. This research design can provide timely evidence to help ensure the most cost-effective strategy (ES or NES) is provided according to the patients’ prognosis, and to help reduce pressure on scarce resources from unnecessary ES or subsequent hospital re-admissions.

Contributions of authors

Richard Grieve (https://orcid.org/0000-0001-8899-1301) (Professor, Health Economics) designed the ESORT study; directed the acquisition and analysis of data and interpretation of the results; and drafted the report manuscript.

Andrew Hutchings (https://orcid.org/0000-0003-0215-9923) (Assistant Professor, Health Services Research) contributed to the study design; undertook the analysis and interpreted the results; helped draft the report manuscript.

Silvia Moler Zapata (https://orcid.org/0000-0003-4733-5601) (PhD student, Health Services Research) undertook the analysis and interpreted the results; helped draft the report manuscript.

Stephen O’Neill (https://orcid.org/0000-0002-0022-0500) (Associate Professor, Health Economics) contributed to the study design; undertook the analysis and interpreted the results; helped draft the report manuscript.

David G Lugo-Palacios (https://orcid.org/0000-0003-2621-5627) (Assistant Professor, Health Economics) contributed to the study design; undertook the analysis and interpreted the results; helped draft the report manuscript.

Richard Silverwood (https://orcid.org/0000-0002-2744-1194) (Associate Professor, Statistics) contributed to the study design; interpreted the results; helped draft the report manuscript.

David Cromwell (https://orcid.org/0000-0002-6516-8125) (Professor, Health Services Research) contributed to the study design; interpreted the results; helped draft the report manuscript.

Tommaso Kircheis (https://orcid.org/0000-0001-7162-2518) (Master’s Degree Student, Public Health) assisted in interpreting the results; helped draft the report manuscript.

Elizabeth Silver (https://orcid.org/0000-0003-0154-423X) (Senior Research Manager) coordinated the ESORT study; organised the PPI and clinical panels; designed the study website; assisted in interpreting the results; helped draft the report manuscript.

Claire Snowdon (https://orcid.org/0000-0002-9133-5476) (Assistant Professor, Qualitative Research) co-led the PPI work; interpreted the results; assisted in interpreting the results; helped draft the PPI elements of the report manuscript.

Paul Charlton (https://orcid.org/0000-0002-1682-3074) (NIHR Patient Ambassador) co-led the PPI work; led the production of accessible ‘easy-read’ guidance; assisted in interpreting the results; helped draft the PPI elements of the report manuscript.

Geoff Bellingan (https://orcid.org/0000-0003-4193-4689) (Professor and Consultant Surgeon, Acute Medicine) assisted in interpreting the results; critically reviewed material in report for important intellectual content.

Ramani Moonesinghe (https://orcid.org/0000-0002-6730-5824) (Professor, Perioperative Medicine) assisted in interpreting the results; critically reviewed material in report for important intellectual content.

Luke Keele (https://orcid.org/0000-0002-3859-2713) (Associate Professor, Statistics) assisted in interpreting the results; critically reviewed material in report for important intellectual content.

Neil Smart (https://orcid.org/0000-0002-3043-8324) (Consultant Surgeon) contributed to the study design; interpreted the results; helped draft the report manuscript.

Robert Hinchliffe (https://orcid.org/0000-0002-6370-0800) (Professor, Consultant Surgeon) contributed to the study design; interpreted the results; helped draft the report manuscript.

Publications

Hutchings A, Moler Zapata S, O’Neill S, Smart N, Cromwell D, Hinchliffe R, Grieve R. Variation in the rates of emergency surgery amongst emergency admissions to hospital for common acute conditions. BJS Open 2021;5:zrab094. https://doi.org/10.1093/bjsopen/zrab094

Hutchings A, O’Neill S, Lugo-Palacios D, Moler Zapata S, Silverwood R, Cromwell D, et al. Effectiveness of emergency surgery for five common acute conditions: an instrumental variable analysis of a national routine database. Anaesthesia 2022;77:865–81.

Moler-Zapata S, Grieve R, Lugo-Palacios D, Hutchings A, Silverwood R, Keele K, et al. Local instrumental variable methods to address confounding and heterogeneity when using electronic health records: an application to emergency surgery. Med Decis Making 2022;42:1010–26.

Data-sharing statement

All data requests should be submitted to the corresponding author for consideration. Access to available anonymised or aggregated data may be granted following review and is subject to Data Sharing Agreement NIC-185179, which expires on 1 November 2023.

Patient data

This work uses data provided by patients and collected by the NHS as part of their care and support. Using patient data is vital to improve health and care for everyone. There is huge potential to make better use of information from people’s patient records, to understand more about disease, develop new treatments, monitor safety, and plan NHS services. Patient data should be kept safe and secure, to protect everyone’s privacy, and it’s important that there are safeguards to make sure that it is stored and used responsibly. Everyone should be able to find out about how patient data are used. #datasaveslives You can find out more about the background to this citation here: https://understandingpatientdata.org.uk/data-citation.

Information about the ESORT’s study’s use of data and privacy statements are available via URL: www.lshtm.ac.uk/research/centres-projects-groups/esort#privacy (accessed 13 September 2022).

Disclaimers

This report presents independent research funded by the National Institute for Health and Care Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, the HSDR programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, the HSDR programme or the Department of Health and Social Care.

Tendency to operate as an instrumental variable

The ESORT study uses a preference-based IV design, with the IV as the hospital’s TTO. 19 The TTO is defined for each qualifying emergency admission, as the proportion of eligible emergency admissions in the 12 months prior to each admission in the specific hospital, who received ES rather than NES strategies (Figure 28). A precedent study by Keele et al. took a similar approach, using TTO at the surgeon level, to evaluate ES compared with NES strategies using US claims data. 20 A recent systematic review identified 185 studies that applied preference-based IV methods within health research and reported that such methods were most commonly applied to cancer, cardiovascular diseases and mental health. 90 As noted by the authors, a valid IV must (1) strongly predict treatment status (‘relevance’), (2) affect outcomes only through exposure (‘exclusion’), (3) not share any unmeasured causes with the outcome (‘unconfoundedness’) and (4) affect treatment status in only one direction (‘monotonocity’). 90

FIGURE 28.

Variation in the TTO across 175 NHS hospitals in the 1 year prior to emergency hospitals admissions that meet the inclusion criteria for each of the five acute conditions (2009–19). (a) Acute appendicitis; (b) cholelithiasis; (c) diverticular disease; (d) abdominal wall hernia; and (e) intestinal obstruction.

Hospitals that differ in their TTO may be expected to differ in the probability that they will provide ES to a particular patient. We find considerable variation across hospitals in their TTO, even after adjusting for observable characteristics of the patients. The hospital’s TTO was strongly correlated with ES receipt for each of the five conditions, even after controlling for a rich set of covariates, with F-statistics that ranged from 450 (diverticular disease) to 24,517 (cholelithiasis) (Table 22). A commonly applied threshold for ‘weak’ instruments is an F-statistic less than 10. 53 Therefore, the hospital’s past preference for ES strongly predicts treatment choice for the current patient.

The validity of the IV analyses rests on the assumption that, conditional on the included variables, the TTO does not influence the outcomes except through changing the uptake of ES [i.e. assumptions 1 and 2. See Chapter 3, Sensitivity analyses for further information]. To increase the plausibility of this assumption, we adjust for fixed effects (indicators) for each financial year, observable patient characteristics and proxies for hospital quality, a key unobservable confounder. Including hospital fixed effects would result in greatly inflated standard errors, as they would remove all between-hospital variation. We did not include these fixed effects, as they aim to control for time-invariant confounders, captured by the proxies for hospital quality (i.e. past mortality or re-admissions either at baseline or over the preceding year). Moreover, the approach taken, of controlling for baseline and a moving window of mortality and re-admissions, reduces the concern that bias may be introduced if sicker patients tend to attend hospitals with a higher (or a lower) TTO, as we anticipate that this is captured in the measured covariates, including hospital quality. Although the assumption that the IV is not correlated with outcomes or unobserved confounders, conditional on the variables controlled for, is fundamentally untestable, but some reassurance is provided if observed covariates do not vary across levels of the instrument. Figure 29 illustrates that observed confounders are similar across levels of the TTO, which provides support for the underlying assumptions.

FIGURE 29.

Mean level of rescaled baseline covariates according to the level of the IV. (a) Acute appendicitis; (b) cholelithiasis; (c) diverticular disease; (d) abdominal wall hernia; and (e) intestinal obstruction.

Person-level instrumental variable approach

The IV approach begins by estimating treatment effects for ‘marginal’ patients, that is patients for whom there was equipoise about the ES decision according to these measured characteristics (e.g. age, TTO), as well as characteristics unmeasured in the data (e.g. physiology). For these (hypothetical) marginal patients, the IV approach estimates treatment effects for patients for whom a small change (or nudge) in the TTO (the instrument) can ‘tip the balance’ towards ES, but does not change the level of any risk factors, including factors that are not unmeasured. Comparing outcomes for patients defined according to small differences in the TTO, therefore, provides an estimate of the causal effect of ES compared with NES for similar patients. By repeating this contrast across different levels of TTO, the study can estimate treatment effects for sets of marginal patients with different combinations of confounders (e.g. frailty levels).

Each individual in the data set, given their observed and unobserved confounders, would be a marginal patient at some level of the TTO. In the absence of further information, we might simply calculate individual-level treatment effects as the average across all marginal patients with similar characteristics to the individual to obtain an effect estimate for each individual. However, the observed decision on whether or not an individual patient has ES provides some information on the extent to which the levels of unobserved confounders for that individual either encouraged or discouraged ES. For instance, if an individual who, according to the observed baseline measures, is at low risk and presents to a hospital with a low TTO does actually receive ES, then we can infer that their unobserved characteristics were such that they influenced the decision to have ES. Therefore, for each individual, a treatment effect is obtained by averaging the treatment effects for those marginal patients who share the same observed characteristics, and who has estimated levels of unobserved confounders that are consistent with the observed ES decision, for that individual, given the TTO in the hospital that they were admitted to, at the time they were first seen by the surgical team (i.e. time zero). The estimated individual-level treatment effects can then be averaged over any sample characteristics to report the effectiveness of ES at the subgroup level, or for the full sample, to obtain an average treatment effect estimate.

This LIV approach was implemented as follows. First, each patient’s propensity for ES was estimated according to their observed characteristics and the TTO. Second, an outcome model was estimated relating the observed outcome to the individuals’ observed characteristics and their propensity for ES, along with interactions between them. After estimating the model, the marginal treatment effect was obtained by considering the impact of a marginal change in the propensity for ES on outcomes. Third, numerical integration was used to obtain individual-level treatment effects. The steps were bootstrapped 300 times (200 times for sensitivity analyses because of computational complexity) to obtain standard errors and CIs. (For further details on the estimation steps, see Basu. 55)

Proxies for the quality of acute care

The validity of an IV analysis assumes that, conditional on the observed baseline covariates, the IV does not have a direct effect on the outcome, except through influencing the receipt of treatment. The requisite assumptions could be violated if the quality of the acute care, which is an unmeasured variable, was associated with the hospital’s TTO. Information was, therefore, collated to proxy the quality of acute care for emergency admissions with each acute condition, by extracting from the HES data rates of all-cause mortality and emergency re-admissions up to 90 days for each hospital. The proxy measures for quality were chosen to adjust for both time-constant differences in quality across hospitals, and those that differed over time. This information was reported for each condition for the 2009–10 financial year to provide a baseline, time-invariant proxies for care quality in each hospital. The information was also reported for the one year preceding each qualifying emergency hospital admission, to provide time-varying proxies for care quality.

We chose to include these proxies for care quality rather than hospital fixed effects to provide a more specific proxy for care quality and to avoid removing all between-hospital variation in rates of ES, which would lead to inflated standard errors in the estimated effectiveness of ES.

As part of the sensitivity analyses, we considered external measures of hospital quality using data from NELA. 59–61 As data were not available for all years of the study, and definitions changed over time, we constructed an average (weighted by volume) using data from 2016–18 for the following seven indicators of quality of perioperative management for emergency laparotomy patients, which we anticipate would capture the influence of any potential time-invariant observed confounders associated with hospital quality:

1. adjusted mortality rate

2. proportion of patients in whom a risk assessment was documented preoperatively

3. proportion of patients arriving in theatre within a time appropriate for the urgency of surgery

4. proportion of patients with a calculated preoperative risk of death > 5% for whom a consultant surgeon and anaesthetist were present in theatre

5. admission to critical care when risk of death is ≥ 5%

6. unplanned returns to theatre

7. unplanned returns to critical care.

Sensitivity analyses

We assessed the extent to which the findings from the main analyses were sensitive to alternative definitions and assumptions. We consider alternative definitions of ES by considering a more conservative panel definition of ES (i.e. SA1), and a reduced time window for the procedure to qualify as ES by considering only procedures that occurred before the 75th percentile of the time of ES used in the main analysis (i.e. SA2). We explored the sensitivity of the findings to alternative proxies for hospital quality (i.e. SA3) by using an external proxy for hospital quality (i.e. the NELA quality measures). In addition, we considered the impact of removing hospitals with low volume for the procedures of interest (i.e. SA4). We excluded hospitals whose volume of eligible procedures was less that thresholds of one IQR below the median, which, in this SA, led to the exclusion of 22 , 23, 27, 25 and 16 hospitals for the appendicitis, cholelithiasis, diverticular disease, hernia and intestinal obstruction cohorts, respectively, and to between 2.84% (Intestinal obstruction) and 6.25% (diverticular disease) of observations. We considered an alternative definition of the primary outcome of DAOH (i.e. SA5), which weights the number of DAOH prior to death for patients who died before day 90. 62 Finally, we used regression adjustment to report estimates of relative effectiveness under the assumption of no unobserved confounding (i.e. SA6).

Search for appropriate health-related quality-of-life scores and adjustment

Findings of the literature review

The literature review (up to September 2021) identified five studies for appendicitis,7,8,67,133,134 one study for diverticular disease,94 24 studies for cholelithiasis,83,135–158 124 studies for hernia,84,108,159–281 and four studies for intestinal obstruction. 85,282–284 In addition to these studies, one additional study that met the inclusion criteria for diverticular disease was identified by the team on a separate search14 and, therefore, the total number of studies reviewed for diverticular disease was two. 14,94

The main reasons why studies above failed to meet the inclusion criteria were that (1) the study had a different definition of study population compared with the ESORT study,135 (2) the study had an insufficient duration of follow-up period,7 (3) the study interventions considered were not performed in the emergency setting247 and (4) the instrument used to measure HRQoL was not the EQ-5D index. 160

Of the studies identified, only two studies8,67 met the criteria for appendicitis, one study met the criteria for diverticular disease,14 three studies met the criteria for cholelithiasis,83,136,147 one study met the criteria for hernia84 and one study met the criteria for intestinal obstruction. 85 For appendicitis, the study by O’Leary et al. 8 was chosen over the study by Sippola et al. 67 because the latter study reported outcomes for a median follow-up of 7 years, whereas O’Leary et al. 8 reported HRQoL at baseline and at 1 year. None of the studies identified for cholelithiasis and abdominal wall hernia was restricted to patients admitted into hospital as an emergency. The studies by Sutherland et al. 83 for cholelithiasis and Rutegård et al. 84 for abdominal wall hernia were favoured over other studies (e.g. Karimuddin et al. 136 and Rosenmüller et al. 147) because they restricted the population to symptomatic patients. Although it was not possible to find studies conducted in the UK, the studies that met the selection criteria were from similar health-care systems and from populations with similar demographics and cultural norms to the UK. The study by O’Leary et al. 8 was undertaken in Ireland, the study by Thornell et al. 14 in Sweden and Denmark, the study by Sutherland et al. 83 in Canada, the study by Rutegård et al. 84 in Sweden and the study by Young and Zahid85 in Australia.

Four14,83–85 out of the five studies selected compared different forms of ES, rather than ES compared with NES strategies. For this reason, as a unified approach across all five conditions, we applied the same HRQoL scores at baseline and at 1 year to both comparison groups (see Chapter 4, Outcomes), in keeping with the assumption noted above, that is any differences in HRQoL between the comparison groups would be captured by differences in 1-year mortality and in the rate and duration of emergency re-admissions (Figure 30).

FIGURE 30.

Health-related quality of life trajectory following initial (index) emergency admission and emergency re-admission for the base case, which assumes that HRQoL reaches follow-up levels following hospital discharge (panel A) and linear interpolation (B) (SA4). (a) Immediate interpolation. Baseline HRQoL is assumed to apply constantly for the duration of the index admission and any emergency re-admission. Following the index admission, the HRQoL is assumed to apply constantly for the duration of the period before the final (1-year) end point, which is accrued immediately after discharge. (b) Linear interpolation. Baseline HRQoL is assumed to apply constantly for the duration of the index admission and any emergency re-admission. HRQoL between the end points is assumed to increase linearly.