Heparin versus citrate anticoagulation for continuous renal replacement therapy in intensive care: the RRAM observational study

Identification of intensive care units

We identified adult general ICUs among those participating in the ICNARC CMP, which covers 100% of adult general ICUs in England and Wales. We defined a general ICU as a stand-alone ICU or combined ICU–high-dependency unit (HDU). We excluded stand-alone HDUs, specialist ICUs (e.g. cardiothoracic ICUs and neuroscience ICUs) and ICUs in specialist centres (e.g. orthopaedic hospitals).

Survey of anticoagulation practice

The aim of the survey was to identify which ICUs had switched to RCA for CRRT and when that switch occurred. We used online software SurveyMonkey [SVMK Inc., San Mateo, CA, USA; www.surveymonkey.com (accessed 6 October 2020)] to create an online survey (see Report Supplementary Material 1). We asked respondents whether their ICU was using SHA, RCA or a combination of both anticoagulation modes for CRRT. Dependent on the response to this question, the total number of questions to be completed varied from 6 to 13 and covered the following:

• for those using SHA

  • current effluent flow rate

  • which CRRT system is in operation

• for those using RCA

  • dates corresponding to start and completion of RCA implementation

  • effluent flow rate before and after the switch to RCA

  • CRRT system in use before and after the switch to RCA

• for those using both RCA and SHA

  • dates corresponding to start and completion of RCA implementation

  • effluent flow rate before and after the switch to RCA

  • CRRT system in use for both SHA and RCA

  • proportion of patients receiving SHA

  • conditions/indications that identify patients receiving SHA.

We piloted the survey among pharmacists and ICU clinicians to ensure that the questions could be understood and would provide the desired information.

Survey administration

The survey was administered via an e-mail containing a link to the online survey and instructions for completion. In June 2018, we sent simultaneous e-mails to clinical directors/lead clinicians of all identified ICUs (via ICNARC) and the UK Clinical Pharmacy Association critical care network. We asked e-mail recipients to complete the survey or forward the survey to the most relevant colleague to reply on behalf of their ICU. Respondents provided contact details and selected their ICU from a drop-down list of ICUs participating in the ICNARC CMP. This allowed identification of duplicate submissions for the same ICU.

Survey data management

Responses were entered directly into SurveyMonkey and subsequently downloaded into a Microsoft Excel^® (Microsoft Corporation, Redmond, WA, USA) spreadsheet. Once downloaded, we checked individual responses for completeness and, where a survey had been initiated but abandoned before completion, we sent an e-mail to the respondent asking them to complete their entry or provide further information. In the case of contradictory submissions for the same ICU, we sent an e-mail to all respondents from the individual ICU requesting further clarification until agreement was reached.

Identification of intensive care units using regional citrate anticoagulation for continuous renal replacement therapy

In the case of ICUs completing the switch from CRRT to RCA before 1 January 2017, we confirmed the switch dates in one of two ways: we either requested purchase details of the consumables involved in CRRT (including anticoagulants and fluids) or asked the clinician to cross-check on computerised information systems.

Where consumables/confirmatory information conflicted with the original survey entry, we contacted sites for further clarification to allow resolution.

Identification of patients exposed to regional citrate anticoagulation

The final linked data set was combined with the survey of CRRT anticoagulation practices to identify patients who were admitted during periods in which the treating ICU used either the exposure or the comparator. We identified exposed patients as patients who received CRRT in an ICU after the date (as reported by the ICU) on which the ICU had completed the transition from SHA to RCA. We identified comparator patients as patients who received CRRT in an ICU prior to the date (as reported by the ICU) that the ICU started to transition to RCA or in an ICU that reported using SHA throughout.

Inclusion criteria

• Age ≥ 16 years.

• Admitted to an adult general ICU in England or Wales participating in the ICNARC CMP between 1 April 2009 and 31 March 2017.

• Receipt of CRRT in ICU, identified by the recording of renal support, as defined by the UK Department of Health and Social Care Critical Care Minimum Data Set (CCMDS),19 on at least 1 calendar day during the ICU stay.

Exclusion criteria

• Pre-existing ESRD, identified by the recording of a requirement for CRRT in the ICNARC CMP data set (including, but not limited to, chronic haemodialysis, chronic haemodiafiltration and chronic peritoneal dialysis for irreversible renal disease, within 6 months prior to admission to the ICU).

• Admitted to an ICU after kidney or multiorgan (including kidney) transplantation identified from the primary and secondary reasons for admission to ICU recorded in the ICNARC CMP data set.

• Admitted with acute hepatic failure identified from the primary reason for admission to the ICU recorded in the ICNARC CMP data set.

The following additional exclusion criteria were applied to the linked data prior to analysis:

• no linked records from HES/PEDW (because of either incomplete NHS numbers or application of the national data opt-out)

• admitted to hospitals that did not complete the survey of anticoagulation practice, or responses to essential questions in the survey were incomplete

• discrepant data for date of death from different sources (excluding discrepancies of a single day, which were accepted, with the date of death from the ICNARC CMP taken as the correct date)

• pre-existing ESRD, prior transplant or nephrectomy, identified by having a record in the UKRR with a date of first ever RRT for ESRD or date of transplant prior to the date of admission to ICU, and/or presence of an International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10),20 code or Office of Population Censuses and Surveys (OPCS) procedure code indicating ESRD, kidney transplant or nephrectomy in any HES/PEDW episode prior to the date of admission to ICU (ICD-10 codes: I12.0, I12.9, N18.6, Z94.0, Z99.2; OPCS codes: M02.2, M02.3, M02.4, M02.5, M02.6, M02.7, M02.8, M02.9, M03.8, M03.9).

Exposure

Patients receiving CRRT in an ICU after the date (as reported by the ICU) on which the ICU completed the transition from SHA to RCA.

Comparator

Patients receiving CRRT in an ICU before the date (as reported by the ICU) on which the ICU started to transition from SHA to RCA or receiving CRRT in an ICU that had not transitioned to RCA.

Primary clinical effectiveness outcome

The primary clinical effectiveness outcome was all-cause mortality 90 days after the first ICU admission at which CRRT was received.

Secondary clinical effectiveness outcomes

Secondary economic outcomes

Subgroup analysis

Owing to the reported immunomodulatory effects of citrate,9 we identified a prespecified subgroup of patients with sepsis (defined according to the Sepsis-3 criteria). 21

Micro-costing task analysis

A micro-costing analysis of the set-up and running of CRRT using SHA and RCA was conducted at eight ICUs. To estimate the cost of delivering CRRT at individual ICU level, ICUs were chosen to represent the common manufacturer systems identified from the survey. At each ICU, we conducted cognitive ‘walk-throughs’ (including hierarchical task analysis; see Report Supplementary Material 2) of CRRT set-up with representative clinicians, which allowed staff time and consumables for each task element to be estimated. 22 Staff time costs were obtained from the Unit Costs of Health and Social Care 2018,23 unit costs of anticoagulation drugs were based on the NHS Business Services Authority Drug Tariff24 and CRRT fluid costs on the manufacturers’ quoted prices. NHS Supply Chain costs were used for consumables. 25

Continuous renal replacement therapy system set-up time and frequency

Continuous renal replacement therapy system set-up time and the frequency of system failures were estimated from anonymised electronically held individual patient data from the Post-Intensive Care Risk-Adjusted Alerting and Monitoring (PICRAM) study (containing anonymised electronically held records of all patients treated at both general ICUs in Oxford, UK, and the Royal Berkshire Hospital ICU in Reading from 2009 to 2015) and from the electronic clinical information system for patients treated in Oxford, UK, following completion of PICRAM (2015–17). From these data, the number and distribution of intervals between one CRRT system failing and the next being in place for periods in which RCA and SHA were in use were determined alongside the number of transfusion episodes in the two periods.

Intensive care unit, hospital and longer-term costs

Unit costs of ICU and hospital LOS and dialysis were obtained from NHS Reference Costs 2017–1826 calculated as total cost per patient to 1 year following the index ICU admission. The cost of RRT for ESRD was also estimated for patients identified from UKRR as having ESRD treated using RRT, based on an assumption of three dialysis sessions per week using unit costs of dialysis from NHS Reference Costs 2017–18. 26

Health-related quality of life and quality-adjusted life-years

EuroQol-3 Dimensions, three-level version (EQ-5D-3L), health-related quality-of-life (HRQoL) data for patients at 3 months and 1 year after ICU discharge were obtained from the 8000-patient UK Intensive Care Outcome Network (ICON) study. 27 Eligible patients meeting the inclusion criteria were identified and divided into quartiles of age. Averaged EQ-5D-3L-based utility weights were calculated by age quartile at 3 months and 1 year and used as the measure of HRQoL. All patients developing ESRD and requiring dialysis were assigned a decrement of 0.14 to their HRQoL based on European norms28 compared with the equivalent age group from the Health Survey for England 2016. 29 This decrement was applied from the date of first RRT for ESRD forward. HRQoL data at 3 months and 1 year were then combined with the survival data to calculate QALYs at 1 year, assuming zero HRQoL at baseline and death, and linear interpolation between the time points.

Ethics

An application for approval by the South Central – Oxford B Research Ethics Committee (REC) and Health Research Authority (HRA) was submitted on 23 March 2018. An application to the Confidentiality Advisory Group (CAG) was submitted at the same time to seek support for approval under Section 251 of the NHS Act 2006 to permit the use of identifiable patient data without consent. 30 Favourable opinions from the REC and CAG were received on 17 April 2018 and 21 June 2018, respectively (REC reference 18/SC/0204; CAG reference 18/CAG/0070). HRA approval was received on 22 June 2018.

Protocol amendments

Following receipt of initial favourable opinion, three non-substantial protocol amendments were submitted for REC approval, two of which also required CAG approval owing to amended access to patient identifiable data without consent. These were:

• to include data linkage to the PEDW data set to allow for identification of admissions to Welsh hospitals following discharge from ICU (protocol version 2.0)

• to obtain electronically held data from Oxford University Hospitals NHS Foundation Trust for the period following the completion of the PICRAM study (2015 to 31 March 2017), required for the health economic analysis (protocol version 2.1)

• editorial changes and the addition of a prespecified subgroup and secondary outcome in line with the statistical analysis plan (protocol version 2.2).

Additional amendments to the initial HRA/REC approvals were submitted for the addition of the research sites for the micro-costing study.

Study registration

The RRAM study was prospectively registered on ClinicalTrials.gov, published online on 4 June 2018, as NCT03545750.

Study Steering Committee

The National Institute for Health Research (NIHR) convened an independently chaired and majority (at least 75%) independent Study Steering Committee (SSC). Dr Steve Harris chaired the SSC, which also included clinicians, methodologists and patient and public involvement (PPI) representatives.

Sponsorship

The RRAM study was sponsored by the ICNARC and managed by the ICNARC Clinical Trials Unit.

Patient and public involvement

Engagement with patients was vital to ensuring the successful delivery of the RRAM study. A former critical care patient was a co-investigator on the study and contributed to all aspects of the study, including design, conduct, management, analysis, interpretation of results and dissemination as members of the study management group. In addition, the SSC included an independent patient representative.

Power calculation

Based on preliminary analysis of ICNARC CMP data, we anticipated a total available sample size of ≈ 85,000 patients from 184 ICUs. Prior to study commencement, UK suppliers indicated that 90 ICUs were currently using RCA. To assess the likely power of the available data to address the research question of interest, we simulated 1000 replications of the study using available ICNARC CMP data under the following assumptions:

• Thirty-five changes from SHA to RCA will be observed within the available data. This was a conservative assumption from the 90 ICUs across the UK reported to be using RCA, to allow for use in ICUs outside England, specialist ICUs and changes that occurred when ICUs were not participating in the ICNARC CMP. In each simulation, 35 ICUs were selected at random to represent the observed changes.

• Changes from SHA to RCA will be evenly distributed over the time period of the study. In the simulations, the changeover quarter for each of the 35 randomly selected ICUs was sampled from a uniform distribution from between their second and penultimate quarters.

• Fifteen ICUs will have changed from SHA to RCA prior to the start of the study. In each simulation, 15 ICUs were selected at random to contribute data to the RCA group throughout. In the simulations, the indicator t_ij is used to indicate ICU i using RCA in quarter j.

• The distribution of risk of 90-day mortality for patients receiving RRT in UK ICUs will follow that of the ICNARC_H–2015 model for acute hospital mortality in critical care32 (with a mean of 50%). This model has excellent discrimination (area under the receiver operating characteristic curve ≈ 0.9) and calibration in this population. In the simulation, the patient-level risk of death for patient k admitted in quarter j to ICU i, p_ijk, was calculated using this model.

• The between-ICU standard deviation (SD) for 90-day mortality will be 0.22. This value was estimated as the observed value for risk-adjusted acute hospital mortality in the ICNARC CMP among patients receiving RRT and corresponds to an intraclass correlation coefficient of 0.015. In each simulation, an ICU-level effect for ICU i, u_i, was sampled from a normal distribution with a mean of 0 and a SD of 0.22. For the purpose of the simulations, no clustering of observations for patients within quarters in the same ICU was assumed.

• Changing from SHA to RCA will be associated with an odds ratio (OR) for 90-day mortality of 0.9. For the purpose of simulation, a change in level was considered with no change in slope only.

In each simulation, the ‘observed’ outcome for each patient, y_ijk, was sampled from a Bernoulli distribution based on the following model:

The estimated treatment effect within each simulation was then estimated using a multilevel logistic regression with robust standard errors. Simulations were undertaken using Stata^®/SE version 14.2 (StataCorp LP, College Station, TX, USA). The random number seed was set prior to analysis to ensure reproducibility of results.

The results of the simulations showed that this sample will have ≈ 81% power (p < 0.05) to detect a step change in 90-day mortality corresponding to an OR of 0.9.

Analysis of clinical effectiveness

An analysis of individual patient data, following techniques based on interrupted time series (ITS) analysis was used to assess clinical effectiveness, where the interruption corresponds to the change from SHA to RCA for CRRT. The analysis followed the eight quality criteria for ITS design and analysis described by Ramsay et al. 33

Random-effects multilevel generalised linear models (patients nested within ICUs) were used to estimate the ICU-level effect of transitioning to RCA on patient-level outcomes. Logistic models were used for binary outcomes and linear models for continuous outcomes. The study includes periods both before and after the switch from SHA to RCA in individual ICUs, and a comparator group of ICUs that did not change anticoagulation mode. The effect estimate is the within-ICU change in trends, with the control ICUs primarily improving estimates of patient-level confounders and the underlying secular trend. Models were fitted with robust standard errors to allow for model misspecification, including autocorrelation and heteroscedasticity.

The primary impact model for the effect of the change from SHA to RCA allowed for a change in both level and slope. Linear trends were assumed in both pre-intervention and post-intervention periods. The period of data from the start of the transition from SHA to RCA until 6 months later was omitted from the model to allow for potential imprecision in the reporting of the time of change and time to transition from one modality to the other in the initial survey, and to allow time for the new modality to be embedded into routine practice. As per the prespecified statistical analysis plan, the choice of 6 months was extended from the originally planned 3 months to account for the substantial number of ICUs taking > 3 months to transition (6 months was the 80th percentile of the time to transition). Where longer transition times occurred, these were accounted for by excluding the corresponding window. The results of the regression models are reported as the OR (or, for continuous outcomes, difference in means) with 95% confidence interval (CI) for the change in level and the OR per year (difference in means per year) with 95% CI for the change in slope associated with the change from SHA to RCA. The overall significance of the change from SHA to RCA was assessed by the joint test of the two parameters for the change in level and change in slope.

The models were adjusted for patient covariates selected based on importance in previous analyses of long-term outcomes following ICU, including acute hospital mortality,32 90-day mortality, 1-year mortality and development of ESRD. 34 The covariates included were:

• age (linear)

• comorbidities identified from HES/PEDW based on the Royal College of Surgeons adaptation of the Charlson Comorbidity Index

  • congestive cardiac failure

  • peripheral vascular disease

  • cerebrovascular disease

  • chronic pulmonary disease

  • chronic liver disease

  • malignancy

• severe conditions in medical history recorded in the ICNARC CMP

  • haematological malignancy

  • immunocompromise [radiotherapy, chemotherapy or daily high-dose steroid treatment during the 6 months prior to ICU admission; recording of human immunodeficiency virus (HIV)/acquired immunodeficiency syndrome (AIDS) or a congenital immunohumoral or combined immune deficiency state]

  • severe liver disease (biopsy-proven cirrhosis, portal hypertension or hepatic encephalopathy)

  • metastatic disease

  • severe respiratory disease (shortness of breath with light activity or a requirement for home ventilation)

  • very severe cardiovascular disease (New York Heart Association functional class IV)

• dependency prior to admission to acute hospital (no, some or total requirement for assistance with activities of daily living)

• body mass index (restricted cubic splines with three knots)

• location prior to admission to the ICU [emergency department or not in hospital, other hospital (not critical care), other critical care unit, planned admission from theatre following elective/scheduled surgery, unplanned admission from theatre following elective/scheduled surgery, admission from theatre following urgent/emergency surgery, ward or intermediate care area]

• cardiopulmonary resuscitation (in or out of hospital) within 24 hours prior to ICU admission

• primary reason for admission to ICU (body system)

• receipt of mechanical ventilation during the first 24 hours in ICU

• physiology recorded during the first 24 hours in the ICU

  • highest heart rate (restricted cubic splines with four knots)

  • lowest systolic blood pressure (restricted cubic splines with four knots)

  • highest central temperature, or highest non-central temperature + 1 °C if no central temperature recorded (restricted cubic splines with four knots)

  • lowest respiratory rate (right-restricted cubic splines with four knots)

  • total urine output (restricted cubic splines with four knots)

  • lowest total Glasgow Coma Scale score (3, 4–6, 7–13, 14, 15, or sedated for entirety of first 24 hours)

  • PaO₂/FiO₂ level from the arterial blood gas with the lowest PaO₂ level (restricted cubic splines with four knots)

  • lowest pH level (restricted cubic splines with four knots)

  • PaCO₂ level from the arterial blood gas with the lowest pH level (restricted cubic splines with three knots)

  • highest blood lactate level (restricted cubic splines with four knots)

  • highest creatinine level (right-restricted cubic splines with four knots)

  • highest urea level (restricted cubic splines with four knots)

  • highest sodium level (restricted cubic splines with three knots)

  • lowest white blood cell count (restricted cubic splines with four knots)

  • lowest platelet count (restricted cubic splines with four knots).

As a sensitivity analysis, all analyses were repeated using data from only those ICUs with data from both before and after a transition from SHA to RCA.

Analysis of cost-effectiveness

The cost-effectiveness analysis (CEA) took a health services perspective and reports mean (95% CI) incremental costs and QALYs at 1 year associated with a change from SHA to RCA for CRRT, overall and for prespecified subgroups. The CEA used multilevel generalised linear models that allow for clustering of patients in ICUs. Separate multilevel models were fitted for costs and QALYs, adjusted for the same patient covariates as the models for clinical effectiveness. INMB at 1 year associated with a change from SHA to RCA was estimated valuing incremental QALYs according to a National Institute for Health and Care Excellence (NICE)-recommended willingness-to-pay threshold for a QALY gain (£20,000) and subtracting the incremental costs from this value. The INMB calculation assumed no correlation between costs and QALYs; a single-level, bivariate, seemingly unrelated regression model for costs and QALYs indicated that the impact of not accounting for correlation would be small and conservative (≈ 3% increase in standard error of INMB). Missing data were addressed as described in Handling of missing data and were assumed to be missing at random, conditional on baseline covariates, resource use and observed end points.

The economic analysis projected lifetime cost-effectiveness by summarising the relative effects of the alternative strategies on long-term survival and HRQoL, compared with that of the age- and gender-matched general population. 38,39 There is evidence to suggest that critical care survivors face a higher probability of death and lower quality of life after the critical care episode than the general population (age and gender matched). 40–42 However, there is no clear evidence on the duration of this excess mortality; the strongest evidence is in support of an excess mortality of up to 5 years. The long-term survival of each patient in the RRAM study was calculated from observed survival within 1 year and their predicted survival from age- and gender-matched general population survival by applying excess death rates for years 2 to 5, after which we assumed that all-cause death rates were those of the age- and gender-matched general population. The HRQoL of RRAM study patients at 1 year was approximately 73% of that of the age- and gender-matched general population. 43 We assumed that HRQoL would improve over 5 years to match the HRQoL of the age- and gender-matched general population. After 5 years, we applied HRQoL values for the age- and gender-matched general population. Lifetime QALYs were reported by combining life-years and HRQoL. Lifetime costs were projected by applying chronic morbidity costs associated with use of dialysis for ESRD over the lifetime. No further differential costs between the groups at 1 year were considered.

In addition to mirroring the subgroup and sensitivity analyses conducted for the analysis of clinical effectiveness, further sensitivity analyses tested the robustness of the results to the selection of sites for the micro-costing exercise. A ‘best-case’ scenario was constructed by combining the lowest treatment-related costs for CRRT with RCA from any unit participating in the cognitive walk-through with the highest treatment-related cost for CRRT with SHA from any unit. A ‘worst-case’ analysis combined the highest derived cost for CRRT with RCA from any unit with the lowest derived cost for CRRT with SHA.

Units switching from systemic heparin anticoagulation to regional citrate anticoagulation

Across the 111 ICUs that reported changing to RCA, the median time taken to complete the transition to fully implementing RCA was 31 days [interquartile range (IQR) 0–91 days]. Among these ICUs, 32 (28.8%) reported that the effluent flow rate used during RCA was lower than that used during SHA, whereas six (5.4%) reported an increased effluent flow rate during RCA.

Residual systemic heparin anticoagulation use

Residual use of SHA was reported by 64 (57.7%) of the 111 ICUs that changed to RCA. In these ICUs, the median proportion of patients estimated to receive SHA was 10% (IQR 5–20%). The most commonly reported reason for patients receiving SHA in a unit that had switched to RCA was severe liver disease with a perceived inability to metabolise citrate. Other reasons reported for patients receiving SHA after changing to RCA are listed in Table 3.

Patient characteristics

The characteristics of patients in the SHA and RCA groups were similar (Table 4). In both groups, the median age was 67 years, ≈ 60% were male and > 80% were medical admissions. Illness severity scores during the first 24 hours of admission to the ICU were similar in both groups, and ≈ 47% of patients in both groups had sepsis. Patients in the RCA group, however, tended to have higher rates of comorbidities and a slightly higher predicted mortality.

Primary outcome

Unadjusted mortality at 90 days following the index admission to ICU was 53.3% among the SHA group and 54.0% among the RCA group (Table 5). There was no significant trend over time for 90-day mortality during either the SHA or RCA periods (OR 1.00, 95% CI 0.99 to 1.01, and OR 1.00, 95% CI 0.96 to 1.04, respectively). When adjusted for patient covariates, there was no step change associated with the change to RCA, OR of 0.98 (95% CI 0.89 to 1.08) for 90-day mortality (Figure 5 and Table 6).

FIGURE 5.

Effect of introduction of citrate on the primary outcome of 90-day mortality in (a) all eligible patients; and (b) restricted to ICUs with data from both before and after a transition. Points are observed values plotted in 3-month intervals and lines are fitted values from models. Please note that observed and fitted values may not correspond because fitted values are adjusted for other covariates. For units that transitioned from SHA to RCA, time = 0 is set to 6 months after transition (the end of the censoring period). For units that did not transition, time = 0 is set to the end of the observation period.

Secondary outcomes

Calendar days of organ support

The median unadjusted number of days of renal support was 1 day higher in the RCA group (see Table 5). During the SHA period the trend was a small increase in the number of days of renal support (difference in means per year 0.06 days, 95% CI 0.04 to 0.09 days), whereas there was no trend over time during the RCA period (difference in means per year –0.08 days, 95% CI –0.19 to 0.04 days). The change to RCA was associated with a ≈ 0.5-day step change increase in the number of days of renal support (difference in means 0.53 days, 95% CI 0.28 to 0.79 days) when adjusted for patient covariates (see Table 6). Consistent with this, we observed similar step change increases in number of days of advanced cardiovascular support (difference in means 0.23 days, 95% CI 0.09 to 0.38 days) and advanced respiratory support (difference in means 0.53 days, 95% CI 0.03 to 1.03 days) (Figure 6).

FIGURE 6.

Effect of introduction of citrate on secondary outcomes: days of organ support. Days of renal support (a) for all eligible patients and (b) restricted to ICUs with data from both before and after a transition; days of advanced cardiovascular support (c) for all eligible patients and (d) restricted to ICUs with data from both before and after a transition; and days of advanced respiratory support (e) for all eligible patients and (f) restricted to ICUs with data from both before and after a transition. Points are observed values plotted in 3-month intervals and lines are fitted values from models. Please note that observed and fitted values may not correspond as fitted values are adjusted for other covariates. For units that transitioned from SHA to RCA, time = 0 is set to 6 months after transition (the end of the censoring period). For units that did not transition, time = 0 is set to the end of the observation period.

Lengths of stay

During the index hospital admission, unadjusted LOS in the ICU were longer in the RCA group but the number of subsequent days in hospital were shorter (see Table 5). In both groups, there were no significant trends over time for ICU LOS; however, both subsequent days in hospital and total hospital LOS showed downwards trends during both SHA and RCA periods (Figure 7). When adjusted for patient covariates, the change to RCA was associated with a step change increase in ICU length of stay (difference in means 0.86 days, 95% CI 0.24 to 1.49 days). Subsequent days in hospital or total hospital length of stay showed a non-significant step change increase (Table 6).

FIGURE 7.

Effect of introduction of citrate on secondary outcomes: days of treatment. Days of treatment in ICU (a) for all eligible patients and (b) restricted to ICUs with data from both before and after a transition; subsequent days in hospital (c) for all eligible patients and (d) restricted to ICUs with data from both before and after a transition; and total days in hospital for (e) all eligible patients and (f) restricted to ICUs with data from both before and after a transition. Points are observed values plotted in 3-month intervals and lines are fitted values from models. Please note that observed and fitted values may not correspond as fitted values are adjusted for other covariates. For units that transitioned from SHA to RCA, time = 0 is set to 6 months after transition (the end of the censoring period). For units that did not transition, time = 0 is set to the end of the observation period.

Adverse events

The unadjusted proportion of patients experiencing bleeding during the ICU stay was 6.5% and 6.4% for SHA and RCA, respectively (see Table 5). Trends over time showed increasing rates of bleeding during SHA; however, this was not apparent during the RCA period (Figure 8). When modelled adjusting for prespecified factors, the change to RCA was not associated with a non-significant step change towards fewer bleeding episodes (OR 0.90, 95% CI 0.76 to 1.06) (see Table 6). Similarly, trends over time suggest that thromboembolic rates were increasing over time with SHA, but this was not observed with RCA (see Figure 8). When adjusting for patient covariates, the change to RCA had no step effect on thromboembolic episodes (OR 0.94, 95% CI 0.78 to 1.13) (see Table 6).

FIGURE 8.

Effect of introduction of citrate on secondary outcomes: bleeding episodes, thrombolic events and dialysis-dependent renal disease or transplant. Bleeding during ICU stay (a) for all eligible patients and (b) restricted to ICUs with data from both before and after a transition; thrombolic events before 90 days after hospital discharge (c) for all eligible patients and (d) restricted to ICUs with data from both before and after a transition; dialysis-dependent renal disease or transplant within 90 days of ICU admission (e) for all eligible patients and (f) restricted to ICUs with data from both before and after a transition; and dialysis-dependent renal disease or transplant within 1 year of ICU admission (g) for all eligible patients and (h) restricted to ICUs with data from both before and after a transition. Points are observed values plotted in 3-month intervals and lines are the fitted values from models. Please note that observed and fitted values may not correspond as fitted values are adjusted for other covariates. For units that transitioned from SHA to RCA, time = 0 is set to 6 months after transition (the end of the censoring period). For units that did not transition, time = 0 is set to the end of the observation period.

The unadjusted proportion of patients who developed ESRD treated by RRT at 1 year after the ICU admission was similar in both groups (see Table 5). There were no trends over time during both the SHA and the RCA periods except for a marginal increase in ESRD treated by RRT at 1 year (but not at 90 days) in the SHA period (see Figure 8 and Table 6). When adjusted for patient covariates, the change to RCA was not associated with a step change in the proportion of patients receiving RRT for ESRD at 1 year (OR 1.00, 95% CI 0.74 to 1.36) (see Table 6). Because 10,060 admissions were excluded for incomplete follow-up data in the UKRR at the 1-year time point, we conducted a post hoc analysis at 90 days (excluding 2497 patients with incomplete follow-up data) that produced results consistent with those seen at 1 year (see Table 6).

Subgroup analyses

The prespecified subgroup of patients meeting Sepsis-3 criteria in the first 24 hours following admission were less likely to be elective admissions, more likely to be mechanically ventilated at admission or during the first 24 hours, and had higher predicted mortality (Table 7). In line with the primary analysis, patients in the RCA group had more comorbidities and higher predicted mortality (Table 8). Unadjusted outcomes in the SHA and RCA groups were similar in terms of duration of organ support, length of stay, adverse events and mortality. Analyses of the primary outcome and secondary outcomes were consistent with the primary results (Table 9).

Sensitivity analyses

The characteristics of patients (Table 10) included in the sensitivity analyses and unadjusted outcomes (Table 11) mirrored those of the overall cohort. Sensitivity analyses of the primary and secondary outcomes, restricted to the 57 ICUs contributing data from before and after the change to RCA, were consistent with the primary results (Table 12), although step changes in days of respiratory and cardiovascular support and ICU length of stay became non-significant. Figures 5–9 show the trends over time and the step changes in outcomes associated with the change to RCA for sensitivity analyses, alongside their counterparts in the primary analysis.

FIGURE 9.

Effect of introduction of citrate on secondary outcomes: mortality. Mortality at acute hospital discharge (a) for all eligible patients and (b) restricted to ICUs with data from both before and after a transition; 30-day mortality (c) for all eligible patients and (d) restricted to ICUs with data from both before and after a transition; and 1-year mortality (e) for all eligible patients and (f) restricted to ICUs with data from both before and after a transition. Points are observed values plotted in 3-month intervals and lines are the fitted values from models. Please note that observed and fitted values may not correspond as fitted values are adjusted for other covariates. For units that transitioned from SHA to RCA, time = 0 is set to 6 months after transition (the end of the censoring period). For units that did not transition, time = 0 is set to the end of the observation period.

Absolute estimated costs and benefits

Analysis of age- and inclusion criteria-matched quartiles of ICON study participants provided HRQoL weights of 0.467 (ages 16–54 years), 0.525 (ages 55–66 years), 0.524 (ages 67–74 years) and 0.565 (ages > 75 years) at 3 months; and 0.552 (ages 16–54 years), 0.595 (ages 55–66 years), 0.581 (ages 67–74 years) and 0.577 (ages > 75 years) at 12 months. Unadjusted costs and QALYs are summarised in Table 14.

The main contributions to costs at 1 year were hospital bed-days, which corresponded to observed differences in mortality. Patients who died in hospital spent a mean of 9.7 days in ICU and 6.0 days in hospital post discharge from ICU, whereas patients who survived their hospital stay spent a mean of 12.7 days in ICU and 29.0 days in hospital post discharge from ICU. Patients admitted to units using RCA had longer lengths of stay in ICU and, therefore, higher costs relating to ICU bed-days, but also higher mortality, which translated to shorter lengths of stay post discharge from ICU and, therefore, lower costs relating to non-ICU hospital bed-days.

Despite only 2.8% of patients being alive and receiving dialysis at 1 year, dialysis was an additional driver of lifetime cost projections. Because patients admitted to units after transition to RCA had higher rates of dialysis at 1 year, the cost differential widened when projected to lifetime costs. When estimating QALYs, the absolute difference between groups was negligible at 1 year (bearing in mind that the 53.4% of participants who died within 90 days contributed zeros to the calculations of mean QALYs at 1 year), but widened slightly when projected over a lifetime and accounting for differences in age and dialysis outcomes.

Primary economic outcome: cost-effectiveness at 1 year

After adjustment for prespecified covariates, the cost-effectiveness models all consistently indicated increased costs associated with transition to citrate, with benefits centring on zero (Table 15). For the primary CEA, at 1 year, there was close to zero trend in costs during both SHA and RCA periods, but the transition to RCA was associated with estimated step-increase in costs of £2456 (95% CI £999 to £3912). Benefits in terms of QALYs at 1 year were estimated to be negligible and stable over time. The resulting estimated INMB at 1 year associated with a change to RCA was –£2376 (95% CI –£3841 to –£911). The estimated likelihood of cost-effectiveness at 1 year with a willingness-to-pay threshold of £20,000 per QALY was less than 0.1 (Figures 12 and 13), and this did not increase at higher willingness-to-pay thresholds.

FIGURE 12.

Uncertainty in the mean cost (£) and QALY differences and their distribution for the step change from SHA to RCA (at 1 year following admission).

FIGURE 13.

Cost-effectiveness acceptability curve for the step change from SHA to RCA (at 1 year following admission).

Secondary economic outcome: cost-effectiveness extrapolated to lifetime

Predicted long-term survival, HRQoL and costs beyond 1 year were pooled across treatment groups given that there was no statistically significant evidence of treatment effect on survival at 1 year. Trends in costs extrapolated to lifetime were similarly small during both SHA and RCA periods, but the transition to RCA was associated with a larger estimated step increase in costs of £4714 (95% CI £496 to £8933) (see Table 15). Trends for lifetime extrapolated benefits were close to zero during both SHA and RCA periods and the estimated step change during transition had a wide degree of uncertainty. The resulting estimated lifetime INMB associated with a change from SHA to RCA was –£724, with a corresponding wide degree of uncertainty (95% CI –£9446 to £7997). The wide degree of uncertainty resulted in estimated likelihood of cost-effectiveness of 44% at a willingness-to-pay threshold of £20,000 per QALY, increasing to 78% at a willingness-to-pay threshold of £100,000 per QALY (Figures 14 and 15).

FIGURE 14.

Uncertainty in the mean cost (£) and QALY differences and their distribution for the step change from SHA to RCA (extrapolation to lifetime).

FIGURE 15.

Cost-effectiveness acceptability curve for the step change from SHA to RCA (extrapolation to lifetime).

Strengths

To the best of our knowledge, this study is by far the largest linked cohort study undertaken comparing SHA and RCA. Among the 200 general ICUs in England and Wales participating in the ICNARC CMP (which had 100% coverage of general ICUs in the two countries by the end of the study period), 94% completed responses to the survey, allowing us to identify whether or not units had changed from SHA to RCA during the study period. We checked these transition dates in multiple ways to ensure reliability. This high response rate to our survey allowed us to include individual patient data from 181 ICUs in England and Wales, of which 57 contributed patient data during both pre and post transition periods. We believe that this presents a clear picture of what happens in clinical practice across the two nations when RCA is introduced. Our micro-costing study is based on practice from seven geographically distinct ICUs. With EQ-5D-3L health-related quality-of-life data from the multicentre 8000-patient ICON study, we believe that our health economic analysis is relatively robust.

Our linkage to the UKRR, HES and national mortality data allowed us to present longer-term follow-up data on very large numbers of patients. This allows us to deliver real-world evidence on the clinical effectiveness and cost-effectiveness of the national moves towards RCA therapy. We chose to maximise the number of patients and ICUs that had received RCA by including patients up to 31 March 2017. We knew that this would mean that patients in the last year would have incomplete data in the UKRR, which meant having to censor ESRD outcomes for the last year of inclusion. Because most patients registered with UKRR within 1 year of admission were registered within 90 days, we undertook a post hoc analysis of ESRD at 90 days rather that at 1 year to maximise use of the available data. The post hoc analysis was consistent with the 1-year outcome.

The SHA and RCA groups were relatively well matched, but the RCA group tended to have slightly higher rates of severe comorbidity and predicted mortality. We accounted for comorbidities and the covariates underlying the predicted mortality in our adjustment models.

Limitations

Our study has some important limitations. Foremost is the lack of patient-level data on treatment; we have studied patient effects when an ICU changes to using RCA for the majority of patients. Although we excluded patients with the most common reason ICUs reported reverting to SHA (primary admissions with liver failure), some patients included in the RCA group will undoubtedly have received SHA rather than RCA. Our results are, therefore, better interpreted as a pragmatic analysis of what happens to renally replaced ICU populations when units change to RCA than as an analysis of what would happen to individual patients.

The ambiguities surrounding testing and interpretation of ITS analyses were highlighted by the fact that, for many outcomes, we observed competing changes in trend and level. The (prespecified) joint tests draw on the statistical power provided by both differences, and so are considerably more sensitive than the p-values that would have been obtained from tests of either the change in trend or the step change alone. However, they do not account for the concordance in meaningful direction of the two component changes, so changes that would have opposing interpretations of clinical value (i.e. one harmful and one beneficial) are considered of equal statistical significance to changes that have concordant clinical value (are both harmful or both beneficial). They are appropriate tests of an impact model in which the interruption (switching from SHA to RCA) results in causal effects on both the level at the point of transition and the trend moving forwards. 63 This may not be an appropriate impact model for the effects of a change in clinical practice in ICU.

As our analysis is of routinely collected data, one difficult-to-assess but potentially strong contributor to observed trends in outcomes is trends in data quality. These might include the propensity for events to be recorded, the population coverage of data sets and the accuracy of linkage over time. Nearly all administrative data are subject to improvements in quality over time. The ICNARC CMP, for example, is known to have less than complete coverage of adult general ICUs prior to 2015, but this variation in completeness was expected to limit statistical power only and not produce bias. The NHS patient identification numbers required for linkage were collected only from 2006 and increase in completeness over time, with similar implications. However, similar variation in completeness or linkage to Civil Registrations or UKRR over time would be expected to produce variation in derived clinical outcomes of (post discharge) mortality and ESRD, respectively, because an absence of link is meaningfully interpreted as implying the absence of the outcome. 64 Such variation in the quality of administrative data can strongly bias analysis of trends. 65 What implications it may have for the analysis of step changes in the context of trends in ITS analysis is much less clear; it is plausible that, by accounting for trends both pre and post interruption, the estimates of step changes are robust to the influence of trends in data quality.

Although most of our clinical effectiveness analyses are based on many ICUs, the difference in time between filtration periods, though based on large patient numbers, comes from a database of two ICUs, which may limit its generalisability. The numbers of units of blood transfused and other items relevant to treatment costs were obtained from a sample of seven ICUs that participated in the cognitive walk-through. Both of these parameters were used in our health economic analyses, but neither were drivers of overall costs differences.

Last, it is worth highlighting that our study contains very large numbers of patients (though smaller numbers of ICUs – the unit of change) on which we have undertaken multiple analyses. As a result, it is possible to achieve high levels of statistical significance with little clinical meaning. Our sensitivity, subgroup and sepsis-cohort results in particular should be interpreted in this light.

Recommendation 1: research prioritisation

Our study demonstrates the possibilities of tracking the effects of changes in ICU practice in England and Wales at scale across routinely collected databases. Our findings strongly support the need to do this for other significant changes in practice, provided that the methodological issues identified above can be addressed. We therefore recommend that review of practice changes be conducted to identify candidates for evaluation of clinical and cost-effectiveness evaluation using methods such as those employed here.

Recommendation 2: methodology and data quality

Methodological work is required to understand the most appropriate way of measuring and testing the effects of changes in clinical practice in ICU and other health-care services. This includes how to interpret information about step changes in the context of associated changes in trend; if and how to allow for non-linearity in trends; and the potential implications of misspecification of trend shape parameters for interpretation of step changes. Given that most such analyses are likely to be based on routinely collected administrative data and patient registries, the potential implications of trends in data quality need to be understood and appropriate methods for handling these identified or developed. When linking to registries such as ONS mortality and UKRR, there is generally no way of knowing when a patient or event has failed to be recorded or when the record has failed to be linked. Indirect measures and quantitative bias analysis (involving error models or simulations) are likely to be required.