Point-of-care creatinine tests to assess kidney function for outpatients requiring contrast-enhanced CT imaging: systematic reviews and economic evaluation

Clinical effectiveness

• To perform a systematic review of studies that compare the results of POC creatinine tests with laboratory-based tests to assess kidney function in a non-emergency setting.

• To perform a systematic review of the clinical impacts and implementation of POC creatinine tests to assess kidney function before CT imaging. This will include assessment of the associated mortality and morbidity, patient-centred outcomes, adverse events, acceptability to clinicians and patients, and compliance.

Cost-effectiveness

• To perform a systematic review of published cost-effectiveness studies of the use of POC creatinine tests in a secondary care setting to assess kidney function before contrast-enhanced imaging.

• To develop a decision model to estimate the cost-effectiveness of the use of POC creatinine tests to assess kidney function before contrast-enhanced imaging. The relevant population is people who need contrast-enhanced imaging in a non-emergency situation and who do not have a recent SCr measurement.

• The objective of the decision model will link the diagnostic accuracy of POC creatinine tests to short-term costs and consequences (e.g. the impact on cancelled or delayed appointments, use and volume of contrast media and associated risks, such as PC-AKI). Short-term risks of PC-AKI will be linked to potential longer-term costs and consequences (e.g. CKD, end-stage renal disease and death) using the best-available evidence. Depending on the robustness of the evidence, additional exploratory analyses using assumptions and expert opinion may be also undertaken.

• The feasibility of extending the decision model to include other clinical outcomes that could be affected by any changes in the imaging decision based on the POC tests will also be assessed. These outcomes could include (i) any anxiety associated with having a delayed or cancelled CT scan and (ii) morbidity and mortality implications of performing unenhanced scans, or using lower doses of contrast agent. However, given that these outcomes will differ depending on the specific population and the underlying reason for imaging, it is envisaged that any extension of this nature will need to be constrained to a specific population/reason for the scan. The practicalities and value of developing a specific ‘exemplar’ application (with potentially limited generalisability) will be considered versus using a simpler and more generic approach (e.g. using threshold analysis to determine the magnitude of any impact necessary to result in a different decision based on conventional cost-effectiveness decision rules).

• The cost-effectiveness of the alternative POC tests will be expressed in terms of incremental cost per quality-adjusted life-year (QALY) and/or net health (or monetary) benefits.

Participants

To maximise the number of data on test accuracy, the eligible population for test accuracy studies was any adult patient group receiving POC creatinine testing compared with laboratory testing in a non-emergency/intensive care setting.

For studies reporting clinical or implementation outcomes, only studies of adult patients receiving POC tests before CT imaging in a non-emergency, outpatient setting were included.

Interventions

For test accuracy studies, details of the POC devices eligible for the review are presented in Table 1. This list is broader than those reported in the NICE scope and in the study protocol, which were restricted to devices that reported eGFRs. This was done to maximise the available evidence base because early on during the screening process it became evident that many studies were of devices that did not calculate eGFR (i.e. creatinine was measured), with eGFR being calculated manually by the study investigators. These studies were included where it was thought (following clinical and technical advice) that the model in question was sufficiently similar to the most recent version of the device (all the most recent models have the facility to present eGFR results). New versions of a device may sometimes incorporate software improvements (to allow eGFR outputs), a different interface or improved functionality, rather than changes in the way creatinine is analysed. For example, the recently released i-STAT Alinity was ‘built on the proven technology of the i-STAT System’,17 and hence studies were included that used an ‘i-STAT’ device.

All the eligible devices measure whole-blood creatinine using an enzymatic method. The devices are either handheld, tabletop or portable and need very small volumes of blood. Creatinine levels may be analysed either as one component of a panel of parameters or as a single measurement via a test card or specific cartridge.

Reference standard

• Non-urgent (results available after 1 hour) laboratory-based SCr measurement:

  • Jaffe method

  • enzymatic method.

• Urgent (results available within an hour) laboratory-based SCr measurement:

  • Jaffe method

  • enzymatic method.

• No testing, clinical judgement alone.

Outcomes

The eligible intermediate outcome measures were:

• diagnostic accuracy of POC creatinine devices compared with laboratory-based creatinine devices

• correlation between POC creatinine devices and laboratory-based creatinine devices

• test failure rates

• number of delayed or cancelled and rescheduled scans

• volume of intravenous contrast material used

• number of unenhanced scans

• number of hospital admissions

• hospital length of stay.

All relevant outcome definitions and cut-off points were extracted.

In addition, the following clinical outcomes were eligible:

• AKI (either PC-AKI or CI-AKI)

• fall in baseline eGFR or rise of baseline creatinine

• temporary renal replacement therapy

• new-onset CKD (stage 3 or worse)

• end-stage renal disease with the need for permanent renal replacement therapy

• health-related quality of life (HRQoL)

• mortality.

Eligible outcomes related to the implementation of the interventions of interest and related practical issues included:

• acceptability of POC devices (to clinicians and patients)

• patient satisfaction

• training requirements

• uptake and compliance.

Study designs

Synthesis of diagnostic accuracy data

For each device, estimates of the probabilities that individuals are classified by the POC device as having an eGFR in one of the four categories in Table 2 given their true eGFR is in one of those categories were required. These probabilities relate to the sensitivity and specificity of each device, which were used to populate the economic model in Diagnostic accuracy of point-of-care creatinine tests. Individuals are categorised as being at risk of PC-AKI if their eGFR is < 30 ml/minute/1.73 m² (i.e. category 1 in Table 2). Therefore, the probability that each POC device correctly classifies individuals in this category will reflect their sensitivity to detecting individuals at risk. To calculate the specificity of each POC device it is necessary to know the underlying distribution of patients across the different eGFR categories (see Diagnostic accuracy of point-of-care creatinine tests for details).

Separate syntheses were carried out for POC devices for which two or more studies reported data on individuals classified into the different categories by laboratory reference test and POC device. Devices with sufficient data were StatSensor (including StatSensor, StatSensor-i and StatSensor Xpress-i), i-STAT (including i-STAT and i-STAT1) and ABL (including ABL827 and ABL800 FLEX); hence, three separate analyses were carried out, pooling the data on three devices (i.e. StatSensor, i-STAT and ABL), assuming that the different specifications of each device does not differ in their diagnostic characteristics.

For each study i reporting data on all cells of Table 38 in Appendix 2 the number of individuals classified by a POC device as belonging to eGFR category k = 1, . . . ., 4, given true eGFR category (as determined by the laboratory reference test) j = 1, . . . ., 4, r_ijk were assumed to follow a multinomial distribution, which is a generalisation of the binomial distribution to more than two categories:

with n_ij defining the number of individuals with true eGFR in category j in study i, and p_jk defining the probabilities of being classified by a POC device in eGFR category k, when the true category is j (j, k = 1, . . . ., 4), which were assumed common to all studies.

The model was estimated in a Bayesian framework using Markov chain Monte Carlo in OpenBUGS (version 3.2.3; OpenBUGS Foundation, Imperial College London, London, UK),21,22 in which the probabilities were given a non-informative Dirichlet prior distribution:

The Dirichlet distribution is an extension of the beta distribution to multiple dimensions and ensures that the estimated probabilities always add to one. 21,23 Setting all the parameters equal to one, as in Equation 2, assigns equal density a priori to any vector of probabilities that sums to one.

Studies reporting only on collapsed categories were assumed to provide information on a function of the probabilities p_jk. This function varied depending on which categories were collapsed, with relationships determined using partitioning properties of conditional probabilities. Estimation of the probability that an individual in an included study (as opposed to the underlying population of interest for this assessment – see Diagnostic accuracy of point-of-care creatinine tests) has true eGFR in category j, T[j] was also required. For details see Appendix 3, Model for the probability that an individual has a true estimated glomerular filtration rate in each category.

As the posterior distributions of the probabilities are bounded at zero and one, they are expected to be highly skewed. Therefore, results are reported as posterior medians with 95% credible intervals (CrIs) and plotted as density strips. In Diagnostic accuracy of point-of-care creatinine tests, the mean probability estimates, calculated from 1000 simulated values from the posterior distribution obtained by thinning the 30,000 posterior values generated in each analysis of the evidence synthesis, were used to derive specificity and sensitivity. Density strips are horizontal rectangles that can represent an entire probability distribution in one dimension: the rectangle is darkest at the point of highest probability density, then shaded with darkness proportional to the density, gradually fading to white at points of zero density. 24 The width of the rectangle itself has no meaning, and is used only to distinguish between distributions arising from different analyses. Standard lines representing point and interval estimates tend to give the impression that the data equally support all points in the interval, whereas density strips give a better description of the uncertainty in a probability distribution, particularly for non-symmetric distributions.

Each model was run until convergence was satisfactory and then the results were based on a further sample of iterations from two separate chains. Convergence was assessed by inspecting history and Brooks–Gelman–Rubin plots. 25,26

Data from different studies were pooled under the assumption that they estimate common probabilities, given a true eGFR category (i.e. using a fixed-effects model). Extension to a model allowing for between-study heterogeneity in probabilities was considered, but as a result of the small number of studies reporting data on all categories and the small number of individuals in some categories (including several zeros), this was not deemed feasible. The OpenBUGS code and data used are given in Appendix 4.

Quantity and quality of research available

Figure 1 presents the study selection process in a Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram. The literature searches identified a total of 3350 unique records. After title and abstract screening, 171 references were retrieved and 54 unique studies were included in the review. Of these, 12 studies reported diagnostic accuracy data (expressed as, or allowing calculation of, sensitivity and specificity) for eGFRs,27–38 seven reported diagnostic accuracy data for only SCr,39–45 and 50 studies presented data on correlation and/or measurement bias between a POC device and a laboratory reference test. 14,27,28,30–76 Six studies reported data on workflow or clinical outcomes. 29,59,62,77–79

FIGURE 1.

Study identification process: PRISMA flow diagram.

All studies that reported data on diagnostic accuracy of either eGFR or SCr also reported correlation/measurement bias results, except one. 29 Three of the studies that reported data on workflow or clinical outcomes also reported data on diagnostic accuracy or correlation/bias. 29,59,62

Risk-of-bias assessment

Table 3 summarises the results of the QUADAS-2 assessment, split by POC device. Full results, including all signalling questions, are reported in Appendix 5.

Six studies were rated as being at low risk across all risk-of-bias domains, including two studies of ABL800,31,37 three studies of i-STAT33,37,38 and three studies of StatSensor. 28,30,37 Among the six studies27,29,34–36 with at least one domain rated as being at unclear or high risk of bias, three used correction factors after comparing initial POC results with laboratory reference test results from the same samples, including two studies of i-STAT34,35 and one StatSensor study. 36 Correction factors can be entered into StatSensor devices to correct for measurement bias (see Table 1). However, in these studies the correction was applied to align POC test results with the reference standard results using the same samples. Therefore, adjusted analyses reported in these studies may overestimate the accuracy of the POC devices. None of the ABL studies reported using its offset correction functionalities. Four studies27,34–36 (including three conference abstracts27,34,35) reported insufficient information to assess bias related to patient selection. Other risk-of-bias issues included the use of different MDRD equations between the index test and the reference standard,31 and the use of a Jaffe method for the laboratory reference test (vs. an enzymatic method for the POC test). 32

Only two studies had low applicability concerns across all domains, including one study of ABL800, i-STAT and StatSensor,37 and one study of i-STAT. 38 The most common applicability concern was the use of eGFR threshold. Three studies of i-STAT,31,33,35 three of StatSensor28,31,36 and one ABL800 study31 used an eGFR cut-off point of 60 ml/minute/1.73 m² or above (see Background). Several studies included disease-specific populations, including two StatSensor studies28,36 and two i-STAT studies;29,34 therefore, their applicability to a broader population of outpatients referred to CT without a recent eGFR may be limited. One study used a non-standard CKD staging34 and one study30 used a country-specific Japanese equation to calculate eGFR, which limits their applicability to the review question.

Overall, two studies were rated as being at low risk of bias and had low applicability concerns across all domains assessed, including one that evaluated ABL800, i-STAT and StatSensor,37 and one of i-STAT only. 38

Some studies are presented in several lines as they compare multiple devices (e.g. the 2018 publication by Snaith et al. 37).

Studies reporting bias or correlation outcomes

Fifty studies reported bias or correlation outcomes. 14,27,28,30–76 Eighteen studies were available only as conference abstracts (Table 4). Where reported, sample sizes ranged from 10 to 3087 patients. Four studies were set in the UK37,38,43,53 and 11 studies were reported as being conducted in a radiology or CT setting14,27,30,31,38,40,41,46,59,62,74

Studies reporting diagnostic accuracy results based on creatinine thresholds

Seven studies reported diagnostic accuracy data relating to creatinine thresholds (Table 5), with four being reported as published papers39,41,42,44 and three as conference abstracts. 40,43,45 Where reported, sample sizes ranged from 33 to 401 patients. Population details were limited with one study (appearing to be) set in the UK43 and one reported as being of patients due to receive CT scans. 41 All the studies were of StatSensor POC devices. Six studies used a Jaffe method39–41,43–45 for the laboratory reference standard and in one study this was unclear. 42

The creatinine thresholds used in the studies (to calculate sensitivity and specificity) ranged from 1.1 mg/dl to 1.5 mg/dl (i.e. 97 µmol/l to 133 µmol/l). As eGFR (rather than creatinine alone) is used to estimate kidney function in clinical practice, diagnostic accuracy results based on creatinine thresholds are not as clinically relevant or useful than those based on eGFR thresholds. Moreover, all these (creatinine) studies are of the StatSensor POC device, which allows users to implement offset adjustment of biased results. Two of the seven studies explicitly reported results that incorporated an offset adjustment. 39,41 The other five studies did not report using offset adjustment. 40,42–45 Notwithstanding these limitations, most studies reported unadjusted sensitivities that were higher than specificities, indicating that StatSensor tended to overestimate creatinine levels compared with laboratory Jaffe results. The exceptions were the study by Haneder et al. ,41 which reported much lower (unadjusted) sensitivities than specificities in the two devices tested, and the small UK study which reported both a sensitivity and specificity of 100%. 43 Although most studies indicated overestimation of creatinine by StatSensor, the Haneder et al. study41 illustrated that some StatSensor devices may underestimate creatinine. This variation in over- or underestimation was also seen across the studies that reported results for creatinine level bias (see Studies reporting bias or correlation outcomes).

The results of the Griffin et al. 39 and Haneder et al. studies41 indicate that, even after offset adjustment of creatinine results, StatSensor can produce false-negative (FN) and false-positive (FP) results. This has the potential to result in unnecessary prophylactic treatment or scans without contrast (i.e. FP) or to unnecessarily expose high-risk patients to contrast (i.e. FN). The laboratory reference standards used in these studies also limits their value, as the adjustments may themselves be inaccurate, being based on Jaffe methods rather than more accurate enzymatic methods.

Studies reporting diagnostic accuracy results using estimated glomerular filtration rate thresholds

Table 6 summarises the characteristics of the 12 studies that reported diagnostic accuracy data of eGFR measurements with POC creatinine test devices.

All included studies were observational. The sample size ranged from 50 to 2042 participants. Two studies included outpatients referred for a contrast-enhanced CT scan. 30,38 Two studies included patients undergoing a radiological examination, but did not specify what proportion were outpatients. 27,31 Four studies included disease-specific populations, including people with CKD,36 cancer,33 diabetes mellitus34 or infected with human immunodeficiency virus (HIV). 28 One study focused on women referred for contrast-enhanced spectral mammography. 29 Other studies included phlebotomy outpatients37 and mixed-ancestry South African patients. 32

Three studies were conducted in the USA. 27,31,33 Two studies each were conducted in the UK,37,38 Australia35,36 and South Africa. 28,32 A single study was conducted in the following countries: the Netherlands,29 Japan30 and Mexico. 34 Three studies were reported only as a conference abstract. 27,34,35

Seven studies evaluated i-STAT27,31,33–35,37,38 and seven studies evaluated a StatSensor device. 28–32,36,37 Three studies included Radiometer Ltd’s POC device, including ABL80031,37 and ABL827. 27 Two studies evaluated three POC devices (ABL, i-STAT and StatSensor)31,37 and one study evaluated two devices (ABL and i-STAT). 27 There were no studies of other eligible POC tests, such as ABL90 FLEX PLUS, DRI-CHEM NX 500, epoc and Piccolo Xpress.

All sample types used with StatSensor were capillary,28–30,32,36,37 except in one study (which used a venous sample). 31 Conversely, most i-STAT devices used venous samples31,33,35,37,38 except in one study (which used a capillary sample). 34 Another i-STAT study did not specify the sample type used. 27 None of the studies compared the accuracy of a single device using two different sample types.

Three StatSensor30,31,36 and two i-STAT studies34,35 reported using an offset correction to estimate concordance between the POC test and laboratory reference test derived from the study sample. Adjusted and unadjusted results were reported in all three StatSensor studies, but only adjusted results were presented by the two i-STAT studies.

The laboratory reference method was Jaffe in two studies32,33 and not reported in one study. 34 All other studies used an enzymatic method. Equations used to calculate eGFR varied across the studies, and only three studies used CKD-EPI. 34,37,38

Individual study results, including contingency tables, are presented in Table 6. Eight studies reported sufficient data to calculate accuracy at an eGFR threshold of 30 ml/minute/1.73 m². 27,29,30,32,33,36–38 Four studies only reported results using higher eGFR thresholds: two studies used an eGFR cut-off point of 60 ml/minute/1.73 m²;31,35 one study used an eGFR threshold of 90 ml/minute/1.73 m² (although some limited data on an eGFR threshold of 60 ml/minute/1.73 m² were extractable);28 and one study only reported eGFR results according to a non-standard CKD classification (stages 0–4). 34 Two studies were conference abstracts and did not provide sufficient data to be included in the synthesis. 34,35 Both studies evaluated i-STAT and reported accuracy results following an offset correction.

Shephard et al. 35 compared the accuracy of i-STAT against an enzymatic method using 101 venous blood samples. After correction of a mean positive bias of 5.6% and alignment to the IDMS reference method, i-STAT had 96% sensitivity and 96% specificity for an eGFR threshold of 60 ml/minute/1.73 m² compared with the laboratory reference test.

Obrador et al. 34 evaluated the accuracy of i-STAT in 257 diabetic patients. Concordance with the laboratory reference test was evaluated according to a CKD classification ranging from 0 to 4, with 0 indicating no CKD. No further details were provided on the CKD classification; therefore, it is not clear how these results compare to the standard Kidney Disease Improving Global Outcomes (KDIGO) classification, as presented in Table 2. The study used a simple linear regression to estimate a correction factor to align i-STAT SCr to IDMS SCr. After this correction, the Obrador et al. 3 study found that all patients with CKD (stages 1–4) were correctly classified by the POC test (100% sensitivity) and all but one were correctly classified as CKD free (99.4% specificity).

Available data for quantitative synthesis

Results: assessment of diagnostic accuracy

Convergence was achieved for all synthesis models at (or before) 5000 iterations. A further 30,000 iterations on two chains were run; therefore, all results are based on 60,000 post-convergence iterations.

Probability of classification by point-of-care device, given a laboratory-defined category

The pooled probabilities of being classified by a POC device in category k, given the laboratory classification j, p_jk = p[j,k], with j,k = 1,2,3,4, are given in Table 9 and plotted as density strips in Figure 2.

TABLE 9 - Pooled probabilities for the three types of device

Notes

p[i,j] is the probability of being classified in category j by the POC device when the laboratory category is i.

Categories are described in Table 2.

Probability	Device, pooled probabilitity
	StatSensor		i-STAT		ABL
	Median	95% CrI	Median	95% CrI	Median	95% CrI
p[1,1]	0.74	0.61 to 0.85	0.85	0.69 to 0.94	0.87	0.75 to 0.95
p[1,2]	0.18	0.08 to 0.30	0.04	0.00 to 0.18	0.03	0.00 to 0.14
p[1,3]	0.03	0.00 to 0.12	0.04	0.00 to 0.18	0.03	0.00 to 0.14
p[1,4]	0.04	0.01 to 0.11	0.04	0.00 to 0.16	0.04	0.00 to 0.15
p[2,1]	0.09	0.03 to 0.19	0.10	0.04 to 0.21	0.02	0.00 to 0.11
p[2,2]	0.57	0.42 to 0.71	0.77	0.64 to 0.87	0.78	0.61 to 0.90
p[2,3]	0.22	0.12 to 0.36	0.10	0.04 to 0.21	0.15	0.05 to 0.29
p[2,4]	0.10	0.03 to 0.24	0.01	0.00 to 0.06	0.03	0.00 to 0.15
p[3,1]	0.01	0.00 to 0.03	0.01	0.00 to 0.05	0.02	0.00 to 0.08
p[3,2]	0.14	0.09 to 0.20	0.10	0.04 to 0.17	0.06	0.01 to 0.16
p[3,3]	0.25	0.16 to 0.34	0.81	0.72 to 0.88	0.74	0.62 to 0.85
p[3,4]	0.60	0.51 to 0.69	0.08	0.04 to 0.13	0.17	0.09 to 0.26
p[4,1]	0.00	0.00 to 0.01	0.00	0.00 to 0.01	0.00	0.00 to 0.01
p[4,2]	0.00	0.00 to 0.01	0.01	0.00 to 0.02	0.00	0.00 to 0.01
p[4,3]	0.06	0.04 to 0.08	0.08	0.06 to 0.10	0.01	0.00 to 0.01
p[4,4]	0.94	0.91 to 0.95	0.91	0.89 to 0.93	0.99	0.98 to 0.99

FIGURE 2.

Density strips for classification probabilities for each device with vertical lines defining the median and 95% CrI. (a) StatSensor; (b) i-Stat; and (c) ABL. p[j,k], probabilities of being classified in category k, given a laboratory classification j. Categories are described in Table 2.

The i-STAT and ABL devices have higher median probabilities of correct classification in each of the three lowest categories (i.e. p[1,1], p[2,2], p[3,3]) than the StatSensor, with StatSensor appearing particularly poor at correctly classifying individuals in category 3 (i.e. individuals with an eGFR of 45–59 ml/minute/1.73 m²). However, there is considerable uncertainty in these probabilities for all devices.

The median probabilities of being correctly classified as being at risk of PC-AKI (i.e. defined as an eGFR of < 30 ml/minute/1.73 m², sensitivity) using i-STAT or ABL devices are similar (85% and 87%, respectively), whereas for StatSensor devices this median probability is lower (74%). The median probability of being incorrectly classified as being at risk of PC-AKI by the POC device for individuals with an eGFR of 30–44 ml/minute/1.73 m² ranges from 2% for ABL devices to 9–10% for StatSensor and i-STAT devices; however, there is some uncertainty around these values. The probabilities of being incorrectly classified as at risk reduce considerably for individuals with an eGFR ≥ 45 ml/minute/1.73 m².

StatSensor results

Figure 3 presents density strips for the probabilities obtained for StatSensor using only the CKD-EPI data (orange, narrow). These results broadly agree with the adjusted data analysis (light blue, narrow), although uncertainty in the probabilities for an eGFR < 30 ml/minute/1.73 m² is larger in the CKD-EPI analysis as only one study37 is used with only a few individuals in this category.

FIGURE 3.

StatSensor: density strips for classification probabilities for the main analysis (dark blue, wide), the adjusted data analysis (light blue, narrow) and the analysis including only CKD-EPI data (orange, narrow). Vertical lines define the medians and 95% CrIs. p[j,k] is the probability of being classified by the POC device in eGFR category k, given laboratory classification in category j. Categories are described in Table 2.

i-STAT results

Figure 4 presents density strips for the probabilities obtained for i-STAT in the main analysis (dark blue, wide) and the analysis using only the CKD-EPI data37,38 (orange, narrow). There is good overlap of all density strips, with the main analysis producing slightly more precise results.

FIGURE 4.

i-STAT: density strips for classification probabilities for the main analysis (dark blue, wide) and the sensitivity analysis including only CKD-EPI data (orange, narrow). Vertical lines define the medians and 95% CrIs. p[j,k] is the probability of being classified by the POC device in eGFR category k, given laboratory classification in category j. Categories are described in Table 2.

ABL results

Figure 5 presents density strips for the probabilities obtained for ABL devices in the main analysis (dark blue, wide) and the analysis using only the CKD-EPI data37 (orange, narrow). There is good overlap of all density strips, with the main analysis producing slightly more precise results, particularly for the probabilities of being correctly classified as at risk of PC-AKI (eGFR < 30 ml/minute/1.73 m²).

FIGURE 5.

ABL800 FLEX: density strips for classification probabilities for the main analysis (dark blue, wide) and the sensitivity analysis including only data from Snaith et al. 78 (orange, narrow). Vertical lines define the medians and 95% CrIs. p[j,k] is the probability of being classified by the POC device in eGFR category k, given laboratory classification in category j. Categories are described in Table 2.

Studies reporting clinical, workflow or implementation outcomes

Six studies reported clinical, workflow or implementation outcomes relating to POC devices (Table 13). 29,59,62,77–79 One study was available only as a conference abstract. 79 Patient sample sizes ranged from 113 to 3087 and one study was a survey of staff at 68 NHS trust sites. 78 Any POC device was eligible to be included in this section of the review: three studies used StatSensor,29,62,79 one used an i-STAT device59 and one used a Reflotron^® Plus (Roche Holding AG, Basel, Switzerland) POC device (and a screening questionnaire). 77

In Lee-Lewandrowski et al. ’s59 US study, an average of 5.3% of patients presented for a CT or magnetic resonance imaging (MRI) requiring a contrast agent, but without a recent creatinine or eGFR result. A 1-month audit of these patients (n = 384) found that the i-STAT POC device identified 74% of patients as having normal results (defined as an eGFR ≥ 60 ml/minute/1.73 m²), with the CT/MRI study proceeding as planned. Of the patients with an abnormal eGFR (i.e. an eGFR of < 60 ml/minute/1.73 m²), 74% of scans were performed with contrast and 26% without contrast. The authors commented that the decision to use contrast agents in patients with abnormal eGFRs considered the type of study being performed (vascular vs. non-vascular) and an assessment of the overall risk/benefit of administering or not administering a contrast agent. Houben et al. 29 also used an eGFR threshold of < 60 ml/minute/1.73 m² for identifying abnormal results, with StatSensor failing to identify six of the seven patients with abnormal results as measured in the laboratory reference test. This resulted in unwanted contrast agent administration. Two patients subsequently developed PC-AKI after 2–5 days, which was normalised after 30 days.

Ledermann et al. 77 studied 1766 patients referred for contrast-enhanced CT at a private Swiss radiology facility. Only 3.5% of patients had external SCr values on their referral forms (as was requested). A Reflotron POC device was used on patients who had risk factors for PC-AKI (identified using a questionnaire). No fixed eGFR threshold on which to base decisions was adopted; although 116 the 796 patients with a risk factor had a POC-measured eGFR of < 60 ml/minute/1.73 m², the diagnostic procedure was modified in 132 patients. The most frequently adopted changes in the management of these 132 patients was a reduction in contrast agent volume (in 64% of patients) and CT scanning performed without a contrast agent (30%). Morita et al. 62 studied the effect of using a StatSensor device on 113 Japanese patients awaiting CT or MRI examinations who did not have a recent eGFR. 62 Twenty-one patients had an eGFR of < 60 ml/minute/1.73 m². The seven patients who had an eGFR of 30–50 ml/minute/1.73 m² underwent IVH with 500 ml of saline.

Snaith et al. 78 considered implementation issues in a survey that examined adherence of UK hospitals to guidance on the use of gadolinium-based contrast agents in MRI; the risk of nephrogenic systemic fibrosis is elevated in patients with impaired renal function. Six out of 68 sites indicated that POC creatinine testing would be carried out where recent blood test results were unavailable. Twelve sites had rejected using a POC device as an adjunct, mostly for cost reasons.

Stahr et al. ’s79 study reported the proportion of scans involving intravenous contrast agent before and after the introduction of a StatSensor device. However, the results are limited by the study design used, the small sample size and the details reported (it was available only as a conference abstract).

Together, the results of these studies illustrate variation in practice both in terms of the proportions of patients who do not have a recent eGFR result and in the management decisions taken when a POC device indicates an ‘abnormal’ eGFR. However, many of these studies were undertaken several years ago, so the value of their results is somewhat limited because the eGFR thresholds for defining an abnormal result have decreased over time.

Evidence of the risk of acute kidney injury from contrast agents

Patients who need contrast-based imaging sometimes have other risk factors for AKI that make it difficult to ascribe a causative role to contrast agents. Determining the true incidence of CI-AKI from the published literature can be difficult as many studies do not include a control group of patients not receiving contrast agents. Such studies will probably also include kidney injuries unrelated to contrast agents. Another important issue when considering the risk of kidney injury following administration of contrast agents is the outcomes being evaluated. AKI is typically defined as a specific change (relative or absolute) in SCr levels, which makes it a surrogate outcome. The clinical significance of surrogate events can be questionable as they sometimes resolve spontaneously without the patient being aware of their existence. Wherever possible, the identification of the risk of real clinical outcomes – such as mortality or the need for dialysis – is more important and useful to patients, clinicians and researchers alike.

These issues seem particularly important in patients with high SCr levels. In a retrospective study81 of 32,161 patients who had not received iodinated contrast material, researchers analysed SCr levels over 5 consecutive days. The study found that, during the 5-day period, more than two-fifths of patients showed a change in level (up or down) of at least 0.4 mg/dl, with higher initial creatinine values being associated with a higher frequency of a given absolute change. 81 These results are important given that some commonly used definitions of AKI cover absolute increases in SCr of ≥ 0.3 to 0.5 mg/dl. 6 Similarly, a retrospective study in a more relevant population (11,588 patients undergoing CT investigations either with or without contrast agents) found that the incidence of AKI increased with increasing baseline creatinine concentration in both contrast and no-contrast groups, concluding that much of the creatinine elevation was attributable to background fluctuation, underlying disease or treatment. 82 Finally, a prospective study of 716 CT or MRI outpatients found that eGFR values varied independently of whether or not patients received a contrast agent. When comparing pre-imaging values with those 3 days after, 45% of CT patients had a change > ± 10 ml/minute/1.73 m² in the contrast group (n = 237), compared with 59% in the smaller control group (n = 97). 83

It is anticipated that a large number of studies would report on the risk of kidney injury after contrast agent administration; therefore, initially it was sought to identify any recent reviews on the subject. A search of MEDLINE was undertaken for reviews reporting data on the risk of AKI in CT patients. The search was run to identify papers published from 2012 to present; the start year was chosen pragmatically to keep the review manageable and to restrict it to the more up-to-date evidence (literature search strategy details are presented in Appendix 1). From the 291 titles and abstracts retrieved, five potentially relevant reviews were identified. However, the results from three reviews had limited applicability to the outpatient population considered in this assessment, as they were of kidney transplant patients,84 critically ill patients,85 and a mixture of emergency, ICU and inpatients. 86 In the two remaining reviews, the quality of included studies was limited because the studies lacked non-contrast agent control groups. 87,88

Therefore, we focused on the most recent of the five reviews identified (i.e. Aycock et al. 86), which was also the largest study in terms of patient numbers and the broadest in terms of populations. The study reported that, compared with non-contrast CT, intravenous contrast-enhanced CT was not significantly associated with AKI [odds ratio (OR) 0.94, 95% CI 0.83 to 1.07], need for renal replacement therapy (OR 0.83, 95% CI 0.59 to 1.16) or all-cause mortality (OR 1.0, 95% CI 0.73 to 1.36). Although all the studies in the Aycock et al. 86 review had control groups, many studies were small and most did not attempt to match groups on factors associated with outcomes. Therefore, the largest studies were identified with matched control groups included in this review: retrospective studies by McDonald et al. (n = 21,346)89 and Davenport et al. (n = 20,242). 90 The McDonald et al. 89 study looked at AKI, mortality and the need for renal replacement therapy, reporting similar results to the pooled results reported in the Aycock et al. 86 review (described above). The Davenport et al. 90 study reported results by subgroups based on SCr thresholds, concluding that iodinated contrast material is a nephrotoxic risk factor for AKI, but not in patients with a stable SCr levels < 1.5 mg/dl.

In outpatient clinical practice it is eGFR, not creatinine alone, that is used to estimate kidney function (and make decisions on whether or not to use contrast agents), so studies that quantify the risk of AKI in populations subgrouped by baseline eGFR thresholds are more relevant to this assessment. Citation searching using Google Scholar (Google Inc., Mountain View, CA, USA), together with reference lists searches, identified large propensity score-matched studies by the same research groups that reported results risk stratified by eGFR thresholds. 91,92 The characteristics and results of these two studies are presented in Table 14.

Propensity score-matching attempts to account for the selection bias inherent in non-randomised studies by accounting for patient characteristics that are associated with the development of AKI and other clinical outcomes, and which can affect decisions on whether or not to use a contrast agent. Matched propensity score analyses matches patients based on risk factors, which predict both whether or not a contrast-enhanced scan is given and the outcome, by calculating a propensity score that reflects the likelihood that a patient is offered a contrast-enhanced scan, if the risk factors are present. The choice of covariates used to calculate the propensity score is crucial: all covariates believed to be related to both the decision to use contrast agent and the outcome should be measured and included. Propensity score analyses can only adjust for known and measured covariates, as opposed to randomised studies, in which both known and unknown confounders tend to be balanced across groups, thus the possibility of residual confounding cannot be completely ruled out. Inclusion of covariates that are related to contrast assignment but not outcome may reduce efficiency of the method, although this is not a serious limitation in large data sets. 93 In addition, the choice of matching method can affect the amount of residual bias. 94

Although the eGFR thresholds to define subgroups mostly differ, the studies’ results are concordant for the risk of AKI in patients with an eGFR ≥ 45 ml/minute/1.73 m², with contrast agent not being associated with increased risk. The results differ most notably for the eGFR < 30 ml/minute/1.73 m² subgroups, with the McDonald et al. 92 study reporting no increased risk and the Davenport et al. 91 study reporting a statistically significant increase in risk in patients receiving contrast agent (see Table 14). Although the Davenport et al. study91 has the largest overall sample size, it has far fewer patients in the eGFR < 30 ml/minute/1.73 m² subgroup (i.e. 116 vs. 1486), which is reflected in its very wide CIs for the estimated ORs.

Another factor that may have contributed to the eGFR < 30 ml/minute/1.73 m² subgroup results being different is the difference in AKI definitions. Davenport et al. 91 used a lower absolute SCr increase of 0.3 mg/dl, compared with the 0.5 mg/dl increase used by McDonald et al. 92 Given the (previously discussed) natural fluctuation in SCr levels, the use of a lower threshold is likely to detect more AKI events in patients with higher baseline SCrs. These events may be less likely to be clinically significant (in terms of their impact on real clinical outcomes) than AKIs defined using larger increases in SCr. This ‘noise’ of excess events may hamper interpretation of the Davenport et al. 91 study results, given the very small denominators in the eGFR < 30 ml/minute/1.73 m² subgroups. The difference in propensity score adjustment methods and matching may also contribute to the differences in results. McDonald et al. 92 derived the propensity scores separately for each eGFR subgroup, which will better account for the different clinical characteristics expected in patients with a lower eGFR score and lead to better matching. In contrast, Davenport et al. 91 derived propensity scores for the whole cohort, with mixed eGFR scores, which may explain why differences between two covariates (whether or not CT was performed in the intensive care unit and whether or not the patient had type 1 diabetes mellitus) remained statistically significant after matching.

The Davenport et al. 91 study required that patients have both a baseline and an index pre-CT creatinine measurement – around 16,000 patients were excluded as a result of unstable kidney function. McDonald et al. 92 did not require a baseline creatinine measurement, although the study excluded patients with pre-existing dialysis requirements and patients with acute renal failure in the preceding 14 days. These different criteria may explain the differing numbers across the two studies of patients with an eGFR < 30 ml/minute/1.73 m². The small numbers mean that the Davenport et al. 91 study eGFR < 30 ml/minute/1.73 m² results may be prone to chance effects. This can be investigated by calculating the fragility index of the eGFR < 30 ml/minute/1.73 m² subgroup result. The fragility index is the minimum number of patients whose status would have to change from a non-event to an event in order to turn a statistically significant result to a non-significant result: the smaller the fragility index, the more ‘fragile’ the result. 95 The fragility index is calculated using a Fisher’s exact test, although other methods, such as a chi-squared test, are often used in studies. The p-value from a Fisher’s exact test can be discrepant from a chi-squared test, especially for small studies. In cases in which a Fisher’s exact test produces a non-significant p-value (i.e. without converting a patient from a non-event to an event), the fragility index is reported as zero, indicating a lack of robustness of the result. For the Davenport et al. 91 study, for an eGFR < 30 ml/minute/1.73 m² result based on the published summary patient data, the fragility index is zero (i.e. the result is not statistically significant using Fisher’s exact test). However, as mentioned previously, following propensity matching the Davenport et al. 91 study OR was adjusted, and the fragility index for the statistically significant OR of 2.96 cannot be calculated from the data available.

If it was assumed that the Davenport et al. 91 study eGFR < 30 ml/minute/1.73 m² subgroup result was robust, then the ‘number need to harm’ is six, that is, for every six inpatients with an eGFR < 30 ml/minute/1.73 m² who receive contrast, one inpatient will have an AKI caused by the contrast agent. However, it should be remembered that this result is for a surrogate outcome – it is unclear to what extent increases of 0.3 mg/dl in the SCr level of patients with a baseline eGFR < 30 ml/minute/1.73 m² translate into real clinical outcomes, such as mortality or the need for dialysis. The McDonald et al. 92 study identified in the in the Aycock et al. 86 systematic review reported data on real clinical outcomes – the results suggested that there was no association between the use of contrast agents and need for dialysis, or death, for all eGFR subgroup analyses (eGFR subgroups were based on stages of chronic renal failure). 89 Although the number of clinical events in this study were quite small, particularly for the dialysis outcome. Moreover, if there is a risk of CI-AKI associated with an eGFR < 30 ml/minute/1.73 m², it is likely to be lower in the outpatient population of interest in this assessment, given that inpatients are more likely to have other AKI risk factors (including acute illness and exposure to nephrotoxic treatments). Nevertheless, uncertainty about the level of risk remains, primarily because of the unmeasured clinical characteristics, which could not contribute to the propensity scores – most notably the level of prophylactic measures (e.g. IVH) used in the contrast groups and the prevalence of potentially nephrotoxic medication use at time of scanning.

The citation and reference searching identified three further publications of interest on the risk of AKI from contrast agents. 96–98 The first was a review of propensity score-matching studies on AKI after contrast,96 which lists several studies by McDonald et al. 92 and Davenport et al. 91 research groups. This review96 also cited a large study (n = 17,934) by a different research group that reported results by baseline eGFR subgroups. 97 The study setting was an emergency department – different from the inpatients studied by McDonald et al. 92 and Davenport et al. 91 – with comparisons made between contrast-enhanced CT, unenhanced CT and no-CT groups. The results were similar to those reported by McDonald et al. ,92 with rates of AKI being similar among all groups, including the eGFR 15–30 ml/minute/1.73 m² subgroups.

The review of propensity score-matching studies96 also cited a further study by McDonald et al. 98 that reported the effect of contrast agents on dialysis and mortality, reported by baseline eGFR subgroups. The study was of 5758 inpatients, emergency patients and outpatients who had a CT scan either with or without contrast agents. Contrast agents were not associated with higher rates of dialysis or mortality for any subgroup comparisons, including the CKD stages 4–5 subgroup (i.e. patients with an eGFR < 30 ml/minute/1.73 m²), although the last results are limited by the small number of patients in the contrast group (90, falling to 76 after propensity score matching).

Evidence on prophylactic interventions for post-contrast acute kidney injury

Pragmatic searches of MEDLINE and recent guidelines were conducted to identify evidence on the clinical effectiveness and safety of standard prophylaxis intravenous saline hydration for preventing PC-AKI in high-risk patients. Recent systematic reviews (from 2012 onwards) of RCTs comparing IVH with oral hydration, placebo or no treatment for preventing PC-AKI in patients with chronic renal failure (defined as an eGFR < 60 ml per min/1.73 m²) undergoing radiological procedures requiring low-osmolality contrast media were included. Risk of bias was assessed using the Cochrane risk-of-bias tool. 99

Evidence of practice variation in renal function assessment

Two quite recent studies that have evaluated how renal function assessment practice varies in the UK were identified by reference list searching and citation searching. A survey undertaken in 2015 by Cope et al. 13 assessed compliance with UK 2013 guidelines107,108 for the prevention, recognition and management of CI-AKI. All UK acute NHS providers with a clinical radiology audit lead registered with the Royal College of Radiologists (RCR) were invited to complete a questionnaire. In order to demonstrate guidance compliance in daily practice, audit data on 40 consecutive stable outpatients who had undergone CT with intravenous iodine-based contrast agents were also requested from each NHS provider.

Eighty-nine of the 172 (52%) health service providers responded to the questionnaire and 91 out of 212 (43%) hospitals provided audit data. In general, the paper by Cope et al. 13 noted wide variation in clinical practice and poor compliance with guidelines. Although kidney function test results within 3 months of the scan were available for 86% of outpatients, eGFR results (as recommended in the guidelines) were available for only 66%. Responsibility for checking baseline kidney function was taken by the radiology department in 49% of departments; in 51%, the responsibility was either devolved to the referring clinician or was not clearly defined. Only 30% of radiology departments had a policy for management of patients who developed PC-AKI or had locally agreed arrangements in place for the care of patients when repeat blood tests demonstrate PC-AKI. The requirement for intravenous volume expansion for high-risk patients prior to the scan was met by 64% of departments.

Audit data were available for 3590 fit outpatients. Analyses were reported for a subgroup of 513 patients with a baseline eGFR < 60 ml/minute/1.73 m²; 288 (56%) had pre- and post-contrast kidney function tests – no change was seen in the median SCr level 2 days post contrast. The incidence of clinically significant (requiring treatment or resulting in death) PC-AKI was zero in 3590 outpatients.

Harris et al. 12 also undertook a UK survey in 2015, requesting data from CT managers in 174 NHS trusts to identify screening practices prior to outpatient contrast-enhanced CT. The response rate (47%) was similar to that reported in Cope et al. ’s13 survey. The RCR guideline109 was most frequently used, although 20% of responders did not cite the use of a specific guideline. Most responding sites (75/82, 92%) required renal function to be assessed via a blood test; most sites did this for all patients, although 20% of sites assessed only ‘high-risk’ patients. Variation in how blood tests were organised was found, with most radiology departments sharing the responsibility with the referring clinician. Most radiology departments removed or minimised the risk of patients attending radiology without a recent kidney function result by checking blood results either before booking appointments (56%) or when appointments were made (16%), with blood tests booked if needed. Just over one-quarter of radiology departments (28%) indicated that results are reviewed on the day of the scan (or the night before).

Variation was also evident in the eGFR or SCr thresholds at which contrast was deemed to be contraindicated; 19 different threshold levels were identified, each leading to different prophylactic strategies. The most frequently used threshold was an eGFR of < 30 ml/minute/1.73 m², which was used in 35 of the 77 (45%) NHS trusts. Blood test results were not checked by 7 out of 82 (9%) sites – sites indicated that it was the referrer’s responsibility. For patients attending without a recent blood result, 45% send the patient away to have a blood test and scan either on the same day (if possible) or on a different day, and 11% of sites use POC devices to get a quick blood test result. Most of the remaining sites said that they would seek advice from a consultant radiologist. Data on practice variation in obtaining follow-up (post-contrast) blood tests were also reported. The authors concluded that the wide variation in practice is a reflection of inconsistencies in published guidance and that an evidence-based consensus on risk thresholds was needed.

Searches

The literature search reported in Chapter 3, Assessment of clinical effectiveness, Searches, was also used to identify studies reporting on the cost-effectiveness of POC creatinine testing in an outpatient non-emergency setting before contrast-enhanced CT imaging.

Selection process

A broad range of studies were considered in the review, including economic evaluations conducted alongside trials, modelling studies and analyses of administrative databases. Only full economic evaluations that compared two or more options and considered both costs and consequences (i.e. cost-minimisation, cost-effectiveness, cost–utility and cost–benefit analyses) were included in the review. The inclusion criteria also defined the relevant population as non-emergency outpatients scheduled to receive intravenous contrast-enhanced CT imaging.

The selection of relevant studies was performed in two stages:

1. Titles and abstracts identified by the search strategy were examined and screened for possible inclusion.

2. Full texts of the potentially relevant studies were obtained and screened for inclusion.

Two researchers (AD and JA) independently screened the titles and abstracts of all reports identified by the bibliographic searches and full-text papers were subsequently obtained for assessment and screened by at least two researchers. Any disagreement was resolved by consensus.

Confidential information

This report contains reference to confidential information provided as part of the NICE appraisal process. This information has been removed from the report and the results, discussions and conclusions of the report do not include the confidential information. These sections are clearly marked in the report.

Results

A total of 3628 records were identified by the initial search of economic databases. Three studies were identified as potentially relevant from their titles and/or abstracts. The full-text articles of these records were assessed for eligibility; however, none was found to meet the inclusion criteria. Figure 6 presents a flow diagram of the selection process. Table 49 in Appendix 6 lists excluded studies alongside reasons for exclusion.

FIGURE 6.

Assessment of cost-effectiveness: summary of study selection and exclusion.

Although no published studies were identified from the systematic review, one unpublished economic study was identified, which was considered potentially relevant (Professor Beverley Snaith, Mid Yorkshire Hospitals NHS Trust, 2019, personal communication). Following discussion with the lead author, a draft version of the manuscript was provided. This draft was provided by the lead author in academic in confidence.

Overview

(Confidential information has been removed.)

FIGURE 7.

Confidential information has been removed.

Relevance of findings

(Confidential information has been removed.)

Resourcing implications identified in the Medtech innovation briefing

The MIB (specifically MIB13615) identifies POC testing technologies as an alternative to laboratory-based testing in those patients who present for contrast CT scanning without a recent eGFR measurement. In the absence of a recent creatinine measurement, these patients may otherwise have their imaging cancelled or rescheduled – given national guidelines for a recent eGFR to be available before imaging. 108,110 If the scan is not cancelled, the authors of MIB suggest that the patient would either undergo non-contrast-enhanced scanning or continue with contrast scanning as planned, thus putting the patient at risk of kidney injury.

The authors therefore identify a key benefit of POC devices to be reducing the incidence of cancelled CT scans as a result of the expectation of a reduced patient waiting time for eGFR measurements for those patients who present without a recent eGFR measurement. MIB specialist commentators note the administrative cost of cancelling or rescheduling scans, and the impact of cancellations on overall scanning capacity. The other benefit is more accurately identifying the subset of patients without a recent eGFR measurement who should not proceed with a contrast CT scan because of their elevated risk of kidney disease (i.e. those patients with an eGFR < 30 ml/minute/1.73 m²). These patients are most likely to suffer adverse effects of contrast-induced kidney injury and, thus, should not generally proceed to contrast CT scanning unless appropriate prophylaxis is provided, their contrast dose is reviewed or they are in urgent need of diagnostic information provided only by contrast-enhanced imaging.

The MIB authors note that POC devices are expected to deliver eGFR results from a whole-blood sample in ≤ 9 minutes, compared with laboratory testing, which can take between 60 minutes and 24 hours. The specialist clinical group that was consulted note that this reduction in waiting time would reduce the need for additional appointments, delayed appointments and increase patient throughput. The MIB authors note that POC devices would be most useful in assessing kidney function in the subgroup of the overall patient population at highest risk of kidney disease, including those patients with diabetes mellitus, people taking metformin and older people.

The MIB authors note that POC testing would increase upfront costs compared with standard laboratory-based testing. The unit cost of a laboratory test for blood/serum/plasma creatinine was £1.29 at 2015/16 prices (i.e. reference cost DAPS04111). The MIB authors note that the unit cost per POC test for the devices that they consider vary between £0.17 and £4.75. The authors also note the significant upfront capital costs of POC devices. On a practical front, the authors note the potential requirement for staff training and compliance and quality assurance policies, as well as an increase in storage space for POC consumables; however, the MIB authors also note that all of these requirements would be unlikely to be a significant change. The MIB authors also note that additional resources may be required for participation in external quality assurance schemes, with specialist commentators also suggesting potential costs for the integration of recording POC results with the existing hospital reporting system. The specialist group of clinical advisors held divergent opinions on whether or not POC testing would replace central laboratory testing or supplement it.

The MIB authors note some economic benefits of early diagnosis of CKD through the use of POC testing as opposed to waiting for GP testing; however, the authors note that these savings would be minimal. The authors also cite a US-based study112 that showed a reduction in waiting times for eGFR results from an average of 1 hour 54 minutes to 5 minutes following the introduction of radiology POC testing. This US study also suggested that the volume of contrast material used was also reduced for 26.4% of patients (33/125 patients). Although not directly reported in the MIB, this study suggests that rapid testing will enable radiology departments to reduce costs by reducing the number of full-time-equivalent administrative positions needed for checking laboratory results prior to testing. In addition, the rapid testing will also reduce technician overtime as a result of the reduced need to accommodate delayed examination times due to waiting for laboratory results.

Implications for the care pathway identified in the King’s Technology Evaluation Centre’s report

As part of the report produced by KiTEC to support the EAG’s report,113 clinical experts were also interviewed regarding their views on the implications of introducing POC creatinine testing within the current CT imaging pathway. The KiTEC report noted that all the clinical experts that were interviewed expressed concerns regarding the use of these devices in their departments. The report highlighted two main reasons for these concerns. First, the clinical experts highlighted that referring clinicians would rely even more on the radiology department to check patients’ eGFR. As a result of this behavioural change, the clinical experts thought that this would result in an increase in the number of patients referred for a CT appointment without a recent eGFR measurement. Second, the clinicians noted that this would increase the responsibility and resourcing required by the radiology department not only to action upon a low eGFR but also to explain to the attending patient that their result was abnormal and may require further investigations and changes in management.

Overview

The model evaluates the cost and health outcomes of a cohort of outpatients presenting for a non-emergency contrast-enhanced CT scan without a recent eGFR measurement. The model is populated using the results from the quantitative synthesis of the diagnostic accuracy of POC testing as described in Chapter 3, Results: assessment of diagnostic accuracy. Other relevant parameters were informed by a series of additional reviews described throughout this section. These parameters are used to provide a link between the diagnostic accuracy of a given testing strategy, the impact on subsequent treatment decisions and the ultimate effect on health outcomes and costs.

Costs are presented from the perspective of the NHS and Personal Social Services (PSS) and are reported in Great British pounds at a 2018 price base. Outcomes are expressed in terms of QALYs. Outcomes beyond the first year are discounted at a rate of 3.5% per annum.

The model uses a decision tree cohort approach to estimate, based on best available data, the costs and health outcomes of the relevant testing and treatment strategies. The model structure captures:

1. individuals’ true eGFR status (with the cohort dichotomised based on a cut-off point of 30 ml/minute/1.73 m²)

2. how these individuals are subsequently classified by different testing strategies (with classification dichotomised on the same eGFR cut-off point of 30 ml/minute/1.73 m² and probabilities conditional on true eGFR status)

3. any actions taken to mediate PC-AKI risk in patients identified (correctly or incorrectly) as below the eGFR cut-off point

4. the subsequent risk of PC-AKI (conditional on eGFR status and any actions taken to mediate PC-AKI risk)

5. the risk of renal replacement therapy (conditional on whether or not a patient experienced PC-AKI).

Costs and QALYs are linked to the use of screening tests, mediating actions taken and the use of RRT.

A simplified model schematic is shown in Figure 8. Patients are defined as true positives (TPs), FPs, true negatives (TNs) and FNs according to their overall classification across each testing strategy and not in relation to individual tests in the sequence. Testing approaches may combine up to three testing elements to identify patients. The elements of testing considered were:

1. screening on the basis of a risk factor questionnaire

2. testing with a POC device

3. testing with a laboratory test (urgent or non-urgent).

Patients identified as negative by the testing approach will receive no alternative management and undergo contrast-enhanced CT. Patients identified as positive will receive mediating action, which in the base case is assumed to be the use of IVH prior to undergoing contrast-enhanced CT. Following their scan, patients may experience a PC-AKI and may subsequently undergo RRT.

FIGURE 8.

Decision tree general schematics.

A key assumption in the base-case analysis is that all individuals will eventually proceed to contrast-enhanced CT. Hence, the only difference between the alternative testing strategies that were evaluated concerns the costs and potential health impact of delayed or rescheduled CT, whether or not any mediating action is taken to reduce the risk of PC-AKI (i.e. use of IVH) and the consequences of PC-AKI. The base-case analysis does not attempt to include other clinical outcomes that could be affected by changes to the imaging decision itself. These outcomes could include anxiety associated with delayed or cancelled scanning, and morbidity and mortality implications of performing unenhanced scanning or using an alternative imaging modality. This simplification was considered necessary given the limited data available and the challenges of characterising the heterogeneity in the overall population and the underlying reason for imaging and linking this to individualised clinical decision-making and associated outcomes.

The challenges of linking different decisions regarding the use of contrast media in imaging to patient outcomes were also highlighted in KiTEC’s report. 113 Clinical experts interviewed in the KiTEC report stated that it is difficult to quantify the impact of decisions regarding the use of contrast media on patient outcomes as the benefits of using intravenous contrast vary depending on the underlying population and scanning indication. The use of intravenous contrast was considered by the clinical experts to be well-established practice, but none was aware of any landmark study that could be used to quantify the benefits compared with alternative imaging decisions.

Although the base-case analysis imposes boundaries around the specific clinical outcomes assessed because of practical considerations and data gaps, a series of additional scenario analyses were undertaken to explore the robustness of the base-case analysis to alternative assumptions concerning the potential impact of alternative imaging decisions on costs and outcomes. These scenarios considered the potential costs as well as any anxiety effects associated with delayed CT scan or a use of an alternative imaging modality. The full set of scenarios are discussed in more detail in later sections.

The model evaluates the cost-effectiveness of 14 alternative testing strategies to identify and manage patients with an eGFR < 30 ml/minute/1.73 m². The likelihood of an individual being classified as positive (i.e. with an eGFR < 30 ml/minute/1.73 m²) or negative (i.e. with an eGFR ≥ 30 ml/minute/1.73 m²) is estimated for each strategy based on an individual’s true eGFR status and the diagnostic accuracy (sensitivities and specificities) of the different elements of testing that constitute the overall testing strategy. Where a strategy involves multiple tests, an individual will progress from one test to the next if the first test classifies them as positive, which in the case of risk factor screening will involve them being classified as at risk or, in the case of a POC device, of having an eGFR < 30 ml/minute/1.73 m². An individual will be identified as positive (either TP or FP) if the final test in the strategy classifies them as having an eGFR < 30 ml/minute/1.73 m².

The risk of PC-AKI is conditioned on an individual’s true eGFR value, with higher risk assumed in patients with an eGFR < 30 ml/minute/1.73 m². This risk is assumed to be modifiable by providing either prophylactic measures prior to the provision of contrast agent or changing the imaging modality. Individuals who test negative are managed with their planned contrast-enhanced CT scan, whereas those individuals who test positive are managed to reduce their risk of PC-AKI. The model assumes that the risk of PC-AKI is modifiable only for patients who have a true eGFR measure < 30 ml/minute/1.73 m². Therefore, individuals who are misclassified as positive (i.e. FP) will incur the costs of the actions taken to reduce their perceived PC-AKI risk, but do not derive any health benefit in terms of a reduction in PC-AKI risk and subsequent clinical events. Individuals who are misclassified as negative (i.e. FN) will not incur the cost of these mediating actions, but will fail to realise the health benefits of receiving an action that would reduce their risk of PC-AKI. In the base case, the mediating action is assumed to be IVH prior to a full-contrast CT scan. In a scenario analysis, individuals were considered to receive a range of possible mediating actions, with a proportion of patients receiving IVH prior to a full-contrast CT scan, a proportion receiving an unenhanced CT scan and a proportion receiving a MRI scan.

Based on evidence from a series of reviews, all individuals are assumed to be at risk of requiring temporary RRT within 6 months of imaging and this risk is assumed to be conditional solely on experiencing a PC-AKI. Based on this evidence, it is also assumed that PC-AKI has no impact on mortality, and that there are no differences between strategies in terms of patients’ costs and HRQoL after 6 months post imaging.

The model considers the costs of testing patients according to the combination of testing components in each strategy. In the base case, undertaking a laboratory test was assumed to always cause a delay and cancellation of the initial CT scan, with consequent loss of the imaging time slot and associated costs. Scenario analyses explored the robustness of the results to alternative assumptions, including that a proportion of the laboratory tests would be urgent and would not result in a delay unless a positive test result was obtained requiring mediating action. Risk factor screening and POC testing would cause the delay and cancellation of the initial CT scan only if they are the final testing component in that strategy and the final result was positive resulting in mediating action being taken. For individuals who undergo mediating actions (i.e. IVH in the base case), the cost of the action taken and any associated adverse events were captured. PC-AKI events are assumed to impose no costs, although PC-AKI events do alter the risk of a patient requiring RRT, which was costed.

Outcomes of patients are captured in QALYs over their remaining lifetime. All patients in the model are assumed to have the same life expectancy and HRQoL as the age- and sex-adjusted general population, with HRQoL decrements applied to those patients who require RRT for a duration of 3 months. No further HRQoL impacts are assumed in the base-case analysis. A scenario analysis considered a HRQoL decrement as a result of anxiety caused by any delay of the CT scan or use of an alternative imaging modality.

Further details of the main structural and input assumptions and the sources of evidence considered for each are discussed in detail in later sections of the report.

Strategies

The strategies included in the model represent the potential pathways that are either part of current clinical practice or represent ways in which POC testing could be integrated into clinical practice. These can be grouped into six types of strategy, according to the testing approach followed:

1. laboratory testing only

2. risk factor screening combined with POC testing

3. risk factor screening combined with laboratory testing

4. risk factor screening combined with POC testing and laboratory testing

5. POC testing only

6. POC testing combined with laboratory testing.

A strategy of ‘no testing and manage all with contrast-enhanced CT’ was not included in the base-case analysis. Although this represents a potentially feasible strategy, it was not deemed to be clinically appropriate given the consistent recommendations reported across clinical guidelines recommending the use of some form of screening or testing to identify individuals at risk of PC-AKI. However, for completeness and to aid the overall interpretation of the results, this strategy was included in a separate scenario. Similarly, a strategy of risk factor screening alone was initially considered but then excluded, as this strategy was not deemed to be clinically feasible as a result of the high rate of FPs that would require IVH and the limited capacity to provide this.

Laboratory testing consists of performing a blood test on all individuals presenting without a recent eGFR measurement prior to imaging. Although the NICE scope distinguished between urgent and non-urgent laboratory tests, no evidence was subsequently identified concerning differences in test performance or unit costs. However, access to urgent laboratory testing has important implications for the timing of clinical decisions and the impact on scanning decisions (i.e. whether or not the scan can be rescheduled within the same day or requires the scan to be rebooked for a separate day). Inevitably, there exists significant heterogeneity across NHS sites in terms of provision and access to urgent laboratory testing. In the base-case analysis, it was assumed that laboratory testing would require the CT scan to be rescheduled on a separate day (i.e. only non-urgent testing). A series of scenarios were also undertaken that assumed that a proportion of patients (i.e. 25%, 50%, 75% and 100%) would receive urgent laboratory testing, allowing their CT scan to be rescheduled for the same day and hence avoiding the full opportunity cost of a lost CT scan appointment.

Individuals who test negative with laboratory testing are assumed to be managed with the planned contrast-enhanced CT scan. Those individuals who test positive receive mediating action to reduce their PC-AKI risk, with management consisting of IVH followed by contrast-enhanced CT in the base-case analysis.

Figure 9 provides a schematic of the model structure for the laboratory testing strategy.

FIGURE 9.

Model structure: laboratory testing. p1, probability of an eGFR < 30 ml/minute/1.73 m²; p2, probability of AKI conditional on an eGFR ≥ 30 ml/minute/1.73 m² and contrast-enhanced CT scan; and p3, probability of AKI conditional on an eGFR < 30 ml/minute/1.73 m² and contrast-enhanced CT scan with prophylactic IVH.

Risk factor screening combined with POC testing consists of screening individuals with a risk factor questionnaire followed by a POC test for individuals identified with at least one risk factor (risk factor positive). Individuals who screen risk factor negative or test negative with the POC test are assumed to proceed with the planned contrast-enhanced CT scan. Individuals who screen positive and have an eGFR measurement of < 30 ml/minute/1.73 m² with the POC device receive IVH to reduce their PC-AKI risk.

Figure 10 provides a schematic of the model structure for the risk factor screening combined with POC testing strategy.

FIGURE 10.

Model structure: risk factor screening combined with POC testing. p1, probability of an eGFR < 30 ml/minute/1.73 m²; p2, probability of AKI conditional on an eGFR ≥ 30 ml/minute/1.73 m² and contrast-enhanced CT scan; p3, probability of AKI conditional on an eGFR ≥ 30 ml/minute/1.73 m² and contrast-enhanced CT scan with prophylactic IVH; p4, probability of AKI conditional on an eGFR < 30 ml/minute/1.73 m² and contrast-enhanced CT scan; and p5, probability of AKI conditional on an eGFR < 30 ml/minute/1.73 m² and contrast-enhanced CT scan with prophylactic IVH. RF, risk factor.

Risk factor screening combined with laboratory testing consists of screening individuals with a risk factor questionnaire followed by a laboratory test for those individuals who screen positive for at least one risk factor. Individuals who have no risk factors, and those who test negative on the laboratory test, receive contrast-enhanced CT scanning. Individuals who screen and test positive receive additional management to reduce their risk of PC-AKI.

Figure 11 provides a schematic of the model structure for the risk factor screening combined with laboratory testing strategy.

FIGURE 11.

Model structure: risk factor screening combined with laboratory testing. p1, probability of an eGFR < 30 ml/minute/1.73 m²; p2, probability of AKI conditional on an eGFR ≥ 30 ml/minute/1.73 m² and contrast-enhanced CT scan; p3, probability of AKI conditional on an eGFR < 30 ml/minute/1.73 m² and contrast-enhanced CT scan; and p4, probability of AKI conditional on an eGFR < 30 ml/minute/1.73 m² and contrast-enhanced CT scan with prophylactic IVH. RF, risk factor.

Risk factor screening combined with POC and laboratory testing comprises a three-step testing sequence that involves screening all individuals for risk factors, testing with POC devices those with at least one risk factor, and providing individuals who screen and test positive (with POC devices) with a confirmatory laboratory test. All individuals that have a negative result at any point in the testing sequence are managed with a contrast-enhanced CT scan. Individuals who test positive at all three steps of the testing sequence receive management to reduce their risk of PC-AKI.

Figure 12 provides a schematic of the model structure for the risk factor screening combined with POC and laboratory testing strategy.

FIGURE 12.

Model structure: risk factor screening combined with POC and laboratory testing. p1, probability of an eGFR < 30 ml/minute/1.73 m²; p2, probability of AKI conditional on an eGFR ≥ 30 ml/minute/1.73 m² and contrast-enhanced CT scan; p3, probability of AKI conditional on an eGFR < 30 ml/minute/1.73 m² and contrast-enhanced CT scan; and p4, probability of AKI conditional on an eGFR < 30 ml/minute/1.73 m² and contrast-enhanced CT scan with prophylactic IVH. RF, risk factor.

Point-of-care testing consists of testing all individuals with a POC device, with those individuals testing negative managed with a contrast-enhanced CT scan and those individuals testing positive receiving mediating action to reduce their risk of PC-AKI.

Figure 13 provides a schematic of the model structure for the POC testing strategy.

FIGURE 13.

Model structure: POC testing. p1, probability of an eGFR < 30 ml/minute/1.73 m²; p2, probability of AKI conditional on an eGFR ≥ 30 ml/minute/1.73 m² and contrast-enhanced CT scan; p3, probability of AKI conditional on an eGFR ≥ 30 ml/minute/1.73 m² and contrast-enhanced CT scan with prophylactic IVH; p4, probability of AKI conditional on an eGFR < 30 ml/minute/1.73 m² and contrast-enhanced CT scan; and p5, probability of AKI conditional on an eGFR < 30 ml/minute/1.73 m² and contrast-enhanced CT scan with prophylactic IVH.

The last strategy type combines POC testing with laboratory testing. Individuals who test positive on the POC test receive a confirmatory laboratory test. Those testing negative on either test receive a contrast-enhanced CT scan, and those testing positive on both sequences receive mediating action to reduce their risk of PC-AKI.

Figure 14 provides a schematic of the model structure for the POC testing combined with laboratory testing strategy.

FIGURE 14.

Model structure: POC testing combined with laboratory testing. p1, probability of an eGFR < 30 ml/minute/1.73 m²; p2, probability of AKI conditional on an eGFR ≥ 30 ml/minute/1.73 m² and contrast-enhanced CT scan; p3, probability of AKI conditional on an eGFR < 30 ml/minute/1.73 m² and contrast-enhanced CT scan; and p4, probability of AKI conditional on an eGFR < 30 ml/minute/1.73 m² and contrast-enhanced CT scan with prophylactic IVH.

For each type of strategy that includes POC testing, the model considers separate strategies for each of the POC devices. The POC devices considered in the cost-effectiveness analysis are restricted to those that reported diagnostic accuracy data using eGFR thresholds reported in the quantitative synthesis (see Chapter 3, Results: assessment of diagnostic accuracy). The three devices considered in the model are i-STAT Alinity, ABL800 FLEX and StatSensor. In line with the clinical effectiveness review, the different models of i-STAT, ABL 800 series and StatSensor are assumed to be equivalent in terms of diagnostic accuracy data within brand, whereas the costs are derived for the models that are commercially available in the UK, according to the manufacturer.

Although different types of laboratory-based SCr tests are used in clinical practice to derive eGFR values, it is assumed that these values are all equivalent in terms of diagnostic accuracy and costs. The laboratory test is assumed to have perfect diagnostic accuracy (i.e. 100% sensitivity and specificity) and, therefore, laboratory-measured eGFR is assumed equivalent to a ‘true’ eGFR value.

Clinical guidelines recommend that only individuals considered at high risk of PC-AKI have their eGFR measured prior to undergoing contrast-enhanced CT. 6,10–12 However, these guidelines do not recommend the use of any particular screening tool, and there is a lack of consistency across this literature regarding the specific criteria that would allow the identification of high-risk individuals. Therefore, screening in the model was assumed to be conducted with a generic risk factor questionnaire.

Laboratory testing requires time for the test to be processed, which means that some individuals may not be able to undergo CT on the same day. In the base case it was assumed that all individuals undergoing a laboratory test would have their CT scan cancelled. However, a scenario analysis allowed for a proportion of patients to receive a rapid laboratory test, and those patients who test negative are assumed not to have their CT scan cancelled.

Risk factor screening and POC testing are assumed to be conducted within the original CT scan time slot and, therefore, do not introduce any further delays (and associated costs). However, if individuals are identified as requiring alternative management to mitigate the PC-AKI risk, it may also be unfeasible to conduct this within the same day for which their original CT scan was planned. The base case assumes that all patients who require a laboratory test or test positive at the last step of the testing sequence will incur the costs of delay. The proportions requiring delay are varied in scenario analyses.

The model considers three alternative management options for patients who are identified as having an eGFR < 30 ml/minute/1.73 m² by any of the testing approaches described above. These management approaches are:

1. IVH followed by contrast-enhanced CT scan

2. unenhanced CT scan

3. unenhanced MRI scan.

It is assumed that all approaches are equivalent in terms of diagnostic accuracy of the imaging modality, but differ in terms of cost and effect on the risk of PC-AKI. As previously stated, all patients in the base-case analysis identified as being at high risk of PC-AKI are assumed to be managed with prophylactic IVH and proceed with a full-contrast dose CT scan. It is assumed that adverse events from IVH are associated only with costs and not with any HRQoL loss. Separate scenarios are presented assuming alternative management approaches.

Table 17 summarises the 14 strategies evaluated in the base-case cost-effectiveness analysis.

Population characteristics

The cost-effectiveness of the alternative strategies will be dependent on the characteristics of the patient population being considered, including the distribution of eGFR and the number of patients who are likely to present without a recent eGFR measurement. The population considered here is non-emergency adult outpatients presenting for intravenous contrast-enhanced CT scanning without an available eGFR measurement at the radiology department.

Diagnostic accuracy

Diagnostic accuracy of risk screening questionnaires

Clinical guidelines recommend risk factor screening for patients without prior eGFR measurements presenting for contrast-enhanced CT scans to avoid unnecessary blood testing. ^6,10,11,62 However, these guidelines do not recommend the use of any particular screening tool, and there is a lack of consistency across this literature regarding the specific criteria to identify high-risk patients. 12 Furthermore, survey data of UK radiology departments suggest that different guidelines are followed in clinical practice, resulting in heterogeneity of clinical practice behaviour to prevent PC-AKI.

Another issue relates to the evidence context in which the guidelines and risk factor questionnaires were developed. The eGFR cut-off point at which PC-AKI risk is considered to increase to clinically relevant values has altered over time, and patients are now considered to be at high risk of PC-AKI only at eGFR values < 30 ml/minute/1.73 m². Therefore, it is unclear if existing screening tools would accurately identify patients at risk of PC-AKI under the currently used diagnostic criterion, especially in patient populations in which the average eGFR is expected to be high, as is the case for non-emergency CT scan outpatients.

Studies identified through reference list searching and citation searching, conducted as part of the pragmatic reviews described in Pragmatic reviews of further evidence to inform the economic model, were examined to identify diagnostic accuracy evidence for risk factor questionnaires. Four studies14,75,114,116 that examined the diagnostic accuracy of risk factor screening questionnaires in an outpatient setting were identified as potentially relevant. In addition, unpublished risk factor screening diagnostic accuracy data were obtained from the 2019 Snaith et al. 38 study (Professor Beverley Snaith, personal communication).

Table 56 (in Appendix 8) summarises the risk factors included in each of the questionnaires. The studies examined 12 different questionnaires used to identify individuals at increased risk of PC-AKI. None of the questionnaires included exactly the same risk factors, but all questionnaires considered previous renal disease and diabetes mellitus as risk factors.

Three of the studies14,38,75 compared the diagnostic accuracy of risk factor screening questionnaires against POC devices, whereas three had a laboratory test as a reference test. 38,114,116 Only three studies14,38,75 included exclusively outpatients and all included patients presenting for a contrast-enhanced CT scan. Data for the relevant eGFR cut-off point (i.e. eGFR < 30 ml/minute/1.73 m²) were reported for three of the studies. 14,38,75

Diagnostic accuracy estimates at different eGFR thresholds are reported, alongside study characteristics, for studies using laboratory and POC test as a reference test in Table 57 (in Appendix 8) and Table 22, respectively.

Although diagnostic accuracy data comparing risk factor questionnaires to a gold standard reference test would have been preferable to inform the model, no studies reported these data for the diagnostic cut-off point of interest (i.e. eGFR < 30 ml/minute/1.73 m²). However, the data reported for the eGFR < 45 and < 60 ml/minute/1.73 m² cut-off points in the studies against a laboratory reference suggest that the sensitivity of the questionnaires is high; sensitivity becomes 100% for the majority of most questionnaires as there is a move from a higher to a lower eGFR cut-off point. The only questionnaires that do not have a sensitivity of 100% at an eGFR < 45 ml/minute/1.73 m² are those applied in the Snaith et al. study38 (Professor Beverley Snaith, personal communication).

The diagnostic accuracy data from studies comparing risk factor questionnaires to POC devices also suggest that high sensitivity tends to be 100% at the lower eGFR cut-off point (i.e. < 30 ml/minute/1.73 m²). The questionnaire based on The RANZCR guidelines11 is the exception with a sensitivity of 0%, but it is also worth noting that only one patient in Snaith et al. 38 had an eGFR < 30 ml/minute/1.73 m² and, thus, results are very uncertain. Specificity at an eGFR < 30 ml/minute/1.73 m² varies between 45.2% and 82.9%. The questionnaire with the lowest overall diagnostic accuracy was that examined by the Azzouz et al. study. 14

In the base-case analysis, the diagnostic accuracy estimates for risk factor screening data were derived from the study by Too et al. 75 This study reported a sensitivity of 100%, which is consistent with the data reported for the studies that used laboratory test as a reference (albeit at higher diagnostic cut-off points). As uncertainty about the diagnostic performance of screening tools remains, a scenario analysis was conducted with data from the Azzouz et al. questionnaire. 14

Table 23 summarises the risk factor screening diagnostic accuracy estimates applied in the model. Beta distributions were fitted to the sensitivity and specificity data to generate random distributions of these parameters in the probabilistic sensitivity analysis.

Risks of post-contrast acute kidney injury

Clinical guidelines have highlighted that individuals with an eGFR < 30 ml/minute/1.73 m² are potentially at an increased risk of PC-AKI following contrast-enhanced CT and that actions should be taken to mitigate that risk, such as considering an alternative imaging method not using iodine-based contrast media or by providing IVH prophylaxis prior to undertaking contrast-enhanced CT. 6,10,11 Whether or not there is an elevated risk in those individuals with an eGFR between 30 and 45 ml/minute/1.73 m² undergoing contrast-enhanced CT remains unclear. 6,11

For the purposes of modelling the impact of identifying patients with low eGFRs, it is important to establish the risk of PC-AKI conditional on eGFR and any actions taken to mitigate the risk (e.g. providing IVH). This section considers the evidence for the risk of PC-AKI conditional on eGFRs in individuals receiving contrast-enhanced CT, the effect of IVH on that risk and the effect of removing intravenous contrast on that risk.

Acute kidney injury consequences and overall mortality

A separate review of published models focusing on the management and consequences of AKI was conducted to further inform the model structure, parameter inputs and assumptions. Further details of the review are reported in Appendix 9.

Based on the review’s findings, the main consequences of PC-AKI include potential mortality risks and the need for RRT. The literature reviewed to inform the risks of PC-AKI in the model was examined for evidence on mortality and risk of RRT conditional on PC-AKI. Park et al. 115 was considered the most relevant to characterise the consequences of PC-AKI in outpatients presenting for CT scan, as the publication reports risks of mortality and initiation of RRT over time by PC-AKI status.

Park et al. 115 present Kaplan–Meier curves by PC-AKI status (PC-AKI vs. no PC-AKI) for time from CT scan until event for (1) death and (2) initiation of RRT (renal survival). Two analyses are presented for each outcome: before and after propensity score matching. The study by Park et al. 115 also reports HRs comparing PC-AKI with no PC-AKI for the full study sample and subgroups by eGFR category (i.e. < 30 vs. ≥ 30 ml/minute/1.73 m²) and timing of events (within 6 months vs. after 6 months of contrast-enhanced CT scan), which are reported in Table 59 (in Appendix 8).

The published Kaplan–Meier curves suggest no difference in terms of mortality for patients who had PC-AKI compared with those who did not, as the curves are largely overlapping for the two groups of patients. This is further supported by the mortality HRs comparing PC-AKI with no PC-AKI, which are consistently non-statistically significant across all analyses; therefore, mortality in the model is assumed to be the same for all patients regardless of PC-AKI status. Mortality was incorporated in the model by applying the costs and QALYs to the PC-AKI pay-offs in the model to the proportion of patients alive at 6 months in Park et al. 115 (i.e. 94.5%). This proportion is assumed to be the same for patients with and without PC-AKI. As baseline mortality risks were not reported by eGFR category, mortality was also assumed to be independent of eGFR levels.

A significant effect of PC-AKI was identified on the probability of RRT initiation. Statistically significant HRs for RRT initiation for the full follow-up period and when events occurring only within 6 months of a CT scan are considered (see Table 59 in Appendix 8). The effect of PC-AKI on the probability of RRT initiation does not appear to be statistically significant in the analysis excluding patients with events after the first 6 months, suggesting that any impact of PC-AKI on the rates of RRT initiation occurs within 6 months of contrast-enhanced CT scanning.

The baseline probability of RRT initiation in the model (i.e. 0.014) is derived from the probability of not having started RRT at 6 months, which is derived from the Kaplan–Meier figure reported for the group who did not experience PC-AKI. The HR for the within-6-months subgroup (i.e. 8.61) is applied to the baseline risk of RRT initiation to estimate the probability of RRT initiation for individuals who experience a PC-AKI event (i.e. 0.111). The HR for RRT initiation for PC-AKI compared with no PC-AKI was set up probabilistically in the model by fitting a log-normal distribution to the data reported in Park et al. 115

Mortality and health-related quality of life

Quality-adjusted life-years were estimated based on estimated mortality and HRQoL. QALYs were discounted at an annual rate of 3.5%. Mortality over 6 months was estimated from a study of post-CT scan patients,115 with mortality post 6 months based on the general population (age and sex adjusted). HRQoL was based on the general population (age and sex adjusted) with utility decrements applied for adverse outcomes, namely undergoing RRT or anxiety resulting from delayed scans. In the base case, RRT is considered the only source of disutility. A scenario analysis also considers the disutility associated with anxiety from delayed scans.

The proportion of patients expected to be alive 6 months post CT scan was derived from Park et al. 115 (i.e. 94.5%) and was estimated as a weighted average of the proportion of patients alive in this study at 6 months post contrast-enhanced CT scan by PC-AKI status (PC-AKI and no PC-AKI). A beta distribution was fitted to the proportion of patients alive at 6 months to derive probabilistic estimates for this parameter. UK life tables were sourced from the Office for National Statistics118 for mortality post 6 months.

Age- and sex-specific general population HRQoL was derived using the equation proposed by Ara and Brazier,119 and applied to the proportion of patients expected to be alive each year (from start age in the model until 100 years old).

Renal replacement therapy was assumed to consist of haemodialysis, based on the study by Kim et al. 117 reporting an earlier data cut-off value of Park et al. 115 The disutility associated with RRT was sourced from a meta-analysis and a metaregression of utilities in CKD patients120 that was identified on the reference list of one of the studies (i.e. Hall et al. 121) examined in the context of the AKI models systematic review. The estimate of –0.11 represents the disutility from dialysis. A gamma distribution was fitted to the utility estimate in the model to generate random draws of the parameter for the probabilistic sensitivity analysis. The disutility is applied for 3 months in the model based on NICE’s Clinical Guideline 169. 108,110 Disutility from anxiety was calculated by assuming that patients would incur the disutility from a EuroQol-5 Dimensions, three-level version (EQ-5D-3L), questionnaire score change from level 1 to level 3 (i.e. –0.236) in the depression/anxiety domain for 2 weeks. The 2-week duration of anxiety was assumed to be the maximum time that patients would have to wait before they could have a CT scan after cancellation of the originally planned scan.

Table 27 details the disutility estimates applied in the model alongside the respective sources and assumptions.

The model does not consider the impact from the delay of the planned CT scan on patient outcomes as a result of any change in their underlying condition during the waiting period. Given the heterogeneity in reasons for referral for a CT scan in the relevant population, and the lack of data sources to characterise the potential impact of delay on HRQoL and disease progression across a wide range of conditions, it was considered unfeasible to include this element in the model. No disutility from PC-AKI was considered, as clinical opinion suggests that the majority of PC-AKI events are asymptomatic.

The potential disutility from adverse events associated with IVH was also not included in the model. The AMACING trial, which compared the cost-effectiveness of IVH to prevent PC-AKI in patients with an eGFR between 30 and 60 ml/minute/1.73 m² with no IVH, found a small difference in excess hospitalisation days due to adverse events from IVH between treatment arms (i.e. 0.06 days). 104 Therefore, it was considered that any adverse events from IVH would have a short duration and have a very limited impact on HRQoL.

Resource use and costs

Overview

The decision-analytic model is evaluated deterministically and probabilistically for the base-case analysis using 1000 Monte Carlo simulations to reflect the joint uncertainty across all of the inputs according to the probability distributions assigned to each input. The parameters set up probabilistically in the model are POC devices diagnostic accuracy data; risk factor questionnaire diagnostic accuracy data; risks of PC-AKI; the HR for the initiation of RRT; the proportion of patients alive at 6 month post contrast; and disutility from RRT.

Following conventional decision rules for cost-effectiveness, the mean costs and QALYs for the various strategies are presented and cost-effectiveness compared by estimating the incremental cost-effectiveness ratios (ICERs), as appropriate.

A limitation of conventional ICER decision rules is that the interpretation of negative and positive ICERs is ambiguous without reference to the cost-effectiveness plane. In contrast to conventional ICER decision rules, the net benefit approach provides an unambiguous decision rule. Net benefits can be expressed on the effect scale [i.e. net health benefits (NHBs)] or the cost scale [i.e. net monetary benefits (NMBs)] and are estimated by rearranging the elements of the conventional ICER equation, where:

In contrast to conventional ICER decision rules, the net benefit approach provides an unambiguous decision rule. For a given cost-effectiveness threshold, the strategy with the highest net benefit is the same strategy that would be considered cost-effective when comparing ICERs against the threshold. A further advantage of using the net benefit framework in the current appraisal is that it may provide a more useful way to summarise results when there are very small differences in QALYs between strategies. In this situation ICERs can be highly sensitive to very small changes in the denominator (i.e. QALY differences).

Uncertainty regarding the appropriate source of data, the appropriate assumptions or model structure and other scenarios are explored using a series of deterministic scenario analysis, as described further in Scenario analyses.

Base-case analysis

The parameters and main assumptions used within the base-case economic model, and their characteristics, are summarised in Table 66 (in Appendix 8).

Scenario analyses

To investigate the impact of several key parameter and structural assumptions, a series of deterministic scenario analyses were undertaken. These scenarios are summarised in Table 32.

Model validation

The model was developed by one researcher (AD) and the programming was checked by a second researcher (MS). A separate version of the model was independently programmed by a third researcher (SW), who successfully replicated the base-case results.

Base case

Deterministic and probabilistic results expressed in NMB and NHB at a cost-effectiveness threshold of £20,000 per QALY are presented in Tables 33 and 34, respectively. Strategy ranking from the highest (1) to the lowest (14) average net benefit is presented in both tables. Incremental net benefit was calculated for each strategy compared with laboratory testing (‘Lab’). Results for the upper bound of the cost-effectiveness threshold recommended by NICE, that is, £30,000 per additional QALY, are not presented, with the exception of probabilities, which are presented for the range of cost-effectiveness thresholds. Results were consistent across the range of cost-effectiveness thresholds considered, and for both deterministic and probabilistic analyses.