Non-invasive imaging software to assess the functional significance of coronary stenoses: a systematic review and economic evaluation

QAngio XA 3D/QFR

QAngio^® XA 3D/QFR^® (three-dimensional/quantitative flow ratio) (Medis Medical Imaging Systems BV, Leiden, the Netherlands) imaging software is used to perform quantitative flow ratio (QFR) assessment of coronary artery obstructions. It is designed to be used with all ICA systems: biplane or monoplane. It uses two standard two-dimensional (2D) angiographic projections, taken at least 25° apart – ideally between 35° and 50° apart – to create a three-dimensional (3D) reconstruction of a coronary artery; this shows the QFR values across the artery. QFR is an assessment (by frame count) of the pressure (blood flow velocity) drop over the artery, with a value of 1 representing a normally functioning artery with no pressure drop. A drop of ≥ 20 mmHg in blood pressure (QFR value of ≤ 0.8) is considered a significant obstruction where revascularisation should be considered. QAngio XA 3D/QFR software is installed on a laptop or workstation that is connected to the ICA system. The Digital Imaging and Communication in Medicine (DICOM) data from ICA projections are immediately uploaded and viewable on the connected workstation. The total time for data acquisition and analysis is about 4 to 5 minutes (as reported by the company).

AngioPLUS [Pulse Medical Imaging Technology (Shanghai) Co. Ltd, Shanghai, China] is an equivalent Conformité Européenne-marked version marketed in Asia.

The QAngio XA 3D/QFR software offers two different flow models to calculate QFR:

1. fixed-flow quantitative flow ratio (fQFR), using fixed-flow velocity

2. contrast-flow quantitative flow ratio (cQFR), using contrast frame count in an angiogram without hyperaemia.

Fixed-flow quantitative flow ratio is faster to compute, but may be less accurate than cQFR.

Furthermore, the QAngio XA 3D/QFR software provides four different QFR indices along the analysed coronary segment:

1. vessel quantitative flow ratio (vQFR): the QFR value at the distal location of the analysed vessel segment

2. index quantitative flow ratio (iQFR): a point that can be moved along the QFR pullback curve

3. lesion quantitative flow ratio (lQFR): the contribution to the QFR drop by the selected lesion alone

4. residual vQFR: an indication of the vQFR, if the selected lesion is resolved.

CAAS vFFR

The CAAS^® vFFR^® (vessel fractional flow reserve) (Pie Medical Imaging BV, Maastricht, the Netherlands) workflow builds a 3D reconstruction of a coronary artery based on two standard angiograms and assesses the pressure drop across the stenosis, and quantitative coronary arteriography (QCA) determines a vFFR value. It gives both anatomical and functional assessment of the stenosis and can be integrated into catheter laboratories. According to the company, the total time for analysis is approximately 2 minutes per artery.

All available versions of CAAS (i.e. 8.0, 8.1 and 8.2) use the same algorithm for calculating vFFR. The CAAS workstation provides various modules (e.g. QCA and left ventricular analysis), and the vFFR module can be added to the CAAS workstation. In addition to the vFFR, CAAS vFFR provides measurements at the end of the lesion and at a chosen position in the coronary artery.

Comparators

Invasive coronary angiography may differentiate between arteries with inadequate blood supply (which need revascularisation) and those with adequate supply that do not need treatment.

During an ICA procedure, a coronary diagnostic catheter is inserted into an artery and moved up the aorta and into the coronary arteries. A special type of dye called contrast medium is injected through the catheter into the coronary artery and angiograms are taken. Although providing valuable information on coronary artery anatomy, visual assessment of angiograms taken during ICA may have limited ability to differentiate between functionally significant (causing inadequate blood supply) and non-significant (not significantly affecting blood supply) coronary stenoses.

When ICA is inconclusive, it may be combined with the invasive measurement of FFR. In these procedures FFR is assessed invasively by advancing a pressure wire towards the stenosis and measuring the ratio in pressure between the two sides of the stenosis during maximum blood flow (induced by adenosine infusion). This is associated with risks related to the passage of a guide wire, side effects of adenosine and additional radiation exposure. The invasive FFR measurement is also associated with increased procedural time and costs compared with ICA alone. As an alternative to invasive FFR, the instantaneous wave-free ratio (iFR) may be used. This also uses inserted pressure wires to assess flow but does not require vasodilator drugs, such as adenosine.

Searches

Comprehensive searches of the literature were conducted to systematically identify all studies relating to QAngio XA 3D/QFR and CAAS vFFR imaging software.

The searches were carried out during October 2019, with a further updated search undertaken on 2 January 2020. The following databases were searched: MEDLINE, PubMed, EMBASE™ (Elsevier, Amsterdam, the Netherlands), the Science Citation Index (Clarivate Analytics, Philadelphia, PA, USA), Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials (CENTRAL), Health Technology Assessment (HTA) database and EconLit (American Economic Association, Nashville, TN, USA).

Ongoing and unpublished studies were identified by searches of ClinicalTrials.gov; Conference Proceedings Citation Index: Science (Clarivate Analytics); EU Clinical Trials Register; Open Access Theses and Dissertations; ProQuest^® (ProQuest LLC, Ann Arbor, MI, USA) Dissertations & Theses A&I; PROSPERO; the World Health Organization’s International Clinical Trials Registry Platform portal; and manufacturer websites. Abstracts from any recent conferences that are thought to be relevant to the review were also consulted.

A search strategy for Ovid^® (Wolters Kluwer, Alphen aan den Rijn, the Netherlands) MEDLINE is reported in Appendix 1. The MEDLINE strategy was translated to run appropriately on the other databases and resources. No language or date restrictions were applied to the searches. No study design search filters were used.

Reference lists of relevant recent reviews8 were checked to identify additional potentially relevant reports.

Database searches were carried out to identify cost-effectiveness studies where ICA (alone and/or with FFR) was one of the interventions under comparison. The following databases were searched: EconLit, EMBASE, HTA database, MEDLINE and NHS Economic Evaluation Database (NHS EED). The search strategies for EconLit, EMBASE and MEDLINE are reported in Appendix 1.

Pragmatic supplementary PubMed and Google Scholar (Google Inc., Mountain View, CA, USA) searches were carried out to identify studies of diagnostic data on ICA compared with FFR.

Contact with study authors and manufacturers and request for individual participant data

An individual participant data (IPD) meta-analysis of four studies that has previously been performed was eligible for this review. 8 The review authors contacted the study authors prior to commencing this assessment, and the study authors agreed, in principle, to share the collected IPD with the review authors for the purposes of this work. However, because of the slow response from the study authors, the IPD could not be supplied in time for this report, and the decision was made not to pursue an IPD analysis. Instead, published data and data presented in figures were used. Where possible, IPD-equivalent data were extracted from plots using a digitising software. See Data extraction for further detail.

Selection criteria

Two reviewers independently screened all titles and abstracts. Full papers of any titles and abstracts that were thought to be relevant were obtained where possible, and the relevance of each study assessed independently by two reviewers according to the criteria below. Any disagreements were resolved by consensus or by consulting a third reviewer. Conference abstracts were included where sufficient data were reported to confirm eligibility. Authors were contacted where insufficient data were reported to confirm inclusion (for instance, to clarify what index test was used in the study, or to provide complete 2 × 2 data) and where it was unclear whether or not the same diagnostic accuracy results were presented in more than one report (e.g. conference abstracts linked to a publication).

Data extraction

A standardised data extraction form was developed, piloted and finalised to data-extract both study and patient characteristics and eligible outcomes. For studies reporting diagnostic accuracy data, the number of true-positive (TP), true-negative (TN), false-positive (FP) and false-negative (FN) results were extracted for each index test evaluated in each study, along with sensitivity and specificity data, the area under the curve (AUC) and 95% confidence intervals (CIs) and PPVs and NPVs. Whether diagnostic accuracy was determined per patient, vessel or lesion was recorded.

Where not reported, sensitivity and specificity were calculated if data allowed. Further data were requested from study authors when required. Correlation and MD between QFR/vFFR and FFR were recorded along with reasons for any excluded, failed or inconclusive results and any other relevant clinical outcomes from the studies.

As IPD could not be supplied, digitised data were extracted using WebPlotDigitizer (Ankit Rohatg, Pacifica, CA, USA) software to approximately reconstruct the individual-level data from included studies. Data were extracted for all studies that presented a Bland–Altman or correlation plot. Bland–Altman plots were preferred for extraction, as these were found to be generally clearer and easier to extract. The extracted averages and differences between QFR and FFR were converted into QFR and corresponding FFR values for each study. For some studies, the quality of published figures was not sufficient to extract data.

Data were extracted by one reviewer using a standardised data extraction form and independently checked by a second reviewer. Discrepancies were resolved by discussion, with involvement of a third reviewer when necessary. Data from relevant studies with multiple publications were extracted and reported as a single study. The most recent or most complete publication was used in situations where we could not exclude the possibility of overlapping populations across separate study reports.

Critical appraisal

The quality of the diagnostic accuracy studies was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool. The QUADAS-2 tool evaluates both risk of bias (associated with the population selection, index test, reference standard and patient flow) and study applicability (population selection, index test and reference standard) to the review question. The tool was piloted on a sample of studies. Signalling questions and criteria for decisions were finalised following piloting.

The quality assessments were performed by one reviewer and independently checked by a second reviewer. Disagreements were resolved through consensus and, when necessary, by consulting a third reviewer.

Methods of data synthesis

The results of data extraction were presented in structured tables and as a narrative summary, grouped by population and test characteristics. The diagnostic accuracy was calculated for each study based on extracted data, using the usual index test of QFR ≤ 0.8 and reference standard of FFR ≤ 0.8 as defining patients in need of stenting. Where sufficient clinically and statistically homogenous data were available, data were pooled using appropriate meta-analytic techniques. Studies that did not report sufficient information to derive 2 × 2 data (from tables, text or plots) were not included in the meta-analysis and were synthesised narratively.

Statistical analysis of diagnostic accuracy

Characteristics of included studies

Table 1 presents the characteristics of the studies included in the systematic review. Only seven studies used QFR prospectively as part of the ICA examination preceding FFR. 14,44,48–52 Fifteen studies were conducted in multiple centres. 15,22,24,26,34,37,38,41,45–47,50–52

Most studies were conducted in Asia, including Japan (33 studies),20–24,26–53 China (five studies),14,26,45,46,52 the Republic of Korea22,24,34,37 and Singapore. 16

Twenty-one studies were conducted in Europe, including Belgium,15,45,46 Denmark,14,50,51 France,38 Germany,42,46,50 Hungary,45 Italy,41,46,50 Lithuania,54 the Netherlands,15,18,35,37,40,41,43,46–48,50 Poland,32,33,50 Spain15,22,34,37,41,50 and the UK. 15,26,47 Two studies were conducted in the USA,19,46 one in Brazil,27 one in Australia36 and one in Canada. 47 Eleven studies included an international cohort. 14,15,22,26,34,37,41,45–47,50

The QAngio XA 3D/QFR studies analysed a total of 5440 patients (over 6524 vessels or lesions), and CAAS vFFR studies analysed a total of 500 patients (over 519 vessels or lesions). Most studies included a mixed population of stable and unstable CAD, although 11 studies focused only on patients with stable CAD. 23,27,31–33,35,44,46,48,49,53 Three studies evaluated non-culprit vessels in patients with MI,17,28,41 two focused exclusively on patients with three-vessel disease,15,23 one study included only patients with intermediate left main stenosis (mostly left main bifurcation)22 and one focused specifically on in-stent restenosis. 34 Where reported, mean age ranged from 59.0 to 72.5 years, and most participants were male (60–93%). Patient history and stenosis severity varied widely across studies. The prevalence of diabetes ranged from 6% to 48%, rates of previous MI from 4% to 58%, and previous PCI 4% to 65% (not accounting for one study with 100% in-stent restenosis). 34 The mean/median FFR ranged from 0.75 to 0.88, and mean DS from 37% to 66%.

Quality of diagnostic accuracy studies

Table 2 summarises the results of the risk-of-bias and applicability assessment for QAngio XA 3D/QFR for the 24 diagnostic accuracy studies reported in a full-text manuscript, with further details reported in Appendix 3, Tables 36 and 37. The risk of bias from the 15 studies included in the diagnostic accuracy review that were reported only as conference abstracts was not formally assessed because of insufficient reporting. 14,16,18,22,25–31,35,36,38,39,44,54 Just as with FAST-EXTEND,18 the extension of the FAST study was reported as conference abstract only; only the quality of the earlier FAST study was assessed. 56

A total of 11 out of the 22 QAngio XA 3D/QFR studies were rated as being at low risk of bias across all domains. 20,21,37,41,42,45,46,49–52 The main source of bias was related to study participant selection; four studies were considered at high risk of patient selection bias because of high rates of patient exclusions or significant exclusion of potentially harder to diagnose patients,17,23,24,32 and three studies did not provide sufficient information on patient selection to assess risk of selection bias (unclear risk). 34,43,48 Exclusion rates and reasons are reported in Appendix 3, Table 37. Risk of bias was rated as being generally low for other domains, although three studies were rated as being at high risk of bias because of the conduct of the index test or reference standard (e.g. no reporting of blinding between QFR and FFR results),33,48,53 and one study was rated as being at high risk of bias because of patient flow concerns, as FFR was performed only in iFR grey-zone patients. 15

The ILUMIEN I19 trial was the only CAAS vFFR complete study with a full-text manuscript. The study was rated as being at high risk of bias because of the large percentage of lesions excluded from the study (65%). In an earlier published report of the FAST-EXTEND study, Masdjedi et al. 56 also reported a high rate of exclusions (54%). Although most of these failed tests appear to have been due to angiographic image processing issues rather than limitations inherent in CAAS vFFR (see Test failures: rates and reasons), the large exclusion rates reported mean that the risk of selection bias cannot be excluded.

Only three studies raised no concerns about their applicability to the review question. 48–50 The main concern about applicability related to QFR being used retrospectively (offline) rather than as part of the ICA examination and before FFR; only five studies (all of QAngio XA 3D/QFR) were conducted prospectively and raised no significant concerns regarding the applicability of the index test. 48–52 There were no significant concerns regarding the applicability of the reference standard in any of the studies. A total of 12 out of the 22 QAngio XA 3D/QFR studies did not raise significant concerns about the applicability of their population to the review question;23,32,33,37,40,42,45,46,48–50,53 concerns about study population applicability were primarily related to the under-representation of patients with stable CAD. We note that, because only patients with a FFR measurement could be included in the diagnostic accuracy review, a subset of patients with intermediate stenosis (including those examined in a diagnostic-only setting, or with a counter-indication to adenosine) are not represented in the included evidence.

Seven studies of QAngio XA 3D/QFR14,15,41,45,48,51,52 and one of CAAS vFFR18 reported a conflict of interest with their respective manufacturers.

Meta-analysis of invasive coronary angiography studies

Five studies included in the meta-analysis also reported 2 × 2 table data on the diagnostic accuracy of using ICA alone, using 50% DS as the cut-off point with FFR < 0.8 as the reference standard. These five studies are summarised in Table 4. We note that reporting of diagnostic data on ICA may be subject to selection bias, as only a small subset of studies reported it, and they are likely to do so to demonstrate the superiority of using QFR over relying on ICA alone.

Given the limited number of studies, and because 2D and 3D ICAs may have very different performance levels, no bivariate meta-analysis of these data are presented here. Based on the results of individual studies, the diagnostic accuracy of ICA appears to be poorer than that of QFR.

Twelve included studies reported AUC estimates for diagnostic accuracy of using ICA alone. A meta-analysis of these studies gave a summary AUC for 3D ICA of 0.71 (95% CI 0.66 to 0.76). For 2D ICA, the summary AUC was 0.63 (95% CI 0.59 to 0.67). Both 2D and 3D ICA have lower AUC values than QFR, and it appears that 2D ICA may be inferior to 3D ICA.

Bivariate meta-analysis to compare tests

Eight studies in the meta-analysis compared two or more testing approaches: five of these compared using 2D or 3D ICA to QFR, and five compared fQFR to cQFR. A ROC plot of results from studies reporting two or more tests is shown in Appendix 4, Figure 25. In all five studies, ICA performed more poorly than QFR, with lower sensitivity and specificity. Differences between fQFR and cQFR were more mixed, with three studies suggesting that cQFR has slightly higher sensitivity than fQFR, but the other two were not consistent with this.

An indirect comparative bivariate meta-analysis accounting for these comparisons between studies is presented in Table 5 and Figure 6. These analyses show the clear inferiority of using ICA alone when compared with FFR as a reference standard. It is clearly inferior to using QFR in both sensitivity and specificity, with a sensitivity of only 51.2% and a specificity of 71.0%.

FIGURE 6.

A ROC plot of bivariate with comparisons of tests.

Unlike the earlier bivariate meta-analysis (see Appendix 4, Figure 24), the comparative analysis suggests that fQFR is slightly inferior to cQFR, mainly due to an inferior specificity (83.9% instead of 89.6%). This suggests that fQFR produces slightly too many FP results (where QFR ≤ 0.8 but FFR > 0.8). This might suggest that if an initial fQFR produces a result less than 0.8 it should be followed up by a confirmatory cQFR.

Impact of study characteristics

Receiver operating characteristic plots differentiating between studies reporting at patient, vessel or lesion level found no evidence that this affects diagnostic accuracy (see Appendix 4, Figure 26). There was also no evidence of any impact on diagnostic accuracy in studies where more than one approach was reported. We note that, where there was more than one lesion assessed, ‘by-patient’ and ‘by-vessel’ analyses selected a single lesion (either at random or based on clinical importance), so a lack of difference is unsurprising, as it would only arise if the choice of lesion was biased. It should also be noted that by-lesion analysis could be biased because of correlation between lesions within patients. Without full patient-level data, the impact this might have cannot be assessed.

There was no evidence of difference in diagnostic accuracy between prospective and retrospective analyses of QFR (see Appendix 4, Figure 27).

Impact of patient factors

Few studies reported diagnostic accuracy data in any form according to different patient characteristics (such as distinguishing between people with and without diabetes, or with and without multivessel disease). The limited evidence reported is discussed in Clinical outcomes.

Given this lack of evidence, to investigate the impact on diagnostic accuracy of key patient factors we have performed meta-regressions of sensitivity, specificity and DOR against the mean value of these factors, where reported in papers. These analyses are obviously limited by being meta-regressions of study-level proportions, rather than true analyses of patient-level data, and because of limited reporting of these factors across studies. For these analyses we did not separate fQFR from cQFR but used one test per study (cQFR for preference) to maximise data. This was considered reasonable given that diagnostic accuracy does not strongly depend on the mode of QFR used.

Appendix 4, Table 38, shows the regression parameter estimates (change in log-DOR, sensitivity or specificity per unit of the covariate), their 95% CIs and p-values from these metaregression analyses. For most parameters there is no evidence of any association with diagnostic accuracy. However, this may be due to a lack of data rather than no association.

Four patient factors (i.e. diabetes, stable CAD, multivessel disease and mean FFR) suggest a possibility of association, as all have at least one p-value below 0.05. Plots of the proportions of patients with these factors, against estimated sensitivity, specificity and log-DOR are shown in Appendix 4, Figures 28–31.

The association between diabetes and diagnostic accuracy is partly driven by one study where nearly all patients had diabetes, but the trend for studies with more diabetic patients to have higher sensitivity and DOR remains even if that study is removed. There is a trend for specificity and DOR to decline as higher proportions of patients have stable CAD. Conversely, specificity and DOR increase as more patients have multivessel disease (although this is based on only five studies32,37,41,42,51).

There is evidence that the lower the average FFR in a study, the higher the sensitivity and the lower the specificity (but with no impact on the overall accuracy in terms of the DOR). We might therefore also expect some variation in diagnostic accuracy with any factor that lowers FFR (DS, medical history, etc.) but the data are too limited to confirm this.

Meta-analysis of diagnostic accuracy

We calculated the diagnostic accuracy for each study based on extracted data, using the usual index test of QFR ≤ 0.8 and reference standard of FFR ≤ 0.8 as defining patients in need of coronary intervention. To investigate whether or not the extracted data could be used for analysis, we compared these diagnostic accuracy results to the results from 2 × 2 tables [used in previous meta-analyses in Bivariate meta-analysis (QAngio XA 3D/QFR)]. The extracted sensitivity and specificity estimates are summarised in Appendix 4, Table 39. Overall, 30 studies reported either 2 × 2 table data or data that could be extracted from a figure. Nine studies did not present an extractable figure, and three studies presented a figure, but no summary data.

In general, the number of data points from the extracted figure data was smaller than that reported in the studies. This is to be expected, as overlapping points will be missed when extracting from figures. There is mostly good agreement in diagnostic accuracy between the data sources, except for a few cases where the figure data have lower sensitivity and mostly lower specificity. 24,40,52,61 Only one study28 had better diagnostic accuracy when using data extracted from figures. This consistency suggests that using the extracted figure data for diagnostic analysis is reasonable, even though it represents a smaller sample size. The results of performing a bivariate meta-analysis for diagnostic accuracy using the extracted figure data are shown in Appendix 4, Figure 37. The black points are the results in each study, and the blue dot is the result of the meta-analysis (with its HSROC curve). The summary sensitivity is 84.6% (95% CI 80.7% to 87.8%) and specificity is 87.2% (95% CI 83.4% to 90.3%). This is similar to the results from the main analysis when cQFR and non-specified QFR were combined (sensitivity 84.3%, specificity 89.8%), albeit with slightly lower specificity, further confirming that analysing the extracted figure data are reasonable.

We note that this bivariate meta-analysis is presented to confirm that the extracted data reasonably represent the properties of the included studies; the bivariate meta-analyses in Bivariate meta-analysis (QAngio XA 3D/QFR) should be taken as the primary analyses.

Grey-zone analysis

The main purpose of extracting data from figures is to permit an analysis where testing includes a grey zone of intermediate QFR values for which a FFR would be performed as a confirmatory test. The grey-zone diagnostic procedure is:

• perform QFR

• if the QFR is > 0.84, continue without stenting/bypass and defer FFR (test negative)

• if the QFR is ≤ 0.78, proceed directly to stenting/bypass without FFR (test positive)

• otherwise, perform a FFR and proceed based on that result (i.e. at a 0.8 cut-off point) (the grey zone).

This means that for anyone within the grey zone there is perfect diagnostic accuracy, so FPs and FNs occur only in patients outside the grey zone.

Appendix 4, Figure 38, shows the FFR and QFR data again, with the proposed grey zone highlighted. In total, across studies, 20.1% of all patients lie within the grey zone (accordingly, up to 79.9% of patients would theoretically have a wire-free and adenosine-free procedure in this scenario). Of these grey-zone patients, 19.1% are TPs with both QFR and FFR below 0.8, and 50.2% are TNs with both tests above 0.8. Only 18.3% are FNs and 12.4% FPs. Hence, only 30.4% of patients in the grey zone have discordant FFR and QFR results (relative to the 0.8 threshold).

Within the grey zone, differences between FFR and QFR are small. This is shown in Appendix 4, Figure 39, categorised by TPs, FPs, etc. Very few patients in the grey zone differ in test values by > 0.1, and most differ by ≤ 0.05.

The diagnostic accuracy when using the grey zone improves, as would be expected, to a sensitivity of 93.1% (95% CI 90.1% to 94.9%) and a specificity of 92.1% (95% CI 88.3% to 94.5%). Appendix 4, Figure 40, shows the result of this meta-analysis (with its HSROC curve) compared with the meta-analysis without the grey zone presented in Bivariate meta-analysis (QAngio XA 3D/QFR). Clearly using the grey zone improves diagnostic accuracy compared with QFR alone because of the 3.7% of patients reclassified from test negative to positive and 2.5% who are reclassified in the opposite direction. However, this improvement depends on assuming that the 0.8 threshold of FFR genuinely separates those who need intervention from those who do not.

As an alternative to using the manufacturer-specified grey zone, we also examined what grey-zone thresholds would be required to achieve a sensitivity and specificity of 90% and 95% respectively. This is summarised in Appendix 4, Table 40. This suggests that the manufacturer-recommended grey zone favours high sensitivity over high specificity.

Alternative fractional flow reserve thresholds

The IRIS-FFR study13 found that only for a FFR ≤ 0.75 did the risk of MACEs become significantly lower in patients with revascularised lesions than in those in whom revascularisation was deferred. This suggests that the current threshold of 0.8 for planning revascularisation may not be clinically appropriate. Using the extracted figure data, we can investigate the diagnostic accuracy of QFR compared with FFR at other thresholds. For example, if the threshold for both QFR and FFR is 0.75 then the diagnostic accuracy becomes a sensitivity of 75.4% (95% CI 69.0% to 80.8%) and a specificity of 90.6% (95% CI 87.9% to 92.7%). This is compared with the previous meta-analysis at the threshold of 0.8 in Appendix 4, Figure 41. Using a 0.75 threshold leads to slightly lower sensitivity, but higher specificity. The two ROC curves, however, are almost identical, suggesting no overall change in diagnostic accuracy.

Meta-analysis of extracted figure data for two-dimensional invasive coronary angiography

To inform the economic analyses an additional pragmatic search for studies that compared 2D ICA with FFR assessment was performed to identify studies that presented sufficient granular data (such as scatterplots or Bland–Altman plots) from which ICA and FFR data could be extracted. This search identified four such studies (see Appendix 4, Table 41). 62–65

Figure 9 shows the plot of all extracted data from these four studies. The colours of the dots indicate the studies. It can be seen that, when compared with the equivalent figure for QFR (see Figure 7), 2D ICA is much more weakly correlated with FFR (correlation coefficient –0.432). There are many FNs (bottom-left shaded region, 13.0% of patients) and FPs (top-right shaded region, 25.5% of patients) when using 50% DS and the index test and FFR ≤ 0.8 as the reference standard.

FIGURE 9.

Extracted data on 2D ICA compared with FFR. Each colour represents a separate study. Black line indicates where FFR = ICA. The lower-left shaded region shows the FNs where ICA < 50% but FFR ≤ 0.8; the upper-right shaded region shows the FPs where ICA > 50% but FFR > 0.8.

We performed a bivariate meta-analysis of these extracted data, using the same approach as for QAngio XA 3D/QFR. The summary sensitivity was 62.6% (95% CI 51.5% to 72.5%) and specificity was 61.6% (95% CI 53.1% to 69.4%). This is a substantially lower diagnostic accuracy than for QAngio XA 3D/QFR.

QAngio XA 3D/QFR: studies not included in meta-analysis

Appendix 5, Table 42, presents results from the six studies of QAngio XA 3D/QFR that reported diagnostic accuracy results but were not included in the meta-analysis because of insufficient data. 14,22,33,36,38,47 All studies were reported as conference abstracts only (although one was subsequently published after the cut-off date for conducting meta-analyses). 33 One QAngio XA 3D/QFR prototype study recorded QAngio XA 3D/QFR analyses prospectively (on-site analysis), and re-ran analyses retrospectively after ‘essential modifications’ (no further details reported). All other studies were retrospective and did not report which version of QAngio XA 3D/QFR was used. 22,33,36,38,47

The results broadly reflected the findings of the meta-analysis. All studies reported moderate to high diagnostic accuracy for QAngio XA 3D/QFR compared with FFR. There was significant heterogeneity in reported diagnostic accuracy estimates. Sensitivity ranged from 64.0% to 91.8%, and specificity from 68.2% to 97.3%. Where reported, PPV estimates ranged from 74.0% to 84.8% and NPV from 68.2% to 93.0%. AUC ranged from 0.77 (95% CI 0.67 to 0.87) to 0.99 (95% CI 0.97 to 1.00). Correlation coefficients (r) also varied significantly, ranging from 0.578 to 0.801.

The wire-free invasive functional imaging (WIFI) prototype study14 reported moderate sensitivity (0.64, 95% CI 0.48 to 0.77) and specificity (0.8, 95% CI 0.66 to 0.89) in its initial analyses. Following ‘essential modifications’ (no further details reported) a blinded in-centre core laboratory reanalysis was performed, and this improved both sensitivity (0.66, 95% CI 0.51 to 0.79) and specificity (0.86, 95% CI 0.73 to 0.93).

QAngio XA 3D/QFR: other modes

Three studies reported results for QAngio XA 3D/QFR modes other than cQFR and fQFR. 32,46,48 Their results are presented in Appendix 5, Table 43. Two small studies (n = 15 and 84 vessels) reported results for adenosine–flow quantitative flow ratio (aQFR),46,48 and one larger study (n = 306 lesions) tested iQFR, lQFR and vQFR. 32 Sensitivity of aQFR per vessel ranged from 78% to 100%, and specificity from 91% to 93%; one study reported a high AUC (0.90, 95% CI 0.81 to 0.96) for aQFR, and similar results in per-patient analyses. One study found that iQFR had higher diagnostic accuracy overall (sensitivity: 83.3%; specificity: 86.6%; AUC: 0.936) than vQFR (sensitivity: 90.5%; specificity: 69.7%; AUC: 0.900) and lQFR (sensitivity: 91.1%; specificity: 71.7%; AUC: 0.822).

Bivariate meta-analysis (CAAS vFFR)

The results of bivariate meta-analysis of these studies are presented in Table 7. It should be noted that, because there are only three independent studies, and as data had to be imputed, meta-analyses of these studies may not be robust, and are included only to permit some comparison with QAngio XA 3D/QFR analyses.

As the FAST56 and FAST-EXTEND18 studies overlap, we report meta-analysis using each of these (and excluding the other). Although diagnostic accuracy is reasonable in both analyses, CIs are wide, reflecting the limited data and high heterogeneity. In both cases, specificity is lower than estimated for QFR (around 89%). When using FAST-EXTEND, sensitivity is similar to QFR (around 84%), but when using the earlier FAST study, sensitivity for CAAS vFFR is lower than for QFR. These meta-analyses should be interpreted with caution because they required imputation of data for two studies on the prevalence of FFR results below and above the cut-off point of ≤ 0.80, and because of the high heterogeneity across studies.

Only one study26 has directly compared CAAS vFFR with QFR, and this is currently reported only as a conference abstract. That study concluded that diagnostic performance of vFFR was poorer than for QFR, with AUCs of 0.719 (95% CI 0.621 to 0.804) for vFFR and 0.886 (95% CI 0.807 to 0.940) for cQFR.

Subgroup and sensitivity analyses (CAAS vFFR)

There were insufficient data to conduct any subgroup analyses or meta-regressions to investigate whether the diagnostic accuracy of CAAS vFFR varied by patient or study characteristics. Sensitivity analyses according to study quality were not feasible.

As only one study presented a figure with extractable data, analyses of these data were not performed. No further data suitable for narrative review or synthesis were identified.

Morbidity, mortality and major adverse cardiac events

Three cohort studies reported mortality or major clinical outcomes in eligible patients with QFR (QAngio XA 3D/QFR) measurements. 30,41 All found that a clinically significant QFR was associated with a higher incidence of long-term MACEs. No data were reported for CAAS vFFR. Results are summarised in Appendix 5, Table 48, and below.

Spitaleri et al. 41 included patients with multivessel disease who underwent revascularisation as part of a large randomised trial of PCI in 1498 STEMI patients in whom at least one non-culprit lesion (NCL) was left untreated. 66 QFR was calculated in NCLs in a subgroup of 110 patients following revascularisation. Patients with QFR values > 0.80 in all NCLs were classified as having functional complete revascularisation (n = 54), and those with at least one NCL with QFR value ≤ 0.80 were classified as having functional incomplete revascularisation (n = 56). Patient-oriented cardiac events (POCEs, defined as cumulative occurrence of all-cause death, any MI and any coronary revascularisation) were measured at the 5-year follow-up. A total of 39 (35%) patients experienced an adverse event. The cumulative incidence of POCEs was higher in the group with QFR ≤ 0.80 (46%) than in the group with QFR > 0.80 (24%) (HR 2.3, 95% CI 1.2 to 4.5; p = 0.01). Further individual POCE outcomes are reported in Appendix 5, Table 48.

Kanno et al. 30 (conference abstract only) evaluated 212 de novo intermediate coronary lesions in 212 patients with deferred revascularisation based on FFR values above 0.80. Baseline and physiological indices including cQFR were compared between patients with and without MACEs (cardiovascular death, non-fatal MI, target vascular revascularisation and non-target vascular revascularisation) during the 4-year follow-up. MACE incidence at the 4-year follow-up was 5.7%. In patients with MACEs, cQFR was lower than that in patients without MACEs (mean or median 0.80 vs. 0.88; p = 0.030). On logistic regression analysis, cQFR≤ 0.8 was a significant predictor of MACEs (OR 5.60, 95% CI 1.69 to 18.6; p = 0.005).

Hamaya et al. 23 included a population of 549 patients with stable three-vessel disease who underwent cQFR. At the median 2.2 years’ follow-up, patients with MACEs had lower cQFR in all three vessels than those without MACEs [2.76 (95% CI 2.64 to 2.88) vs. 2.64 (95% CI 2.49 to 2.73); p < 0.001], and three-vessel cQFR was a statistically significant predictor of MACEs in multivariate analyses (HR 0.97, 95% CI 0.96 to 0.99). cQFR was also a better predictor of remote revascularisation (≥ 3 months) than DS [AUC 0.73 (95% CI 0.65 to 0.79) vs. AUC 0.66 (95% CI 0.56 to 0.74); p = 0.043].

Subsequent use of invasive pressure wire fractional flow reserve

No studies of QFR prospectively evaluated the impact of QFR use and subsequent reductions in the use of adenosine and pressure wire procedures. However, five studies included in the diagnostic accuracy review retrospectively derived a grey-zone strategy based on their diagnostic accuracy results to model a potential reduction in adenosine and FFR use. 29,37,40,50,51

Results are summarised in Appendix 5, Table 49. None of these studies used the grey-zone boundaries recommended by the manufacturer (0.78–0.84), and only two studies used the same grey zone. 40,50 All studies derived their grey-zone boundaries from their own cohort, except one40 that used boundaries defined by another study. 50 Diagnostic accuracy criteria of QFR against FFR used to derive grey-zone boundaries varied across the studies (e.g. minimum sensitivity and specificity of the grey zone was > 95% in one study50 and > 90% in another51). Each study retrospectively modelled a QFR–FFR hybrid strategy using QFR as the main diagnostic method and only performing FFR measurements in their defined grey zone. Despite the variety of choice of grey zones and how they were defined, all are broadly similar to each other and to the manufacturer-specified definition used in the meta-analysis in Bivariate meta-analysis (QAngio XA 3D/QFR).

All simulated grey-zone strategies were associated with a large percentage of adenosine/FFR procedures (hypothetically) avoided, ranging from 42% to 68%. The widest grey-zone area (0.71–0.90)51 was associated with the lowest proportion of adenosine/FFR-free procedures (42%), and the narrowest boundaries (0.77–0.86, 0.78–0.87) associated with the highest proportion of procedures avoided (61–68%). 40,50,51 None of the simulations modelled the clinical impact of delayed FFR in patients with a FFR below 0.8.

Interobserver variability

Eight studies reported outcomes data on the reproducibility of QFR readings between two different analysts (see Appendix 5, Table 50). One study directly compared QAngio XA 3D/QFR and CAAS vFFR,26 six studies evaluated QAngio XA 3D/QFR only17,24,25,27,33,45 and one CAAS vFFR only. 56 Three studies were reported only as conference abstracts. 25–27 The number of single measurements analysed ranged from 10 to 101 vessels. All QFR measurements were performed and compared retrospectively (offline) where reported. Only two studies explicitly reported blinding analysts to each other’s readings. 45,56

It was found that QFR had a moderate to high level of inter-rater reliability. Two studies of QAngio XA 3D/QFR reported MDs ≤ 0.01 in repeated QFR measurements between analysts. 24,45 One study reported a moderate correlation between three separate raters [mean intraclass correlation (ICC) 0.614, 95% CI 0.464 to 0.728]. Two studies reported high ICC results, one in stable angina patients (r = 0.990)33 and the other in NCLs of STEMI patients (r = 0.991). 17 Another study found that inter-rater reliability was higher for cQFR (R² = 0.82) than for fQFR (R² = 0.70) and ICA DS (R² = 0.67). 27 One study found high inter-rater repeatability for QAngio XA 3D/QFR [fQFR 0.001 (SD 0.036) and cQFR 0.001 (SD 0.049)] as well as CAAS vFFR [0.005 (SD 0.037)] and no statistically significant differences between raters’ measurements. Inter-rater reliability was also high in the FAST study across 100 repeated CAAS vFFR measurements (r = 0.95). 56

Intraobserver variability

Eight studies reported outcomes data on intraobserver reproducibility of QFR readings (see Appendix 5, Table 51). Seven studies evaluated QAngio XA 3D/QFR only,16,17,25,27,33,45,54 and one study directly compared QAngio XA 3D/QFR and CAAS vFFR. 26 Five studies were reported only as conference abstracts. 16,25–27,54 Where reported, all measurements were performed retrospectively (offline). The time gap between initial and repeated measurements was reported in four studies and ranged from 3 days to 2 weeks. 16,27,45,54

All studies except one25 reported a high level of intrarater reliability for QFR. One study that assessed QFR readings independently by three analysts twice among 100 vessels reported a moderate ICC coefficient (r = 0.428). Where reported, r coefficients for QAngio XA 3D/QFR in other studies ranged from 0.958 to 0.997, and MDs between repeat measurements from 0.00 (SD 0.03) to 0.016 (SD 0.06). One study found that R² was higher for fQFR (0.91) and cQFR (0.94) than DS measured by ICA (0.76). 27 One study that evaluated both QAngio XA 3D/QFR and CAAS vFFR found high levels of repeatability and no statistically significant changes between repeated tests (cQFR: MD 0.009 ± 0.053, p = 0.230; fQFR: MD 0.016 ± 0.060, p = 0.066; vFFR: MD 0.008 ± 0.040, p = 0.175).

Test failures: rates and reasons

Appendix 5, Table 52, reports rates of exclusions from diagnostic accuracy studies and reasons for exclusion. Sixteen studies did not report rates of patient exclusions or reasons for exclusion. 14,16,18,22,23,25,29,31,33,35,36,38,39,44,48,54 Exclusion rates varied widely, from 6% to 92%, although this is partly due to differences in patient selection criteria, reporting and methods of calculating exclusions rates (e.g. out of total population considered for eligibility vs. out of total number of patients with FFR). This limits the comparability of exclusion rates across the studies.

Issues with the acquisition and quality of angiographic images (e.g. lack of at least two projections with a 25% degree angle in between, or poor image quality) were the most reported cause of exclusion, with 15 studies reporting it as their main reason for excluding patients from QFR analyses. 15,17,19–21,24,28,32,37,42,43,46,49,50,56 Anatomical features of arteries (e.g. excessive overlapping or foreshortening, ostial lesions, severe tortuosity) were the second most commonly listed reason for exclusion. Rates of exclusions were higher overall in retrospective studies (median 28%, range 6–92%) compared with prospective studies (median 17%, range 7–52%). This may be partly explained by the fact that ICA images in retrospective studies were less likely to have been collected following manufacturer instructions to acquire images suitable for QFR.

Both CAAS vFFR that reported reasons for exclusion reported high exclusion rates (63% and 65%), although both studies were retrospective. 19,56 In both studies, the majority of exclusions were explained by angiographic image processing issues (rather than CAAS vFFR directly). For instance, 83% of exclusions in ILUMIEN I19 were due to the lack of at least two angiographic projections, table movement during ICA or pixel resolution incompatibility. ILUMIEN I19 concluded that careful adaptions in acquisitions of ICA images could reduce test failure.

Other outcomes

No evidence was reported in QAngio XA 3D/QFR and CAAS vFFR studies for any of the following protocol-specified outcomes: impact of QFR on the rate revascularisation procedures, adverse events related to the diagnostic procedure, adverse events related to revascularisation, distress, anxiety and similar harms caused by QFR, vFFR, invasive FFR or iFR, number of vessels with stent placements, HRQoL and radiation exposure.

Simulation study of clinical effectiveness

Given the very limited data on clinical effectiveness of QAngio XA 3D/QFR reported in publications, we performed a simulation study to investigate the possible impact of using QAngio XA 3D/QFR, compared with FFR, on actual coronary outcomes. This simulation study treats the complete data extracted from figures (3192 observations) as a representative sample from the true population of FFR and QFR measurements. To predict coronary outcomes we used the results of the recent IRIS-FFR registry report, representing 5846 patients who were either revascularised (stent or bypass surgery) or deferred (continued with current management without surgery) based on their measured FFR result. The full methods are set out in Statistical analysis of clinical effectiveness.

The IRIS-FFR study used major cardiovascular events (MACE, a composite of cardiac death, MI and repeated/emergency revascularisation) as its primary outcome. The reported hazard of MACEs by FFR value was used to estimate the risk for each person in the extracted data. Based on those risks we simulated whether or not each person had a MACE if they were ‘deferred’ or if they were revascularised. Note that this assumes that risk is solely a function of FFR values, and that knowing the QFR has no impact on risk of MACEs.

We investigated three strategies for deciding on whether or not to revascularise:

• FFR only – perform FFR on all and revascularise if the FFR is ≤ 0.8

• QFR only – perform QFR on all and revascularise if the QFR is ≤ 0.8, without FFR measurement

• grey zone – perform a QFR and:

  • revascularise if the QFR is ≤ 0.78

  • defer if the QFR is > 0.84

  • if the QFR is between 0.78 and 0.84, perform FFR and revascularise if the FFR is ≤ 0.8.

Results of the simulation study

Appendix 6, Figure 42, presents an example simulation, showing the distribution of simulated MACEs according to FFR and QFR. For ease of interpretation, the majority of patients with no MACE are excluded and only patients with MACE are shown. Preventable MACEs (i.e. patients who would have MACE if not revascularised) are evenly distributed across both FFR and QFR ranges. MACEs caused by revascularisation (i.e. where MACE occurs if revascularised, but would be avoided if deferred) are concentrated above values of 0.75 for both FFR and QFR, in line with the suggestion in IRIS-FFR that deferral is preferable for a FFR > 0.75.

In Appendix 6, Figure 42, most events occur in the white regions, where the same revascularisation decision would be made using either FFR or QFR. There are few patients, and hence few MACEs, in the FP region (upper-left shaded area), where patients would be revascularised based on FFR but not if using QFR. Hence, using QFR would miss out on preventing some events in this region (green dots) but equally would avoid causing MACEs due to revascularisation (blue dots).

In Appendix 6, Figure 42, in the FP region (lower-right shaded area), where patients would be revascularised based on QFR but not if using FFR, there are also few events. QFR prevents some events in this region (green dots) that would be missed by FFR, but equally would cause MACEs due to revascularisation (blue dots). The ‘preventable’ and ‘caused’ events in these two regions approximately balance each other out.

Based on the data extracted from figures, if using the ‘FFR only’ strategy 40.2% of patients would be revascularised; using the ‘QFR-only’ strategy 42.0% would be revascularised; and using the grey-zone strategy 43.2% would be revascularised. So, using QFR moderately increases the revascularisation rate, and using it in combination with a grey zone increases it further.

Table 8 summarises the key results of the simulation. FFR is slightly more effective at preventing MACEs, but QFR leads to only about one extra unprevented MACE per 1000 patients. If the grey zone is used the total number of MACEs is closer to that of using FFR for everyone. Using QFR results in around 1.3 to 1.6 more revascularisations to prevent one MACE.

TABLE 8 - Key results of the simulation study

Strategy	Percentage with MACE	Percentage with prevented MACE	Percentage with MACE caused by revascularisation	Percentage with unprevented MACE	Number of revascularisations per MACE prevented
FFR	1.75	1.60	0.91	0.78	25.18
QFR	1.85	1.57	0.97	0.81	26.80
Grey zone	1.82	1.63	1.00	0.75	26.50

Using QFR with or without with a grey zone leads to more revascularisations, and so more MACEs caused by revascularisation (6 or 9 per 10,000 more, respectively), which leads to a larger number of revascularisations per MACE prevented.

Table 8 presents only the median values across all simulations. Figure 11 shows the distribution of revascularisations per MACE prevented. Appendix 6, Figures 43–45, show the prevented and unprevented events, and events caused by revascularisation across all simulations. These show the substantial overlap between the distributions, so, although the results in Table 8 suggest some difference between strategies, it is not clear if these are genuine differences that would be observed in actual clinical practice.

FIGURE 11.

Estimated revascularisations per MACE prevented across all simulations.

Overall, these simulations suggest that there is little conclusive clinical difference between using QFR and FFR to make revascularisation decisions. Using FFR may prevent slightly more MACEs, at around 1 event per 1000 patients, but the overlap in simulated distributions means it is highly uncertain whether or not the difference is genuine. By contrast, the simulation suggests that QAngio XA 3D/QFR increases the number of revascularisations performed, without substantially improving the number of MACEs prevented.

The simulation has numerous limitations as a result of its assumptions. Most important is that the risk of MACE depends only on a patient’s FFR. The simulation could not account for any other key patient factors, and there is the possibility that knowing the QFR as well as the FFR might alter the predicted risk. The IRIS-FFR risks may not match the risks in the UK population eligible for FFR or QFR assessment. The simulation is also based only on the data extracted from figures, which is a small sample and may not represent the patients seen in practice. The simulation considers only a single lesion per patient, when QFR may be used to assess multiple stenoses in a patient.

Timing of results from data acquisition

Six studies of QAngio XA 3D/QFR reported measuring the time required to complete QFR analysis. 14,32,45,50,52,53 The results are summarised in Appendix 5, Table 53. Two studies were prospective,14,52 and one was reported only as a conference abstract. 14 Sample size ranged from 68 to 268 patients. The reporting of methods for calculating time to QFR acquisition differed among the studies. For instance, only two studies specified that calculations included time required to select appropriate angiographic images for generating 3D images. 50,53

Time to QFR data acquisition ranged from an average of 2 minutes 7 seconds to 10 minutes (SD 3 minutes). One study of 268 patients reported that time to image acquisition significantly decreased with the number of ICAs analysed, from 5 minutes 59 seconds to 2 minutes 7 seconds, between the first and last 50 cases. One conference abstract of an earlier prototype version of QAngio XA 3D/QFR reported a mean total time to QFR of 10 minutes (SD 3 minutes). The study reported that the application required essential modifications during the study and retrospective reanalysis of ICA and QFR was performed with the final version of QFR, although it was not clear which analysis was used to derive mean time to data acquisition.

Other outcomes

No evidence was found for any of the following review protocol-specified implementation outcomes: acceptability of QFR, vFFR and invasive FFR (to clinicians and patients), referral times, patient satisfaction, training requirements, test uptake and compliance.

Conclusions and recommendations for research from included studies

Most studies concluded that QAngio XA 3D/QFR had good diagnostic accuracy for detecting significant coronary stenosis and good correlation and agreement with both wire-based FFR14,15,20,21,23,24,26–30,32,34–46,48,50–54,56,67 and iFR,16,21,24,36,49 and is able to improve angiographic assessment for evaluation of intermediary coronary artery stenosis. 14,50,52

Studies of CAAS also concluded that QFR had good correlation and agreement with wire-based FFR,18,19,24,56 although one concluded that only one-third of routinely acquired coronary angiographic images were appropriate for retrospective vFFR analysis. 19

Studies conducted in patients with acute coronary syndrome concluded that QFR was safe and accurate in assessment of non-culprit vessels. 17,28,31,41 Some studies suggested that diagnostic accuracy of QFR may be affected by specific clinical characteristics, namely small vessels,15,32 presence of bifurcated lesions and trifurcated lesions,15,38 left main stenosis,22 prior MI-related coronary arteries20 and microvascular function. 29,41

Several studies concluded that QFR may be a good alternative tool for identifying significant coronary stenosis in various clinical settings or may complement invasive wire-based options;23,31,37,49 it is applicable to patients allergic to adenosine and adenosine triphosphate (ATP) vasodilators and may avoid procedural risks or patient discomfort associated with invasive wire-based options. 23,52 Some studies noted that QFR may reduce procedure time, be associated with reduced cost and allow for wider adoption of functional assessment of coronary stenosis. 23,33,46

Some studies recommended using a hybrid approach to reduce the need for invasive FFR, although there was no consensus on an optimal grey zone. 16,33,43 In some cases, patients may be unsuitable for evaluation of stenosis severity using angiography, including diffuse tandem disease, tandem lesions, lesions with angiographic haziness caused by calcification or thrombus and lesions with ulceration cause by plaque rupture. 20 Diagnostic accuracy could be affected in patients with bifurcation lesions,32,50 patients with prior MI-related coronary arteries20 and patients with left main location of stenoses. 22 Some studies suggested that confirmation with FFR may be required close to values of 0.8. 33,53 One CAAS study noted that careful adaptations in image acquisition will be required to reduce the risk of test failures if used in daily clinical practice.

Further prospective online investigation into the clinical benefit of QFR-based revascularisation was recommended by multiple studies,17,37,42,43,45,46,48,50 including using appropriately powered randomised controlled trials (RCTs) with relevant clinical end points before implementing the device as a definite alternative to invasive FFR,24,32,37,40,50 such as the ongoing FAVOR III China trial (NCT03656848). Some recommended further testing of modelled hybrid QFR/FFR approaches. 51

A number of studies recommended that that testing of the diagnostic accuracy and feasibility of QFR in clinical practice in different settings is needed. 17,32,36–38,40,43,45,48,49

Further investigation of diagnostic precision and the application of the current QFR methodology in patients with different lesion subtypes,38 including bifurcation lesion,32,50 patients with prior MI-related coronary arteries20 and patients with left main location of stenoses22 was recommended.

Searches

The bibliographic search detailed in Chapter 3, Searches, was used to identify studies reporting on the cost-effectiveness of QAngio XA 3D/QFR and CAAS vFFR.

Selection process

The review considered a broad range of economic studies including economic evaluations conducted alongside trials, modelling studies and analyses of administrative databases. The inclusion criteria considered were full economic evaluations comparing two or more alternatives and considering both costs and consequences (i.e. cost-minimisation, cost-effectiveness, cost–utility and cost–benefit analyses).

The protocol for the selection of relevant studies defined two selection stages: (1) assessment and screening for possible inclusion of titles and abstracts identified by the search strategy, and (2) acquisition and screening for inclusion of the full texts of potentially relevant studies. Two researchers independently screened the titles and abstracts of all reports identified by the bibliographic searches. Full-text papers were to be subsequently obtained for assessment and screened by at least two researchers, with any disagreement resolved by consensus.

Searches

Targeted searches were conducted in October 2019 in the following databases: MEDLINE databases (i.e. MEDLINE Epub Ahead of Print, In-Process & Other Non-Indexed Citations, Ovid MEDLINE Daily, and Ovid MEDLINE), EconLit, EMBASE, NHS EED and the HTA database. Search strategies are detailed in Appendix 1.

Study selection

Cost-effectiveness studies published after the year 2000 where ICA (alone and/or with FFR) was one of the interventions under comparison were considered for inclusion. Only cost-effectiveness, cost–utility and cost–benefit analyses were considered eligible. Studies that presented results as a cost per diagnosis were not considered for inclusion, as the key aim of the review was to assess how the link between short-term diagnostic outcomes and longer-term impact and subsequent prognosis associated with the diagnostic pathways in the management of CAD and associated costs and outcomes had been established in the literature. The patient population of this review was defined as patients with stable chest pain and suspected or known CAD. Studies in patients with acute coronary syndromes and NSTEMI as the primary diagnosis were excluded. The inclusion criteria further specified that only titles in English would be considered eligible. Titles that were books, editorials, letters to the editor and reviews that did not include a de novo model were excluded from the review.

One researcher (AD) conducted the two-step selection process consisting of screening for inclusion (1) the titles and abstracts of studies identified by the bibliographic searches, and (2) the full-text articles identified at the previous step as potentially relevant.

Walker et al.77

Walker et al. 77 developed a decision tree and Markov model structure to evaluate the cost-effectiveness of eight alternative testing sequences, including different combinations of exercise treadmill testing, SPECT, cardiovascular MR and coronary angiography, to identify patients with angina who require revascularisation (i.e. those with significant stenosis) derived from the CE-MARC (Clinical Evaluation of Magnetic Resonance Imaging in Coronary Heart Disease) trial. 89 The study population included patients with angina (with and without significant stenosis) and those without angina, based on characteristics of patients in the CE-MARC trial. 89 The base-case analysis considered the case of a 60-year-old man, classified as grade 2 on the Canadian Cardiovascular Society (CCS) scale, with a prior likelihood of significant stenosis requiring revascularisation of 39.5%. Patients with angina were assumed to have had no previous MI. The Markov model had a 50-year time horizon with a 3-month cycle length. The perspective of the study was NHS and Personal Social Services (PSS), and health outcomes were measured in terms of QALYs. Costs were expressed in term Great British pounds (GBP) (2010/11 price year), and costs and health outcomes were discounted at a rate of 3.5% per annum.

The aim of the diagnostic testing was to identify patients with significant coronary artery stenosis who require revascularisation [either PCI or coronary artery bypass graft (CABG)]. It was assumed that all patients suspected of having significant coronary stenosis would undergo coronary angiography as a definitive test before revascularisation. ICA was considered the reference standard test with perfect sensitivity and specificity. As ICA was performed on all patients indicated for revascularisation, the model did not consider any FP test results. The diagnostic component of the model divided the patient cohort according to their underlying disease status based on characteristics of patients in the CE-MARC trial,89 survival to interventional and diagnostic procedures, test results and subsequent clinical management conditional on test results. All patients with positive and inconclusive test results progressed to a further test in the sequence, although the type of the next test depended on whether the result was positive or inconclusive for some strategies (e.g. in strategy 8 a positive exercise treadmill test result would be followed by ICA, whereas inconclusive test results would be followed by a SPECT test). Patients whose overall testing sequence resulted in a positive result were managed with either PCI or CABG. The relative proportion of patients who underwent each type of revascularisation was sourced from UK clinical registries. Patients who tested negative at any point in the test sequence were managed with optimal medication if they had angina, or with no further medical therapy for those without angina. The decision tree captures mortality associated with both invasive tests and revascularisation, and separately applies procedure-specific mortality rates for ICA, PCI and CABG. At the end of the decision tree, patients with significant stenosis could be classified as TP, FN or dead. Patients without significant stenosis could be classified as TN with angina, TN without angina or dead. All testing strategies are assumed to take the same time and do not account for delays to revascularisation resulting from strategies that involve more tests.

The diagnostic accuracy estimates for the different tests considered in the alternative strategies were conditional on positive/inconclusive results in previous tests in the strategy, thus accounting for correlations between tests within diagnostic strategies. This is possible only with access to IPD from studies that include all the tests used across the full set of diagnostic strategies, as was the case for the CE-MARC study,89 which informed diagnostic accuracy in this model. However, the people interpreting each test were blinded to the results of previous tests in each diagnostic sequence, so the data would not have captured the influence of knowledge on previous tests on the diagnostic accuracy estimates of subsequent tests.

The long-term model is composed of three submodels. Patients with significant stenosis enter one submodel at either the TP or FN state. The key difference between TP and FN patients is that TP patients have undergone revascularisation. In the base-case analysis, the treatment effect of revascularisation is limited to a reduction from angina symptoms, with improved HRQoL for TP patients compared with FN patients, whereas the same baseline risk of cardiovascular events is applied for TP and FN patients. A proportion of FN patients are assumed be correctly diagnosed over time (conditional on their CCS grade), and transition to the TP health state. Patients can remain event free, have a primary non-fatal cardiovascular event, or die from a cardiovascular event or other causes. Patients who survive a primary non-fatal cardiovascular event transition to the non-fatal cardiovascular event state and have an increased risk of further cardiovascular events for 12 months, after which they transition to the non-fatal event post 12 months state. The risk of cardiovascular events in this state is lower than in the non-fatal event post 12 months state, but higher than the baseline risk (TP and FN states). Patients in all health states are subject to a mortality risk from non-cardiovascular death, which is sourced from UK life tables (with cardiovascular deaths removed to avoid double counting). A similar submodel to the one described above, this is used to estimate the cost and health outcomes of TN patients with angina. TN patients without angina go into a two health states (alive and dead) submodel that derived transition probabilities from sex- and age-adjusted UK life tables for all-cause mortality.

The probabilities of fatal and non-fatal cardiovascular events in the submodels for patients with angina were estimated based on risk equations from the EUROPA (EUropean trial on Reduction Of cardiac events with Perindopril in stable coronary Artery) trial. 90 This study estimated risk equations to predict (1) the risk of a first primary event, cardiovascular death, MI or cardiac arrest (see equation 1), (2) the odds of that event being fatal (see equation 2) and (3) the risk of a further primary event in the first year after a first non-fatal event (see equation 3). The equations allow for the adjustment of the rate of events dependent on the patient characteristics (age, sex, medication, comorbidities, etc.) and, importantly, accounting for the occurrence of previous MI. Walker et al. 77 applied a fourth equation to model the risk of secondary cardiovascular events, which captures the excess cardiovascular risk for patients who had had a previous MI.

The model also considers cancer-related mortality due to radiation exposure during some testing procedures (ICA and SPECT) and PCI (assumed to be performed at the same time as ICA). The model quantified the average radiation exposure in each test sequence; these radiation dosages were then combined with cancer incidence and mortality estimates from the literature to calculate lifetime incidence and mortality conditional on the patient’s age when they were tested. The costs and morbidity associated with cancer were not modelled.

The HRQoL in the model was dependent on age, sex, CCS grade and whether or not the patient had undergone revascularisation. EuroQol-5 Dimensions (EQ-5D) utility weights by CCS grade from a study on angina were combined with UK-population norm EQ-5D estimates by age and sex to obtain age- and CCS-specific HRQoL estimates. The underlying assumption was that the relative impact of CCS grade on HRQoL compared with the population is the same across all age groups.

One important base-case assumption of the Walker et al. 77 model is that revascularisation has no impact on the risk of cardiovascular events, and provides relief only from angina symptoms (captured by change in CCS score). HRQoL scores for patients with angina (with and without significant stenosis) are based on age- and sex-adjusted UK population scores with a relative adjustment made based on CCS grade. Data from a RCT comparing coronary angioplasty with medical management was used to link CCS scores at baseline and 6 months after intervention with the two treatments. Patients with angina and significant stenosis who receive revascularisation (TP) are attributed the HRQoL based on the average CCS grade of those following treatment with angioplasty conditional on initial CCS grade. Patients with angina and significant stenosis who are misclassified (FN) are attributed the HRQoL based on the average CCS grade of those following treatment with medical management conditional on initial CCS grade. It was assumed that angina patients without significant stenosis received the same HRQoL as FN patients, whereas the HRQoL of the other TN patients without angina was based on age- and sex-adjusted UK population scores.

Costs included in the model were those of tests and interventional procedures, treatment costs in the long-term model and health-state costs (namely fatal and non-fatal cardiovascular events, and other-cause mortality). Treatment and health-state costs were also sourced from the EUROPA trial90 (with a price year inflation adjustment). Background treatment costs were the same for all patients with angina and an additional background cost was applied for patients after a cardiovascular event. Patients without angina were assumed to have no costs in the long-term model.

The authors considered uncertainty by performing probability sensitivity analysis and scenario analysis where they varied assumptions on baseline characteristics (CCS grade, sex and age), prior likelihood of coronary heart disease requiring revascularisation, rediagnosis rate of FN patients, clinical management of TP patients, the impact of radiation exposure on cancer (risk assumed to be zero), risk of cardiovascular events following revascularisation (treatment effect from the EUROPA trial90), HRQoL decrements and the cost of diagnostic tests. The model was sensitive to prior likelihood of disease, reducing the starting age and increasing baseline CCS grade in the model, use of absolute HRQoL decrements by CCS grade, allowing for a proportion of TP patients to not receive revascularisation, reidentification rate of FN patients, and costs of tests. The prior likelihood of coronary heart disease requiring revascularisation was considered a key driver of cost-effectiveness.

Genders et al.74

The model developed by Genders et al. 74 was a microsimulation model comprising a decision tree and a lifetime state-transition model to assess the cost-effectiveness of invasive and non-invasive testing strategies for patients with stable chest pain. The base-case population consisted of 60-year-old patients with a 30% pretest probability of obstructive CAD (defined as ≥ 50% stenosis on at least one vessel) who had never undergone revascularisation procedures and had no prior history of CAD. The study presents cost-effectiveness results for the separate jurisdictions. We refer here to inputs and results specific to the analyses under the UK NHS perspective, as they are more relevant to our study. Costs were calculated in GBP (2011 price year), and health outcomes were calculated as QALYs. Both costs and QALYs were discounted at an annual rate of 3.5%.

The diagnostic strategies in the model are evaluated under two different diagnostic workups. In the invasive workup, patients with obstructive CAD on CCTA and patients with inducible ischaemia on cardiac stress imaging were referred for ICA prior to a decision regarding medical management. In the conservative workup, only patients identified as having higher CAD severity by CCTA or cardiac stress imaging would be referred to ICA and patients with milder forms of the disease managed with OMT. Patients with normal arteries or mild CAD (< 50% stenosis) received no further testing under either diagnostic workup.

The decision tree starts by classifying patients according to eight categories of disease severity based on percentage stenosis, number of vessels affected, location of lesion (left main trunk or not) and severity of inducible stenosis (where present). Patient distribution across disease severity categories was sourced from hospital records for patients who had undergone CCTA and ICA. Diagnostic accuracy estimates derived from published meta-analyses were then applied to split patients according to the test results for each diagnostic strategy. For the purpose of applying these estimates, patients who were considered correctly classified with a negative result had normal coronary arteries or mild CAD (< 50% stenosis). Patients correctly identified with a positive result had moderate CAD, severe CAD or three-vessel disease/left main coronary stenosis. The model did not consider inconclusive test results. The authors assumed independence of diagnostic accuracy estimates for CCTA and cardiac stress imaging, and further assumed that FP results were possible only for mild CAD and mild inducible ischaemia (under the conservative diagnostic workup). Patients with FP results are assumed to receive unnecessary optimal medication for the full time horizon, incurring a treatment cost and utility decrement in the long-term model. As in Walker et al. ,77 adverse events from testing and revascularisation procedures were considered. However, in this model adverse events are not limited to procedural mortality, but also include non-fatal MI with ICA. This adverse event had a cost attributed to it, but did not translate into an increased risk of further events in the long-term model.

The decision tree splits the patient population according to disease severity, test results and survival to testing (ICA and FFR) and revascularisation procedures. It also allows quantifying the average exposure to radiation with the different tests and PCI.

In the ICA strategy, all patients were tested with ICA. Those who tested negative received risk factor management and those who tested positive would be tested with FFR to decide treatment. ICA is assumed to be a perfect test, and FFR appears to allow prefect distinction between disease severity categories, although this is not explicitly stated in the paper. OMT was then given to patients with mild ischaemia and moderate to severe CAD, PCI was given to patients with severe CAD and severe ischaemia, and CABG was given to patients with three-vessel disease or left main coronary stenosis. Revascularised patients would also receive OMT, and all individuals in the model received risk factor management.

Subsequent to the decision tree, patients entered a state-transition model comprising three health states: alive, post MI and dead. Patients enter the model through the alive state, where they could remain until death or suffering a non-fatal MI. Patients who suffered a non-fatal MI would transition to the post-MI state, where they could remain or transition to the dead state. The transition probabilities were derived from published trial data that reported risk of MACE (cardiovascular death, non-fatal MI and repeated revascularisation) in patients treated with CABG, PCI and OMT. The rates of MACE were dependent on disease severity and whether patients were treated with optimal medication or revascularisation. All FN patients were assumed to be correctly identified and treated by the end of the first year, with the exception of those with moderate CAD without ischaemia, of whom only 25% were rediagnosed. Patients who experienced a primary cardiovascular event would have a higher risk of subsequent cardiovascular events, which was modelled by applying a HR of 1.44 to their baseline risk. The model also considered mortality from non-cardiovascular causes. This was estimated based on age- and sex-specific general mortality data from which deaths attributed to cardiovascular causes had been removed to avoid double counting. The mortality, morbidity and costs due to cancer incidence were not modelled, although the model calculated cumulative radiation exposure over the time horizon.

The risk of MACE was estimated from the trial data separately for the first year and all subsequent years to allow for a higher event rate in the first year after starting treatment. The rates were estimated based on the CABG arm of the SYNTAX trial59 for patients with three-vessel disease or left main coronary stenosis, and the optimal medication and PCI arms of the COURAGE trial91 for the patients with suspected or mild inducible ischaemia and moderate to severe CAD (treated with optimal medication) and patients with severe CAD and severe inducible ischaemia (treated with PCI). The reciprocal of the treatment HR was applied to this risk to estimate the baseline probability of cardiovascular events for untreated patients (FN), who have a higher rate of events until they are correctly diagnosed. A single treatment effect hazard for optimal medication, PCI and CABG (HR 0.70) was sourced from three meta-analyses of OMT comparing treatment with no treatment, but it is unclear how this estimate was calculated. The rates of MACE applied in the model are summarised in Appendix 7, Table 57.

We note that the MACE rates without treatment seem counterintuitive (e.g. higher MACE rates for untreated moderate CAD with mild ischaemia compared with untreated severe CAD with severe ischaemia). The authors did not comment on the MACE rates.

If the treatment effect applied in the model is indeed the same for optimal medication and revascularisation, this is similar to the absence of a treatment effect of revascularisation in addition to optimal medication in Walker et al. 77 This is an important interpretation of the clinical evidence on the treatment effect of revascularisation, and one that is discussed further in Chapter 5, Treatment effect of revascularisation. In previous studies where a treatment effect on the rate of cardiovascular events for revascularisation compared with optimal medication was considered explicitly for comparable patients (e.g. same disease severity), seven models included the existence of a treatment effect68,69,76,83,85,86,88 and six studies did not,70–72,79,84,87 in line with Walker et al. 77 and Genders et al. 74

The HRQoL in the model was assigned to individuals according to disease severity and treatment received. Patients without CAD or inducible ischaemia were assumed to have the HRQoL of the general population based on age- and sex-specific EQ-5D estimates for the US population. For patients with CAD and inducible ischaemia who underwent active treatment (optimal medication or revascularisation), mapped EQ-5D utility decrements were applied to the general population HRQoL estimates. In the first year of treatment, the utility decrements of treatment relative to the general population were derived from the average utility decrement as observed in the same trial data that informed the rates of MACE for treated patients, whereas for the subsequent year the last observed value in the trials was carried forward. The authors state that a disutility was considered for patients with FP results. It was not clear how the utility decrements for FN patients were estimated. Appendix 7, Table 58, summarises the utility values for the start age in the model conditional on treatment and disease severity. The HRQoL estimates for the first year and subsequent years of treatment are presented for the same age solely for ease of comparison.

The model considers costs of tests, test adverse events, medication, MI in the long-term model, and incidental findings from CCTA. Unit costs were mostly sourced from UK published data. Based on the description of the unit cost selected for PCI, this procedure was assumed to take place in an outpatient setting. It is not, however, clear what assumptions were made regarding the setting for ICA, CABG and treatment of non-fatal MIs. The unit cost for FFR was sourced from a previous cost-effectiveness study in a US setting. 87 An annual cost of medication was included in the model according to disease severity and treatment received (OMT, PCI or CABG). The resource use assumed for patients who received optimal medication alone and in addition to PCI was sourced from the COURAGE trial,91 whereas for those who received CABG and optimal medication it was taken from the SYNTAX trial. 59 The distribution of medication use applied in the model is shown in Appendix 7, Table 59.

Model parameters were entered as distributions, and probabilistic sensitivity analysis was performed to incorporate joint parameter uncertainty. Scenario analysis was performed to test assumptions on diagnostic accuracy of stress echocardiography, cost of tests, alternative diagnostic pathways, probability of CAD, time to rediagnose FN patients, and treatment effect of optimal medication for FP patients. A subgroup analysis by sex was also performed. The authors do not identify any drivers of cost-effectiveness, but note that the assumption that FP patients will remain misclassified over the time horizon and that FN patients will be rediagnosed after 1 year is likely to have biased results against strategies with low specificity.

Diagnostic model

Diagnostic outcomes are modelled with a decision tree, which takes account of the diagnostic accuracy of the tests and subsequent treatment pathway. The decision tree is constructed to compare the TP, FP, TN and FN rates of the alternative diagnostic strategies.

The decision tree starts with the alternative diagnostic strategies that are used to diagnose the functional significance of stenosis. The outcomes of each strategy are governed by the sensitivity and specificity of the particular test strategy. The accuracy of the tests is defined independently of disease prevalence (i.e. underlying prevalence of functionally significant stenosis in the population); however, the expected proportion of tests with positive and negative results in the population is dependent on the underlying prevalence. Therefore, for all strategies, patients are separated into their ‘true’ status of either functionally significant stenosis or functionally non-significant stenosis based on the distribution of the population with a FFR value ≤ 0.8. Patients with functionally significant stenosis requiring revascularisation are allocated to one of three outcome states as a result of the diagnostic strategy: (1) TP, who are correctly identified and treated with revascularisation, (2) FN, who are misidentified and do not receive revascularisation, and (3) death as a result of the mortality risks associated with the diagnostic and revascularisation procedures. Patients without functionally significant stenosis who do not require revascularisation are also allocated to one of three outcome states as a result of the diagnostic strategy: (1) TN, who are correctly identified and treated with OMT, (2) FP, who are misidentified and receive an inappropriate revascularisation, and (3) death as a result of the mortality risks associated with the diagnostic and revascularisation procedures. For revascularisation, a proportion of patients are assumed to be treated with either PCI or CABG, in addition to OMT.

A schematic of the diagnostic model for each of the five strategies is shown in Figures 12–16 (with outcome of death not shown in the figures for simplicity). Strategies 1 (ICA alone), 3 (ICA plus QFR) and 5 (ICA plus vFFR) have four possible diagnostic test results of TP, FN, FP and TN based on the diagnostic accuracy of the tests relative to the reference standard test of FFR ≤ 0.8. Strategy 2 is the reference standard test (ICA plus FFR), and TP and TN are the only diagnostic outcomes under this strategy because FFR/iFR is assumed to be a perfect test (100% sensitivity and specificity). Strategy 4 is the hybrid approach of QFR where the possible diagnostic outcomes are TP, FN, FP and TN for those considered to have conclusive QFR (values outside the grey zone), whereas TP and TN are the only possible outcomes for those with inconclusive QFR (grey zone) because these patients undergo confirmatory FFR/iFR. Death can occur as a result of the diagnostic and revascularisation procedures.

FIGURE 12.

Strategy 1 of ICA alone, without additional testing to assess the functional significance of stenosis.

FIGURE 13.

Strategy 2 of ICA, followed by confirmatory FFR/iFR.

FIGURE 14.

Strategy 3 of ICA with QFR.

FIGURE 15.

Strategy 4 of ICA with QFR, followed by confirmatory FFR/iFR when QFR is inconclusive.

FIGURE 16.

Strategy 5 of ICA with vFFR.

The model assumes that all diagnostic tests in each strategy are performed in the same medical appointment, and that revascularisation procedures are performed either immediately after testing or without a delay that might lead to a deterioration of the patient’s condition. Therefore, the base-case analysis is more representative of an interventional setting. The assumption that all diagnostic tests in each strategy are performed in the same medical appointment is relaxed in a scenario analysis so as to explore the cost-effectiveness of the strategies in a diagnostic-only setting.

For the diagnostic model, costs are incurred according to the type of diagnostic test, adverse events associated with FFR, and treatment received. Procedural HRQoL loss is included for FFR and revascularisation. Costs, HRQoL and mortality effects associated with ICA are excluded from the model because these are incurred equally across all strategies.

The diagnostic model represents the start of the long-term prognostic model. The proportion of patients starting in the health states in the prognostic model is based on the expected proportion of tests with positive and negative results in the population (TP, TN, FP, FN).

Prognostic model

The prognostic implications of receiving treatment (either revascularisation in addition to OMT or OMT alone) based on being in one of the four diagnostic outcome states (TP, FN, FP or TN) is quantified using a Markov model that captures the progression of disease through the risk of MACEs and associated costs and consequences, and the risk of death from non-cardiac causes, over a lifetime horizon. A cycle length of 1 year is used in the model. A schematic of the model structure is shown in Figure 17.

FIGURE 17.

Schematic of prognostic model. CV, cardiovascular.

Patients with stable CAD and intermediate stenosis enter the model in one of four diagnostic outcome health states: TN (functionally non-significant stenosis with FFR > 0.8 and received OMT), FN (functionally significant stenosis with FFR ≤ 0.8 and received OMT), TP (functionally significant stenosis with FFR ≤ 0.8 and have undergone revascularisation), FP (functionally non-significant stenosis with FFR > 0.8 and have undergone revascularisation) or death.

All patients alive may remain in their initial health state over time with no MACEs, or have a primary MACE. The primary MACE is defined as cardiovascular death, non-fatal MI and unplanned (urgent) revascularisation, where the risk differs between the first year and subsequent years from model entry. Each MACE is assumed to be mutually exclusive. If the primary MACE is fatal, the patient enters an absorbing state of cardiovascular death. If the event is a non-fatal MI, they enter the post-MI health state where the risk of a subsequent cardiovascular event (cardiovascular death or MI) is increased as a result of having had a previous MI. Patients in the post-MI state are assumed to not be at further risk of unplanned revascularisation. If the primary MACE is an unplanned revascularisation, it is assumed that patients will receive an appropriate revascularisation procedure, with the same associated costs and risks as those patients who entered the model in the TP state (i.e. it is assumed that patients who require a subsequent targeted revascularisation have the same risk of MACE as patients who enter the model in the TP state, under the assumption that the need for urgent revascularisation is indicative of functionally significant stenosis with FFR ≤ 0.8). Patients enter the unplanned revascularisation event state for one cycle only and, for the following 12 months, are at an increased risk of cardiovascular death or MI compared with subsequent years. Those who have a cardiovascular death or non-fatal MI during this period enter the cardiovascular death state or post-MI state, respectively. If no event occurs in 12 months following an unplanned revascularisation, the patient moves into the post-revascularisation health state, where the risk of cardiovascular death or MI relates to the risk post 12 months from the TP state. From any of the states in the model where patients are alive, there is a competing risk of a non-cardiovascular death.

Patient population

The population consists of patients with stable CAD whose angiograms taken during ICA show intermediate stenosis. The age and sex distribution varies across the studies informing the diagnostic accuracy of the technologies (mean age ranging from 61 to 72 years and proportion of males from 67% to 81%). The IRIS-FFR registry is the largest registry to investigate the prognosis of coronary stenosis assessed by FFR. The mean age and proportion of males in the IRIS-FFR registry was 64 years and 72%, respectively. These values are within the range reported in the diagnostic accuracy studies and the patient populations of the largest RCTs undertaken to evaluate clinical outcomes of revascularisation for patients with stable CAD [e.g. the International Study of Comparative Health Effectiveness With Medical and Invasive Approaches (ISCHEMIA),95 mean age 64 years and 77% male; the Bypass Angioplasty Revascularization Investigation 2 Diabetes (BARI 2D) trial,96 mean age 62 years and 70% male; and the COURAGE trial,91 mean age 62 years and 85% male]. These studies, however, were mainly or wholly undertaken outside the UK. The smaller ORBITA (Objective Randomised Blinded Investigation with optimal medical Therapy of Angioplasty in stable angina) trial,97 which enrolled patients with stable angina and angiographically severe single-vessel CAD at five UK sites, had a mean age of 66 years and was 73% male. The IRIS-FFR registry includes mostly patients with stable angina (76%), although 18% had unstable angina and nearly 6% had NSTEMI/STEMI.

Given that the IRIS-FFR registry is the largest registry to investigate the prognosis of coronary stenosis assessed by FFR and that the mean age and proportion of males is very similar to the ORBITA trial,97 this is used to inform the base-case population in the model (mean age 64 years and 72% male).

Setting

In UK clinical practice, ICA can be used in two settings: diagnostic-only and interventional catheter laboratories. QAngio XA 3D/QFR and CAAS vFFR can be used in the same settings as ICA, whereas FFR/iFR is currently performed only in an interventional setting.

The base-case analysis assumes that the tests in each of the five diagnostic strategies are performed at the same medical appointment and, thus, implicitly assumes an interventional setting. Although the large majority of ICA procedures in the NHS are conducted in an interventional setting (see Appendix 8, Table 60),98 it is important to consider the differences between settings that may affect the cost-effectiveness of the strategies under comparison.

The key difference between diagnostic-only and interventional catheter laboratories is that FFR/iFR can be performed only in the latter. In a diagnostic-only setting, strategies that include FFR/iFR in the testing sequence (strategies 2 and 4) require that at least some patients undergo two separate diagnostic catheterisation procedures. The initial catheterisation corresponds to the ICA that is common to all strategies in the model and is performed in the diagnostic-only catheter laboratory. Patients who have an inconclusive QFR measurement with QAngio XA 3D/QFR (patients in the grey zone for strategy 4) and all patients in strategy 2 are referred to an interventional catheter laboratory where they undergo a second catheterisation to obtain a FFR/iFR measurement. This is in contrast with how patients would be tested in an interventional setting, as all tests could be performed with a single catheterisation, and at the same point in time. One of the implications of conducting two separate catheterisations is that strategies that involve FFR/iFR will be more costly in a diagnostic-only setting than in an interventional setting. Another potential consequence of this is that the condition of patients initially tested in a diagnostic-only setting deteriorates while waiting for the referral to the interventional catheter laboratory and subsequent clinical management with revascularisation where appropriate. However, the delays to patient management are unlikely to result in significant condition deterioration with impact on patients’ health outcomes (Dr Gerald Clesham, personal communication). This is supported by evidence of the ORBITA trial97 (see Treatment effect of revascularisation), which showed that PCI compared with a placebo (mock PCI) did not demonstrate a statistically significant increase in the exercise time or change in HRQoL of patients with medically treated angina and severe coronary stenosis at 6 weeks post procedure. 97

Another difference between settings is the expected annual patient throughput in diagnostic-only compared with interventional catheter laboratories. Appendix 8, Table 60, shows that, on average, 584 patients undergo ICA in a diagnostic-only setting compared with 2093 in an interventional setting. However, the proportion of patients with stable CAD who undergo ICA in a diagnostic-only setting is likely to be higher than in an interventional catheter laboratory. The expected patient throughput in a diagnostic-only setting is unknown but needs to be considered when evaluating the cost-effectiveness of the alternative diagnostic strategies, as it determines the cost of the QAngio XA 3D/QFR and CAAS vFFR tests. This is discussed further in Test costs.

It is possible that the diagnostic accuracy of QAngio XA 3D/QFR and CAAS vFFR is also linked to the diagnostic setting. The diagnostic accuracy studies identified in Chapter 3 do not provide evidence to ascertain whether or not there are any differences in the diagnostic accuracy of these technologies across settings, as the patient population in the review may not represent patients examined in a diagnostic-only setting. This is because only patients with a FFR measurement could be included in the diagnostic accuracy review (see Chapter 3, Selection criteria).

Finally, it is possible that there are additional training requirements for QAngio XA 3D/QFR and CAAS vFFR in diagnostic-only catheter laboratories, as staff in these centres may need more training and support to correctly calculate and interpret the QFR and vFFR measurements. It is, however, uncertain what resource use is associated with these additional training requirements, so this could not be reflected in the cost-effective analysis.

The base-case scenario assumes that all diagnostic procedures take place in an interventional setting. The diagnostic-only setting is considered in scenario analyses where the impact on cost-effectiveness estimates of the following is explored: (1) additional costs due to the need to refer patients who require FFR/iFR measurements to an interventional catheter laboratory and (2) alternative throughput assumptions (see Results of the alternative scenario analyses).

Diagnostic accuracy

The model considers the diagnostic accuracy of ICA, QFR and vFFR, and FFR/iFR is the reference standard test with 100% sensitivity and specificity. The diagnostic accuracy of iFR is assumed to be equivalent to that of FFR. The definition of functionally significant stenosis is based on a FFR value ≤ 0.8. The following sections present the diagnostic accuracy estimates used in the model.

Procedural adverse events

Procedures involving catheterisation for diagnostic testing (ICA and FFR/iFR) or revascularisation (PCI and CABG) have associated complications that may result in health-care resource and HRQoL loss. The diagnostic model considers the impact of serious procedural complications from FFR/iFR and revascularisation. The procedural complications of ICA are excluded from the model because all patients undergo this procedure in all strategies and, therefore, procedural complications associated with ICA do not result in differences in costs and HRQoL across strategies.

The diagnostic pathway explicitly distinguishes between complications associated with invasive testing with FFR/iFR and revascularisation so that the potential benefits of less invasive testing can be captured (i.e. non-invasive testing with QFR and vFFR in strategies 3 and 5, respectively). However, revascularisation is often performed immediately after FFR/iFR and, therefore, the rates are often not reported separately in the literature by type of procedure.

Risk of major adverse cardiac events and treatment effects of revascularisation

Other-cause mortality

Mortality due to non-cardiovascular causes was estimated based on age- and sex-specific UK life tables120 and by deducting the mortality due to ischaemic heart disease (International Classification of Diseases, codes 20–25). 121 The age-specific probability of death was estimated as a weighted average of male and female mortality.

Health-related quality of life

To estimate QALYs, it is necessary to quality-adjust the period of time for which the average patient is alive within the model using an appropriate utility or preference score. QALYs are calculated by summing the time spent in a health state weighted by the utility value associated with the health state. Additional adjustments are made to QALYs to reflect a decrement in utility associated with an acute or adverse event. The NICE methods guide advocates a preference for EQ-5D data with utility values using UK population weights when available. 92

In the diagnostic model a one-off utility decrement is applied to patients undergoing invasive FFR/iFR and those who undergo revascularisation (known as procedural disutility). At the end of the diagnostic model, patients who survive enter the long-term prognostic model in one of the four health states of TP, FN, FP or TN. The implications on HRQoL of receiving treatment (either revascularisation in addition to OMT or OMT alone) based on being in one of the four diagnostic health states is quantified by attaching a utility value to each of the health states. A one-off utility decrement is also applied in the prognostic model to those who experience a non-fatal MI or require an unplanned revascularisation. To reflect a decrease in HRQoL for those with a history of MI, a separate utility decrement is applied to the post-MI health state. For those patients who experience an unplanned revascularisation, the utility value associated with the TP health state is applied based on the assumption that patients who undergo a targeted revascularisation achieve the same benefits of revascularisation, in terms of symptom relief, as patients who had a successful initial revascularisation procedure.

The utility values are used to calculate the expected number of QALYs for each diagnostic strategy over the duration of the model.

Resource use and costs

This section details the resource use and costs applied in the model. The diagnostic model considers the costs of diagnostic testing, revascularisation and treatment of procedural complications. The prognostic model considers the costs of OMT, health state and clinical events. Costs in the model are fixed estimates. Details by category of resource use and costs are presented in the sections below.

Overview

The cost-effectiveness of the QAngio XA 3D/QFR and CAAS vFFR imaging software used during ICA for assessing the functional significance of coronary obstructions in patients with intermediate stenosis is evaluated by comparing the total expected costs and QALYs with those obtained using pressure wire FFR/iFR measurement or visual interpretation of angiographic images alone.

Summarising cost-effectiveness using conventional incremental cost-effectiveness ratios (ICERs) requires consideration of which pairwise comparisons are appropriate when calculating incremental costs and effects. With multiple strategies there are several different comparisons that could be made, each resulting in different incremental QALYs, costs and ICERs. In this case, a fully incremental ICER comparison is required to rule out any strategies based on the principles of dominance or extended dominance. However, even with a fully incremental ICER comparison, strategies that are dominated and would not be considered cost-effective may be ‘better’ than other non-dominated (but not cost-effective) strategies. The ICER comparison is also particularly sensitive to small differences between the strategies (e.g. small changes to the denominator in terms of QALY differences may result in highly variable ICERs). For these reasons, the cost-effectiveness of the interventions is summarised in terms of net benefit, which applies a single unambiguous decision rule when there are multiple strategies; the cost-effective strategy is the one with the highest net benefit for a given cost-effectiveness threshold.

In this analysis, the net benefit is expressed in terms of net health benefit (NHB), estimated by rearranging the ICER equation into the following equation:

Strategies are ranked in terms of NHB from highest to lowest, which is used to identify the cost-effective strategy (highest NHB), but NHB is also used to interpret the next best choice (second highest NHB) and so on. A cost-effectiveness threshold of £20,000 per additional QALY is used in the analysis.

Uncertainty in the estimates of cost-effectiveness of the alternative strategies is reflected in probabilistic analysis, and the probability that each strategy is more cost-effective than the others at the cost-effectiveness threshold of £20,000 per QALY is presented for the base-case scenario.

Base-case analysis

The base-case parameters and associated assumptions are listed alongside their sources in Appendix 8, Table 73.

Scenario analyses

Alternative assumptions to those of the base-case analysis are made in a number of scenarios analyses. The aim of these analyses is to assess the robustness of the base-case results to alternative assumptions and sources of data used to parametrise the model.

Table 13 summarises the alternative scenarios analyses undertaken. The position in the base-case analysis and the alternative assumption applied is described for each element.

Model validation

The model was developed in Microsoft Excel by two analysts (AD and LS) and the programming checked by a third analyst (CR). As part of an overall quality assurance process, the internal validity of the model was assessed by extensively exploring logical consistency in the model results. A separate version of the prognostic model was also independently programmed by the third analyst (CR) to successfully replicate the base-case results.

Results of the base-case scenario

Deterministic and probabilistic cost-effectiveness results expressed in terms of NHB at a cost-effectiveness threshold of £20,000 per additional QALY for the base-case scenario are presented in Tables 14 and 15, respectively. Strategy ranking from highest to lowest NHB is presented in both tables. The incremental net health benefit (INHB) is calculated for each strategy relative to ICA alone. The probability that each strategy is cost-effective at a threshold of £20,000 per additional QALY is presented in Table 15 for the probabilistic analysis. The results are consistent for both the deterministic and probabilistic analyses.

The strategy with the highest NHB is strategy 2 (ICA plus FFR) but the difference between all the strategies is small. Strategy 2 is also the strategy with the highest probability of being cost-effective (27.8%). The least costly strategy is strategy 1 (ICA alone), which also has the lowest QALY gain, whereas the most costly strategy is strategy 5 (ICA plus vFFR) but this has the highest QALY gain. The INHB per patient diagnosed for each of the strategies relative to ICA alone is 0.027 QALYs (or equivalently £544) for ICA plus FFR; 0.020 QALYs (or equivalently £400) for ICA plus QFR; 0.015 QALYs (or equivalently £298) for ICA plus QFR, followed by confirmatory FFR when QFR is inconclusive; and 0.016 QALYs (or equivalently £316) for ICA plus vFFR.

To understand the difference in NHB between the alternative strategies, the disaggregated results for total expected costs and QALYs are informative. Table 16 shows the expected costs and QALYs for each strategy from the diagnostic and prognostic components of the model separately. The diagnostic model includes the costs of the diagnostic tests, revascularisation, and costs associated with treating adverse events related to FFR, and the prognostic model includes costs related to unplanned/repeat revascularisation, MI events and long-term OMT use. The diagnostic model includes a procedural disutility associated with invasive FFR/iFR and revascularisation, and the prognostic model includes the long-term symptom-free benefits of revascularisation, a disutility associated with unplanned/repeat revascularisation, MI events and history of MI, applied to a baseline utility for patients with intermediate stenosis. Note that costs, adverse events and disutility associated with invasive ICA are excluded from the model because all strategies undergo ICA. The disaggregated costs and QALYs from the diagnostic component of the model are shown in Table 17, and the proportion of patients who enter the long-term prognostic model in each of the TN, FN, TP and FP entry states for each of the alternative strategies (based on the diagnostic accuracy results) is shown in Table 18.

Certain strategies have among the highest diagnostic model costs, as they result in a greater proportion of revascularisations. The percentage of revascularisation is dependent on the rate of TP and FP results. The PPV is highest for strategies involving QFR [strategies 3 (84.8%) and 4 (86.8%)], compared with vFFR (71.5%) or ICA alone (52.3%), whereas FFR is assumed to have perfect PPV. This means that there are more unnecessary revascularisations for vFFR and ICA alone, than for QFR or FFR, which increases the costs of revascularisation for these tests. Some of the total diagnostic model cost is offset by differences in costs of testing. The costs of the diagnostic tests are dependent on the level of patient throughput, and for the base-case scenario (throughput assumed to be 200) vFFR has lower costs (£172 per patient tested) than QFR (£431) and FFR (£437). The cost of adverse events associated with FFR increases the total cost of FFR but this is very marginal because of a low likelihood of adverse events occurring. The difference in total diagnostic model costs between strategies 2 (ICA plus FFR) and 3 (ICA plus QFR) is £15 per patient diagnosed, whereas the total diagnostic model cost for strategy 4, with the addition of FFR when QFR is inconclusive, increases the cost of a QFR strategy by £215 per patient because of a higher rate of TP results. The difference in total diagnostic model costs of strategy 5 relative to strategies 2 and 3 is £314 and £329 per patient, respectively, and this is because of a higher percentage of revascularisation as a result of more FP test results. The procedural QALY loss associated with invasive FFR and revascularisation ranges from 0.0037 for ICA plus QFR (no additional adverse events for QFR relative to ICA) to 0.0093 QALYs for ICA plus FFR.

The difference in total costs between the strategies from the prognostic model is much smaller than the diagnostic model because the base-case scenario assumes that there is no treatment effect associated with revascularisation on the rates of MACE. The only difference between the strategies in terms of costs associated with MACE is because of the inverse relationship between underlying FFR and subsequent risk of MACE, that is those with lower FFR have a higher risk of a cardiovascular event, whereas those with higher FFR have a lower risk of a cardiovascular event. The prognostic model also includes a difference in annual costs for OMT between those who receive OMT in addition to revascularisation (£150 for PCI and £126 for CABG) and those who receive OMT alone (£163 for TN and £169 for FN). Strategies 5 (ICA plus vFFR) and 1 (ICA alone) have the lowest prognostic costs, which is mainly due to the lower total costs of OMT associated with a greater number of revascularisations in these strategies. The difference in total prognostic costs between strategy 3 (ICA plus QFR), with the highest cost, and strategy 5 (ICA plus vFFR), with the lowest cost, is £22 per patient.

The difference in total QALYs between the strategies from the prognostic model is largely due to the HRQoL gain associated with TP test results. The base-case scenario assumes that there is no change in baseline HRQoL associated with TN and FP test results, whereas there is a small non-statistically significant change associated with FN test results. This means that the total expected QALYs is greater for strategies with more TP test results (better sensitivity), whereas there are no HRQoL benefits associated with more TN test results (better specificity). As a result, strategies 2 (ICA plus FFR) and 5 (ICA plus vFFR) have the highest prognostic model QALY gains, and strategy 1 (ICA alone) has the lowest. These QALY gains, however, are offset by the disutility associated with diagnostic testing that is highest for strategies 2 and 5 and lowest for strategy 1.

The benefits of revascularisation, in terms of improved HRQoL, mean that the sensitivity of test results is a more important driver of cost-effectiveness than specificity because TP test results translate into higher QALY gains than mismanagement of FN results. The base-case cost-effectiveness results are largely driven by the balance between the costs of the diagnostic tests and the costs and benefits of revascularisation.

Results of the alternative scenario analyses

Scenarios 23 and 24: diagnostic-only setting

Setting details key differences between diagnostic-only and interventional settings. Scenarios 23 and 24 reflect the diagnostic-only setting by considering (1) the additional costs due to the need to refer patients who require FFR/iFR measurements to an interventional catheter laboratory and (2) alternative throughput assumptions.

In scenario 23, the unit cost of FFR/iFR corresponds to the cost of a complex catheterisation (£2202) so as to account for the additional catheterisation that would be required under this scenario (see Invasive coronary angiography and fractional flow reserve). In the base-case scenario the cost of FFR/iFR only includes the incremental cost of FFR/iFR compared with ICA (£437), as a single catheterisation allows both procedures to be performed. Scenario 24 builds on the assumptions of scenario 23 on the cost of FFR/iFR and further assumes an average annual throughput of 500 patients per diagnostic-only centre. This is the average annual throughput value at which (with everything else held equal to the base-case scenario) strategies 3 and 5 became equivalent in terms of NHB, as identified by a previous exploratory analysis. This throughput estimate is close to the average annual number of ICA procedures per diagnostic-only centre (584; see Model input parameters, Patient population, Patient throughput). Cost-effectiveness results for scenarios 23 and 24 are presented in Table 32.

In scenario 23, the cost of testing increases considerably for strategies 2 (£1765) and 4 (£357) compared with the base-case scenario. The sharp increase in cost for these strategies reduces their NHB, with strategy 1 now having the lowest NHB followed by strategy 4. In this scenario, strategy 3 has the highest NHB across all strategies.

In scenario 24, in addition to the changes to the testing costs of strategy 2 and 4 found for scenario 23, the increase of annual patient throughput reduces the testing costs of strategy 5 (£99 less than in the base-case scenario), and the testing costs of strategy 3 remain £431. The difference in total costs between strategy 5 and strategy 3 is reduced to £206, which is offset by the QALY gains of strategy 5 compared with strategy 3 (0.011). In this scenario, strategy 5 has the highest NHB across all strategies, but this result needs to be interpreted cautiously given the uncertainties in the diagnostic accuracy of CAAS vFFR.

The deterministic cost-effectiveness results of the base-case and scenario analyses are summarised in Appendix 9, Table 74.

Diagnostic accuracy

The diagnostic accuracy of QFR has been widely studied, with 39 studies in this review, including 5940 patients (over 7043 vessels or lesions). QFR, as measured using QAngio, is highly correlated with FFR measured with an invasive pressure wire. The average difference between FFR and QFR measurements is almost zero, and they rarely differ by more than 0.1, with about 50% of measurements differing by less than 0.04.

QAngio XA 3D/QFR at a cut-off point of 0.8 has good diagnostic accuracy to predict FFR (also at a cut-off point of 0.8), cQFR mode had a sensitivity of 85% (95% CI 78% to 90%) and a specificity of 91% (95% CI 85% to 95%) and fQFR mode had a sensitivity of 82% (95% CI 68% to 91%) and a specificity of 89% (95% CI 77% to 95%). Although there is some discordance between QFR and FFR, most FPs or FNs arise near the boundary (e.g. where one is 0.81 and the other 0.79), and the discordance may not be clinically meaningful. Data on how this accuracy may vary by key patient characteristics was very limited, and no conclusive variation could be found.

The use of a ‘grey-zone’ strategy, where patients with a QFR between 0.78 and 0.84 receive confirmatory FFR, improves diagnostic accuracy, compared with using QFR alone, to a sensitivity of 93.1% and a specificity of 92.1%. However, this improvement is dependent on assuming that the exact FFR cut-off point of 0.8 is clinically meaningful. Most FFR and QFR values differ by 0.05 or less; therefore, the grey-zone approach is mainly identifying discordant FFR and QFR results very close to the 0.8 boundary; 30.4% of patients with QFR results in the grey zone have results that are discordant with their FFR.

Data on the diagnostic accuracy of CAAS vFFR were limited to only three studies. Owing to variable reporting of results and apparent substantial heterogeneity in results across studies, a full meta-analysis was not feasible.

Although assessing the diagnostic accuracy of using standard ICA alone was not the focus of this report, studies that reported data on ICA, and targeted searches for additional data, found that ICA alone had poor diagnostic accuracy when compared with FFR. All studies that compared QFR with ICA found QFR to be superior in diagnostic accuracy.

Clinical value and implementation

This review found limited evidence on the clinical impact of using QFR. The use of a grey zone could significantly reduce the proportion of adenosine and pressure-wire-free procedures compared with universal use of FFR, without significantly affecting diagnostic accuracy. Evidence on the applicability of QAngio XA 3D/QFR suggests that the technology is applicable in a clinical context.

Given the limitations in the evidence, a simulation study was performed to investigate the impact of using QFR, with or without a grey zone, on future coronary events. The simulation found that using QFR slightly increased the revascularisation rate compared with using FFR for all, from 40.2% to 42%. Using a grey-zone strategy increased it further to 43.2%. However, all three strategies had similar numbers of resulting coronary events, suggesting that all have a broadly similar benefit when making decisions as to who should receive revascularisation.

Although CAAS vFFR appears promising, its clinical value is currently uncertain because of limited evidence and a lack of on-site prospective studies.

Cost-effectiveness

The base-case cost-effectiveness results showed that the test strategy with the highest net benefit (most cost-effective strategy) was ICA followed by confirmatory FFR/iFR (strategy 2) at a cost-effectiveness threshold of £20,000 per QALY gained. However, the difference in net benefit between this strategy and the next best strategies was relatively small at 0.007 QALYs (or equivalently £140) per patient diagnosed for ICA with QFR (strategy 3), 0.012 QALYs (or equivalently £240) per patient diagnosed for ICA with QFR followed by confirmatory FFR/iFR when QFR is inconclusive (strategy 4) and 0.011 QALYs (or equivalently £220) per patient diagnosed for ICA with vFFR (strategy 5).

A number of alternative scenarios were considered in which the assumptions used as part of the base-case results were varied. These alternative scenarios showed that the cost-effectiveness results were robust to the mode of QFR measurement (contrast-flow QFR or fixed-flow QFR), the use of an alternative diagnostic threshold of 0.75 for FFR and QFR, the use of a wider definition of the grey-zone region for confirmatory FFR/iFR when QFR is inconclusive, throughput assumptions for QFR and vFFR, alternative estimates of procedural complication rates for FFR/iFR, and dependency of MACE risk on FFR. The scenarios were also used to identify the main drivers of cost-effectiveness. The key drivers identified were (1) the sensitivity (rather than specificity) of test results because TP test results translated into higher QALY gains than mismanagement of FN test results, (2) the procedural QALY loss associated with FFR/iFR, (3) the magnitude and duration of the QALY gains associated with revascularisation and (4) the additional costs associated with confirmatory testing with FFR/iFR in strategy 4. Strategy 1 of ICA alone, without additional testing, appeared cost-effective relative to the other strategies only when it was assumed that there were no benefits of revascularisation.

Overall, the differences in net benefit at a cost-effectiveness threshold of £20,000 per QALY between ICA followed by confirmatory FFR/iFR (strategy 2), and ICA with QFR (strategy 3) are small in an interventional setting. When considering a diagnostic-only setting, ICA with QFR may result in higher net benefit at a cost-effectiveness threshold of £20,000 per QALY than strategy 1, assuming QAngio XA 3D/QFR has similar diagnostic accuracy across settings.

Strengths

This review includes a comprehensive systematic review of all the published literature on QFR as assessed by QAngio XA 3D/QFR and CAASS vFFR technologies, and has been conducted following recognised guidelines to ensure high quality.

The review identified a substantial literature on the diagnostic accuracy of QAngio XA 3D/QFR (37 studies and > 5000 patients) and, despite evidence of heterogeneity and variable quality in the evidence, future research is unlikely to significantly change the overall diagnostic accuracy review findings. Study authors were contacted to provide additional data, and the review includes additional data from published studies and data from as yet unpublished studies.

This review has made best use of all available data, including extracting data from published figures, to maximise the range of analyses, including analysing the diagnostic impact of using a grey zone with QFR, and performing a simulation study to assess the clinical impact of QFR on future coronary events. To our knowledge, this goes beyond any previous review or meta-analysis in the field.

This is the first study, to our knowledge, to assess the cost-effectiveness of QAngio XA 3D/QFR and CAAS vFFR. The decision model comprehensively assessed both the short-term costs and consequences associated with diagnostic testing and the longer-term impact of treatment on both costs and consequences to ensure that lifetime differences (e.g. the risk of major adverse cardiovascular events and HRQoL benefits associated with revascularisations) were appropriately quantified.

Limitations

Evidence on the CAAS vFFR technology was limited to four studies, which varied in their reporting, and appeared to have heterogeneous results. This prevented any full meta-analyses of diagnostic accuracy for CAAS vFFR or any assessment of its clinical effectiveness.

There were insufficient data allowing exploration of the impact of key patient characteristics (such as multivessel disease or diabetes) on diagnostic accuracy or clinical effectiveness, so these could not be fully investigated.

As is common in reviews of diagnostic tests, data beyond basic diagnostic accuracy, such data on the clinical impact of QAngio XA 3D/QFR, or its practical implementation, were extremely limited and could not be fully reviewed. Although a simulation study was performed to address this, it was innately limited by having to make strong assumptions about the relevant population, and the risk of events, in that population.

The cost-effectiveness results for strategy 5 with vFFR must be interpreted with caution because of very limited data available from diagnostic accuracy studies of vFFR. The use of alternative diagnostic accuracy estimates for vFFR highlighted the substantial uncertainty surrounding the cost-effectiveness of vFFR. The cost-effectiveness results were very sensitive to the procedural disutility assumed in the model for FFR/iFR and the duration of HRQoL benefits associated with revascularisation.

Clinical implications

The results of this review suggest that making revascularisation decisions using QFR as measured with QAngio XA 3D/QFR is preferable to making decisions based on DS assessment using standard ICA alone.

The high level of correlation between QFR and FFR, and the high level of diagnostic accuracy of QFR, suggest that QFR assessment could potentially replace FFR entirely, and hence remove the need for invasive pressure wire and adenosine use. Simulations suggest that replacing FFR with QFR entirely might slightly increase the number of patients who are revascularised, but would have minimal or no impact on the incidence of coronary events.

The use of a grey zone, where patients with borderline QFR values proceed to a FFR assessment, would require around 20% of patients to have a FFR. However, for around 70% of these patients the FFR assessment would agree with the existing QFR assessment. The use of a grey zone might increase the number of patients being revascularised but would appear to have almost the same future incidence of coronary events as if FFR had been used in all patients.

The current evidence on CAAS vFFR appears to be too limited for it to be used in clinical practice at this time.

Economic implications

The economic evidence suggests small differences in net benefit at a cost-effectiveness threshold of £20,000 per QALY between ICA followed by confirmatory FFR/iFR (strategy 2), and ICA with QFR (strategy 3) in an interventional setting. In a diagnostic-only setting, ICA with QFR may yield a higher net benefit at a cost-effectiveness threshold of £20,000 per QALY than ICA followed by confirmatory FFR/iFR, provided that the diagnostic accuracy of QAngio XA 3D/QFR is comparable across settings. Therefore, the use of QAngio XA 3D/QFR in line with strategy 3 is potentially a good use of NHS resources, particularly in a diagnostic-only setting.

Search strategy

1. QANGIO$.ti,ab,kw. (8)

2. quantitative flow ratio$.ti,ab,kw. (36)

3. QFR.ti,ab,kw. (82)

4. “3D/QFR”.ti,ab,kw. (1)

5. aQFR.ti,ab,kw. (2)

6. adenosine-flow QFR.ti,ab,kw. (2)

7. cQFR.ti,ab,kw. (6)

8. contrast-flow QFR.ti,ab,kw. (7)

9. fQFR.ti,ab,kw. (5)

10. fixed-flow QFR.ti,ab,kw. (5)

11. iQFR.ti,ab,kw. (1)

12. index QFR.ti,ab,kw. (1)

13. LQFR.ti,ab,kw. (4)

14. lesion QFR.ti,ab,kw. (1)

15. vQFR.ti,ab,kw. (1)

16. vessel QFR.ti,ab,kw. (1)

17. 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15 or 16 (99)

18. vessel FFR.ti,ab,kw. (11)

19. vFFR.ti,ab,kw. (8)

20. CAAS vFFR.ti,ab,kw. (0)

21. 18 or 19 or 20 (19)

22. 17 or 21 (118)

23. animals/not (humans/and animals/) (4,586,208)

24. 22 not 23 (107)

25. “quinol:fumarate reductase”.ti,ab. (29)

26. 24 not 25 (88).