Evaluation of venous thromboembolism risk assessment models for hospital inpatients: the VTEAM evidence synthesis

Objectives

We aimed to evaluate the cost-effectiveness of VTE RAMs in hospital inpatients, determine the optimal approach to providing thromboprophylaxis and determine how changing the risk threshold for prophylaxis affects cost-effectiveness. We also sought to evaluate the potential performance of different RAMs in an NHS setting, assess the use of efficient methods to capture data on risk characteristics and to determine the accuracy of efficient methods to capture relevant outcomes in a future trial. Our specific objectives were as follows:

1. Update recent systematic reviews to identify VTE RAMs for hospital inpatients and estimate prognostic accuracy.

2. Undertake decision-analytic modelling to determine the cost-effectiveness of VTE risk assessment compared to thromboprophylaxis for all and thromboprophylaxis for none, specifically determining the risk threshold that optimises effectiveness (QALYs) and cost-effectiveness (i.e. maximises net benefit assuming willingness to pay according to NICE thresholds).

3. Use the decision-analytic model to identify key areas of uncertainty and determine the value of gathering additional information to reduce uncertainty.

4. Pilot the use of efficient methods alongside routine practice to determine the feasibility of a future implementation study of VTE risk assessment tools in hospital inpatients.

5. Estimate key parameters for planning a future implementation study.

Objectives 1, 2 and 3 are addressed by workstream 1 (systematic review and economic modelling). Objectives 4 and 5 are addressed by workstream 2 (cohort study).

Eligibility criteria

We sought studies evaluating RAMs which could be applied to a general inpatient population (medical, surgical or trauma), rather than disease-specific models. All primary validation studies (as derivation studies may give an overoptimistic assessment of model performance measures) that evaluated the accuracy (e.g. sensitivity, specificity, concordance statistic) of a multivariable RAM (or scoring system) for predicting the risk of developing VTE were eligible for inclusion. We selected studies that included validation of the model in a group of patients that were not involved in model derivation. This involved either splitting the study cohort (internal) or using a new cohort (external). The study could have reported derivation of the model, but we only used the validation data to estimate accuracy. The study population consisted of hospital inpatients including those who required medical care, undergoing any surgery (excluding day surgery) or received care following an injury. Studies that primarily focused on children (aged under 16 years), women admitted to hospital for pregnancy-related reasons and any patient admitted to a level 2 or above critical care environment (e.g. patients requiring more detailed observation or intervention including support for a single failing organ system or postoperative care and those ‘stepping down’ from higher levels of care) were excluded. These patient groups have VTE risk profiles that differ markedly from the general inpatient population, making the use of a generic model inappropriate.

Data sources and searches

Potentially relevant studies were identified by searching the following electronic databases and research registers:

• Ovid MEDLINE^® Epub Ahead of Print, In-process and Other Non-indexed Citations, Ovid MEDLINE^® Daily, MEDLINE and Versions^® (OvidSP) 1946 to February 2021

• EMBASE (OvidSP) 1974 to February 2021

• Cochrane Database of Systematic Reviews (www.cochranelibrary.com/) 1996 to February 2021

• Cochrane Central Register of Controlled Trials (www.cochranelibrary.com/) 1898 to April 2017

• ClinicalTrials.gov (US NIH) 2000 to February 2021

• International Clinical Trials Registry Platform (WHO) 1990 to February 2021

The search strategy used free text and thesaurus terms and combined synonyms relating to the condition (e.g. VTE in medical inpatients) with risk prediction modelling terms. No language restrictions were used. However, as the current review updated three previous systematic reviews,8,9,10 searches were limited by date from 2017 (last search date from earlier reviews)8 to February 2021. Searches were supplemented by hand-searching the reference lists of all relevant studies (including existing systematic reviews); forward citation searching of included studies (using the Web of Science Citation Index Expanded and Conference Proceedings Citation Index – Science) to identify articles that cite the relevant articles; contacting key experts in the field; and undertaking targeted searches of the World Wide Web using the Google search engine. Further details on the search strategy can be found in Appendix 1.

All identified citations from the electronic searches and other resources were imported into and managed using the EndNote bibliographic software (version X8; Clarivate Analytics, Philadelphia, PA, USA).

Study selection process

The inclusion of potentially relevant articles was undertaken using a two-step process. First, all titles were examined for inclusion by one reviewer Katie Sworn (KS) and any citations that clearly did not meet the inclusion criteria (e.g. non-human, unrelated to VTE inpatients) were excluded. Second, all abstracts and full-text articles were then examined independently by two reviewers (KS and AP). Any disagreements in the selection process were resolved through discussion or if necessary, arbitration by a third reviewer (SG) and included by consensus.

Data abstraction and quality assessment strategy

Data relating to study design, methodological quality and outcomes were extracted by one reviewer (KS) into a standardised data extraction form and independently checked for accuracy by a second [AP or Michael Tonkins (MT)]. Any discrepancies were resolved through discussion or if necessary, arbitration by a third reviewer (SG) and included by consensus. Where multiple publications of the same study were identified, data were extracted and reported as a single study.

The methodological quality of each included study was assessed using PROBAST (Prediction model Risk Of Bias Assessment Tool). 19,20 This instrument evaluates four key domains: patient selection, predictors, outcome and analysis. Each domain is assessed in terms of risk of bias and the concern regarding applicability to the review (first three domains only). To guide the overall domain-level judgement about whether a study is at high, low or an unclear (in the event of insufficient data in the publication to answer the corresponding question) risk of bias, subdomains within each domain include a number of signalling questions to help judge with bias and applicability concerns. An overall risk of bias for each individual study was defined as low risk when all domains were judged as low; and high risk of bias when one or more domains were considered as high. Studies were assigned an unclear risk of bias if one or more domains were unclear and all other domains were low. Further details on the PROBAST tool can be found in Appendix 1, Table 20.

The methodological quality of each included study was independently evaluated by two reviewers (KS and AP). Any discrepancies were resolved through discussion or if necessary, with involvement of a third reviewer (MT). Blinding of the quality assessor to author, institution or journal was not considered necessary.

Data synthesis and analysis

We were unable to perform meta-analysis due to significant levels of heterogeneity between studies (participants, inclusion criteria, clinical condition) and variable reporting of items. As a result, a pre-specified narrative synthesis approach21,22 was undertaken, with data being summarised in tables with accompanying narrative summaries that included a description of the included variables, statistical methods and performance measures [e.g. sensitivity, specificity and C-statistic (a value between 0.7–0.8 and > 0.8 indicated good and excellent discrimination, respectively; and values < 0.7 were considered weak23)], where applicable. All analyses were conducted using Microsoft Excel 2010 (Microsoft Corporation, Redmond, WA, USA).

Patient and public involvement

Patients and the public were not involved in the design of this systematic review. However, all patient representatives reviewed the most widely validated risk models identified, to comment on the appropriateness of criteria, acceptability and potential for face-to-face and remote (patient-led) completion.

Quantity and quality of research available

The literature searches identified 6355 citations. Of these, 51 studies15,24–73 investigating 24 unique RAMs met the inclusion criteria. A flow chart describing the process of identifying relevant literature can be found in Figure 1. A total of 60 full-text articles were excluded as they did not meet all the pre-specified inclusion criteria. The majority of the articles were excluded primarily for not using a RAM for predicting the risk of developing VTE, having no useable or relevant outcome data or an inappropriate study design (e.g. derivation study, reviews, commentaries or editorials). A full list of excluded studies with reasons for exclusion is presented in Report Supplementary Material 1.

FIGURE 1.

Study flow chart (adapted from free online resources hosted by the EQUATOR network https://www.equator-network.org).

Description of included studies (design and patient characteristics)

The design and participant characteristics of the 51 included studies15,24–73 that provided data on the comparative accuracy of RAMs for predicting VTE in patients admitted to hospital is summarised in Table 1. All studies were published between 2003 and 2020 and were undertaken in North America (n = 24),26,27,35–42,45,49,50,54–61,67,70,71 Asia (n = 13),31,32,44,46–48,62–65,69,72,73 Europe (n = 9),15,25,28–30,33,51,53,68 the Middle-East (n = 2),24,65 South America (n = 1),34 Australia (n = 1)43 and one study was intercontinental. 52 Sample sizes ranged from 7042 to 1,099,09345 patients in 37 observational cohort studies [11 prospective15,24,25,30,31,34,46,53,54 (5 of which were multicentre) and 26 retrospective26–29,35,36,38–43,45,48,51,52,55–57,60,61,64,65,67,70,71 (16 of which were multicentre) in design]. Sample sizes in 14 case–control studies32,33,37,44,47,49,50,59,62,63,68,69,72,73 (4 of which were multicentre) ranged from 14863 to 19,21759 patients.

The vast majority of studies evaluated VTE risk assessment in hospital inpatients who required medical care (n = 21),15,28–30,33,34,38,39,47,49,51,53,59,60,63,68,69,71–73 were undergoing surgery (n = 15)26,27,35,37,40,42,45,48,50,54,58,61,65,67,70 or were a mixed medical and surgical cohort (n = 4). 25,31,32,36 The remaining studies focused on patients receiving care for trauma (n = 4),41,43,57,64 cancer (n = 4),24,44,56,62 stroke (n = 1),46 burn injuries (n = 1)55 and sepsis (n = 1). 66 The mean age ranged from 45 years41 to 76 years52 (not reported in 29 studies)15,25–27,32,33,35,37,40,42–47,49,50,54,57–60,63,64,67,68,72,73 and the proportion of female subjects ranged from 17%42 to 81%61 (not reported in 12 studies). 25–27,35,37,45,50,54,57,58,60,63

The majority of studies (n = 37)15,24,26,28–34,38,40,42–49,51,54,57–60,62,64–66,68,69,72,73 defined the VTE end point [deep-vein thrombosis (DVT) and or pulmonary embolism (PE)] as being objectively confirmed. Of the remainder, three studies36,56,71 had no objective confirmation of VTE and 11 studies25,27,35,37,41,50,55,61,63,67,68 did not report the methods for diagnosis confirmation. In terms of VTE risk period, half of the studies (n = 23)15,24–27,28,30,37–40,42,45,46,49,51–54,58,59,61,71 utilised the RAMs to predict the occurrence of VTE within 3 months of the index hospitalisation. The remaining studies did not report the VTE risk period. The reported incidence of VTE ranged widely from 0.3%39,70 to 27.9%48 depending on definition, study design and study participants (e.g. medical, surgical or trauma).

The studies included in this review evaluated 24 validated unique RAMs. The most widely evaluated models were the Caprini RAM (22 studies),24,26,31–35,37,38,40,42,44,45,47,48,50,51,61,62,65,72,73 Padua Prediction Score (16 studies),15,29,30,32,33,36,39,47,50,51,65,66,68,69,72,74 International Medical Prevention Registry on Venous Thromboembolism (IMPROVE) models (8 studies),29,30,33,39,49,51,52,59 the Geneva risk score (4 studies)28,29,30,33 and the Kucher score (4 studies). 33,39,68,71 A summary of their associated characteristics and composite clinical variables is provided in Appendix 1, Table 21. Our patient and public involvement (PPI) group reviewed the most widely validated RAMs and identified all as appropriate and acceptable, in the context of face-to-face risk assessment. They did not feel patient-led completion of any RAM in isolation, to be feasible.

Statistical methods varied significantly between studies. Most studies reported the discrimination of the RAMs using a combination of the C-statistic and sensitivity or specificity.

Thromboprophylaxis was employed in about half (n = 25) of the studies, with the proportion receiving thromboprophylaxis ranging from 3.8% to 100%. It was not employed in three studies, and 23 studies did not report on thromboprophylaxis use. The use of thromboprophylaxis may lead to underestimation of predictive accuracy if a given RAM were to predict VTE events that were subsequently prevented by thromboprophylaxis.

Risk of bias and applicability assessments of included studies

The overall methodological quality of the 51 included studies15,24–73 is summarised in Table 2 and Figures 2 and 3. The methodological quality of the included studies was variable, with most studies having high or unclear risk of bias in at least one item of the PROBAST tool. The main sources of potential bias were related to the following domains: (1) patient selection factors, such as retrospective data collection, incomplete patient enrolment or unclear criteria for patients receiving VTE prophylaxis; (2) predictor and outcome bias arising from inappropriate inclusion of predictors within RAMs, unclear methods of outcome definition, low event rates and missing predictor or outcome data; (3) analysis factors, such as small sample sizes, inappropriate handling of missing data and failure in reporting relevant performance measures such as calibration.

FIGURE 2.

PROBAST assessment summary graph – review authors’ judgements on risk of bias.

FIGURE 3.

PROBAST assessment summary graph – review authors’ judgements on applicability.

Assessment of applicability to the review question led to the majority of studies being classed either as high (n = 35)24,25,31,32,34,36,37,40–51,54–57,61–70,72,73 or unclear (n = 12)15,26,29,30,33,35,52,53,58,60,71 risk of inapplicability. These assessments were generally related to patient selection (highly selected study populations, e.g. single pathologies, single site settings), predictors (inconsistency in definition, assessment or timing of predictors) and outcome determination.

Quantitative data synthesis (summary of results)

As there were a reasonable number of studies to compare, a summary of the C-statistics for studies involving medical, surgical and trauma patients, respectively, is presented in Figure 4, with the results grouped by RAM. Results of other hospital inpatients are presented in Appendix 1, Table 22. C-statistics varied markedly between these studies and between models, with no RAM performing obviously better than other models. In studies evaluating a single model, C-statistics23 were sometimes weak (< 0.7; 10 studies with 17 data points), often good (0.7–0.8; 17 studies with 20 data points) and a few were excellent (> 0.8; 5 studies with 5 data points). There was marked heterogeneity between multiple studies evaluating the same model. Studies evaluating multiple (more than 3) models33,39 tended to report weak accuracy across all the models (C-statistic < 0.7; 2 studies with 16 data points).

FIGURE 4.

C-statistics by model for studies involving (a) medical, (b) surgical and (c) trauma inpatients.

ACS NSQIP, American College of Surgeons National Surgical Quality Improvement Program; CI, confidence interval; NR, not reported; RAP, risk assessment profile.

Table 3 shows the sensitivity and specificity at various thresholds for studies involving medical, surgical and trauma patients, respectively, with the results grouped by RAM. Interpretation was again limited by marked heterogeneity, which was exacerbated when different thresholds were reported by different studies evaluating the same model. Model accuracy was generally poor, with high sensitivity usually reflecting a threshold effect, as evidenced by corresponding low specificity (and vice versa).

Summary of key findings

• A number of RAMs have been evaluated for potential VTE risk stratification of hospitalised adult patients.

• In general, external validation studies have poor designs and limited generalisability.

• Available data suggest that RAMs have generally weak predictive accuracy for VTE.

• Use of VTE prophylaxis varied across the study populations and was not reported for many studies.

• There is insufficient evidence and too much heterogeneity to recommend the use of any particular RAM for predicting the risk of developing VTE in hospital inpatients.

Aim

The aim of the independent economic evaluation is to assess the cost-effectiveness of different strategies for selecting hospitalised patients for pharmacological thromboprophylaxis including prophylaxis for all, prophylaxis for none and prophylaxis based on a VTE RAM. Sections of this chapter have been reproduced from Pandor et al. under licence CC BY 4.0. 74

Population

The target population is hospital inpatients, including medical, surgical and trauma patients but excluding critical care patients, children and women admitted to hospital for pregnancy-related reasons. In order to allow for differences in patient characteristics and risks within this very broad population, the economic analysis considered two broad groups as follows:

• acutely ill medical inpatients

• surgical inpatients including both elective and emergency surgery.

The analysis focused on those populations where NG89 (NICE Guideline 89: Venous thromboembolism in over 16s: reducing the risk of hospital-acquired DVT or PE)8 recommends the use of thromboprophylaxis based on the balance of VTE and bleeding risks. The analysis is not expected to be applicable to very specialist patient groups, such as neurosurgical patients, where the decision whether to use prophylaxis is based on a very individualised approach and the use of a RAM designed for general hospitalised patients is unlikely to be relevant.

Patients identified to be at increased risk of bleeding, such as those identified on the Department of Health risk assessment tool, are assumed not to receive pharmacological thromboprophylaxis under any strategy and are excluded from the model. Therefore, when we refer to a strategy of ‘prophylaxis for all’, this does not mean offering pharmacological thromboprophylaxis to those identified as being at high risk of bleeding.

We considered whether to model elective and emergency surgical patients separately. However, one of the more established risk-scoring tools for surgical patients, Caprini, has been validated in a mixed population of elective and emergency surgical patients and includes questions that would differentiate emergency surgical patients, such as those who have had fractures, into higher-risk groups. It therefore seemed reasonable to evaluate RAMs for use across the broad category of surgical inpatients and to treat the reason for surgery as a risk factor rather than modelling separate decision-making processes in elective and emergency surgical inpatients.

Trauma patients are not modelled as an explicit subgroup as trauma patients are highly complex, often have critical site bleeding injuries and are not managed within a specific trauma speciality in the UK. Managing speciality is often determined by their main complaint, which means they usually fall under a neurosurgical, spinal, general or orthopaedic speciality and, therefore, fall within the emergency surgical population. In addition, many of the RAMs which have been developed specifically for trauma patients, have been developed and validated in countries where trauma patients are treated within a critical care environment; this cohort is excluded from project scope.

Patients having elective hip or knee replacements are examined as a specific subset within the surgical inpatient cohort because of procedure-specific national recommendations (NG89); these patients are recommended to have a longer duration of prophylaxis than is typical in other surgical populations.

Strategies for prophylaxis

The prophylaxis interventions examined were chosen based on the recommendations in NG89 as these were selected based on cost-effectiveness modelling and, therefore, should represent the most cost-effective strategies. For most types of surgery, NG89 recommends that low-molecular-weight-heparin (LMWH) is considered if the person’s VTE risk outweighs their risk of bleeding. Although the use of mechanical prophylaxis in the form of antiembolism stockings (AES) is generally recommended for surgical patients within NG89, a recent study has shown that LMWH alone is non-inferior to LMWH with AES in elective surgical patients who were assessed as being at moderate or high risk of VTE. 75 Given that it is the use of LMWH that is being driven by risk assessment and not the use of AES, the comparison of interest for the modelling is LMWH versus no LMWH. The conclusions from the modelling should apply equally to patients having mechanical prophylaxis or no mechanical prophylaxis. This is because the model estimates whether it is cost-effective to use a RAM to decide whether to add LMWH to the treatment of patients who would not otherwise receive it and that is not dependent on any underlying care being given, provided that the underlying care is not altered by the addition of LMWH. Further discussion of mechanical thromboprophylaxis was, therefore, considered to be outside the scope of the modelling.

For acutely ill medical patients, LMWH is recommended in NG89 but mechanical prophylaxis is not recommended in acutely ill medical patients. Therefore, for acutely ill medical patients we compare LMWH to no prophylaxis.

When LMWH is used, we assume that it is given for the duration of admission. Although NG89 recommends that LMWH is given for a minimum of 7 days, information from the National VTE Exemplar Centres Network suggests that a pragmatic approach has been taken to the implementation of NG89 with LMWH generally not being prescribed for post-discharge use. 76 Therefore, we assume in the base-case analysis that LMWH is given for the duration of admission, but we also explore a scenario analysis in which it is given for 7 days even if this means post-discharge administration.

In higher-risk populations, such as elective hip and knee replacement, an extended form of pharmacological prophylaxis is recommended in NG89. Therefore, we have modelled these groups as specific subsets of the surgical population in separate analyses, described in Model inputs for the elective hip replacement and elective knee replacement scenarios, to determine if the threshold for cost-effective intervention is different in these groups. In addition, NG89 recommends that clinicians should ‘consider extending pharmacological VTE prophylaxis to 28 days postoperatively for people who have had major cancer surgery in the abdomen’. Therefore, a scenario analysis is conducted to determine if the threshold for cost-effective intervention is different in general surgical patients having 28 days of LMWH.

In summary, the main comparisons being modelled are:

• acutely ill medical inpatients – LMWH for duration of admission versus no prophylaxis

• surgical inpatients (excluding elective hip and knee replacement) – LMWH for duration of admission (without or without AES) versus no pharmacological prophylaxis (with or without AES).

Although the recommendations in NG89 were informed by modelling to determine the most cost-effective thromboprophylaxis strategy, the analysis did not explore the use of RAMs to determine which patients should be offered prophylaxis. The RAMs have been selected based on the review in the previous chapter. For medical patients, the primary analysis compares a group of several RAMs that have been externally validated within a single cohort as this minimises the risk of introducing bias from comparing tools that have been studied in populations with different characteristics. A secondary analysis then compares the remaining tools that have been externally validated in different medical cohorts. The number of RAMs for surgical patients was more limited. Therefore, for surgical patients we have included in the primary analysis all tools externally validated in either a surgical or mixed medical and surgical population. In all analyses, the comparator strategies of LMWH for all and LMWH for none are included. The tools evaluated are summarised in Table 4.

Context

The model estimates lifetime costs and QALYs for the different thromboprophylaxis strategies and the comparator of no thromboprophylaxis under a NHS and Personal Social Services perspective. Future costs and benefits are both discounted to their net present value at a rate of 3.5% per annum in accordance with the NICE guide to the methods of technology appraisal. 77 Costs are reported in Great British pounds based on 2020 prices. To achieve this, historical prices used as model inputs were uplifted using the hospital and community health services pay and prices index up to 2016 and the NHS cost inflation index thereafter. 78

Conceptual model for medical inpatients

The model consists of a decision tree (see Figure 5) to capture short-term outcomes followed by a lifetime state transition (Markov) model (see Figure 6) to capture the impact of outcomes that result in death or ongoing morbidity. The model estimates outcomes for a cohort of identical patients with average characteristics. In reality, the application of RAMS may lead to treated and untreated patients having different characteristics. Based on clinical advice, age was the patient characteristic considered most likely to result in variation in outcomes within the cohort being risk assessed. We conducted a scenario analysis to explore whether incorporating patient heterogeneity into the model would affect the conclusions by varying the age of the cohort. From this we concluded that the cohort-level modelling approach was adequate and would not introduce significant bias as the optimal strategy was fairly robust to changes in the age.

FIGURE 5.

Short-term (6-month) decision tree model structure.

FIGURE 6.

Long-term Markov model.

The decision tree is used to estimate for each strategy: the number of patients receiving thromboprophylaxis, the impact of thromboprophylaxis on VTE outcomes (PEs and DVTs) and the incidence of major bleeds during either thromboprophylaxis or VTE treatment with anticoagulants. Major bleeds were considered to be those meeting the International Society for Thrombosis and Haemostasis (ISTH) definitions for major bleeding for surgical and non-surgical (medical) patients. 79,80 Major bleeds were divided into fatal bleeds, non-fatal intracranial haemorrhages (ICHs) and other major bleeds (referred to as non-fatal non-ICH major bleeds). Gastrointestinal (GI) bleeds were assumed to be a reasonable proxy for non-fatal non-ICH major bleeds and these were assumed to have no impact on costs or quality of life after 1 month. PEs were divided into fatal and non-fatal events. DVTs were divided first into symptomatic and asymptomatic DVTs and then into proximal and distal DVTs. Symptomatic DVTs and non-fatal PEs are assumed to result in 3 months of anticoagulant treatment. Patients having major bleeds during either prophylaxis or VTE treatment are assumed to stop their anticoagulant medication at the time of the bleed. In our previous analysis of thromboprophylaxis strategies in patients having lower limb immobilisation following injury, we found that the prevention of post-thrombotic syndrome (PTS) following asymptomatic DVT was an important driver of both cost-effectiveness and decision uncertainty due to asymptomatic DVTs being more common than symptomatic DVTs but their long-term consequences being more uncertain. 74 So while asymptomatic DVTs are assumed to remain undetected and untreated, it is important to capture these DVTs in the decision tree phase of the model in order to capture any ongoing morbidity due to PTS in the long-term state-transition model. In the decision tree, any VTE event that is related to the hospital admission is assumed to occur within 90 days of hospital admission. Therefore, a 6-month time frame was considered sufficient to capture the period of VTE risk associated with the hospital admission and the period of treatment following VTE (3 months). Diagnosis of PTS and chronic thromboembolic pulmonary hypertension (CTEPH) is assumed not to occur until the end of the decision-tree phase of the model, as it is difficult to distinguish these chronic complications from acute symptoms during the first 3 months after VTE. The decision tree allows for the possibility that patients not having anticoagulation may have a major bleed during their hospitalisation. The likelihood of VTE and the likelihood of bleeding during treatment for VTE is assumed to be independent of patient characteristics (e.g. age and sex) and independent of whether the patient had major bleeding during hospitalisation. Clinically relevant non-major bleeding (CRNMB) is excluded in the base-case analysis as it was considered that CRNMB occurring during hospitalisation would have a minimal impact on costs and quality of life in patients already hospitalised. Heparin-induced thrombocytopenia (HIT) was not included in the model because the most important consequence of HIT is the increased risk of VTE, but any increase in VTE related to HIT in the LMWH arms of the clinical trials would be included within the efficacy estimates for LMWH versus placebo and would, therefore, already be accounted for in the model.

The key model assumptions for the decision tree phase are as follows:

• Bleeding events following hospitalisation are possible in both those having thromboprophylaxis and those having no thromboprophylaxis.

• Bleeds associated with thromboprophylaxis are assumed to occur during prophylaxis and, therefore, before hospital discharge.

• VTE associated with hospitalisation is assumed to occur within 90 days of admission.

• Patients who have major bleeding will stop thromboprophylaxis immediately, but the treatment effect of thromboprophylaxis is assumed to be the same as for those who completed treatment, as patients who bleed are assumed to be adequately anticoagulated.

• The risk of VTE is the same whether or not thromboprophylaxis caused bleeding.

• All patients with symptomatic DVT receive accurate diagnosis and initiate treatment with anticoagulants [3 months of either direct oral anticoagulants (DOACs) or phased anticoagulation].

• Asymptomatic DVTs are not detected and are not treated.

• All PEs are symptomatic and lead to detection and treatment (3 months of either DOACs or phased anticoagulation) in all cases.

• Patients treated for symptomatic DVT and PE have a bleed risk associated with treatment which is assumed to occur during the 3-month treatment period.

• Chronic complications of VTE (CTEPH following PE and PTS following DVT) are assumed to be diagnosed at least 3 months after VTE and therefore occur after any bleeds associated with VTE treatment.

• Deaths due to PE occur before any bleeding associated with the treatment of PE.

• Risk of bleeding during treatment of VTE is independent of whether the patient bled during prophylaxis.

• Risk of VTE, risk of bleeding and risk of PTS/CTEPH are based on average patient characteristics (e.g. age and sex) for the cohort being risk assessed.

A state-transition model (see Figure 6) was then used to extrapolate lifetime outcomes including overall survival and ongoing morbidity related to either bleeds or VTE. The health states included within the state-transition model capture the risk of PTS following VTE and the risk of CTEPH following PE. The risk of PTS is dependent on whether the DVT is symptomatic and treated or asymptomatic and untreated, and also whether the DVT is proximal or distal. All patients with PTS are combined in a single health state as costs, utilities and survival are not expected to be affected by whether PTS occurred following proximal or distal DVT. The PTS health state is not split into different severity levels as the utility estimates are based on the average utility across severity levels and the costs are not expected to differ by severity. The CTEPH health state is divided according to whether patients receive medical or surgical management to allow for differential costs and survival between these groups. There is also a post-ICH state to capture ongoing morbidity following ICHs. Further adverse outcomes (PTS, CTEPH) are not modelled following ICH, as lifetime costs and QALYs are assumed to be predominantly determined by morbidity related to ICH. Recurrent VTEs are not modelled as it is unclear whether any recurrent VTE would be related to the VTE provoked by hospital admission or whether it is related to underlying risk factors and would therefore have occurred whether or not prophylaxis was initiated during the hospital admission. The state-transition model has one 6-month cycle to extrapolate the outcomes of the decision tree up to 1 year, followed by annual cycles thereafter. All-cause mortality during the first year is applied at exactly 6 months. Thereafter, the health state occupancy is half-cycle corrected such that all transitions between states, including mortality, are assumed to occur mid-cycle. The key model assumptions during the state-transition phase are as follows:

• All symptomatic DVTs are associated with a risk of PTS but the rate is allowed to differ depending on whether the DVT is distal or proximal and whether it is symptomatic and treated or asymptomatic and untreated.

• There is no risk of PTS following PE and CTEPH is possible only after PE.

• Further outcomes (i.e. VTE, CTEPH and PTS) are not modelled for those who experience ICH as lifetime cost and QALYs will be determined predominantly by disability related to the ICH.

• All-cause mortality is applied to all transition states except CTEPH and post-ICH which have state-specific mortality rates.

• Recurrent VTE is not modelled.

Conceptual model for surgical patients

We considered whether it was necessary to add additional wound-related outcomes to the model for surgical patients, such as surgical site bleeding or surgical site infections associated with wound haematoma. The clinical experts advised that the vast majority of major bleeds in surgical patients are distant from the surgical site. In addition, any complications related to surgical site bleeding that require patients to return to theatre or that result in prolonged hospitalisation would fall under the ISTH definition of major bleeding in surgical patients. 79 Therefore, these would already be captured within the decision-tree model as non-fatal non-ICH major bleeds. For these reasons, the conceptual model for surgical patients was kept identical to that for medical inpatients, with the caveat that the definition of major bleeding is different for surgical patients. Extended duration thromboprophylaxis in clinical subgroups (such as surgical patients with abdominal cancer) was evaluated using the same conceptual model with a separate scenario analysis (as per Strategies for prophylaxis).

Data sources

The input parameters used in a previous analysis of thromboprophylaxis during lower limb immobilisation were examined to identify any that were less relevant to patients receiving prophylaxis during hospitalisation. The data related to the population characteristics (age, sex and life expectancy), the incidence of VTE, the incidence of bleeding and the costs of prophylaxis were updated to use data specific to our target population. Other data such as the incidence of chronic sequelae (PTS and CTEPH) following VTE, and the costs and utility values for patients experiencing adverse outcomes were generally based on the same sources with costs updated to reflect changes in prices. One exception is the utility data for DVT and PE which have been updated because a more detailed analysis of the study used previously is now available.

Clinical input parameters are described below in Population characteristics to Mortality risks for adverse outcomes. Those that are specific to the medical and surgical inpatient populations are summarised in Table 5, with full details of all clinical input parameters provided in Appendix 2, Table 23. Cost inputs are described below in Cost of prophylaxis to Costs of managing PTS and CTEPH and summarised in Appendix 2, Table 24 with a detailed breakdown of drug costs for treating VTE and resource use for diagnosing and treating VTE provided in Appendix 2, Tables 25 and 26. Utility inputs are described below in Utility values and summarised in Appendix 2, Table 27 for the decision tree phase of the model and Appendix 2, Table 28 for the state-transition phase of the model. Probabilistic distributions for cost and utility inputs are summarised in Appendix 2, Table 29. The assumed timings of utility inputs are described in Timing and duration of utility decrements applied in the decision tree. Sensitivity and specificity data for the alternative RAMs are described in Sensitivity and specificity of RAMs.

Sensitivity and specificity of RAMs

Sensitivity and specificity data have been extracted from the RAM studies listed in Chapter 3. 29,30,36,38,39,71 For many of the RAM studies we wished to examine in the model, the authors themselves have not reported sensitivity and specificity data. In this case, estimates of sensitivity and specificity have been extracted using reports of the incidence by risk score and graphical data, where possible. In some cases, estimates of sensitivity and specificity were extracted digitally from receiver operating characteristic (ROC) curves where these were reported.

Figure 7 shows the sensitivity and specificity data for RAMS in medical inpatients extracted from various studies. Overall, there is data reported for seven RAMs (Caprini, Kucher, Padua, IMPROVE, Intermountain, Geneva, Rothberg) and all except one (Rothberg) are reported in more than one cohort. It can be seen that there is not always consistency in the estimates of sensitivity and specificity for a particular RAM when reported across different cohorts. In particular, the estimates for IMPROVE and Intermountain obtained by Blondon et al. are more favourable than the estimates from Greene et al. However, several RAMs appear to have similar performance when estimates of sensitivity and specificity were taken from single cohort, as reported in papers by Greene et al. and Grant et al.

FIGURE 7.

ROC curve for multiple risk assessment models evaluated in cohorts of medical inpatients.

Figure 8 shows the sensitivity and specificity data for two RAMs in surgical inpatients. There were data on the Pannucci score reported by Pannucci et al. and data on the Caprini score reported by Bahl et al.

FIGURE 8.

ROC curve for multiple risk assessment models evaluated in cohorts of surgical inpatients.

Figure 9 shows the sensitivity and specificity data available for mixed populations of both surgical and medical inpatients. Only one paper reported this information for a mixed population (Elias et al. 201736). Elias et al. provide ROC curves for patients having prophylaxis and patients not having prophylaxis. They also provide a single estimate of sensitivity and specificity for the cohort as a whole.

FIGURE 9.

ROC curve for the Padua RAM in a mixed cohort of medical and surgical patients reported by Elias et al. 36 [with/without prophylaxis (PPX) and all].

The base-case deterministic analysis estimates the expected costs and QALYs for all available RAMs where performance data are available in that specific population (i.e. surgical or medical patients). In addition, because the usage of prophylaxis within the cohorts will tend to result in RAM performance being underestimated, we have also conducted a secondary analysis using the data from Elias et al. which was estimated in the subset of the cohort not receiving prophylaxis. This cohort included both surgical and medical inpatients and is therefore not used in the base-case scenario for either group.

Available data suggest that RAMs have generally weak predictive accuracy for VTE. There is insufficient evidence and too much heterogeneity to recommend one RAM over another. Hence, we have focused on using data from studies of RAMs to explore the trade-off between sensitivity and specificity, rather than try to identify the most cost-effective RAM. To do this we have used regression to estimate the trade-off between sensitivity and specificity for a typical RAM and have used this to explore how this trade-off affects the cost-effectiveness of using RAMs to select patients for prophylaxis.

Figure 10 shows the sensitivity and specificity data available across all populations with trend lines fitted firstly to all of the estimates across the multiple RAMS reported by Greene et al. and Grant et al. which were estimated in a single cohort, and secondly to the estimates from Elias et al. , which were estimated in the subset of the cohort not having pharmacological prophylaxis. The trend lines have been fitted by assuming that there is a linear relationship between logit (sensitivity) and logit (1-specificity). It should be noted that for the line fitting against the data from Greene et al. and Grant et al. , the fitted line assumes that each point is equally valid and does not take into account (1) the uncertainty around each pair of sensitivity and specificity estimates, (2) the relationship between multiple points that represent the ROC curve for a single RAM in a single study or (3) any relationship between estimates for different RAMs from the same study. The estimates from the Elias cohort were extracted for use in a sensitivity analysis exploring the potential for a more accurate RAM to be cost-effective. This was undertaken specifically to address the concern that use of VTE prophylaxis could lead to underestimation of the accuracy of the RAMs. It should be noted, however, that the Elias study reported higher accuracy for the Padua RAM than other studies in cohorts with and without prophylaxis. It is, therefore, unclear why accuracy was better in the Elias study, but it probably reflects features of the study design.

FIGURE 10.

ROC curve for all studies (surgical, medical and mixed) with trend lines fitted.

Clinical outcomes predicted by the model with and without prophylaxis

Table 7 shows the clinical outcomes predicted by the model with and without prophylaxis in each of the modelled populations.

In medical inpatients, prophylaxis reduces serious adverse outcomes (fatal PEs, fatal bleeds and non-fatal ICHs) from 53 per 10,000 to 42 per 10,000. The reduction in symptomatic DVTs and non-fatal PEs is also higher than the increase in other bleed types. In the long-term outcomes (presented in Table 7) at 5 years, there is a large reduction in the number of patients experiencing PTS from prophylaxis from 787 per 10,000 to 385 per 10,000, but minimal difference in overall survival. Therefore, in medical inpatients, although prophylaxis increases the risk of major bleeding, this is more than outweighed by the number of VTE events avoided and there are additional benefits from preventing long-term VTE-related complications such as PTS.

In surgical inpatients, the risk of a serious adverse outcome (fatal PEs, fatal bleeds and ICH) is low at 7 per 10,000 but it is increased slightly by prophylaxis to 11 per 10,000 due to the additional risk of fatal bleeding or non-fatal ICH. Although the risk of any symptomatic VTE is reduced from 140 per 10,000 to 41 per 10,000, the risk of any major bleeding is increased from 125 per 10,000 to 370 per 10,000. Prophylaxis still reduces the risk of PTS in surgical inpatients, from 367 per 10,000 to 107 per 10,000, but PTS is less common in the surgical cohort than in the medical cohort due to the lower risk of DVT. Therefore, in the surgical population there is a more explicit trade-off between the short-term risk of major bleeding and the benefits of preventing VTEs and their long-term consequence.

In elective hip replacement patients, the risk of a serious adverse clinical outcome (fatal PEs, fatal bleeds and non-fatal ICH) is reduced by prophylaxis from 28 per 10,000 to 17 per 10,000. The risk of any symptomatic VTE is reduced from 346 per 10,000 to 116 per 10,000 and the risk of any major bleeding is increased from 128 per 10,000 to 321 per 10,000. The reduction in PTS is similar to that observed in the general surgical inpatient population (294 per 10,000 reduced to 98 per 10,000 by prophylaxis). Therefore, in the elective hip replacement population, the increase in rates of major bleeding is similar to the decrease in rates of symptomatic VTE, but overall there is a reduction in serious adverse clinical outcomes as only a very small minority of major bleeds are fatal or intracranial.

In elective knee replacement patients, the risk of a serious adverse clinical outcome (fatal PEs, fatal bleeds and non-fatal ICH) is reduced from 20 per 10,000 to 11 per 10,000 by prophylaxis. The risk of any symptomatic VTE is reduced from 397 per 10,000 to 186 per 10,000 but the increase in the risk of any major bleeding is smaller (116 per 10,000 to 125 per 10,000). The reduction in PTS is more similar to that observed in the medical inpatient population (764 per 10,000 reduced to 357 per 10,000), which is largely being driven by the high risk of asymptomatic DVT in the elective knee replacement population which is more similar to that observed in the medical inpatient population.

The 5-year survival is higher in surgical inpatients than in medical inpatients which is to be expected given the lower starting age of 54 years in surgical inpatients compared with 64 years in medical inpatients. In both the elective knee replacement population and the elective hip replacement population, all-cause mortality at 5 years is higher than in the general surgical population due to a higher starting age of 69 years in both groups. However, the impact of prophylaxis on all-cause mortality at 5 years is small regardless of the population selected.

Balance of benefits and harms of prophylaxis at different levels of risk

It can be seen from the clinical outcomes predicted by the model in Table 7, that there is always a trade-off between the benefits of using prophylaxis to reduce VTE risks and the potential harms that may occur due to the increased risk of bleeding. These differ between the medical and surgical populations largely because these groups have different average risks of VTE and major bleeding but also because some specific types of VTE outcomes and major bleeding events are more likely in some populations than others. It would, therefore, be useful to explore the overall impact of prophylaxis on health at differing levels of risk so that the balance of benefits and harms in individuals who have a higher or lower than average risk can be assessed. Figures 11 and 12 present information on whether the overall impact of prophylaxis on QALYs over the patient’s remaining lifetime is positive or negative for different levels of VTE risks and major bleeding risks. Where the overall impact of prophylaxis on lifetime QALYs is positive for a given level of VTE and major bleeding risk, the cell is coloured green and where it is negative the cell is coloured red. It should be noted that the steps in risks between rows and columns are not even across the table. In addition, although the risk levels presented for VTE without prophylaxis, and for major bleeding with prophylaxis are the same in Figures 11 and 12, the risk levels for VTE with prophylaxis and major bleeding without prophylaxis differ because the RRs for these outcomes are different in medical and surgical inpatients. The square outlined in bold is the average level of risk assumed for that population in the base-case analysis.

FIGURE 11.

Balance of benefits and harms for medical inpatients with different levels of risk (blue and red indicate overall QALY gain and loss, respectively).

FIGURE 12.

Balance of benefits and harms for surgical inpatients with different levels of risk (blue and red indicate overall QALY gain and loss, respectively).

In Figure 11, it can be seen that prophylaxis for all results in QALY gains for medical inpatients with average levels of VTE and bleeding risks (3.4% and 1.02%, respectively). Prophylaxis for none would result in greater QALY gains if either the average VTE risk in the medical cohort was much lower (≤ 0.35%) or the average risk of bleeding in the medical cohort was much higher (≥ 11%). However, in a cohort of patients with both a higher risk of bleeding and a lower risk of VTE, for example, a 6% bleeding risk and a 1.5% VTE risk, prophylaxis would also do more harm than good.

In Figure 12, it can be seen that prophylaxis results in overall QALY gains over a much broader range of VTE and bleeding risks in surgical inpatients than in medical inpatients. In surgical inpatients a much lower proportion of major bleeding is fatal bleeding or a non-fatal ICH and, therefore, the overall impact of prophylaxis is positive over a broader range of bleeding risks. In addition, the average age assumed for patients in the surgical population is lower than the average age for patients assumed in the medical population. Therefore, any QALY losses due to long-term consequences of VTE, such as PTS, will be larger as they will have an impact over a longer time frame.

Figures 11 and 12 show exploratory analyses and should not be used to guide decision-making. The analysis assumes that the RR of bleeding with prophylaxis is constant and does not increase as the baseline bleeding risk increases, whereas it is conceivable that risk factors for bleeding also increase the RR of bleeding with prophylaxis. Furthermore, VTE and bleeding risk factors are often inter-related and there may be interactions between risk factors and between risks. These issues mean that the decision to prescribe prophylaxis for an individual patient with an increased bleeding risk is complex and unlikely to lend itself to determination by a simple trade-off.

Figures 11 and 12 show the balance of benefits and harms for prophylaxis in terms of the impact on overall lifetime QALYs, but they do not address the question of whether it is cost-effective to use prophylaxis at different levels of risk and whether it is cost-effective to use a RAM to select patients at higher risk for prophylaxis. These questions are addressed in Deterministic base-case cost-effectiveness results for medical inpatients to Deterministic scenario analyses for elective hip and knee replacement populations below.

Deterministic base-case cost-effectiveness results for medical inpatients

The deterministic base-case results obtained when applying the midpoint parameter estimates to the base-case scenario for medical inpatients using data for the five RAMS that were evaluated in a single cohort of patients (Green et al. 2016; Grant et al. 2016) are presented in Figure 13. For each RAM, several points are plotted which represent the costs and QALYs that can be achieved for different RAM cut-off points. It can be seen in Figure 13 that the strategy of prophylaxis for all dominates all other options because it has a larger expected QALY gain and it also has a lower expected cost. Figure 13 also shows the expected costs and QALYs for a RAM which has the average characteristics of all the RAMS evaluated in the cohort reported by Greene et al. and Grant et al. using the ROC curve regression described in Sensitivity and specificity of RAMs.

FIGURE 13.

Deterministic base-case results for medical inpatients when using RAMs evaluated in a single cohort of patients (Greene et al. 39 and Grant et al. 38).

Figure 14 shows the secondary analysis introducing the data from RAMs evaluated in other cohorts. For some of these RAMs, only a single point is plotted as sensitive and specificity was only reported at a single threshold but for others multiple points are plotted showing different RAM thresholds. Several of these RAMs appear to perform better than the regression based on the average RAM evaluated in the cohort reported by Greene et al. and Grant et al. in that they provide a similar number of QALYs at a lower cost. The best-performing RAM reported in an alternative cohort, is the Geneva score with a cut-off of ≥ 3 (sensitivity of 94.6% and specificity of 44%) reported by Blondon et al. 29 However, prophylaxis for all would still be more cost-effective than using a Geneva score cut-off of ≥ 3, even when applying the performance data from Blondon et al.

FIGURE 14.

Deterministic base-case results for medical inpatients when using RAMs evaluated by Greene et al. 39 and Grant et al. 38 and RAMs evaluated in other cohorts (including the mixed cohort of surgical and medical patients reported by Elias et al. 36).

Also shown in Figure 14 is a sensitivity analysis using the Padua RAM performance data from Elias et al. which was obtained in the subgroup of the cohort not having prophylaxis. 36 When including this data, the optimal strategy would be a Padua score of ≥ 3 (sensitivity 99.9% and specificity of 23.7%), as this has slightly higher QALYs and slightly lower costs than prophylaxis for all. However, the caveat here is that Elias et al. recruited a mixed population of medical and surgical patients and, therefore, it is unclear if this RAM would perform equally well in an exclusively medical cohort. When the Padua score was evaluated by Greene et al. in an exclusively medical cohort, the performance was not as good, with a Padua score of ≥ 3 having a sensitivity and specificity of 49.3% and 73.0%, respectively. However, this sensitivity analysis suggests that if a RAM with sufficiently good performance could be developed and validated in an exclusively medical cohort, then prophylaxis for all would not be the optimal strategy in medical patients. Also shown are the expected costs and QALYs when using a regression fitted to the data points from Elias et al. It can be seen from this that the performance based on the regression is not as good as when using the observed sensitivity and specificity for the actual Padua RAM scores obtained by Elias et al. This is because the regression is not a good fit to the ROC curve for low RAM scores.

Probabilistic base-case results for medical inpatients

Based on these deterministic analyses, the probabilistic model was run for the strategies of no prophylaxis, prophylaxis for all and for the various cut-offs of the Padua score (assuming the RAM performance observed in Greene et al. ). It should be noted that this probabilistic analysis does not explore the uncertainty regarding which RAM is optimal as this would require comparing all of the cut-offs from all of the various RAMs simultaneously. Instead, it uses the Padua RAM as an example RAM to answer the question of what the optimal strategy would be if the RAM chosen performed similarly to the Padua RAM.

The results based on the mean outcomes from 10,000 probabilistic model runs are summarised in Table 8. The broad conclusions are the same, in that prophylaxis for all is the optimal strategy when applying valuing a QALY at £20,000–30,000.

Figure 15 shows that while the average cost saving is £28.44 and the average QALY gain is 0.055 QALYs for prophylaxis for all versus prophylaxis for none, there are a broad range of estimates with 79% of samples resulting in a cost saving and the incremental cost having a CrI of £−113.66 to £46.28. The incremental QALY estimate has a CrI of 0.02 to 0.11, with 99.9% of samples resulting in a QALY gain compared to prophylaxis for none.

FIGURE 15.

Cost-effectiveness plane for prophylaxis for all versus prophylaxis for none in medical inpatients.

The cost-effectiveness acceptability curve (CEAC) is presented in Figure 16 for medical inpatients. There is < 1% chance that prophylaxis for all is not the optimal strategy when valuing a QALY at £20,000. In addition, because prophylaxis for all is cost-saving compared to even the next most effective strategy, there is a 76% chance that it would be the optimal strategy when valuing a QALY at £0.

FIGURE 16.

Cost-effectiveness acceptability curve (CEAC) for medical inpatients (applying performance of Padua RAM from Greene et al. 39). Strategies with < 1% probability across all values of a QALY are not plotted.

The overall expected value of perfect information (EVPI) associated with all parameters included in the probabilistic sensitivity analysis (PSA) when valuing a QALY at £20,000 was £0.06 per patient per year (£59.33 per 1000 patients per year) due to the high probability that prophylaxis for all is the optimal strategy. This suggests that there is minimal decision uncertainty regarding whether a RAM should be used in medical inpatients, when assuming the RAM will perform according to the performance of the Padua RAM in the Greene et al. cohort.

However, the results in Figure 14 suggest that the optimal strategy may not be prophylaxis for all if a better-performing RAM could be developed in medical patients. Therefore, we decided to re-run the EVPI analysis for the medical cohort when using the sensitivity and specificity data from the mixed cohort of medical and surgical patients reported by Elias et al. When using this sensitivity and specific data for medical inpatients, the EVPI is £2.42 per person per year (£2422.90 per 1000 patients per year). Offering prophylaxis at a RAM score of ≥ 3 has a 76.6% chance of being optimal and prophylaxis for all has a 17.6% chance of being optimal when valuing a QALY at £20,000. In addition, a large proportion of the global EVPI is attributable to the uncertainty around the RR of bleeding for prophylaxis versus no prophylaxis; expected value of partial perfect information (EVPPI) of £2.01 for this parameter with no other parameter having an EVPPI > £0.01. This sensitivity analysis suggests that if a better-performing RAM could be developed for medical inpatients, then the decision uncertainty would largely relate to the uncertainty around the increased risk of bleeding attributable to prophylaxis.

Deterministic scenario and sensitivity analyses for medical inpatients

Deterministic scenario and sensitivity analyses were conducted to explore which model assumptions and inputs were key drivers of decision uncertainty. To do this, deterministic results were generated using midpoint parameter inputs when varying individual model inputs or assumptions. These analyses assume that the RAM used is Padua and that it performs as observed in the cohort reported by Greene et al.

The results presented in Table 9 show that the broad conclusion that prophylaxis for all is the optimal strategy (defined as most cost-effective when valuing a QALY at £20,000) does not vary under most of the scenarios and sensitivity analyses considered. The exceptions were scenarios that explored large changes to the risk of VTE and bleeding during PPX, with a sixfold decrease in VTE risk or a sixfold increase in risk of major bleeding needed to alter the optimal strategy.

On the basis of this, further analyses were conducted to explore how the optimal strategy varies when changing the average risks of VTE and major bleeding in the medical cohort. Figure 17 shows that the optimum strategy is prophylaxis for all when the risk of VTE is high and the bleeding risk is low, but the optimum strategy is pharmacological prophylaxis for none when the risk of VTE is low and the bleeding risk is high. There is a band of combined VTE and bleeding risks where using a RAM to target prophylaxis at higher-risk patients would be optimal instead of a blanket strategy of either prophylaxis for all or prophylaxis for none. This suggests that it is important to gather good evidence on the average risks of VTE and major bleeding in the medical cohort as a whole in order to determine whether a RAM should be used to direct prophylaxis.

FIGURE 17.

Optimal risk assessment strategy for cohorts of medical inpatients with varying levels of baseline risks (box indicates base-case scenario). It should be noted that the rows and columns do not represent equal steps in risk; red (10) indicates prophylaxis for none is optimal and blue (0) indicates prophylaxis for all is optimal; the numbers in cells of other colours represent the optimal Padua RAM score at which prophylaxis should be offered at that level of risk to maximise incremental net monetary benefit when valuing a QALY at £20,000.

The sensitivity and scenario analyses also suggest that various individual factors about which there was some uncertainty have no impact on the optimal strategy. These include the incidence of PTS after asymptomatic DVT, the costs and utility losses attributable to PTS, the case-fatality rate for PE and the exact timing of VTE within the 90-day post-admission period. In addition, it was found that the optimal strategy did not change when assuming no additional cost for administering the RAM or when assuming that patients continue LMWH for 7 days including 2 days of post-discharge administration.

Deterministic base-case results for surgical inpatients

The deterministic base-case results obtained when applying the midpoint parameter estimates to the base-case scenario for surgical inpatients using data for the two RAMS that were evaluated in either a surgical population (Caprini by Bahl et al. ,26 Pannucci by Pannucci et al. 54) are presented in Figure 18. For each RAM, several points are plotted which represent the costs and QALYs that can be achieved for different RAM cut-off points. For the Pannucci RAM, the cut-offs shown are ≥ 6, ≥ 3 and ≥ 1. For the Caprini score the cut-offs shown are ≥ 7, ≥ 5, ≥ 3 and ≥ 2. From this it can be seen that the optimal strategy is pharmacological prophylaxis for all, if the analysis is restricted to RAMs evaluated exclusively in this cohort. Figure 18 also includes results for the Padua RAM from the mixed cohort of both surgical and medical inpatients reported by Elias et al. When including the performance data from the Padua score evaluated by Elias et al. , the optimal strategy is a Padua score of ≥ 3 (sensitivity of 99.9% and specificity of 23.7%), which achieves more QALYs at a reduced cost compared to prophylaxis for all.

FIGURE 18.

Deterministic base-case results for surgical inpatients when using RAMS evaluated in either surgical populations (Pannucci et al. 201454, Bahl et al. 201026) or mixed populations of both surgical and medical inpatients [Elias without prophylaxis (PPX)].

Figure 18 also shows the expected cost-effectiveness of a RAM whose performance is based on a regression fitted to the data from the ROC curve for Padua using the data observed in the cohort not having thromboprophylaxis reported Elias et al. The data for the lower Padua score cut-offs (≥ 1 to ≥ 3) were excluded from this regression as they did not fit the assumption that there is a linear relationship between logit (sensitivity) and logit (1-specificity). Therefore, the regression does not closely follow the performance of the Padua score for the cut-offs below 4 and the Padua score is predicted by the regression to perform less well than observed at cut-offs below 4. This results in thromboprophylaxis for all being the optimal strategy when assuming that the RAM would follow the ROC curve predicted by the regression. Based on this, we can say that a RAM would need to have a very high sensitivity, similar to that observed by Elias et al. , to make using a RAM more cost-effective than offering prophylaxis to all.

Probabilistic base-case results for surgical inpatients

Based on these deterministic analyses, the probabilistic model was run for the strategies of no pharmacological thromboprophylaxis, prophylaxis for all and for the various cut-offs of the Pannucci score. It should be noted that this does not capture uncertainty related to which RAM is optimal as this would require a simultaneous comparison of all of the various cut-offs for all three RAMs.

The results based on the mean outcomes from 10,000 probabilistic model runs are summarised in Table 10. The broad conclusions are the same, in that offering prophylaxis to surgical patients with a Pannucci score ≥ 3 is the optimal strategy when applying valuing a QALY at £20,000 to £30,000.

Figure 19 shows the incremental costs and QALYs for pharmacological thromboprophylaxis for all versus prophylaxis for none in surgical inpatients. It can be seen that while the average cost saving is £47.88 and the average QALY gain is 0.035 QALYs, there are a broad range of estimates with 24% of PSA samples resulting in a cost saving and the incremental cost having a CrI of −£96 to £254. The incremental QALY estimate has a CrI of 0.002 to 0.080 with 98% of samples resulting in a QALY gain compared to prophylaxis for none.

FIGURE 19.

Cost-effectiveness plane for prophylaxis for all versus prophylaxis for none in surgical inpatients.

The CEAC is presented in Figure 20 for surgical inpatients. It can be seen from this that giving prophylaxis to all patients has the highest probability of being cost-effective at £20,000 per QALY, with a 70% likelihood of being optimal, but there is also a 17% likelihood that using a Pannucci score of ≥ 3 would be optimal and a 9% likelihood that using a Pannucci score of ≥ 1 would be optimal.

FIGURE 20.

CEAC for surgical inpatients (assuming the RAM used is Pannucci).

The overall EVPI associated with all parameters included in the PSA when valuing a QALY at £20,000 was £16.35 per person. This is calculated across all strategies including no prophylaxis, Pannucci scores of ≥ 6, ≥ 3 and ≥ 1 and thromboprophylaxis for all. It does not include the uncertainty of considering which RAM to use as the EVPPI has been run assuming that the Pannucci score is used. In addition, it should be noted that the EVPI analysis does not include any uncertainty not captured in the PSA such as uncertainty around the model assumptions or the choice of data sources, or uncertainty around the estimates of sensitivity and specificity.

In the analysis of parameter EVPI (EVPPI), only six individual parameters were found to have an EVPPI of more than £0.01 per person (see Table 11 EVPI results for surgical in patients when using Pannucci RAM based on sensitivity and specificity from Pannucci et al. ). The parameter with the largest EVPPI was the RR of VTE for patients having thromboprophylaxis compared to patients not having thromboprophylaxis. Other slightly less important parameters were the likelihood that major bleeding is intracranial (ICH) or fatal, the absolute and RRs of bleeding and the utility of patients experiencing PTS following DVT. These findings reinforce the idea that it is the trade-off between the short-term risks of bleeding in patients receiving prophylaxis and the long-term risks of chronic complications that are most important to the decision regarding which surgical inpatients should be offered prophylaxis.

We also ran the PSA and EVPI analysis assuming that the Padua RAM is used and that it performs as seen in subset of the cohort reported by Elias et al. who did not receive prophylaxis. This was to explore whether further research would be more worthwhile if a RAM could be identified that performed better than the Pannucci RAM in surgical inpatients. The CEAC when assuming the RAM performance for Padua reported by Elias et al. is shown in Figure 21. When using the sensitivity and specificity data from Elias et al. the overall EVPI was £15.90 and the same set of parameters was identified as being the most important drivers of uncertainty.

FIGURE 21.

CEAC for surgical inpatients (assuming the RAM used is Padua from Ellias et al. 36).

Deterministic scenario and sensitivity analyses for surgical inpatients

Deterministic scenario and sensitivity analyses were conducted to determine whether the optimal strategy (when valuing a QALY at £20,000) varies under different assumptions or alternative sources of evidence. These analyses assume that the RAM used is Pannucci. The optimal strategy is defined as the most cost-effective strategy when valuing a QALY at £20,000.

These analyses (see Table 12) identified that prophylaxis for all is no longer the optimal strategy when the risk of VTE is halved, or when the risk of bleeding is doubled. It is also no longer optimal when the utility decrement of PTS is halved or estimated from alternative sources,134 or when assuming there is no risk of PTS in asymptomatic DVTs. It is also no longer optimal when assuming that there are no asymptomatic DVTs. These analyses suggest that the cost-effectiveness of offering prophylaxis to surgical patients is being partly driven by the prevention of PTS in patients having asymptomatic DVT. The scenario exploring an increase in age to 80 changed the optimal strategy to a RAM score of ≥ 3, which suggests that the optimal strategy may differ for patients who have lower life expectancy and lower quality of life prior to admission as these patients have less potential to benefit from reducing the risk of long-term complications. An increase in the duration of prophylaxis to 7 days to cover 5 days of admission and 2 days of post-discharge prophylaxis also altered the optimal strategy to offering prophylaxis at a RAM score of ≥ 1. However, this was largely attributable to the additional 2 days of prophylaxis provided after discharge, as the duration of in-hospital prophylaxis had to be increased to 16 days before the optimal strategy changed. The use of extended prophylaxis for 28 days also resulted in an optimal strategy of offering prophylaxis for those with a RAM score ≥ 3.

The sensitivity and scenario analyses also suggest that various individual factors about which there was some uncertainty have no impact on the optimal strategy. These include the costs attributable to PTS, the case-fatality rate for PE and the exact timing of VTE within the 90-day post-admission period.

As these one-way scenario analyses identified that the conclusions were sensitive to the average risks of VTE and major bleeding in the cohort, we conducted a two-way scenario analysis to determine whether the optimal strategy would be different if the risk of VTE or the risk of major bleeding had been underestimated or overestimated in the base-case analysis. Figure 22 shows, although prophylaxis for all is the optimal strategy at the exact level of VTE and bleed risk assumed in the base-case scenario, the use of a RAM would be optimal across a large range of VTE and bleeding risks if that RAM performed similarly to the Pannucci RAM. This suggests that obtaining good evidence on the average baseline risks of VTE and major bleeding in the surgical cohort being offered risk assessment is important for determining which patients should be offered thromboprophylaxis.

FIGURE 22.

Optimal risk assessment strategy for cohorts of surgical inpatients with varying levels of baseline risks (box indicates base-case scenario). It should be noted that the rows and columns do not represent equal steps in risk; red (10) indicates prophylaxis for none is optimal and blue (0) indicates prophylaxis for all is optimal; the numbers in cells of other colours represent the optimal Pannucci RAM score at which prophylaxis should be offered at that level of risk to maximise incremental net monetary benefit when valuing a QALY at £20,000.

Deterministic scenario analyses for elective hip and knee replacement populations

Figure 23 shows the incremental costs and QALYs versus no prophylaxis for each RAM cut-off point. For each series of points, the point closest to the origin shows the costs and QALYs for a Pannucci ≥ 6, and then each successive point shows the next RAM cut-off (≥ 3, ≥ 1 etc.) until the last point shows prophylaxis for all. Separate lines show the results for the main surgical population and for the two specific populations of patients having either elective hip replacement or elective knee replacement. For the main surgical population, two alternative assumptions regarding whether thromboprophylaxis is stopped at discharge or continued post-discharge are shown for comparison.

FIGURE 23.

Incremental costs and QALYs versus no prophylaxis for various RAM cut-off points (Pannucci ≥ 6, Pannucci ≥ 3, Pannucci ≥ 1, PPX for all) for populations requiring PPX after discharge.

The optimal strategy in the base case for the main surgical population, where LMWH is assumed to stop on discharge at 5 days, is prophylaxis for all. This is also the optimal strategy in the knee replacement population where LMWH is assumed to continue post discharge up to 14 days. For the main surgical population, when allowing patients to continue prophylaxis post discharge up to a maximum of 7 days, the optimal strategy (when valuing a QALY at £20,000) is offering prophylaxis using a RAM score cut-off of ≥ 1. The optimal strategy in the hip replacement population where LMWH is assumed to continue post discharge up to 28 days is offering prophylaxis using a RAM score cut-off of ≥ 3.

When applying the costs for the alternative pharmacological prophylaxis strategies using either aspirin or DOACs, but assuming equivalent efficacy to LMWH, the incremental costs were lower than the base-case scenario as these thromboprophylaxis strategies had lower costs. In the case of elective knee replacement patients, the optimal strategy was thromboprophylaxis for all when assuming either 14 days of aspirin or 14 days of a DOAC were offered. In hip replacement patients, thromboprophylaxis for all was optimal when assuming that DOACs were offered instead of LMWH and offering prophylaxis at a Pannucci score of ≥ 1 was optimal when assuming that prophylaxis consisted of 10 days of LMWH was followed by 28 days of aspirin.

The additional costs of offering extended duration prophylaxis, particularly with LMWH instead of DOACs or aspirin, mean that using a RAM to reduce the population being offered prophylaxis is optimal in patients having an elective hip replacement. However, in patients having elective knee replacement, when the duration of prophylaxis is only 14 days, the optimal strategy is the same as in the main surgical cohort where prophylaxis is recommended up to 7 days.

Summary of key findings

• For medical inpatients, pharmacological prophylaxis for all patients without contraindications is likely to be the more cost-effective strategy, when valuing a QALY at £20,000. This assumes that any RAM applied in medical inpatients would perform in a similar manner to the average RAM performance estimated in a single cohort of medical inpatients.

• The optimal prophylaxis strategy was fairly robust under sensitivity analysis in the medical inpatient cohort. The key factors determining cost-effectiveness appear to be the baseline risks of VTE and major bleeding and the increased risk of major bleeding associated with prophylaxis.

• For surgical inpatients, pharmacological prophylaxis for all patients without contraindications is likely to be the more cost-effective strategy, when valuing a QALY at £20,000 and assuming that any RAM applied in a surgical population performs similarly to either the Pannucci or Caprini RAM.

• In the surgical inpatient cohort, the most cost-effective strategy was sensitive to many of the individual model inputs and assumptions tested, with the optimal strategy being a Pannucci score of ≥ 6 (sensitivity of 40% and specificity of 82%) when assuming no risk of PTS after asymptomatic proximal or distal DVT.

• In specific surgical populations who require longer durations of prophylaxis, it was found that pharmacological prophylaxis for all eligible patients is optimal in those having 14 days of LMWH following elective knee replacement whereas offering prophylaxis at a Pannucci score of ≥ 3 (sensitivity of 84% and specificity of 49%) was optimal when offering 28 days of LMWH to patients having elective hip replacement.

• Using a RAM to select patients for pharmacological prophylaxis in medical or surgical inpatients would only be optimal if that RAM had a high sensitivity, similar to that observed for the Padua RAM with a cut-off of ≥ 3 in a mixed cohort of medical and surgical patients (sensitivity 99.9%, specificity 23.7%).

Population

We identified a consecutive, unselected cohort of general medical and surgical patients requiring hospitalisation at each site during the calendar year from 1 January to 31 December 2019. We chose 2019 because of concern that patients admitted during the COVID-19 pandemic might have been assessed for use of thromboprophylaxis differently to prior guidelines. We collated data on all risk assessments performed prospectively at the point of hospital admission, then scrambled episodes into randomly assorted batches of 50 (‘A’ batches) to ensure diversity in speciality presentation and mitigate seasonal bias. We excluded paediatric patients (age < 16), anyone requiring critical care admission (defined as level 2 care or above) and pregnant/post-partum patients due to differential risk, as outlined in the wider study protocol.

Study design

For each patient episode in each batch, we extracted basic speciality data using business intelligence and VTE risk assessments completed prospectively by clinical teams during the provision of routine care. Risk assessments were captured differently at each site, including the use of paper proformas, dichotomous output on electronic prescribing record, or through a detailed structured note within the electronic healthcare record. Example images/screenshots for each site can be found in Appendix 3, Figure 28. All four sites recommended the Department of Health tool in local guidelines, to facilitate initial VTE risk assessment.

Research assistants at each site were subsequently employed to undertake retrospective case note review for each patient episode through shared primary and secondary care electronic health records (EHRs). We extracted descriptive data on relevant clinical outcomes such as the subsequent diagnosis of HAT, major bleeding and CRNMB events as per internationally agreed definitions. 79,80 Data extractors were trained in identification of these variables/outcomes and followed a detailed workflow diagram (see Appendix 3, Figure 29). Data abstractors were blinded to batch allocation, International Classification of Disease, Tenth Revision (ICD-10) coding/HAT database entries and the final analysis plan.

Following case note review, we collated multiple routine data sources for each patient episode to determine the accuracy of these methods in identifying relevant clinical outcomes, identified by research assistants. Data sources for interrogation were identified a priori and included the following: ICD-10 diagnostic codes judged relevant to thrombosis or bleeding by the project management group (PMG) (see Appendix 3, Table 30); Emergency Care Data Set/Systematised Nomenclature of Medicine (SNOMED) codes relevant to thrombosis or bleeding; and local HAT database entries. HAT database entries are co-ordinated by local thrombosis committees and include a contemporary register of all patients diagnosed with acute VTE (using radiology/pathology data), who are subsequently identified as HAT following local root cause analysis. All data sources were interrogated for the duration of hospital stay and up to 90 days post discharge, for each patient episode. Data sources were obtained through routine local business intelligence requests or direct approach to local coding teams. HAT database entries were obtained through local thrombosis committees, where feasible.

Given the potential for very low numbers of outcome events in the wider study population, we augmented the sample with likely positive thrombosis and bleeding cases. We obtained positive cases through ICD-10 coding identification for VTE events (V batches) and bleeding events (B batches) in accordance with the previously selected code list (see Appendix 3, Table 30) and local HAT database entries (H batches) within the relevant study period. Positive cases were scrambled, batched and reviewed in accordance with the general protocol.

The following feasibility criteria were proposed within our original protocol to determine success and accuracy of efficient data methods to identify relevant clinical outcomes:

1. ethical and regulatory approval for proposed methods

2. proportion of outcome events identified by routine data sources that are confirmed by record review (target 100%)

3. proportion of cases with no outcome event identified by routine data sources that have an event identified on record review (target 0%)

4. proportion of each RAM and individual risk variables collectible using efficient data methods (target 90%)

5. proportion of inpatients with data collected (target 90%).

In addition, we proposed using this sample of patients to estimate key parameters for any future implementation study, including the primary outcome event rates for HAT and major bleeding.

Statistical analysis

The accuracy of routinely recorded HAT and bleeding events was compared against direct case note review data for the cohort. Case note review determination of events was assumed to be the gold standard. Data are presented in contingency tables with sensitivity, specificity, positive and negative predictive values along with confidence intervals calculated using the Wilson score method. 139

The primary analysis included patients identified from all sources (A, V, B or H batches) with bleeding and HAT assumed to have not occurred unless coded as such in the relevant data, or detected following case note review. In addition, two pre-planned sensitivity analyses were undertaken:

1. Inclusion was limited to cases identified in routine case review (i.e. ‘A batch patients’ only), with exclusion of all augmented sample cases.

2. Inclusion limited to participants for whom bleeding or HAT was definitively recorded.

We used case note review identification of HAT and bleeding events as the reference standard and report contingency tables for diagnostic test characteristics of all preidentified efficient data sources. We took a conservative approach and interpreted missing or unknown end points as ‘no event’.

The proposed sample size was reliant on an initial iterative developmental phase, including database refinement, user acceptability testing and site trial. We originally planned to identify 3000 inpatients across 4 hospitals during a 12-month study period within the 2-year project plan, dependent on appointment of research assistants and time required for outcome ascertainment. This sample size was designed to allow key parameters to be estimated with a high degree of precision across the whole cohort (standard error < 1%).

Ethical aspects

The study received a favourable opinion from the Proportionate Review Sub-committee of the London – West London & Gene Therapy Advisory Committee (GTAC) Research Ethics Committee (REC) and approval from the HRA and Care Research Wales (HCRW) on 18 September 2019 (reference 19/LO/1303, IRAS project ID 262220).

Participating sites identified members of the clinical care team (research nurses or assistants predominately) to access patient records and extract clinical data using a predesigned and protected Microsoft Excel database with embedded macro function, hosted at site. All data subsequently underwent local de-identification following completion and were exported to an independent team of statisticians at the Clinical Trials Research Unit (CTRU) in Sheffield, for collation and analysis. The REC approved this process and, as researchers outside the clinical care team only used anonymised data, Clinical Advisory Group approval was not required. We were able to access HES through the sites, and we were, therefore, able to undertake the original proposed analysis without involving NHS Digital.

All aspects of the data collection process, export, analysis and oversight were regularly reviewed by the internal PMG including CTRU representation, and an external Trial Steering Committee (TSC), throughout the duration of the project.

Patient and public involvement

Representatives of two PPI groups, Thrombosis UK and Sheffield Emergency Care Forum (SECF) joined the research team and were involved in developing the initial proposal and undertaking the wider study.

The SECF is a patient and public representative group with an interest in emergency care research. The forum has provided PPI for many emergency care research projects over the last 10 years. 140 Thrombosis UK is a charity that aims to identify, inform and partner with the NHS, healthcare providers and individuals to work to improve prevention of VTE and the management and care of VTE events (see https://www.thrombosisuk.org/).

The PPI members were involved in determining the study design and ensuring that the proposal addressed the needs of patients and the NHS, while respecting the needs of potential participants. Their input regarding the importance of providing thromboprophylaxis for potential participants of any prospective cohort study and the need for such a study to yield reliable findings was instrumental in determining our approach to answering the research question. The PPI members also provided input at project management meetings and, where required, in day-to-day running of the project. The members used meetings and surveys of their wider PPI groups to enhance PPI in the project.

In relation to this particular substudy, the PPI members had specific roles in ensuring the acceptability of methods explored in the feasibility study. The efficient methods required to deliver an implementation study involve use of patient data without consent. We planned our methods so that they are compatible with ethical principles and data protection regulation, and would be acceptable to our PPI representatives. However, the feasibility study involved piloting these methods and the potential to adapt the methods to ensure efficiency. The PPI representatives were involved in project management, monitoring of data collection and specifically considered the acceptability of any proposed changes to the methodology throughout the project.

Baseline sampling and augmentation

Over the 12-month feasibility study period we successfully set up and delivered the project across four sites. Patients (2115) were identified and selected for analysis with an original hospital admission occurring in the calendar year 2019. Of these, 107 patient episodes were ineligible due to being pregnant or post-partum women (n = 49); admitted to a critical care environment of level 2 or above (n = 38); children aged under 16 (n = 13) or for unrecorded reasons (n = 7) leaving 2008 episodes for analysis. All episodes were suitable for data extraction and comparison to routine data sources. Ethical and regulatory approvals were secured for all sites. All sites failed to meet their sample target of 750 for reasons mainly related to the SARS-CoV-2 pandemic, including redeployment of research staff to clinical care, delayed local approvals secondary to prioritised pandemic research and a longer than anticipated time for individual outcome ascertainment per case review. Data were missing or the outcome unknown for fewer than 2% of participants for each outcome/site.

Patient episodes showed an even balance of medical and surgical cases, but with a focus on emergency (73.7%) rather than elective (25.8%) admissions. A broad range of subspeciality interests were represented within the cohort. Median length of stay was 3 days [interquartile range (IQR) 3–8] and mean length of stay 7.75 days [standard deviation (SD) 16.5]. Speciality groups with frequencies and percentages are shown in Table 13. The vast majority of patient episodes (1809, 90.1%) were taken from ‘A’ batches, through patients identified as having a VTE risk assessment. The total sample was augmented by 45 (2.2%) patients with potential bleeding events (one B batch) and 154 (7.7%) patients with potential VTE events (one V and two H batches). Sites contributed evenly to the sample with the exception of Manchester; numbers at this site reflect a significant delay to institutional approval during the pandemic, with this research deprioritised. Site and batch numbers are shown in Table 14.

Accuracy of efficient data methods compared to case note review for ascertainment of key clinical outcomes

The accuracy of routine data sources for HAT and major bleeding events compared to case note review are shown in Table 15. The proportion of outcome events identified by routine data sources that were confirmed by case note review was 71% (95% CI 63% to 79%). The proportion of cases with no HAT outcome event identified by routine data sources that had an event identified on record review (target 0%) was 3% (95% CI 2 to 4) for ICD-10/SNOMED coding and 2% (95% CI 1 to 2) for local HAT database entries.

The sensitivity was 62% (95% CI 54% to 69%) for the use of ICD-10/SNOMED coding data to detect HAT events and 81% (95% CI 75% to 87%) for local HAT database entries. Sensitivity by individual site ranged from 45% (95% CI 28% to 63%) to 72% (95% CI 61% to 82%) using ICD-10/SNOMED coding and 68% (95% CI 51% to 84%) to 94% (95% CI 87% to 100%) using local HAT database entries. The sensitivity of ICD-10/SNOMED coding to detect major bleeding events identified by case note review was 38% (95% CI 27% to 50%). Sensitivity by individual site ranged from 22% (95% CI 0% to 49%) to 56% (95% CI 37% to 75%).

These sensitivity values imply moderate rates of missed HAT and major bleeding events through the use of efficient data methods in isolation. Specificity values were > 90% for both outcomes at all sites, implying reasonable accuracy when an event was identified by efficient data methods.

Sensitivity analyses and estimation of key parameters for future implementation work

A sensitivity analysis was conducted using only patient episodes obtained through ‘A’ batches, to remove augmentation of the sample and mitigate bias. The sensitivity of efficient data methods to detect key outcomes identified at case note review remained poor. These results are summarised in Table 16. This analysis also facilitated estimation of key parameters for any future implementation study. We found the HAT event rate on case note review to be 29/1809 (1.6%, 95% CI 1.0 to 2.2) and the major bleeding event rate to be 45/1809 (2.5%, 95% CI 1.8 to 3.2) within this large cohort of hospitalised patients receiving risk assessment and thromboprophylaxis in the context of routine care.

Key feasibility outcomes

Regarding key feasibility outcomes, we achieved ethical and regulatory approvals and the design of the study was acceptable to all sites. The proportion of HAT events identified by routine data sources that were confirmed by record review (target 100%) was 71% (95% CI 63% to 79%) for ICD-10/SNOMED coding and 100% (95% CI 97% to 100%) for local HAT database entries. The proportion of cases with no HAT outcome event identified by routine data sources that had an event identified on record review (target 0%) was 3% (95% CI 2% to 4%) for ICD-10/SNOMED coding and 2% (95% CI 1% to 2%) for local HAT database entries. The proportion of outcome major bleeding events identified by routine data sources that were confirmed by record review (target 100%) was 20% (95% CI 13% to 27%) for ICD-10/SNOMED coding. The proportion of cases with no major bleeding outcome event identified by routine data sources that have an event identified on record review (target 0%) was 2% (95% CI 1% to 3%) for ICD-10/SNOMED coding.

We were unable to use efficient data methods to collect individual risk variables in three out of four sites, due to use of either paper proformas or dichotomous risk assessment without granular data. As such we did not meet the feasibility target of using routine data to collect individual variables for 90% of patients. We were able to collect outcome data in 1745/2008 (87%) of inpatients (target 90%). This was mainly due to inability at the Manchester site to access the local HAT database. Excluding this issue, the other three sites all managed to collect relevant outcome data in > 98% of patient episodes.

Summary and key findings

• In this multicentre cohort study, routine and efficient data methods used to identify HAT events and major bleeding events performed poorly when compared to direct case note review by trained research staff.

• We were able to demonstrate ethical and regulatory approvals for our chosen design across four NHS sites.

• We were unable to demonstrate overall feasibility of an efficient design and did not meet several prespecified criteria set out in the original research proposal:

  • Proportion of outcome events identified by routine data sources that are confirmed by record review (target 100%); VTE 71%, bleeding 20%: Criteria not met.

  • Proportion of cases with no outcome event identified by routine data sources that have an event identified on record review (target 0%); VTE 3%, bleeding 2%: Criteria not met.

  • Proportion of inpatients with data collected (target 90%); 87% across all sites, 98% across three sites: Partially met.

  • Proportion of each predictor variable recorded (target 90%); only routinely recorded at one site: Criteria not met.

  • Proportion of each risk assessment tool completed using available data (target 90%); only routinely recorded at one site: Criteria not met.

• These findings suggest a future data-enabled study using efficient methods would significantly under-report events, leading to inaccurate RAM evaluation.

• In this multicentre cohort of hospitalised inpatients, the HAT event rate was 1.6% (95% CI 1.0% to 2.2%) and the major bleeding event rate was 2.5% (95% CI 1.8% to 3.2%).

Population

We identified a consecutive cohort of general medical and surgical patients requiring hospitalisation during the calendar year from 1 January to 31 December 2019. We chose 2019 because of concern that patients admitted during the COVID-19 pandemic might have been assessed for VTE prophylaxis differently to prior guidelines. 141 We collated data on all risk assessments performed prospectively at the point of hospital admission, then scrambled episodes into randomly assorted batches of 50 to ensure diversity in speciality presentation and mitigate potential seasonal bias. We excluded all patients with an index presentation of suspected VTE, anticoagulated patients and patients with a hospital stay < 24 hours as they were unlikely to require risk assessment or thromboprophylaxis. We also excluded patients requiring critical care admission, children (< 16) and pregnant/post-partum patients due to differential risk, in keeping with the wider study protocol.

Study design

A single-centre observational cohort study, conducted at Salford Royal NHS Foundation Trust. We extracted initial clinical data on patient episodes using hospital admission records and VTE risk assessments completed prospectively by clinical teams during the provision of routine care. All risk assessments were captured through a structured note within the electronic healthcare record, with granular data capture accessible through business intelligence. The structured note mirrors the Department of Health VTE Risk assessment tool (see Appendix 3, Figure 28).

We identified additional necessary variables a priori to enable further completion and comparison of 6 prespecified RAMs for each patient episode, in addition to the Department of Health tool. These variables are listed in Table 17. We subsequently undertook case note review for each patient episode through shared primary and secondary care EHRs up to 90 days following hospital discharge. We extracted data points necessary to facilitate retrospective completion of the Padua, Caprini, Geneva, Kucher, IMPROVE predictive and IMPROVE associative RAMs. We also extracted descriptive data on relevant clinical outcomes, such as the real-world use of pharmacological thromboprophylaxis, subsequent diagnosis of HAT, length of stay and survival to hospital discharge. Data were also extracted on episodes of major and CRNMB as per internationally agreed definitions. 79,80 Data extractors were trained in identification of these variables/outcomes and followed a detailed workflow diagram (see Appendix 3, Figure 29). Data abstractors were blinded to the analysis plan.

Statistical analysis methods

The primary analysis was a comparison of the proportion of patients within the cohort in whom pharmacological thromboprophylaxis would be recommended during hospitalisation, by each RAM. We applied conventional prescribing thresholds for each RAM to the cohort in order to estimate the proportion of patients who would require medication, prior to any assessment of bleeding risk. Prescribing thresholds were taken from author recommendations, international guidelines and nationally endorsed summary tools for practical use. 8,15,16,28,34,49,53,59,142–149 We used ordinal scores to create distribution plots, clustered histograms and to facilitate descriptive analysis. We used dichotomous prescribing thresholds (as above) to calculate proportions with 95% confidence intervals and allow head-to-head RAM comparison. We used the Wilson method to estimate confidence intervals around a proportion, adapted by Newcombe. 139

Missing data were assumed to be negative (risk factor not present) for the primary analysis. A sensitivity analysis was planned with best-case-worse-case imputation for missing data when calculating the percentage of patients who would meet the criteria for prescribing thromboprophylaxis. We estimated that 80% of admitted patients would be classified as eligible for pharmacological thromboprophylaxis, according to the Department of Health VTE risk assessment tool. 8,145 A sample of 250 patients would, therefore, be sufficient to provide estimates of prescribing with a confidence interval range approaching 10%. We proposed a data review following completion of at least 500 case episodes (10 batches).

Ethical aspects

The study received a favourable opinion from the Proportionate Review Sub-committee of the London – West London & GTAC REC and approval from the HRA and Care Research Wales (HCRW) on 18 September 2019 (reference 19/LO/1303, IRAS project ID 262220).

Patient and public involvement

Representatives of two PPI groups, Thrombosis UK and SECF joined the research team and were involved in developing the proposal and undertaking the study.

The SECF is a patient and public representative group with an interest in emergency care research. The forum has provided PPI for many emergency care research projects over the last 10 years (see https://secf.org.uk/ and https://emj.bmj.com/content/33/9/665). Thrombosis UK is a charity that aims to identify, inform and partner with the NHS, healthcare providers and individuals to work to improve prevention of VTE and the management and care of VTE events (see www.thrombosisuk.org/).

The PPI members were involved in determining the study design and ensuring that the proposal addressed the needs of patients and the NHS, while respecting the needs of potential participants. Their input regarding the importance of providing VTE prophylaxis for potential participants of any prospective cohort study and the need for such a study to yield reliable findings was instrumental in determining our approach to answering the research question. The PPI members also provided PPI at project management meetings and, where required, in day-to-day running of the project. The members used meetings and surveys of their wider PPI groups to enhance PPI in the project.

Comparative risk assessment and prescribing outcomes

We analysed 543 hospital episodes with VTE risk assessment performed during 2019. Two-hundred and eighty-nine episodes met exclusion criteria, leaving 254 episodes suitable for data extraction and comparative analysis.

Distribution plots of ordinal scoring for each RAM are shown in Figure 24. The Caprini and Department of Health RAMs appeared to be normally distributed, with the former showing the widest dispersion. All others were skewed to the left, indicating a higher attribution of low risk.

FIGURE 24.

Histograms for ordinal score (x-axis) by RAM using patient numbers (y-axis), with recommended prescribing thresholds (dashed blue lines). Source: National Institute for Health and Care Excellence8; Barbar et al. 15; Blondon et al. 28; Caprini et al. 142; Chopard et al. 143; Kucher et al. 146; Spyropoulos et al. 148

Overall recommendations for prescribing varied substantially between RAMs, ranging from 13% for the IMPROVE associative score to 91% for the Department of Health VTE risk assessment tool. Variation persisted throughout medical (15–92%) and surgical subgroups (10–88%). Key results and estimates of recommended prescribing by speciality (with 95% CIs) are shown in Table 18. Of note, all RAMs recommended higher rates of prescribing in medically unwell patients within this patient cohort. A bar and dot chart of prescribing recommendations with comparative 95% confidence intervals is shown in Figure 25. The Department of Health VTE risk assessment tool was an outlier within this cohort, recommending pharmacological thromboprophylaxis for over 90% of patients with a confidence interval that did not overlap those of any other RAM.

FIGURE 25.

A visual comparison of prescribing proportions per RAM, with 95% CIs. Source: National Institute for Health and Care Excellence8; Barbar et al. 15; Blondon et al. 28; Caprini et al. 142; Chopard et al. 143; Kucher et al. 146; Spyropoulos et al. 148

On descriptive analysis, the variables used by the Department of Health tool with the highest prevalence within our cohort were as follows: one or more significant medical co-morbidities (76%); age > 60 (57%); dehydration (38%); significantly reduced mobility (29%); obesity (20%). These 5 variables were responsible for > 90% cumulative prescribing decisions within the cohort, with negligible addition from the other 14 variables.

All tools were subject by missing data with up to 12% of patients being partly scored, largely due to BMI being unavailable in 22 (9%) of patients. This affected all scores other than the two IMPROVE scores which were scored in all bar four (2%) patients. The impact of this was minimal in terms of the percentage of patients that would be recommended for thromboprophylaxis, since the missing data generally could not overturn a ‘do not recommend’ into ‘recommend’. For the Padua score, a further seven patients would meet the cut-off of 4 points if all missing data were assumed to be abnormal, taking the percentage from recommended from 30% to 33%. The equivalent assumption on other tools led to increases of less than 2%.

Temporary reference list for Figures 24 and 25 and Table 18:

1. Barbar S, Noventa F, Rossetto V, Ferrari A, Brandolin B, Perlati M, et al. A risk assessment model for the identification of hospitalized medical patients at risk for venous thromboembolism: the Padua Prediction Score. J Thromb Haemost 2010;8:2450–7. https://doi.org/10.1111/j.1538-7836.2010.04044.x (PADUA)

2. Blondon M, Righini M, Nendaz M, Glauser F, Robert-Ebadi H, Prandoni P, et al. External validation of the simplified Geneva risk assessment model for hospital-associated venous thromboembolism in the Padua cohort. J Thromb Haemost 2020;18:676–80. https://doi.org/10.1111/jth.14688 (GENEVA)

3. Caprini JA, Arcelus JI, Hasty JH, Tamhane AC, Fabrega F. Clinical assessment of venous thromboembolic risk in surgical patients. Semin Thromb Hemost 1991;17 Suppl 3:304–12. (CAPRINI)

4. Chopard P, Spirk D, Bounameaux H. Identifying acutely ill medical patients requiring thromboprophylaxis. J Thromb Haemost 2006;4:915–6. https://doi.org/10.1111/j.1538-7836.2006.01818.x (IMPROVE PREDICTIVE)

5. Kucher N, Koo S, Quiroz R, Cooper JM, Paterno MD, Soukonnikov B, et al. Electronic alerts to prevent venous thromboembolism among hospitalized patients. N Engl J Med 2005;352:969–77. https://doi.org/10.1056/NEJMoa041533 (KUCHER)

6. Spyropoulos AC, Anderson FA, Jr., FitzGerald G, Decousus H, Pini M, Chong BH, et al. Predictive and associative models to identify hospitalized medical patients at risk for VTE. Chest 2011;140:706–14. https://doi.org/10.1378/chest.10-1944 (IMPROVE ASSOCIATIVE)

7. National Institute for Health and Care Excellence. Venous thromboembolism in over 16s: reducing the risk of hospital-acquired deep vein thrombosis or pulmonary embolism (NG89). nice.org.uk: NICE; 2018. URL: www.nice.org.uk/guidance/ng89 (accessed 22 July 2022). (DOH SCORE)

We found no cases where an alternate RAM recommended thromboprophylaxis but the Department of Health tool did not, in keeping with the low threshold (≥ 1 risk factor) of the latter tool. Results for each alternate RAM compared against the Department of Health tool are shown in Appendix 3, Figure 30.

Estimation of other key parameters for future implementation research

Baseline demographics and relevant clinical outcomes for the final cohort of patients are shown in Table 19, by speciality. In this cohort, surgical patients appeared to be younger with lower rates of medical comorbidity and lower rates of significant mobility reduction than medical patients.

We found six cases of HAT within this cohort, with a subsequent overall event rate of 2.4% (95% CI 1.1% to 5.1%). All six of the HAT cases in this cohort were diagnosed in patients who were prescribed thromboprophylaxis during hospital admission in line with an initial assessment using the Department of Health tool. It is, therefore, impossible to estimate whether application of other RAMs would have affected relevant clinical outcomes; all patients were receiving thromboprophylaxis.

Summary and key findings

• The majority of sites approached within this substudy had no mechanism for accessing individual variables recorded at VTE risk assessment, other than case note review of written notes. Only one site recorded risk variables electronically. This raises significant challenges for future implementation research looking to compare RAMs using routine data or through retrospective methodology.

• In this single-centre cohort of predominately emergency admissions, the proportion of patients receiving thromboprophylaxis in current practice was 86.2% (95% CI 81.2% to 90.1%). Over 90% of surgical inpatients received prophylaxis in the context of routine care.

• Prior to evaluation of bleeding risk, existing RAMs produce highly variable rates of recommended pharmacological prophylaxis when applied to the same cohort of medical and surgical hospital inpatients, ranging from 13% (95% CI 9.1% to 17.2%) to 91% (95% CI 86.3% to 93.6%).

• The national UK Department of Health VTE risk assessment tool conferred higher rates of potential prescribing in this cohort than any other RAM, by > 25%.

Population

We identified local VTE leads through the National Nursing and Midwifery Network, VTE exemplar centre network and the collated list utilised by the recently published GIRFT project. VTE leads represented a range of clinical disciplines, including anticoagulation pharmacists, specialist nurses, doctors and ward managers.

Survey design and dissemination

We designed the survey using Smartsurvey.co.uk. We utilised the VTEAM PMG to refine the questions through iterative development. The final survey included free text answers, radio button responses and multiselect options. Regarding VTE risk assessment, we included questions on positive characteristics (recording the definitive presence of a risk factor) and negative characteristics (recording the definitive absence of a risk factor). The survey was trialled on three members of the PMG prior to circulation, with incorporated feedback. A copy of the survey is available in Appendix 4.

We used single-stage sampling over a four-week period, from 20 October 2021 to 17 November 2021 inclusive. We disseminated the survey by personal e-mail, with facilitated access through a weblink, QR code and direct web address. We sent weekly reminders to all potential participants throughout the survey duration. We asked for trust identification to guard against multiple participant responses. Given the time frame, need for site identification and direct personal contact through an established list of VTE leads, we did not add security measures to the survey. We did not seek specific ethical approval for this survey work and assumed implied consent from clinicians for participation in the survey.

Statistical analysis

We did not perform a sample size calculation for this survey. All analyses were proportional or descriptive. No weighting was applied to any variables.

Methods of VTE risk assessment and data capture

The majority of respondents (48%) reported using EHR to conduct VTE risk assessment within an NHS setting, although 20% respondents reported using a hybrid approach (paper prof orma and EHR) at site dependent on ward environment. A large proportion of sites (32%) reported use of a paper pro forma only.

The majority of respondents noted their site method of risk assessment to report positive VTE characteristics (57% for paper pro forma, 73% for EHR) and the final decision on thromboprophylaxis (57% for paper pro forma, 64% for EHR). Routine capture of negative risk characteristics was lower overall, with only 19% of sites using paper pro forma and 32% of sites reporting capture.

The majority of sites (52%) responding reported that they were uncertain as to whether they could access granular data from VTE risk assessments conducted within routine care, through local business intelligence requests.

Willingness to participate in future research

Enthusiasm appeared higher for participation in future research overall. More respondents were willing to participate in future observational research on VTE risk assessment (100%) compared to interventional research, including randomisation to different models of assessment (72%). When further questioned on potential methodologies for future interventional research, most respondents favoured either allocation of the whole trust site or certain wards to intervention/control within the context of a wider national cluster RCT, above a longitudinal step wedge design or individual patient randomisation. This preference continued through scenarios involving comparison of RAMs, higher dosing strategies for higher-risk patients and opt-out strategies for default prescribing. Pie chart summaries of proportional response are shown in Figure 26 below.

FIGURE 26.

Proportional summary preferences for research design methods on future prophylaxis studies.

Thematic analysis of free text responses

We received 21 free text responses, listed in Appendix 4. Themes of response included support for potential study designs evaluating opt-out strategies for prophylaxis/mandatory prescribing, higher dosing strategies for higher-risk patients and further refinement of bleeding risk scores. Several responses highlighted support for cluster or broad implementation study designs in this area, citing the existing challenges of standardisation and the potential for individual randomisation strategies to increase confusion. In particular, several respondents referred to the ‘tick box’ nature of VTE risk assessment and highlighted the increasing medical scrutiny on the value of complex risk assessment.

Several respondents highlighted areas of VTE risk assessment they felt required further specific research: exclusion of medical patients at ‘baseline’ mobility; better clarification of dehydration and other subjective risks; evidence that routine care follows national guidance on assessment and prescribing; validity of the Department of Health risk assessment tool; defining higher-risk categories.

Concerns were highlighted regarding the variation in care by speciality group, and the potential for more nuanced risk assessment to benefit certain elective surgical groups. In addition, several responders highlighted the potential delays in adapting EHRs as a concern for future research designs looking to change the manner/standard of risk assessment. Finally, several respondents noted concern about an opt-out strategy as moving away from current evidence themes on individualisation of approach.

Systematic review of RAMs

Sections of this chapter have been reproduced from Pandor et al. under licence CC BY 4.0. 17 In a systematic review of 51 observational studies evaluating RAMs for predicting the risk of developing VTE in hospital inpatients, we found that VTE RAMs have generally weak predictive accuracy. The studies validating these models are heterogeneous and most have a high risk of bias. Lack of methodological clarity was common, leading to difficulty in assessing the applicability of the individual study results. VTE prophylaxis was used in around half the study populations, with markedly varying frequency, and was not reported in most of the other studies. This is an important potential source of bias and heterogeneity between studies.

Our systematic review updated previous reviews to include more recent studies, but did not fundamentally alter the conclusions that the available evidence shows weak predictive accuracy and has significant methodological limitations. 8–10 VTE RAMs may be useful, despite the limited evidence and apparently weak accuracy, if they can identify patients with a higher risk of VTE who are more likely to benefit from prophylaxis. Decision-analytic modelling allowed us to explore whether VTE RAMs can provide more cost-effective use of VTE prophylaxis.

Economic evaluation

Using decision-analytic modelling to address the cost-effectiveness of VTE risk assessment, we made several key findings. For medical inpatients, the balance of VTE risk and risk of major bleeding, combined with the performance observed for RAMs in medical cohorts, means that prophylaxis for all is the optimal strategy within the model. The key factors determining cost-effectiveness in the medical cohort appear to be the baseline risks of VTE and major bleeding and the increased risk of major bleeding associated with prophylaxis. In the analysis for surgical inpatients, the optimal strategy within the model is to offer prophylaxis to all. However, the scenario analyses found that the optimal strategy was sensitive to many of the individual model inputs and assumptions tested, with the optimal strategy varying from a Pannucci score of ≥ 6 (sensitivity of 40% and specificity of 82%) to offering prophylaxis for all.

In the scenario analyses exploring specific surgical populations who require longer durations of prophylaxis, it was found that prophylaxis for all is optimal in patients having elective knee replacement whereas offering prophylaxis at a Pannucci score of ≥ 3 (sensitivity of 84% and specificity of 49%) was optimal when offering extended duration prophylaxis with LMWH to patients having elective hip replacement. The difference in optimal strategy between the elective hip replacement population and the main surgical cohort appear to be driven in part by the high cost of providing extended-duration prophylaxis, as the scenario analysis assuming 28 days of LMWH in the general surgical population also resulted in a Pannucci score of ≥ 3 being optimal. In the elective knee replacement population, the increased risk of bleeding associated with LMWH was lower meaning that there was less of a trade-off between a reduced VTE risk and an increased bleed risk in this group.

Our findings differ slightly from the previous cost-effectiveness analysis from a US health system perspective by Le et al. , which found that LMWH is cost-effective for medical inpatients when the risk of VTE is over 1%. 11 Le et al. discussed the use of RAMs to select patients with a risk of < 1% but did not explicitly model using a RAM to identify patients with a risk of < 1%. The difference is explained by the fact that Le et al. only included proximal symptomatic DVT and PE within their analysis and their average VTE risk in medical inpatients was assumed to be 1.4%. Under these conditions, the optimal strategy using our model would be a Padua score of ≥ 3 (when valuing a QALY at £20,000 or £30,000). Green et al. report a risk of < 1% in those with a Padua score < 3, suggesting the analyses are consistent when the VTE risk is defined in a similar manner. 39

Assessing the feasibility of efficient data methods and estimating key parameters for future implementation research

The findings from our cohort study suggest that currently available, routine and efficient data methods which can potentially identify VTE or major bleeding events during hospital admission or within 90 days of discharge, are not sufficiently sensitive for use in a large data-enabled study. We failed to achieve feasibility for a number of predefined metrics and concluded that use of routine data to identify outcomes would be highly likely to miss important VTE and bleeding events, and may erroneously identify false-positive events. We also identify significant limitations in the ability of efficient methods to identify individual risk variables and facilitate RAM comparison at scale in any future work. The majority of our sites did not collect contemporaneous data on risk assessment in a digital, or easily accessible format.

In addition, we provide estimates of key parameters for any future implementation study, based on an observational cohort study of > 2000 patients across four NHS sites. We report a HAT event rate of 1.6% (95% CI 1.0% to 2.2%) and a major bleeding event rate of 2.5% (95% CI 1.8% to 3.2%) in the context of routine NHS care for hospital inpatients. We also identify high rates of pharmacological prophylaxis in the context of routine care and risk assessment using the Department of Health tool, noting prescribing rates of 86.2%, 80.4% and 93.7% for all, medical and surgical patients, respectively. When comparing RAMS with attempted validation, we demonstrate highly variable rates of potential pharmacological thromboprophylaxis within the same cohort of general medical and surgical patients, in a UK NHS setting. Prescribing recommendation rates at published thresholds varied from 13% to 91% between RAMs. These data provide key estimates of current and potential prescribing rates, and potential event rates for any future studies looking to compare VTE RAMs, or assess implementation of novel strategies.

Finally, recent survey work suggests that > 80% of UK NHS sites continue to use the Department of Health tool for VTE risk assessment and that a high proportion use weight-adjusted dosing of pharmacological thromboprophylaxis. Our survey data also found that many sites continue to conduct risk assessment via paper proforma only and that very few sites record the absence of risk characteristics. These findings suggest that further comparative research on individualised risk assessment reliant on routine data collection, is unlikely to be feasible or reliable. These are important points to consider in the design of any future study.

Systematic review of RAMs

Our systematic review work has a number of strengths. The review was conducted with robust methodology in accordance with the PRISMA statement and the protocol was registered with the PROSPERO register. Clinical experts were involved throughout as checkers and to assess the validity and applicability of research during the review. We reported descriptive statistics to provide insight into the limited evidence base applicable to the subject matter, and the scientific concerns regarding validity of the data. However, there are a number of potential weaknesses. Decisions on study relevance, information gathering and validity were unblinded and could potentially have been influenced by pre-formed opinions. However, masking is resource intensive with uncertain benefits. The studies of risk prediction were a combination of prospective cohorts and retrospective health database registries. Both have significant limitations. Retrospective studies of health database registries may have large numbers but may be limited by poor data quality and failure to accurately ascertain outcomes. Prospective cohorts may have better quality data but with smaller numbers lack statistical power.

We were unable to undertake meta-analysis or statistical examination of the causes of the observed heterogeneity. Potential sources of heterogeneity include variation in study design, the study population, how RAMs are implemented, outcome definition and measurement and the use of thromboprophylaxis. The latter point is highlighted above as a key finding.

Economic evaluation

The main strength of the economic evaluation is that it is able to evaluate the trade-off between costs, benefits and harms in a systematic manner based on the existing evidence base. One of the key uncertainties in the modelling is the incidence of VTE in cohorts not having thromboprophylaxis and the incidence of major bleeds outside of clinical trials. To address this uncertainty, the two-way scenario analyses on VTE and bleeding risk provide information on how robust the conclusions would be if the average risk of bleeding and the average risk of VTE across the medical and surgical cohorts has been underestimated or overestimated. However, these analyses do not necessarily provide information on whether a different strategy would be optimal in a select cohort of patients known to have higher or lower risks of bleeding or VTE. This is because the performance of the RAMs has been measured in broad cohorts of patients and the RAMs may not perform as well in very select cohorts chosen to have a very high or very low risk of VTE or major bleeding.

The economic analysis assumed that the population did not have characteristics associated with an increased risk of bleeding, and therefore had a low risk of bleeding overall. The findings, and specifically the conclusion that prophylaxis for all is the optimal strategy, do not, therefore, apply to populations or individual patients with an increased bleeding risk. The analyses showing whether the overall health impact is positive or negative at varying levels of VTE and bleeding risk (see Figures 11 and 12) are intended to show how bleeding and VTE risks might interact but should not be used to guide individualised decisions for patients. Bleeding and VTE often share the same risk factors, and interactions may occur between risks and between risk factors. Furthermore, our analyses assume that the RR for bleeding associated with prophylaxis is constant and increases in proportion to baseline bleeding risk. It is conceivable, however, that the risk factors for bleeding could increase the RR associated with bleeding. Taken together, these issues mean that the decision to provide VTE prophylaxis in the presence of increased bleeding risk is extremely complex, and best undertaken on an individual patient basis by an experienced clinical decision-maker. In this scenario, there remains a potential option to use mechanical thromboprophylaxis through graduated compression stockings or intermittent pneumatic compression devices. However, the evidence to support use of either device in medically ill patients is limited. We opted not to evaluate alternative methods of mechanical prophylaxis for patients at high risk of bleeding within the model, for the reasons outlined in Chapter 4.

A key limitation of the economic analysis is the heterogeneity in the estimates of RAM performance across the various cohorts identified in the systematic review. The secondary analysis using performance data for the Padua RAM from the cohort reported by Elias et al. provides different conclusions about the optimal strategy than found in the base-case analysis, due to the better performance of the Padua RAM in this cohort compared to the performance of this and other RAMs in other cohorts. The Padua RAM did not perform consistently better than other RAMs in other studies, so the higher accuracy in the Elias study probably reflects study characteristics rather than superiority of the Padua RAM. Our use of these data was intended to explore the potential for an accurate RAM to be cost-effective, rather than draw conclusions regarding a specific RAM or population.

Estimating key parameters for future implementation research and assessing the feasibility of efficient data methods

We engaged a combination of digitally mature and paper-based NHS sites within our cohort study and supplemented the findings with survey work, to obtain an accurate picture of VTE risk assessment conduct in a contemporary UK setting. We based key parameter estimation on an updated systematic review (workstream 1) and strict consensus definitions for VTE and bleeding events. We used statistical support, topic experts and research staff to iteratively develop our data collection tool and designed a clear workflow diagram to ensure compliance with data collection methods and limit subjective interpretation of case note data.

However, there are a number of potential limitations with design which could impact on the outcomes from workstream 2. We recruited only teaching hospital sites with university affiliation, two of which were VTE exemplar centres and three of which were tertiary centres, which may limit the external validity of our key parameter estimates. Regarding the evaluation of risk variables and estimation of key parameters, we opted for a cohort design on the early assumption that VTE risk assessments would be completed prospectively and accurately at all sites, in line with NICE guidance and NHS contract standards,86 and that these data would be retained within the healthcare record. Limitations in electronic data capture at several chosen sites led to this aspect of the project becoming single centre. As such, we were unable to undertake detailed work to determine whether other sites could implement efficient methods of recording risk assessment data, but the issues identified with limited IT and service pressures suggested that substantial work would have been required and success was unlikely.

In addition, the complexity of identified RAMs for comparison necessitated retrospective case note review to identify additional variables. This may have introduced bias in assessment and downgrading of other RAMs due to absence of inquiry and information in the case notes. Other limitations include a relatively small sample size and differential ascertainment for the Department of Health VTE risk tool (collected in real time by clinicians) compared to other RAMs (collected through case note review by research teams). In addition, we do not report estimation of bleeding risk within this work. Although this is a vital aspect of VTE risk assessment, the six RAMs presented for comparison do not integrate any form of bleeding assessment. It is therefore unclear how these RAMs would perform comparatively within a real-world population of medical and surgical inpatients. It is likely that the proportions of patients receiving prophylaxis in practice would be lower than those recommended once bleeding risk is taken into account. The exclusion criteria applied to this cohort limit the generalisability of our results.

Regarding the feasibility of efficient data methods, research assistants across sites varied in seniority and clinical experience. Although all sites had a principal investigator with clinical expertise available for queries and strict working definitions for outcome events, this may have introduced variation in reporting. Finally, we did not achieve our intended target of 3000 patients. Reasons for this are detailed in the above chapters. However, it is important to note that the overall results within our cohort of 2008 patients are well outside of feasibility targets and that sensitivity values using routine data sources to identify key outcomes are universally poor. We do not envisage that further cases would have significantly affected these values to the point of reaching feasibility, even with a high level of accuracy.

Systematic review of RAMs

The present review is the largest and most comprehensive systematic review in this field to date. It includes 18 recent studies published since the completion of the previous systematic review. These studies are consistent with the previous literature in that they report modest performance of the assessed RAMs, with limitations in methodology and reporting preventing further analysis. The conclusion of this review therefore concurs with previous systematic reviews: there is insufficient evidence to recommend one RAM over another.

Economic evaluation

Although the optimal strategy of prophylaxis for all is the same in the medical and surgical cohorts, there is a more evident trade-off between the benefits of reduced VTE and the risks of increased bleeding for surgical patients as shown by the table of clinical outcomes in Table 4. This is why the optimal strategy is more sensitive to changes in the model inputs and assumptions in the surgical population. Parameters related to the long-term benefits of preventing VTE, such as the risk of PTS and the utility decrement of PTS, were identified as being important in the EVPPI analysis and sensitivity analyses for the surgical cohort. Factors related to long-term risks of chronic complications such as PTS do not appear to be important in determining the optimal strategy in the medical cohort, which may be because the medical cohort is older and, therefore, has a shorter life expectancy over which to accrue benefits from avoiding long-term complications. It is also important to recognise that while the risk of any major bleeding is higher in the surgical population, the risk of fatal bleeding or a non-fatal ICH is higher in the medical population. Medical inpatients are also more likely to die following a PE, and more likely to die in the year following admission even without experiencing VTE or major bleeding. However, sensitivity analyses applying a lower case-fatality rate for PE in medical inpatients or a higher case-fatality rate for PE in surgical inpatients do not result in a change in the optimal strategy.

In the secondary analysis using RAM performance data from the study by Elias et al. , the optimal strategy was to use a RAM in both medical and surgical patients. This is because RAM performance from Elias et al. was better than assumed in the base case for either the medical or surgical patients. For example, Elias et al. reported a sensitivity of 99.9% and a specificity of 23.7% for a Padua score of ≥ 3, whereas the equivalent values reported by Greene et al. in an exclusively medical cohort, were 49.3% and 73.0%, respectively. Performance data for the Padua RAM were not evaluated in an exclusively surgical cohort so a direct comparison cannot be made. These results suggest that if a RAM were to be useful for determining which patients should receive prophylaxis, it would need to have very high sensitivity similar to that observed for the Padua RAM in the study by Elias et al. If a future RAM performed similar to the Padua score ≥ 3 and a cut-off of 3 was applied, the model predicts that we would expect 80% of surgical patients and 84% of medical patients to receive prophylaxis. NICE reported that according to national figures, over 70% of medical patients receive prophylaxis using the Department of Health Risk Assessment tool, with some trusts offering prophylaxis to over 90% of medical inpatients. 8

Estimating key parameters for future implementation research and assessing the feasibility of efficient data methods

Our findings are in line with previous comparative research on VTE risk assessment. Systematic review data have reported prescribing recommendation rates ranging from 30% to 90% for medical patients with the application of different RAMs. 10,145,152,153 We have not identified previous work describing the comparative prescribing rates of different RAMS in a cohort of surgical patients and therefore provide new comparisons in this area. The baseline data on hospital length of stay, HAT event rates, major bleeding rates and prescribing rates are all in keeping with published evidence reviews, suggesting internal validity of our cohort. 8

The findings in our study highlight the ongoing variation in clinical care, of both medical and surgical patients, when different RAMs are applied to similar populations. In addition, our findings support previous research regarding the risk thresholds employed using the Department of Health VTE risk tool; this RAM conferred the highest proportion of pharmacological prophylaxis to both medical and surgical patients than any of the other six RAMs, with a 36% increase in recommended prescribing for surgical patients, and a 21% increase in medical patients, compared to the next highest prescribing model. Such high prescribing recommendations invite two key questions. First, does this tool provide better clinical or cost-effectiveness compared to others? Second, if over 90% of patients are likely to receive pharmacological thromboprophylaxis following use of the RAM, is there any real value in asking clinical teams to perform subjective and complex risk assessment? Switching to an ‘opt-out’ approach based on contraindications and bleeding risk has the potential to confer advantage, through mitigation of time and opportunity costs while concurrently encouraging default and reliable prescribing options. Such changes could also mitigate the key recurring issues with preventable HAT, most commonly resulting from errors in assessment, linked prescribing or drug administration. 150

Our findings of limited sensitivity using routine data methods within a UK setting are consistent with previous international work in this area. An early comparison of HES data to general practice records in England reported in 2012, initially concluded reliable identification of vascular disease using HES data (derived from ICD-10 coding data). 154 However, this analysis was restricted to PE from a VTE perspective and sought only to correlate disease states, rather than identify new case episodes. Several authors have also used primary care research data sets correlated to evidence of anticoagulation or other secondary care data to identify VTE events, with reported reliable capture. This work does not seek to discriminate between index presentation of VTE and HAT. 155,156 A systematic review, with searches run in July 2010, published in 2012, summarised findings on this topic from 19 studies. The positive predictive value (PPV) for PE ICD codes ranged from 24% to 92%, with higher values from certain combinations of codes. PPV values for DVT codes ranged from 31% to 97%. More recently, a cross-sectional North American study compared ICD-10 codes for VTE in hospitalised medical patients to a ‘gold standard’ manual review of clinical data in 4000 patients. 157 The authors report a sensitivity of 63%/PPV of 43% for any DVT and a sensitivity of 83%/PPV of 70% for PE, implying further discrepancy between types of VTE.

Several authors have also experimented with composite data sets and diagnostic/procedural/disease coding combinations. One study combined ICD codes for VTE with a common procedural terminology code for a VTE Diagnostic Study plus at least one of the following within 30 days of diagnosis: pharmacy script for anticoagulation, placement of an inferior vena cava filter, or death. 158 This algorithm still lacked sensitivity, reporting a value of 0.67 (0.60, 0.73), a specificity of 0.99 (0.98, 0.99), a PPV of 0.92 (0.87, 0.96) and a NPV of 0.94 (0.93, 0.95). Alotaibi et al. subsequently combined routinely collected ICD-10 coding data with imaging procedure codes to identify VTE events over a 10-year period compared to case note review and reported highly specific results but limited sensitivity, in line with our findings [74.83% (95% CI 67.01 to 81.62) and 75.24% (95% CI 65.86 to 83.14) for PE and DVT, respectively]. 159 Verma et al. report findings using natural language processing (NLP) algorithms for digital interrogation of radiology reports in a large cohort of hospitalised medical patients. 157 The authors conclude a high level of accuracy for NLP algorithms, reporting sensitivities of 94%/91% and PPVs of 90%/89% for DVT and PE, respectively. Such algorithms would require external validation in a UK setting and would be unable to distinguish HAT from index presentation of VTE, in isolation. Reliance on such algorithms in the context of low event rates would also lead to potential bias from false-negative results.

In 2017, Baumgartner et al. highlighted further issues through interrogation of an administrative coding database, looking to determine the accuracy of ICD coding for new episodes of recurrent VTE in patients with a prior history. 160 Only 31.1% of coded encounters were verified by reviewers as true recurrent VTE. More recently, Pellathy et al. have conducted similar work to VTEAM within the USA, comparing accuracy of HAT diagnoses made through administrative coding to manual case note and radiology review. 161 The authors report only 40% of HAT cases identified through routine coding were confirmed by case note review and 45% of HAT confirmed through diagnostic test records lacked corresponding ICD codes. This body of evidence, now including our recent work, suggests caution with the use of routinely collected outcome data within any future study where HAT events are an outcome of significant importance.

There are multiple potential explanations for the limited performance of routine data to identify HAT. The condition is a temporal phenomenon and routine coding data can therefore mistake index presentation with VTE as HAT, or fail to code subsequent HAT following index admission with alternative pathology; there are clear international guidelines supporting outpatient diagnosis and management of VTE, so genuine cases of HAT may not necessarily require hospital admission or relevant administrative coding; and prior diagnosis of VTE can often be coded during repeat hospital attendance and mistaken as HAT. Many UK hospitals attempting root cause analysis of HAT cases in line with NHS contract standards have developed pathways to mitigate these issues and improve identification, through local reporting arrangements with radiology, ultrasound and pathology services. Such arrangements often work well transiently but are reliant on individuals and reporting systems subject to human error. These issues are reflected in our findings from workstream 2, which report a PPV of 100% for HAT RCA database findings, but limited sensitivity (implying missed cases still occur despite a systematic approach).

More generally, these findings raise questions about the current enthusiasm for data-enabled trials and epidemiological research using outcome measures reliant on routine data methods. 162 Such concepts are inherently attractive to researchers and patients, particularly in topic areas with low event rates. However, complex outcome measures which require temporal evaluation and qualification against prior disease states are unlikely to be reliably delivered through use of routinely collected data in isolation. Several explanations have been proposed for current demonstrations of inaccuracy. Relevant data may contain coding errors arising from ambiguous documentation by physicians and inconsistent handling of recurrent cases. 162,164 Recent case studies have reported significant amounts of missing data and poor interobserver agreement between routinely collected EHR data accessible through HES and case report form (CRF) evaluation. 165 Electronic records contain an abundance of free text, but often lack necessary intelligence to classify patient episodes appropriately, or allow processing and comparison of routinely collected data. 166 Recent evidence also suggests that more complex outcomes will be less easily identified through routine coding; a registry study of Medicare claims following mitral valve repair compared to formal adjudication, reported a PPV for mortality of 97%, heart failure requiring hospitalisation of 69%, bleeding of 40% and renal failure of 19%. 167 In addition, the time and effort needed to acquire necessary permissions for national routine coding data or to orchestrate data linkage can be substantial. A UK clinical trials unit recently reported a digital request to facilitate routine data comparison to CRFs in the context of a RCT, highlighting a negotiation process over consent that took several years and receipt of data 15 months following application. 168 Such time frames may only be realistic within the context of continually adaptive design trials and add further concern as to whether the juice is actually worth the squeeze.

In their call to action, Sydes et al. discuss supplementation of trial-specific follow-up as an option to realise the full potential of data-enabled research. 162 Such an approach has potential merit to attempt identification of potential HAT, given the high PPV and high specificity of routine data sources. These cases would require subsequent adjudication and this approach would still fail to address the low sensitivity reported using routine data methods. Our findings suggest the use of routine data sources in isolation would be insufficiently accurate to capture complex VTE and bleeding outcomes within the context of a pragmatic trial context, but supplementation may be a viable use of resources. In addition, routine data sources may have a role in other research contexts, such as identification of cases for qualitative work, case–control studies, targeted individual follow-up or downstream survey work.