Prediction of risk of recurrence of venous thromboembolism following treatment for a first unprovoked venous thromboembolism: systematic review, prognostic model and clinical decision rule, and economic evaluation

Study design

Studies of any design [e.g. cohorts, randomised controlled trials (RCTs)] or systematic reviews that developed, compared or validated a prognostic model (or clinical prediction rule based on a model) utilising multiple (at least two) predictors to predict the risk of recurrent VTE or adverse outcome (mortality or bleeding) following cessation of therapy for a first unprovoked VTE.

Patient group

Patients aged ≥ 18 years with a first unprovoked VTE where the patient has received at least 3 months treatment with an OAC therapy. Studies with mixed populations (including those outside the remit), were included provided that appropriate data for the defined group of patients were extractable.

Setting

Studies in any setting were included.

Potential prognostic models

Studies must have reported a prognostic model utilising multiple prognostic factors to predict the risk of recurrent VTE or adverse outcome following cessation of therapy for a first unprovoked VTE. A prognostic model was defined as a combination of at least two predictors within a statistical model, used to predict an individual’s risk of outcome (e.g. VTE recurrence).

Patient selection

Rodger et al. 9 used a prospective cohort study with consecutive unselected patients from 12 tertiary care centres in four countries. The prospective cohort design ensures that predictor information can be collected blinded of patient outcome. Inclusion criteria were patients treated with low-molecular-weight heparin for at least 5 days, OAC therapy for between 5 and 7 months with an INR between 2 and 3, and with no recurrence during therapy. Patients were excluded if they would not provide consent, were aged < 17 years, had already ceased therapy, required therapy for another reason, were inaccessible for follow-up, were being treated for unprovoked recurrence or known thrombophilia. Thrombophilia testing was not performed as part of the study, but any patients with known thrombophilia prior to the start point were excluded.

The definition of unprovoked VTE used by Rodger et al. 9 was based on an absence of the following provoking factors:

• leg fracture or plaster cast

• immobility for > 3 days

• surgery using general anaesthetic (in the 3 months prior to the index event)

• diagnosis of cancer (in the past 5 years).

This definition of unprovoked VTE therefore includes women who have currently, or previously, used OCs or HRT. Hormone therapy should be considered as a provoking factor for recurrent VTE; though some argue that because the effect of hormone therapy is weak it may be included within the definition. 41,44 Strictly, the definition of unprovoked VTE refers to a VTE in the absence of any transient risk factors, hence why hereditary thrombophilia can be considered unprovoked VTE. 45 As hormone therapy is a transient risk factor patients with a history of hormone therapy could be considered provoked. As a result there may be patients included in the model development who are at potentially lower risk than other unprovoked patients. This could lead to biased estimation of effect sizes within analyses, particularly for the effect of sex, which could lead to poor validation in external populations which may not include patients with hormone-related index events.

Outcomes

The primary outcome of the study was recurrent VTE, with associated deaths also recorded. Suspected DVT was confirmed by compression ultrasound, whereas suspected PE was confirmed by high-probability ventilation/perfusion scanning and/or spiral computed tomography. All events were adjudicated by independent physicians who were blinded to predictor information. As such, outcomes were pre-specified with the same definition and assessment used for all patients reducing the risk of detection bias, where there are differences in the determination of outcomes. The outcome definition excluded any candidate predictors again reducing the risk of detection bias and overall providing a low risk of bias with regard to outcome assessment.

Predictors

Rodger et al. 9 report that information was collected on 69 potential predictors based on evidence from a pre-specified systematic review. Summary information was provided for 21 candidate predictors (with categories equating to 39 candidate predictors) including:

• sex

• ethnicity

• age (years)

• weight (kg)

• height (cm)

• BMI (kg/m²)

• abnormal baseline imaging

• abnormal baseline compression ultrasound

• D-dimer (µg/l)

• homocystine (mmol/l)

• haemoglobin (g/l)

• heterozygous for prothrombin gene mutation

• factor VIII (U/ml)

• factor V Leiden

• ventilation/perfusion scan or pulmonary vascular obstruction result < 95%

• post-thrombotic signs

• history of chronic obstructive pulmonary disease or emphysema

• family history of VTE

• previous secondary VTE

• use of OC in year before event

• HRT in year before the event.

All predictors were measured before outcomes occurred, with laboratory predictors measured from samples taken while still on OAC therapy, and predictor definitions were consistent for all patients. Therefore the risk of selection bias due to differences in the characteristics of patients, and also the risk of reporting bias through differences in the reporting of predictors based on outcomes, could be considered low.

The use of a systematic review to identify potential predictors, the consistent updating of the systematic review and the assessment of all identified predictors provides confidence that there is a low risk of selection bias where patients are selected based on their characteristics, and attrition bias based on exclusions from analyses.

Rodger et al. 9 categorised all continuous predictors that had a p-value < 0.2 at univariable analysis, by dichotomising at various thresholds and identifying ‘optimal’ thresholds as those with the highest chi-squared value. The process of categorisation was therefore completely data driven which often leads to reporting bias, where the most significant results are presented. 46 Rodger et al. 9 indicate that dichotomised predictors were used to enhance the applicability and acceptance of the proposed clinical decision rule. Although it is a valid reasoning, with researchers and clinicians looking for parsimony in any decision rule,12 the dichotomisation of continuous predictors leads to a loss of information. Dichotomisation splits patient’s risks into two groups treating patients on either side of a threshold as distinctly different, when in reality they may be very similar on the original scale. 47 Best practice recommends that continuous predictors remain continuous in any analysis and to investigate non-linear associations. 47–50

Sample size and patient flow

A total of 646 patients had at least one follow-up visit and were included within the analysis. Only 600 of these patients completed follow-up, with 14 lost to follow-up, 10 patients dying after their first follow-up, nine patients withdrawn and 13 patients restarting OAC therapy for another reason. There were 91 events recorded out of the 646 patients included in the analysis, which could be considered insufficient power to investigate the 36 predictors described within the article. As a rough guide at least 10 events are required for each candidate predictor being investigated to give enough power for the analysis to yield valid conclusions. 51 Therefore, to investigate 36 predictors would require roughly 360 events. It is unclear from the article whether or not model building included the 36 predictors summarised within the article, or the 96 predictors for which information was recorded: roughly 960 events would be required to adequately power such an analysis. 51 As the study was substantially underpowered (with a maximum of 2.5 events per predictor), the results of model building and conclusions drawn must be carefully interpreted. Predictor effects could be substantially biased, with overestimation or underestimation of both the effect size and its associated uncertainty. 51

Rodger et al. 9 conducted a complete-case analysis, excluding patients with any missing predictor information. There was missing predictor information quantified for five predictors included in the predictor selection procedure:

• abnormal baseline compression ultrasound (events = 14/non-events = 180)

• factor V Leiden (events = 0/non-events = 1)

• ventilation/perfusion scan or pulmonary vascular obstruction result < 95% (events = 53/non-events = 283)

• post-thrombotic signs (hyperpigmentation, edema or redness in either leg) = (events = 18/non-events = 83)

• family history of VTE (events = 0/non-events = 1).

The use of a complete-case analysis for the final model may introduce attrition bias by excluding patients who have outcome data but are missing information from one or more candidate predictors. The drop in the number of events is also a cause for concern; a potential total of 85 events were missing across the five predictors, enhancing the possibility of spurious relationships being seen or important prognostic effects being missed during the predictor selection procedure. In terms of the final model, only post-thrombotic signs were included and so the complete-case analysis sample size was reduced by 101 patients, of whom 18 were events. Overall the sample size and handling of missing data within the study indicate a potentially high risk of bias within the model development.

Analysis

Rodger et al. 9 used a conditional logistic regression model and selected predictors using a stepwise forward selection process. As the outcome of interest was time to recurrent VTE, a time-to-event analysis may have been more appropriate here as logistic regression does not account for the censoring of patients over time and variable lengths of follow-up. The analysis also did not consider the potential heterogeneity across the three tertiary centres from four different countries, stratification by centre or country would take into account any heterogeneity in the baseline risk of recurrence within these potentially different populations. 52

Predictors were only included in multivariable analysis if univariable analysis yielded a p-value < 0.2, though univariable results were not presented in the article. This exclusion of candidate predictors from multivariable analysis was therefore completely data driven based on univariable results, which could lead to potential bias in results because predictors which may be important in combination (e.g. in a multivariable model) were not considered for multivariable analysis. Univariable analyses are not recommended for decisions about inclusion criteria in a multivariable model. 53

The use of a forward selection procedure in model development can lead to overoptimism in regression coefficients (betas) and therefore a method such as shrinkage or bootstrapping should be used to account for this optimism. Rodger et al. 9 did not use methods to account for optimism in their analyses and as such there is a risk that the performance of their proposed model could be weaker when applied to a new population.

Rodger et al. 9 decided to split patients into two groups based on sex in a post-hoc analysis because a specified low-risk group (< 3% annual risk of recurrence) could not be identified. Post-hoc subgroup analyses such as this are often not considered in any assessment of study power. Stratifying by sex reduced the number of events for females and males to 28 and 63 events, respectively, creating similar issues to those discussed above in the estimation of regression coefficients within the model. 51

Five clinical decision rules were developed for women and two for men, the final decision rule was selected based on criteria including the classification of performance; the proportion of patients identified as low risk; the face validity of the model; ease of use of the model; and more parsimonious models. The performance of decision rules for men was considered poor, particularly in identifying a low-risk group of men which could be considered for cessation of therapy, as such the study recommended that men continue with OAC therapy. The final decision rule for women included predictors for:

• post-thrombotic signs:

  • hyperpigmentation

  • oedema

  • redness in either leg

• D-dimer level ≥ 250 µg/l

• BMI ≥ 30 kg/m²

• aged ≥ 65 years.

Rodger et al. 9 suggested that any female patient with fewer than two of these predictors could potentially safely cease OAC therapy after 5–7 months of initial OAC therapy for an unprovoked VTE. The specification of the model is not described in full, with regression coefficients presented for the final model only, but without any measurement of variability such as a standard error or 95% CI for the coefficients. It is unclear if or how the decision rule is related to the regression coefficients, though it may be that a rounding of the coefficients to the nearest integer was used. There is no indication of the level of risk associated with a particular score using the proposed decision rule; for example, what the risk of recurrence at 1 year post cessation of therapy is for a female with three of the included factors.

The decision rules developed were internally validated using split-sample cross-validation. Five hundred subsamples, half the size of the study sample, consisting of randomly selected patients from the population were used to assess rule performance. The mean annual recurrence risk predicted by the clinical decision rule was recorded within each subsample and showed that for all subsamples the decision rule identified a low-risk group of women with mean annual risk of recurrence between 0% and 3%, suggesting that the rule performed well in internal validation. No measure of performance in terms of calibration or discrimination was presented, and no external validation of the rule was conducted. An external validation of the clinical decision rule is currently being undertaken comparing use of the decision rule to decide on cessation of therapy versus standard therapy. 27,28

Summary

Overall there are significant concerns over the robustness of the proposed clinical decision rule. The testing of many more predictors than the study was powered for could lead to potentially spurious predictor effects. The data-driven dichotomisation of continuous predictors does not allow for non-linear effects, and instead suggests a constant effect in patients at either side of the chosen threshold. 48 The effects of missing predictor information were not assessed which could have resulted in different conclusions had there been more data available. Potentially inappropriate analyses were performed, without accounting for heterogeneity in baseline risk across differing populations and performing post-hoc subgroup analysis. The decision rule was poorly presented, with a lack of explanation linking regression coefficients with the decision rule. Furthermore, there was no reporting of uncertainty surrounding coefficients and no description of calibration or discrimination given. Therefore, there is substantial concern that the proposed decision rule would not perform as presented when applied in a new population and examined in new data independent to that used to develop the model.

Patient selection

Eichinger et al. 2,42 also used a prospective cohort design with consecutive unselected patients recruited from four thrombosis centres in Vienna, between July 1992 and August 2008. Similar to Rodger et al. ,9 a prospective cohort design ensures that predictor information can be collected blinded of patient outcome. Patients were included if they were at least 18 years old and had been treated with OAC therapy for at least 3 months for a first unprovoked VTE.

The definition of unprovoked VTE used by Eichinger et al. 2,42 was based on an absence of the following provoking factors:

• surgery

• trauma

• pregnancy

• hormone intake

• deficiency in antithrombin, protein C, protein S

• presence of lupus anticoagulant, or

• cancer.

This definition of unprovoked VTE therefore follows a standard definition based on an absence of any transient risk factors. As a result the study population should be representative of the unprovoked VTE population and therefore provides a low risk of bias in the model development.

Outcomes

The main outcome of the study was recurrent VTE, with suspected DVT confirmed by venography or colour duplex ultrasonography. Suspected PE was confirmed by ventilation/perfusion scan and/or spiral computed tomography. All events were adjudicated by a committee of independent radiologists. Detection bias was limited by pre-specification of outcome definitions, with the same definition and assessment used for all patients, meaning systematic differences in the determination of outcomes were avoided. Outcomes were pre-specified with the same definition and assessment used for all patients reducing the risk of differences in the diagnosis and reporting of outcomes. The outcome definition excluded any candidate predictors and was also determined blind to any predictor information, again, reducing the risk of detection bias and overall resulting in a low risk of bias based on the study outcomes.

Predictors

Eichinger et al. 2,42 pre-specified a selection of clinical and laboratory predictors based on criteria including independent confirmation of the association with risk of recurrence (literature), simplicity of assessment and reproducibility. Using pre-specified candidate predictors reduces the risk of bias through unnecessary investigations, limiting the number of hypothesis tests and therefore reducing the chance of multiple testing issues. Candidate predictors included:

• sex

• age (years)

• BMI (kg/m²)

• site of index event (distal DVT, proximal DVT, PE)

• D-dimer (µg/l)

• factor V Leiden

• factor II mutation

• peak thrombin.

All predictors were measured blinded to outcome data (with laboratory predictors measured at cessation of OAC therapy), and predictor definitions were consistent for all patients. These methods reduce the risk of selection bias due to differences in the patient characteristics and also the risk of bias in the reporting of predictors based on outcomes.

Eichinger et al. 2,42 investigated linear forms of all continuous predictors including age, BMI and D-dimer level. The study authors also investigated a dichotomisation of BMI based on the standard threshold for obesity (BMI > 30 kg/m²) used in clinical practice. The study therefore avoided introducing bias from data-driven methods of categorising continuous predictors, but this approach can still lead to a loss of information by splitting patients risk into distinct groups. 47

Sample size and flow

A total of 929 patients were included within the analysis with 176 recurrent events being recorded. Given that Eichinger et al. 2,42 investigated a total of eight predictors (15 predictors including categorisations of predictors), the number of events seen could be considered sufficient given the rule of thumb of 10 events per predictor. 51 As the study could be considered suitably powered (with a maximum of 12 events per predictor), the inclusion of predictors and their effects are likely to be at low risk of bias, with effect sizes and associated uncertainty likely to be reliably estimated. 51

Eichinger et al. 2,42 conducted a complete-case analysis, excluding patients with any missing predictor information. There was missing predictor information quantified for five predictors included in predictor selection, but no indication of the number of associated events:

• BMI = 20

• D-dimer = 97

• peak thrombin = 300

• factor V Leiden = 13, and

• factor II G20210A mutation = 14.

Complete-case analysis introduces attrition bias through excluding patients with outcome data because they have missing information related to one or more candidate predictors. The drop in the number of events could have an impact on the regression coefficients and selection of predictors for inclusion within the final model, particularly for peak thrombin with a third of patients missing predictor information. Any analyses including peak thrombin would exclude a third of the study population, markedly reducing sample size and causing issues with the estimation of predictor effects. As D-dimer is the only predictor included in the final model, the complete-case analysis would have reduced the sample size by 97, though it is unclear how many events this represents and therefore could present a high risk of bias in the results presented. Overall the sample size and handling of missing data within the study indicate a potentially high risk of bias within the model development.

Analysis

Eichinger et al. 2,42 used a Cox proportional hazards model and selected predictors using a forwards stepwise selection process. A Cox regression model accounts for the censoring of patients over time and variable length of follow-up in a time-to-event analysis, making it an appropriate choice for the recurrent VTE outcome. No stratification by centre was performed as part of the analysis, which did not account for potential heterogeneity in the baseline risk of recurrence for patients from the four different thrombosis centres. Failing to account for differences in baseline risk (or at least to investigate whether or not stratification is necessary), could lead to biased predictor effects which may not be replicated when the model is applied in a new population. 52

Eichinger et al. 2,42 first fitted a saturated model using all clinical predictors and then performed forward selection to investigate laboratory predictors (using an inclusion threshold of p-value < 0.5). To account for optimism associated with the stepwise selection procedure, Eichinger et al. 2,42 evaluated the statistical significance of included predictors using bootstrap zero-corrected 95% CIs. From 1000 bootstrap resamples of the population, only factors for which the 95% CI did not overlap zero were included in the model. Further to this, Eichinger et al. 2,42 applied a shrinkage factor (calculated by bootstrap resampling), to the final beta coefficients to adjust for optimism which may affect the models performance in new study populations. The extensive use of methods to account for optimism in their analyses suggests that performance of the model is unlikely to be affected when applied to a new population.

Two prognostic models were presented within the study including the following predictors:

• sex (female as the reference category)

• site of index event (distal DVT as the reference category)

• peak thrombin (included in the first model)

• D-dimer (included in the second model).

Both models included sex and site of index event as predictors, but during initial predictor selection peak thrombin was identified as significantly prognostic and D-dimer did not enter the model [hazard ratio (HR) 1.21, 95% CI 0.87 to 1.53; p = 0.622]. The authors then decided to evaluate a model including D-dimer without peak thrombin, and in this model D-dimer was a significant predictor of recurrent VTE. Eichinger et al. 2,42 chose to take forward the model including D-dimer levels as a final model because ‘D-dimer is a well-standardised and widely established parameter’. The ad hoc selection of predictors may be a cause for concern as it was not pre-specified that D-dimer was to be included and there was strong evidence against the prognostic value of D-dimer compared with peak thrombin levels. However, as discussed by the authors, the inclusion of D-dimer within the model could potentially improve the implementation of the model, as D-dimer is an established predictor more readily measured in practice. Prognostic models should aim to include predictors with standard definitions, which are easily available at the time the model is intended for use; however, the performance of the model may have been significantly improved by the inclusion of peak thrombin as a predictor (which showed strong evidence of prognostic value). However, it is also worth noting the substantial number of missing data for peak thrombin (300/929 patients missing peak thrombin data), which could have influenced the estimated effect of peak thrombin within the complete-case analysis performed.

Eichinger et al. 2,42 proposed a nomogram based on the final model including sex, site of index event and D-dimer as predictors. The nomogram can be used to calculate patient’s cumulative recurrence rate at 12 and 60 months from cessation of therapy, with estimated 95% CIs. The relationship between the regression coefficients of the model and the simplified nomogram is not explicitly stated, though it is suggested that the coefficients are first multiplied by the estimated shrinkage factor. No estimate of baseline risk is provided and therefore it is only possible to estimate patients predicted risk of recurrence at the specified time points within the study (1 and 5 years). The use of estimated recurrence risk at specific time points (with associated uncertainty), improves on the HER DOO 2 model9 by allowing practitioners to apply their own judgement based on the predicted risk, using current guidelines and patient consultation, to make an informed decision on treatment strategy for the individual patient.

Internal validation of the model was performed using bootstrap cross-validated risk scores. Patients were randomly drawn with replacement from the original sample, to make a new bootstrap sample of 929 patients. Eichinger et al. 2,42 re-evaluated their model within this new bootstrap sample and validated their model in the sample of patients that were not selected from the original study data. The process was repeated 1000 times and an average risk score for each patient was calculated from the 1000 replications. Using these averaged risk scores the performance of the model (within the validation subsets) was assessed using the AUC, which is a measure of model discrimination. For recurrence risk at 12 and 60 months from cessation of therapy the optimism adjusted (bootstrapped) AUC was estimated at 0.674 and 0.646, respectively, indicating moderate discrimination, which suggests moderate ability of the model to separate groups of patients such as high- and low-risk patients (where AUC = 1 represents perfect discrimination). The study also reported an apparent c-statistic (assessing discrimination across all time points, and without adjustment for optimism) of 0.651 for the developed model as a measure of discrimination (where c-statistic = 1 represents perfect discrimination), which suggests a small reduction in model performance after accounting for optimism. The Vienna model also calculated a bootstrap optimism-adjusted calibration slope (or uniform shrinkage factor), which showed moderate calibration performance of 0.88 (with 1 indicating perfect calibration). This shrinkage factor was also used to adjust the predictor effect values in their final model, to adjust for the overoptimism. However, the performance of a model measured in the same data set used to develop the model will always be biased, showing greater performance than can be expected in an external setting.

An external validation of the Vienna prediction model is currently being undertaken, which should provide a more reliable indication of model performance in a new patient population. It should also be noted that the internal validation of the Vienna prediction model relates to the multivariable Cox regression model developed, and it is unclear whether or not internal validation of the simplified nomogram was conducted.

Summary

In summary, there was a moderate risk of bias associated with the Vienna predication model, mainly because no external validation has yet been performed. Model development itself was undertaken well. Patient selection avoided inappropriate exclusions and outcomes were defined consistently for all patients and blinded to predictor information. Continuous predictors were assessed in their linear form as opposed to categorising, therefore avoiding a loss of information. Overoptimism in the estimated regression coefficients was accounted for by using bootstrapping methods to adjust these coefficients.

However, there were also some areas for concern which could have introduced bias into the model development. The analysis did not investigate potential heterogeneity in the baseline risk of recurrence in the different populations from the four thrombosis centres used, potentially affecting the models applicability to a new population. The study authors made an ad hoc decision to include D-dimer as a predictor in the model, despite its lack of prognostic value during predictor selection. There was strong evidence of an important effect for peak thrombin, which meant D-dimer was originally excluded. The decision to include D-dimer was based on practical implications for model use, as it is a more established predictor. There were also some issues with missing predictor information, with marked reductions in data for both D-dimer and peak thrombin, and no information on the number of events excluded from complete case analyses investigating these predictors. Furthermore, it was not clear how the nomogram was created and whether or not the internal validation of the nomogram was examined.

Overall the Vienna prediction rule was presented well and classed as low risk of bias in terms of model development. External validation is, however, now essential for the proposed nomogram. If performance was found to be acceptable, the model would allow individual prediction of recurrence risk limited to the two specified time points. The model was presented in a nomogram which may facilitate uptake of the model in practice, though this format limits the precision available (e.g. in specifying a patient’s exact D-dimer level). The predicted recurrence rate is provided with associated uncertainty (95% CI), which allows both clinician and patient to make an informed decision regarding treatment.

Finally, the extension to the Vienna model aimed to allow prediction of recurrence risk at further time points post cessation of therapy by measuring D-dimer over time. 42 Measurements were made at 3, 9 and 15 months post cessation of therapy, and three more nomograms were developed to allow risk prediction using the Vienna model at these time points. D-dimer levels did not vary over the observation time and the associated HRs remained very similar (only the point estimate slightly decreased over time). 42 A web-based calculator allows users to predict recurrence risk at any time between baseline (3 weeks) and 15 months post cessation of therapy. 42 The model was adjusted for optimism using leave-one-out resampling to calculate shrinkage factors for 3, 9 and 15 months of 0.79, 0.81 and 0.7, indicating moderate calibration of the model at all time points but reduced performance compared with the original Vienna model (optimism-adjusted calibration slope = 0.88). In terms of discrimination performance (for 5-year predictions) at each time point, optimism adjusted AUC values were 0.61, 0.61 and 0.58, representing a small reduction in performance compared with the original model (AUC = 0.646). 2,42 However, although the earlier Vienna model has recently been externally validated, this model has not been externally validated to date.

Patient selection

Tosetto et al. 41 used IPD from seven prospective cohort studies with consecutive unselected patients described by Douketis et al. 11 previously. The prospective cohort design as used in the previous studies ensures that predictor information can be collected blinded of patient outcome. Patients were included if they were at least 18 years old and had been treated with OAC therapy for at least 3 months for a first unprovoked VTE. Patients were excluded if follow-up ended before D-dimer measurement, or if they had a distal DVT; only proximal DVT and PE were included as valid index sites.

The definition of unprovoked VTE used by Tosetto et al. 41 was based on an absence of the following provoking factors:

• surgery

• trauma

• pregnancy and the puerperium

• immobility

• cancer.

This definition of unprovoked VTE therefore includes women who have currently, or previously, used OCs or HRT. As described for the Rodger et al. 9 study, hormone therapy is sometimes included in the definition of unprovoked VTE because the effect of hormone therapy is weak;41,44 however, hormone therapy should be considered as a provoking factor. The definition of unprovoked VTE refers to a VTE in the absence of any transient risk factors, hence why hereditary thrombophilia can be considered unprovoked VTE. 45 As hormone therapy is a transient risk factor patients with a history of hormone therapy could be considered provoked. This could lead to effect sizes within the analyses being incorrectly estimated, particularly the effect of sex, because there may be subgroups of patients included in the model development who are at potentially lower risk than other unprovoked patients.

Outcomes

The primary outcome of the study was recurrent VTE, with associated deaths also recorded. All suspected outcomes were objectively confirmed and independently adjudicated. 11 Outcomes were pre-specified with the same definition and assessment used for all patients, therefore reducing the risk of differences in the determination of outcomes. The outcome definition excluded any candidate predictors and outcomes were also determined blind to any predictor information again reducing the risk of detection bias.

Predictors

Tosetto et al. 41 used a backward elimination approach, starting with a saturated model including the following candidate predictors:

• sex

• age (years)

• site of index event (DVT alone, or DVT and PE)

• D-dimer (ng/ml)

• hormone use at time of VTE (women)

• previous history of cancer.

All predictors were measured blinded to outcome data (with D-dimer measured 3–5 weeks after cessation of OAC therapy), and predictor definitions were consistent for all patients. These methods reduce the risk of selection bias due to differences in the patient characteristics and also the risk of bias in the reporting of predictors based on outcomes.

Tosetto et al. 41 also stratified their analyses by source study to allow for potential heterogeneity in the baseline risk of recurrence within these seven different populations. The adjustment for underlying differences in the source study populations may make the final model more robust when applied within a new population, not used in the development process.

Tosetto et al. 41 categorised all continuous predictors, creating a dichotomisation of D-dimer (normal vs. abnormal), while categorising age into quartiles. Quartiles of age were used to control for a non-linear relationship between patient age and recurrence risk. The study may have therefore introduced bias from categorisation of continuous predictors, which can lead to a loss of information by separating patients risk into distinct groups. 47 Categorisation of age appeared to be data driven, whereas dichotomisation of D-dimer was likely based on the instrument used (though it was not stated), both inducing the risk of reporting biases.

Sample size and flow

A total of 1818 patients were included within the analysis with 239 recurrent events being recorded. Given that Tosetto et al. 41 investigated a total of six predictors (14 predictors including categorisations of predictors), the number of events seen could be considered sufficient given the rule of thumb of 10 events per predictor. 51 As the study could be considered suitably powered (with a maximum of 17 events per predictor), the inclusion of predictors and their effect estimates may be considered to be at low risk of bias. Predictor effects are likely to be at low risk of bias, with effect sizes and associated uncertainty likely to be reliably estimated. 51

There was no missing predictor information in the analyses performed by Tosetto et al. 41 and therefore it was not necessary to allow for the effects of potential attrition bias. The previous two studies have both suffered with missing predictor information and conducted complete case analyses, potentially introducing attrition bias into their analyses. Tosetto et al. 41 were therefore able to use all patient data in their analyses and thus statistical power remained appropriate to assess all predictors for inclusion. However, the study also used a selection procedure meaning more predictors were considered, resulting in a proportion of missing predictor data. The DASH model considered predictors including BMI, for which only 802 out of 1818 patients had complete predictor information, which may have effected the ability to detect a BMI effect. Overall the sample size and handling of missing data within the study resulted in a low risk of bias associated with the model development.

Analysis

Tosetto et al. 41 used a Cox proportional hazards model and selected predictors using a backwards elimination process. A Cox regression model accounts for variable lengths of follow-up and the censoring of patients over time and in a time-to-event analysis, making it an appropriate choice for the time to recurrent VTE outcome. The analyses were stratified by source study to allow for the differences in the baseline risk across the seven different study populations, therefore avoiding biased predictor effects which may improve the external performance of the model. 52

Tosetto et al. 41 first fitted a saturated model using all clinical predictors, and then performed backward elimination to investigate candidate predictors (using an exclusion threshold of p-value > 0.1). To account for excessive optimism associated with the backward selection procedure, Tosetto et al. 41 evaluated this using a heuristic formula and linear shrinkage by bootstrapping. A correction factor was calculated and applied to the final beta coefficients to adjust for optimism which may affect the models performance in new study populations. The use of shrinkage methods to account for overoptimism in their selection process and analyses provides a low risk that the performance of their proposed model could differ when applied to a new population.

The final DASH score was developed by multiplying the regression coefficients by the calculated correction factor, doubling these coefficients and rounding them to the nearest integer. Giving a final model included the following predictors:

• abnormal D-dimer (post therapy), score = +2

• age (≤ 50 years), score = +1

• sex (male), score = +1

• hormone use (at time of index event, in women), score = –2.

The proposed score can be used to calculate patients’ cumulative recurrence rate at 1, 2 and 5 years from cessation of therapy, with estimated 95% CIs. Despite stratification for source study in the analysis, there is no reported estimate of baseline risk and therefore patients’ predicted risk of recurrence can only be estimated at the specified time points presented. The use of estimated recurrence risk at specific time points (with associated uncertainty) is similar to that presented for the Vienna prediction model. 2,42 This is a substantial improvement compared with the HER DOO 2 model9 as it allows physicians and patients to make an informed decision on treatment duration for the individual patient.

Internal validation of the model was performed using a bootstrap procedure similar to that described for the Eichinger et al. study. 2,42 Patients were randomly drawn with replacement from the original sample, to make a new bootstrap sample of 1818 patients. Tosetto et al. 41 then re-estimated the DASH score within this new bootstrap sample, to confirm the recurrence rate and associated CI for DASH score of < 1 (identified as having an annual recurrence risk < 5%). The process was repeated 500 times and an average risk of recurrence was calculated to be less than the agreed 5% annual recurrence risk. Apparent c-statistics (which represent the discriminatory performance within the development data without adjustment for optimism using, for example, bootstrapping) were between 0.71 and 0.72 for the score and model (beta terms), respectively, indicating moderate discrimination ability even for the simplified score for use in practice; however, apparent performance is likely to be optimistic. The DASH model also provided a bootstrap optimism-adjusted calibration slope (or uniform shrinkage factor) as the Vienna model did, which also showed strong calibration performance of 0.97 for the DASH model (with 1 indicating perfect calibration). The shrinkage factor was then used to adjust the predictor effect values for overoptimism in the final model. However, the performance of a model measured within the development data set is likely to be biased, indicating stronger performance than could be expected in a new population. The use of IPD meta-analysis in the derivation of the model and stratification for source studies may make the DASH score more robust to departures from the development population’s characteristics, but external validation should be sought to identify the true performance of the model in a new patient population.

Summary

In summary the DASH score proposed by Tosetto et al. 41 could be considered at moderate risk of bias, mainly due to the lack of external validation and the categorisation of continuous predictors in the model development. Many aspects of model development were done well. Patient selection avoided inappropriate exclusions and outcomes were defined consistently for all patients and blinded to predictor information. Stratification was used in analyses to account for heterogeneity in the baseline risk of recurrence across the source studies. Missing predictor information was not an issue in the model development, avoiding attrition bias and preserving statistical power. Overoptimism in the selection of predictors and estimated effects was accounted for by a correction factor calculated by bootstrapping. Reporting of the final score was clear with recurrence risks associated with particular scores presented including uncertainty.

However, there were also some areas for concern which could have introduced bias into development of the DASH score. There were issues with categorisation of continuous predictors, which could lead to a loss of important prognostic information. Furthermore, and most importantly, there was no external validation.

External validation is therefore now essential. The DASH score provides individual recurrence risk prediction at specific time points post cessation of therapy. If externally validated and found to perform well, then the score could be useful in practice as the included predictors are well defined and readily available at the time the decision rule would be applied. However, the true performance of the DASH score within a new patient population is unclear given the lack of internal and external validation statistics. Any physician or patient using the DASH score should therefore interpret the predicted recurrence risk with care, and the included 95% CIs are importantly presented within the study, allowing informed decision-making regarding treatment.

RIETE data set

The RIETE database contained factors describing the site of index event but only described categories in terms of either (1) DVT and PE, (2) DVT or (3) PE. The site of index event factor used in the development of the pre D-dimer model included categories for distal DVT, proximal DVT and PE, therefore breaking down the types of DVT into the lower-risk distal DVT and higher-risk proximal DVT. As no other information was provided within the RIETE database, with regard to site of index event, the required categories as described in the development database could not be recreated. Therefore the pre D-dimer model unfortunately could not be validated in the RIETE database.

MEGA data set

Within the MEGA data set there was no variable describing the site of index event using the same categorisation as the RVTEC database, which was essential for validation of the developed models. To overcome this, a separate variable identifying the patients site of index event in categorisations representing the vein or artery location was recategorised according to Martinelli et al. 89 (Table 17). After recategorisation, summary statistics showed a small number of distal DVT’s (6.5%) compared with the higher-risk proximal DVT (29%) and PE (42%) (Table 18).

The median follow-up within the MEGA database was around 69 months providing substantial information on patient’s long-term prognosis. Summary statistics from the MEGA database, for the predictors included within the pre D-dimer model, are presented in Table 18. The average (mean) age of patients was around 49 years old; there were slightly more females within the data set than males (54% females).

The summary statistics appeared to indicate that the population in the MEGA database was different in a number of ways to that of the RVTEC database. The distribution of index events was different between the two databases with the proportion of proximal DVTs around double that seen in the MEGA data set (58.9% to 29.4%). The RVTEC population had a higher average age (approximately 60 years) and greater proportion of males (approximately 60% males) (see Table 18).

To assess the external performance of the pre D-dimer model, predicted risk of recurrence over time was calculated for all individuals in the MEGA data set using the pre D-dimer model equation as shown in Specification and parameter estimates (see Equation 1). The predicted recurrence probabilities were compared with the true observed recurrence probabilities estimated by a Kaplan–Meier survival curve (Figure 15). On visual inspection the expected probability of recurrence appeared to be overpredicted slightly up to 1 year from cessation of therapy and underpredicted beyond 1 year post therapy. This was reflected in the expected minus observed statistics at 6 months, 1 year, 2 years and 3 years post cessation of therapy (Table 19). CIs for S(t) – Ŝ(t) showed increasing uncertainty over time from cessation of therapy, with the greatest difference in probability of recurrence at 3 years from therapy. Predictions appeared to calibrate very well at 1 year from cessation of therapy with a difference of < 0.1 in the percentage of those observed and expected to have a recurrence by 1 year. Before 1 year and after 1 year the calibration is not as close, but the difference in observed and expected percentage with a recurrence is quite small. In terms of discrimination the c-statistic for the pre D-dimer model within the MEGA database remained similar to that seen in the RVTEC database (0.56). This again indicates poor ability to separate high- and low-risk patients. Moreover, the performance seen in the MEGA data set lies well within the 95% prediction interval (0.49 to 0.62) estimated from the random-effects meta-analysis of c-statistics from the IECV approach (see Figure 7).

FIGURE 15.

Calibration of the pre D-dimer model predicted probability of recurrence (expected) compared with observed probabilities (from Kaplan–Meier curve) within the MEGA external data set.

Proportional hazards assumption

A scatterplot of the scaled Schoenfeld residuals against log-time, with a lowess smoother, was used to check the proportional hazards assumption for factors in the post D-dimer model as described previously (see Development of prognostic factor). Plots for log-D-dimer (Figure 30) and log-lag time (Figure 31) show that the proportional hazards assumption is valid for the post D-dimer model; the lowess smoothed line roughly follows the reference line for each covariates log-HR, indicating proportionality. Similar plots testing the proportional hazards assumption were inspected for the remaining covariates in the post D-dimer model, the proportional hazards assumption was valid for all predictors (see Appendix 5).

FIGURE 30.

Scaled Schoenfeld residuals vs. log-time from cessation of therapy for log-D-dimer (HR 0.539).

FIGURE 31.

Scaled Schoenfeld residuals vs. log-time from cessation of therapy for log-lag time (HR –0.19).

Functional form

To check that continuous predictors were included in the model with appropriate functional form, scatterplots of Martingale residuals against the predictors with a lowess smoother applied were inspected. Patient age, log-D-dimer and lag time were the only continuous predictors included in the post D-dimer model and the functional form of these covariates was checked using Martingale residuals. Figures 32 and 33 show a lowess smoother applied to a scatter of Martingale residuals against log-D-dimer and log-lag time respectively. In both cases the smoother appears to follow a linear trend over the covariate values, indicating a linearity assumption (on the log-scale) for both factors was appropriate.

FIGURE 32.

Scatterplot of Martingale residuals against log-D-dimer (the post D-dimer model).

FIGURE 33.

Scatterplot of Martingale residuals against log-lag time (the post D-dimer model).

Outliers

As seen for the pre D-dimer model (see Appendix 5), plots of the deviance residuals against a patient indicator (Figure 34) and against time (Figure 35) indicate some outlying individuals. Figure 34 illustrates a scatter of the deviance residuals for the post D-dimer model; they clearly do not follow a normal distribution and this may again be due to heavy censoring in the data set, a small number of individuals fall above the 1.96 critical z-value.

FIGURE 34.

Scatterplot of deviance residuals vs. patient ID (the post D-dimer model). ID, identification.

FIGURE 35.

Scatterplot of deviance residuals vs. years from cessation of therapy (the post D-dimer model).

A plot of the deviance residuals against years from cessation of therapy allows investigation of any trend in the deviance residuals. In Figure 35 for the post D-dimer model there is again a trend in the deviance residuals over time based on the cumulative hazard at the event time (or censoring time). The deviance residuals which lie in the top left of the plot are, as for the pre D-dimer model, likely to be those individuals who had a recurrence early and therefore did not accumulate much hazard.

Leverage

To check the influence of individuals on the parameter estimates, leverage can be assessed using delta–beta changes for each covariate as seen in the pre D-dimer model (see Appendix 5). Scatterplots of delta–betas for log-D-dimer (Figure 36) and log-lag time (Figure 37) show that even individuals with the greatest leverage on these parameter estimates have very small effects on the log-HR as seen for the pre D-dimer model. Similar, small delta–beta changes were observed for the other covariates included in the post D-dimer model (see Appendix 5).

FIGURE 36.

Scatterplot of delta–beta for log-D-dimer vs. years from cessation of therapy (log-HR 0.666).

FIGURE 37.

Scatterplot of delta–beta for log-lag time vs. years from cessation of therapy (log-HR –0.361).

Multiple imputation of missing data

As the RVTEC data set used for model development included some missing data for some of the potential predictors, a complete-case analysis was performed for model development, excluding any patient with missing data from the analysis. Sensitivity analysis was performed using multiple imputation, to evaluate how model estimates compared with those from the complete-case analysis.

Of the included predictors only D-dimer and lag time had missing values, with 15% and 11.4% incomplete data, respectively, across the whole data set of six trials (see Table 8). It was therefore possible to consider multiple imputation for these two factors within the post D-dimer model. As the RVTEC data set consisted of multiple trial populations for development of the post D-dimer model, it was important to account for this clustering when imputing missing observations; imputation across trials can lead to bias where the association between factors differs by trial. 85 As such, missing data for lag time, which were 100% incomplete within the Baglin et al. trial,61 could not be imputed (see Table 8) and so the same set of six trials (excluding Baglin et al. 61) were used as in the complete data analysis for the post D-dimer model.

Imputation models were selected to include all included predictors from the final post D-dimer model as well as predictors for the observed recurrences (event indicator) and the baseline hazard to account for the time-to-event outcome;78 imputation was performed within trial populations. The largest proportion of incomplete data observed within individual trial populations was 48.1% missing D-dimer observations within the Poli et al. trial;59 therefore, 50 imputed data sets were created to provide the greatest reproducibility. 78 Imputation was performed for 10 cycles within each of the 50 imputed data sets to stabilise the results.

Box plots were used to check that the distributions of the observed and imputed data broadly matched; large differences could indicate an inappropriate imputation model. 78 On inspection, the imputed distributions for both D-dimer and lag time (Figures 38 and 39) appeared to be very similar to the corresponding observed distributions (indicated as zero on the box plots). Therefore the imputation process appeared to be appropriate.

FIGURE 38.

Comparison of observed and imputed data for log-D-dimer (the post D-dimer model).

FIGURE 39.

Comparison of observed and imputed data for log-lag time (the post D-dimer model).

The model, including all predictors identified as important in the complete-case analysis (age, site of index event, sex, D-dimer and lag time), was fitted to the 50 imputed data sets and the coefficients of each were combined using Rubin’s rules79 (Table 32). In comparison with the specification of the post D-dimer model under a complete-case analysis (see Table 27), the estimated HRs after imputation were reasonably similar. The effect of each factor within the model did not have a dramatically different interpretation between the complete case and multiple imputation models. In particular, the effects of D-dimer and lag time are relatively unchanged with HRs of 1.93 and 0.74 compared with 2.01 and 0.75 for the complete case model. In general, the 95% CIs were similar with the exception of site of index event where the multiple imputation model estimated slightly smaller 95% CIs, showing greater precision, likely due to the increased number of observations. The effect of age and lag time were borderline significantly different from null in the complete case model, but appeared to be significant in the multiple imputation model. This adds further weight to the inclusion of age and lag time factors in the prognostic model.

The inclusion of treatment duration as a predictor in the multiple imputation model was investigated given the increased complete patient data; treatment duration did not reach significance within the imputation model with a HR of 1.07 (95% CI 0.85 to 1.35) and p-value of 0.574, providing confirmatory evidence towards the exclusion of treatment duration within the complete case post D-dimer model.

To check that the imputation results were reproducible, such that similar conclusions could be drawn from an identical imputation approach, the MC error of each estimated HR and standard error was checked. The MC error was measured as a percentage of the standard error of the estimated HR, where an MC error lower than 10% of the standard error was considered appropriate. The MC errors observed from the imputation procedure used were all lower than 10%, with the greatest being 5.74% for D-dimer, meaning that it is highly likely that the results of multiple imputation procedure would lead to consistent conclusions across the imputed data sets (Table 33).

Summary of sensitivity analysis

Compared with the complete data model, the multiple imputation approach suggests similar conclusions about the predictors to include and their magnitude of effect. As the complete data model was already validated during the IECV approach, and it performed well in terms of calibration and discrimination, it was decided by the team to retain the post D-dimer model as derived using complete data as the final model. Furthermore, the multiple imputation approach makes the additional assumption about MAR, which may not hold.

External validation in RIETE and MEGA data sets

Two independent databases (RIETE and MEGA) were available for potential external validation of the proposed post D-dimer model (see Identifying, obtaining and cleaning individual patient data). Unfortunately neither included D-dimer or lag time predictors, meaning that the post D-dimer model could not be externally validated in these databases. However, it should be emphasised that through the IECV approach external validation has been conducted already as discussed above (see Internal–external cross-validation).

Model overview

A Markov patient-level simulation was developed in TreeAge version 2014 (TreeAge Software, Inc., Williamstown, MA, USA) to estimate the cost-effectiveness of the use of a decision rule compared with treating no-one (usual care) in patients with a first unprovoked VTE event. In patients no longer receiving therapy, the clinical decision rule used a threshold risk of recurrent VTE in 1 year to make a decision about return to therapy as follows:

• If the post D-dimer model gives a predicted probability of recurrence equal to or above the threshold, an individual should have their therapy resumed.

• If the post D-dimer model gives a predicted probability of recurrence less than the threshold, an individual should remain off therapy

The comparators for the treat no-one strategy were decision rules using threshold risks of 1%, 3%, 5%, 10%, 15% and a strategy where everyone restarts therapy (0% threshold). The decision rule used the risk of recurrent VTE at 1 year provided by the post D-dimer model described in detail in Chapter 4, Final model: post D-dimer model.

A Markov model is the most appropriate model type for this decision problem as the model can represent a clinical situation where patients change health states or experience recurrent events over a long period of time. A patient-level simulation was chosen instead of a cohort model. This allowed individual patients to be created, using data on existing patients where a number of characteristics could vary, with the model predicting individual VTE risks. Model results represent the mean costs and QALYs for a realistic patient population. This is in contrast to running the model for a hypothetical cohort of patients. The model was run for 50,000 simulated patients and had a time cycle of 1 month. The number of simulated patients was determined to be sufficiently large to reduce the variability in the model results. A time cycle of 1 month was chosen as it was agreed through clinical consensus that it is a short enough period of time to allow an assumption of only one clinical event occurring within a time cycle. The base-case analysis used a lifetime time horizon and the evaluation was conducted from a UK NHS/Personal Social Services perspective, to take into account health-care costs and longer-term care costs of recurrent VTE and major bleeds. Costs, utilities and clinical probabilities were converted into monthly equivalents in accordance with the time cycle length.

Model population

Individuals were created from the patient-level data previously used to develop the post D-dimer model (RVTEC database). The patient population consisted of patients aged ≥ 18 years with a first unprovoked VTE, having completed at least 3 months of anticoagulation for the unprovoked VTE. The patient-level data contained information on age, sex, type of index VTE event (distal DVT, proximal DVT and PE) and post-warfarin D-dimer level. Further details on this data set can be found in Chapter 4, Results I: summary characteristics of available data sets.

In order to align the economic model with the post D-dimer model developed in Chapter 4, Results III: development and validation of post D-dimer model, the starting point of the economic model is defined as the point where a patient had received at least three months of anticoagulation therapy after suffering their first unprovoked VTE. The model assumed all individuals entered the model with their D-dimer measured 30 days after stopping their anticoagulation (known as the ‘lag time’ in the post D-dimer model). At model entry, the characteristics of an individual patient were determined by randomly sampling from the patient-level data11 using a uniform distribution. Sex was sampled first, and once this was determined, age was sampled from a sex-specific age distribution, again from the patient level data. D-dimer and site of index VTE were sampled independently of age and sex as no clear relationship was observed in the data set (see Table 9 and Figure 63). After an individual’s characteristics were determined, the risk of a recurrent VTE in 1 year was estimated by entering these characteristics into the risk equation taken from the post D-dimer model. Table 36 contains summary statistics of the patient-level data, which contained information on 1200 patients. Graphical representations of the distributions used for age and D-dimer can be found in Figures 48 and 54 in Appendix 4.

Model pathways and clinical events

In the clinical decision rule strategy, a specific threshold risk was chosen (1%, 3%, 5%, 10% or 15%), then the decision rule was applied. If the patient had a first year risk of a VTE event equal to or higher than the specified threshold risk, it was assumed they resume therapy, otherwise they continued without further therapy. The clinical decision rule was only assumed to be applied at this point in the model. In the comparator with no decision rule, individuals remained off therapy. Once the clinical decision rule was applied, all individuals had the same potential patient pathways, with probabilities of events, costs and utilities determined by their characteristics.

Figure 44 shows the potential patient pathways each simulated individual could progress along, irrespective of whether the clinical decision rule dictated they resumed therapy or remained off therapy. Given the very low probability of more than one clinical event occurring in a month, it was assumed that only one clinical event could occur per time cycle. In 1 month, an individual has a probability of experiencing a clinical event: death from other causes, recurrent VTE (distal or proximal DVT, fatal or non-fatal PE), or fatal or non-fatal major bleed [intracranial bleed, gastrointestinal (GI) bleed, other bleed]. The following describes the details for each possible event.

FIGURE 44.

Model patient pathways.

An individual patient could die from other causes, with other-cause mortality dependent on the current age and sex of the patient. Patients had a risk of a recurrent VTE, with risk determined by the patient’s characteristics, whether or not they had already suffered a recurrent event, length of time since entering the model and whether or not the patient was on anticoagulant therapy. If the patient had a recurrent VTE, this could be a PE, proximal DVT or distal DVT. It was assumed that the type of recurrent VTE was influenced by the patient’s index VTE location. For example, a patient with an index PE had a higher probability of having a further PE that a patient who had index DVT. If the patient suffered a PE, there was a risk the PE was fatal, whereas a DVT alone was assumed not to be fatal. Once a recurrent VTE occurred, an individual was put on lifelong therapy if they were not already on therapy. All VTE events incurred an acute cost of therapy. Surviving patients were assumed to suffer a one-off reduction of quality of life, and a proportion of patients were also assumed to suffer from PTS and had quality of life reduced for life.

An individual was also at risk of a major bleed. The risk of a major bleed depended on age and whether or not the patient was on therapy, and bleeds were split into GI bleeds, intracranial bleeds and other major bleeds. Each of the major bleeds had a risk of death and was associated with acute costs and a short-term reduction in quality of life. An intracranial bleed was assumed to have ongoing health-care costs and a permanent reduction in quality of life. For the purposes of the model and in order to assign costs and utilities, an ‘other major bleed’ was assumed to have the same cost and quality-of-life reduction as a GI bleed. If the patient was on anticoagulation therapy, any bleed led to immediate cessation of therapy. If they suffered a recurrent VTE in a subsequent cycle, they were put back on therapy.

Clinical parameters

This section outlines the sources used to populate the base-case parameters in the model and any related assumptions. Parameter estimates are presented in Table 37. The model assumes the base-case anticoagulation therapy is warfarin. The base-case analysis considers warfarin as the therapy of choice, with newer anticoagulant therapies [dabigatran (Pradaxa^®, Boehringer Ingelheim), rivaroxaban] explored in the sensitivity analysis.

The initial off-therapy risk of a recurrent VTE was calculated using the information on simulated patient characteristics which were then entered into the post D-dimer model. Risks were estimated for 6 months, 1 year, 2 years and 3 years post D-dimer measurement 30 days after initial therapy cessation. Beyond 3 years, the post D-dimer model was considered to have weak calibration statistics and therefore was not used; rather an annual risk of 5% for recurrent VTE off therapy was assumed for all patients after 3 years. This value was taken from a cohort study of patients who had discontinued anticoagulation, using incidence data from patients who had stopped therapy for more than a year. 66 The risk of a VTE on therapy was assumed to be 1.3%, taken from a trial of long-term therapy with rivaroxaban after VTE. 43 For those individuals who had a recurrent VTE event within the model, the risk of a further VTE was taken from the Prevention of Recurrent Venous Thromboembolism (PREVENT) trial, which considered low-intensity warfarin and placebo, and reported recurrence rates with normal and elevated D-dimer in those with two or more prior VTE. 62 The risk of VTE recurrence was estimated as 12% if off therapy and 5% if on therapy, with risks assumed to be between those values for patients with normal and elevated D-dimer. On-therapy VTE risks were assumed to be applicable for warfarin, dabigatran and rivaroxaban. The RVTEC data set11 provided the probabilities on the type of recurrent VTE event by index event, with a probability of 0.4828 for a PE after an index PE and a probability of 0.1456 for a PE after an index DVT. The risk of death from a PE was estimated by clinical consensus and was assumed to be 20% for the base case, with a higher estimate of 30% to be used in the sensitivity analysis. It was assumed that a proportion (1.1%) of patients who suffered a recurrent VTE would develop severe PTS and have a reduced quality of life, and these data were obtained from a study of patients on subtherapeutic warfarin after a first idiopathic VTE. 91

The model assumes that all patients have an age-related risk of major bleeding on and off therapy, with a greater risk on anticoagulation therapy which increases with age. The bleeding risk for patients with a first unprovoked VTE not receiving anticoagulant therapy (0.45%) was taken from a systematic review and meta-analysis of patients who had completed anticoagulant therapy for secondary prevention of VTE. 92 The bleeding risk for those on warfarin by age category is taken from the warfarin arm of the Randomized Evaluation of Long-Term Anticoagulation Therapy (RE-LY) trial which compares warfarin with dabigatran for atrial fibrillation. 93 Bleeding was assumed to increase with age from 2.43% if aged < 65 years to 3.25% for those aged 65–74 years and 4.37% if aged ≥ 75 years. Major bleeds were split into GI bleeds, intracranial bleeds and other major bleeds, with the split between types of bleed 36.5%, 17.9% and 45.6% respectively. All bleeds had a risk of being fatal and the risk of death from each type of bleed in the first month after the bleed was 18.4%, 32.2% and 10.5% respectively. Data on bleeds were obtained from a Spanish registry of patients who have suffered a first VTE. 94 Once an individual suffered a non-fatal intracranial bleed, it was assumed that they had an increased risk of death for the rest of their life, and a standardised mortality ratio of 2.2 was applied, taken from a study looking at long-term survival after an intracerebral haemorrhage. 95 All clinical parameter estimates and their sources are presented in Table 37.

Resource use and costs

The model included costs of therapy (drugs and monitoring) and acute costs for clinical events and long-term costs for intracerebral haemorrhage. Base-case anticoagulation costs included warfarin tablets, assuming an average dose of 4 mg of warfarin per day and INR monitoring. In the sensitivity analysis, dabigatran and rivaroxaban were explored as therapy options. All drug costs were obtained from the British National Formulary. 96 Monitoring was only assumed for warfarin and annual INR test costs were obtained from the economic modelling in a National Institute for Health and Care Excellence (NICE) technology appraisal for dabigatran in the treatment of VTE. 97

One-off acute care costs for DVT and PE, GI bleeds and other major bleeds were obtained from NHS reference costs,98 with costs for DVT and PE calculated as a weighted cost taking into account all Healthcare Resource Group categories for these events. The cost for other bleeds was assumed to be the same as the cost of a GI bleed, due to the huge amount of heterogeneity in the other bleeds category. This assumption was agreed through clinical consensus. Intracranial bleeds were assigned an acute cost and it was assumed that a non-fatal intracranial bleed would lead to lifelong health-care costs. The source of both costs was a study reporting costs of all types of stroke. 99 All unit costs were updated to 2012/13 prices using the Hospital and Community Health Services index100 and are presented in Table 38.

Estimation of quality-adjusted life-years

Utility values were required for all possible health states and clinical events, and were combined with information on survival in order to calculate QALYs. The initial post-index VTE health state was assumed to have a utility value related to the age of each individual as they enter the model using European Quality of Life-5 Dimensions UK normative values. 101 As individuals age within the model, their utility score changes to reflect the score for that age range (Table 39).

Therapy with warfarin was associated with a very small disutility of 0.997102 (applied multiplicatively). DVTs and non-fatal PEs were assumed to reduce quality of life for a month then patients returned to their previous utility level. Non-fatal intracranial bleeds and PTS were assumed to reduce quality of life for the rest of the patient’s lifetime. GI bleeds and other major bleeds reduced quality of life for 2 weeks after the event and the same level of disutility was assumed. Median utility values and their interquartile range for a DVT, PE, GI bleed, intracranial bleeds and PTS are taken from Locadia et al. ,103 who used the time trade-off method to elicit utility weights from a 124 participants who had been affected by a VTE, major bleeding event or PTS. All utility values were applied to the age-related utility multiplicatively. For example, for an individual aged between 55 and 64 years (utility value 0.80), PTS would reduce their utility to a value of 0.66 (0.80 × 0.82 = 0.66). Utility values and their sources can be found in Table 40. In addition to event-specific mortality, other-cause mortality was also included in the model, dependent on the current age and sex of the patient, and the risk of death was taken from UK life tables. 104

Time horizon restricted to 3, 5 and 10 years

In order to determine the impact of a decision rule over a shorter time horizon, the model was run for time horizons of 3, 5 and 10 years. In particular, 3 years was the length of time the post D-dimer model could provide predicted recurrence estimates with good calibration statistics (see Model validation in the internal–external cross-validation cycles). The results, shown in Tables 43–45, show all ICERs were larger for the shorter time horizons than the lifetime time horizon. The 15% threshold strategy could be considered be cost-effective within 3 years; however, the 10% threshold strategy is no longer cost-effective, although is close to the £20,000 per QALY threshold by 10 years.

Cost of warfarin monitoring

The model was run with scenarios where the cost of warfarin monitoring was doubled and where the cost was halved. For the a higher cost of monitoring, the 15% threshold strategy remained potentially cost-effective at £9830 per QALY gained, but the ICER for the 10% threshold strategy increased to £23,597 per QALY gained, above the threshold for cost-effectiveness (Table 46). Lowering the cost of monitoring resulted in lower ICERs for the 15% and 10% threshold strategies, thus becoming more cost-effective. However, the 5% threshold strategy, with an ICER of £42,066 per QALY, was still not a cost-effective option (Table 47).

Disutility of warfarin

Tables 48 and 49 present the results of changing the disutility on warfarin. It is evident the model is sensitive to this variable, with only the 15% threshold decision rule that could be considered cost-effective if there is a greater disutility of 0.95, with all other options dominated by no therapy. However, if there is no disutility, then all ICERs decrease; however, the 15% and 10% threshold strategies still remain the only potentially cost-effective options at the £20,000 per QALY willingness-to-pay threshold.

Newer anticoagulants

The model was run for the newer anticoagulants rivaroxaban and dabigatran, using their higher drug costs, but assuming the same effectiveness and impact on bleeds, and also assuming they resulted in no disutility for individuals. If patients are treated with either therapy, only a strategy with a threshold of 15% could be considered cost-effective, with all ICERs higher than those for warfarin (Tables 50 and 51). The 10% threshold strategy is no longer cost-effective, with ICERs above the £20,000 per QALY willingness-to-pay threshold.

Alternative values for utility losses: all clinical events

Alternative values for utility losses due to VTE and bleeding events were used, as an alternative source for values was available. 106 Results in Table 52 demonstrate that changing values for all types of event does not change the overall result a great deal, rather than just changing values related to therapy.

Alternative values for utility loss: post-thrombotic syndrome

By lessening the quality-of-life impact of one condition (PTS) the values on mean QALYs for different decision rules all increase; however, the overall results do not differ greatly from the base case (Table 53). This is due to only a small proportion of patients in the model being affected by severe PTS.

Alternative value for risk of severe post-thrombotic syndrome after recurrent venous thromboembolism

Increasing the risk of severe PTS after a recurrent VTE decreases all ICERs, with no strategies dominated. This demonstrates the positive impact of avoiding VTE and therefore PTS by treating more patients. However, the 15% and 10% threshold strategies still remain the only cost-effective options (Table 54).

Alternative value for risk of death from pulmonary embolism

The model appears to be very sensitive to the risk of death from PE, with the 5% threshold strategy now potentially cost-effective compared with no therapy, with an ICER of £13,979 per QALY gained (Table 55). This demonstrates how sensitive the model is to small changes in risk of death and a greater risk of PE being fatal makes continuing therapy appear to be worthwhile even at a relatively low 1-year risk of VTE.

Subgroup analysis: patients aged ≥ 60 years

A subgroup analysis was undertaken to determine the cost-effectiveness of alternative strategies when considering only a patient population over the age of 60 years, where risks of bleeding are higher. Only data for those aged ≥ 60 years were used from the patient-level data set for simulating individuals for the model. The 15% threshold strategy could still be considered cost-effective, albeit at a higher ICER of £7337 per QALY (Table 56). The 10% threshold strategy was no longer cost-effective and all other strategies were dominated by treating no-one. This highlights the impact of higher bleeding risks which outweigh the risk of recurrent VTE.

Subgroup analysis: modelling index pulmonary embolism and index deep-vein thrombosis patients separately

The model was run for two separate subgroups, to determine whether the cost-effectiveness of the decision rules strategies were different for different index VTE types. If the model is only run for individuals who have an index PE, all strategies are cost-effective compared with treat no-one, even the treat all option (Table 57). This suggests that the risk of further VTE in this subgroup, which has a higher probability of being a PE (with the risk of being fatal), far outweighs any risk of bleeding on therapy. When the model is run for index DVT patients, the only cost-effective strategy is the 15% threshold strategy, with all other options dominated (Table 58). This suggests that the impact of bleeds and the small disutility on warfarin outweighs the short-term impact of a recurrent VTE in most patients.