Risk assessments and structured care interventions for prevention of foot ulceration in diabetes: development and validation of a prognostic model

Source of data

The data for PODUS 2020 came from a previous research project, PODUS 2015 (see Appendix 3), published in the National Institute for Health Research (NIHR) Health Technology Assessment journal. 15 PODUS 2015 obtained eight studies and had access to another two identified from an IPD SR. Eight studies contributed data to PODUS 2015. 20–27 Access to a ninth study28 was available via a Safe Haven facility; a 10th study29 was not directly available but the PODUS 2015 team could request results of analyses from the data set. After the publication of PODUS 2015, we re-ran the searches to identify new studies and found only one that met the inclusion criteria. Unfortunately, the authors of that study did not respond to requests to share their data. 30 The search strategy to find studies was last run in June 2017 for MEDLINE and in August 2017 for EMBASE, and was published as appendix 3 of the PODUS 2015 Health Technology Assessment journal publication.

Inclusion criteria for development and validation studies

Studies could be included in PODUS 2015 if patients had diabetes mellitus, predictors had been assessed at recruitment, foot ulcer status was assessed at follow-up and the study had recruited at least 100 patients. In addition, for a study to be included in PODUS 2020 development data sets, we required that it collected data on insensitivity to a 10-g monofilament, presence/absence of pedal pulses, history of ulceration or amputation, and the time period in which the ulcer occurs. As we planned to conduct a one-step meta-analysis at the development stage, we needed to merge all of the development data sets, and so required them to be stored on the same server. These criteria reduced the number of eligible development studies to four. 20,21,24,25 Four studies did not provide data on sensitivity to monofilaments and/or the presence or absence of a pedal pulse,22,23,26,27 and the access arrangements for the Leese et al. 28 and Boyko et al. 29 data sets meant that they could not be stored on the same server as those of the other studies. The Boyko et al. 29 data set had been used for validation in PODUS 2015, but included a very small proportion of women (< 2%). We therefore decided to use the Leese et al. 28 data set for validation of the CPR.

We had no date restriction on studies. Recruitment dates ranged from 1 May 1995 to 10 November 2007 in the development data sets, and the final follow-up date was 5 December 2008. In the Leese et al. 28 validation data set, recruitment dates ranged from 28 January 2001 to 8 December 2006 and the final follow-up date was 2007.

Selection of predictors in PODUS 2020

In PODUS 2015, six predictors were selected from a potential candidate list of 22: age, sex, body mass index, smoking, height, weight, alcohol intake, glycated haemoglobin (HbA_1c), insulin regime, duration of diabetes mellitus, eye problems, kidney problems, insensitivity to a 10-g monofilament, absence of pedal pulses, tuning fork, biothesiometer, ankle reflexes, ABI, peak plantar pressure, prior ulcer, prior amputation and foot deformity. Predictors were chosen for clinical plausibility, availability in at least three studies and lack of clinical heterogeneity. Statistical criteria such as small p-values were not used. Six variables were chosen for inclusion in the primary model in PODUS 2015: age, sex, duration of diabetes mellitus, insensitivity to a 10-g monofilament, absence of pedal pulses and prior ulcer or amputation. The analysis was a two-step meta-analysis. In each data set we fitted a logistic regression model with the six predictors, which gave us adjusted estimates for each predictor. We conducted a meta-analysis for each predictor using the generic inverse method. 32 We had used a two-step method so that we could include, in the second stage, aggregate data (i.e. log-odds ratios and their variances) derived from the Leese et al. 28 data set with > 3000 patients. The Leese et al. 28 data set was housed on a different server and so could not be used in a one-step meta-analysis, although this is the preference of some methodologists. 33

We tested these six predictors in the 10th, externally held data set,29 which had 1489 people and 229 ulcer outcomes. We considered the PODUS 2015 results to be replicated in the external data set if the predictor achieved statistical significance, if its effect was in the same direction as the PODUS 2015 estimate and if its CIs overlapped. The predictors that survived this process were insensitivity to a 10-g monofilament, absence of pedal pulses and prior ulcer or amputation.

For the CPR, we decided not to use the three predictors that were not replicated in the Boyko et al. 29 data set: age, sex and duration of diabetes mellitus. Age and duration of diabetes mellitus are credible predictors of any diabetic complication, including foot ulcer. They are also continuous, which means that, in theory, they could be used to generate more precise risk estimates than categorical predictors; however, their inclusion in the CPR would require a calculator, or similar, to estimate risk. CPRs are a form of clinical decision support system that tend not to be used unless they are integrated into the existing workflow. 34 The project did not have access to resources to support a website, or similar, that would calculate risk for health professionals or embed the CPR into NHS information technology systems. However, we could use three binary predictors that were replicated to produce a simple CPR that can be paper based and does not require any calculation from the users to implement. Practicalities as well as the lack of replication in the Boyko et al. 29 data set were reasons to drop age and duration of diabetes mellitus from our CPR model; however, we understand that some individuals will be interested in the six predictor model, and the results from this model are in Appendix 3 and make direct comparisons with the three-predictor model. We also investigated possible reasons why age and duration of diabetes mellitus did not reach statistical significance or were not replicated predictors in the Boyko et al. 29 data set. For simplicity, we also chose not to use the category sex as a predictor. Discussion with potential users of the CPR showed that they were very much in favour of a simpler model.

Definition of the PODUS 2020 predictors

We decided to use the three replicated predictors only (i.e. monofilaments, pulses and history) in the CPR. These three binary predictors were measured at the initial assessment of each patient in each study. In detail, the predictors are:

• Insensitivity to a 10-g monofilament at any site on either foot was defined as test positive. This test is carried out by podiatrists. The podiatrist touches the sole of the patient’s foot with a monofilament and the patient states whether or not he or she felt it.

• In general, there are two pulses tested in each foot: the dorsalis pedis and the posterior tibial pulses. We defined absence of either pulse on either foot as test positive, although it is known that the dorsalis pedis pulse is missing in some healthy individuals. 35

• History of ulceration or amputation was ascertained either at initial assessment or from patient records. Patients were considered test positive for history if they had experienced either ulcer or amputation.

As predictors were measured before outcome in three of the four development studies, the measurement of predictors was blind to outcome. However, assessment of predictors blind to other predictors generally did not occur and would not always be possible; for example, toe amputation would be apparent to any podiatrist assessing monofilaments or pulses.

As in PODUS 2015, we chose to use patient rather than foot as the unit of analysis. The three binary predictors are defined as above, as this was the only way to have a consistent definition in all four development data sets; for example, Crawford et al. 21 recorded the presence or absence of each of the four foot pulses, whereas Abbott et al. 20 recorded the number of foot pulses per person (0–4). The outcome was binary and was defined as the occurrence or not of ulcer by 2 years.

In the PODUS 2015 publication, there is an extensive examination of differences and similarities between the studies as sources of heterogeneity. 15 We repeat some of those analyses here to provide a description of the contributing data sets, with emphasis on the predictors chosen for PODUS 2020.

Sample size considerations

Sample size calculations are generally not carried out for meta-analyses conducted as part of a SR, as the aim is to use not an acceptable minimum but all of the available data. Post hoc sample size calculations are problematic and not recommended by statisticians, and so we did not conduct any. 36 The development data sets have a total of 8255 people with 430 ulcer outcomes, giving 143 events per variable. This is well above the often-cited rule of thumb of 10 events per variable. 37

Statistical models can often give overly optimistic results if the data from which they were derived come from small data sets, if data-driven methods are used to select variables or if too many variables are used. A way of compensating for optimistic results is to use a shrinkage factor:38 a number < 1 by which the coefficients are multiplied. All of the shrinkage factors that we calculated during the model development phase were > 0.9999, which would have resulted in negligible changes, so we did not use shrinkage factors. Shrinkage factors are affected by sample size and complexity of model but our model is simple and our sample size (events) is large relative to the number of included predictors.

The external validation data set had 3324 patients and 128 ulcer outcomes, meeting the recommendation of at least 100 events and 100 non-events to investigate model performance. 39

Missing data

To account for missing data, we would have considered multiple imputation if we thought that data were likely to be missing at random (MAR). 40 However, the proportion of missing data was very small (0–3% in the development data sets and < 2% in the largest data set of > 6000 patients) and so the results of any imputation exercise would not have made any notable difference to our results; therefore, we analysed the data using complete cases only, that is, patients for whom data on monofilaments, pulses, history and ulcer outcomes at 2 years were available.

One reason why outcome information at 2 years might be missing is death of the patient before 2 years. However, death was not consistently recorded across the data sets; for example, in the development studies, the largest study had recorded only one death in 2 years and another did not record deaths at all. The other two development studies were more systematic about including death data. Overall, the proportion of patients recorded as having died was 2%. If a patient had died, but had all information on predictors and outcome, that person was included in our analyses.

Some patients had missing data on previous amputation or ulcer history. However, the clinical context in which these data were collected means that it is very important to record when ulcers and amputations have occurred; therefore, the data were not MAR and are far more likely to be missing if the patients did not have previous ulcers and did not have amputations. Patients who were missing ulcer or amputation history were, therefore, recoded as test negative for these two items. The numbers of patients whose data were recoded are given in each study’s flow chart (Figures 1–4).

FIGURE 1.

Flow of patients in the Abbott et al. 20 data set. All patients had 2-year ulcer outcome recorded. Not all patients are shown at each stage.

FIGURE 2.

Flow of patients in the Crawford et al. 21 data set. Not all patients are shown at each stage.

FIGURE 3.

Flow of patients in the Pham et al. 25 study. Not all patients are shown at each stage.

FIGURE 4.

Flow of patients in the Leese et al. 28 study. Not all patients are shown at each stage.

Length of follow-up in the Crawford et al.21 data set

The PODUS 2020 outcome variable is ulcer occurrence by 2 years, and we knew that the Crawford et al. 21 study had prespecified the follow-up period to be 12 months. We received ethics approval from the Scotland A Research Ethics Committee (reference 16/SS/0213: Integrated Research Approval System project ID 97542), Caldicott approval from NHS Tayside (reference IGTCAL3842) and NHS Tayside approval (reference 2017DM03, NHS Research Scotland reference NRS17/9754) for permission to contact the participants in the study by Crawford et al. 21 to ask if they would consent to include longer-term data in PODUS 2020, some of which were stored on paper records. Despite all these efforts, follow-up data were obtained from only 42% of the original sample (see Appendix 3, Figure 33). Efforts were hampered by the non-retention of patient records for more than 8 years post death, patients being uncontactable and patients who did not consent. The PODUS 2020 Steering Committee discussed this issue and recommended that the data should not be used for the current project.

Statistical analysis methods: choice of model

Given a binary outcome, the obvious method of analysis is logistic regression. Although other methods are available, we chose to base the CPR on a logistic regression model because this model is simple to implement and acceptable to the medical community, and the methods for assessing its performance are well developed. The selection of predictors was described in Definition of the PODUS 2020 predictors. We did not consider adding any interaction terms as these often do not improve the predictive ability of the model19 and would have made the CPR more complex.

As the data came from four studies, we used logistic regression with a separate intercept for each study to allow for clustering of participants within studies and to allow for between-study variation in baseline risk. This was especially important because of the inclusion of the Crawford et al. 21 study, which had a follow-up duration of only 1 year, compared with 2 years in the other three studies. Although ORs of included predictors were similar in studies with 1- and 2-year’ follow-up, the baseline risk was not comparable, as it was higher in those studies with 2-year’ follow-up because of the longer time period.

For defining the intercept for our final CPR based on this logistic regression model, we chose a weighted average of the intercept estimates from the three studies with 2-year follow-up. This weighted average was obtained by using a random-effects meta-analysis of the three intercepts, and fitting using the DerSimonian and Laird method, which allows for both within-study variability (i.e. variance of intercept estimates) and between-study heterogeneity (i.e. genuine differences in baseline risk across studies beyond chance) (see Appendix 3, Figure 42). Therefore, the intercept in our final CPR model was not based on the Crawford et al. 21 study (because of its 1-year follow-up), but predictor effects were based on the four developmental studies.

Statistical analysis methods: transformation of the logistic regression model into a clinical prediction rule

We adapted the method described by Steyerberg41 to generate a CPR from our logistic regression analyses. In brief, Steyerberg’s method is (1) multiply and round regression coefficients, (2) search scores for continuous predictors, (3) estimate the multiplication factor for the scores and (4) estimate the intercept and present a score chart. We omitted the second step because we had no continuous predictors and the third because our multiplication factor was 1. Steyerberg’s method could be applied to many different kinds of statistical model. We made a further modification to allow for the effect of the non-linear logit function used in logistic regression.

The outcome variable in binary logistic regression is the natural logarithm of the odds, or log-odds, of the binary event occurring:

where βs are log-odds ratios and the xs are the predictors. The intercept is the log-odds of the outcome occurring when all the predictors are zero. The probability of the outcome occurring can be calculated from the log-odds. For each unit change in x, the log-odds will increase by the corresponding β (a fixed amount), but the effect on the probability of outcome is not fixed because of the non-linear nature of the log-odds; for example, if the log-odds is 1.3, the corresponding probability is 78.6%. If the log-odds increases by 0.5 to 1.8, the probability becomes 85.8%, an increase in probability of 7.2%. If the log-odds is 2.3 and it is increased again by 0.5 to 2.8, the probability changes from 90.9% to 94.3%, an increase in probability of 3.4%, less than half the change before. The same change in log-odds does not mean the same change in probability given different values of initial log-odds; therefore, when considering the transformation of the logistic regression model into a simpler CPR, we also took account of the probabilities that would result from the scoring system as well as the size of the coefficients. This process was greatly simplified by having only three binary predictors. The number of possible predictor combinations is only eight, and it is not onerous to calculate the probability of ulcer for each combination.

To be explicit, our method was as follows:

1. Fit the logistic regression model with the three risk factor predictors (monofilament, pulses and history) and study. This gives coefficients showing the extent to which the log-odds change for patients who have a test-positive result for monofilaments, pulses or history in comparison with lower-risk test-negative patients. There are also individual estimates for the intercept for each study. The intercept is the baseline risk of ulcer on the log-odds scale. We used SAS^® PROC LOGISTIC (SAS Institute Inc., Cary, NC, USA; SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the US and other countries. ^® indicates USA registration) with maximum likelihood estimation.

2. Conduct a random-effects meta-analysis of the three intercepts from the studies with 2-year follow-up to get a single average intercept.

3. Use this average intercept and the log-odds coefficients for the three predictors to calculate the probability of ulcer for each possible predictor combination; as there are three binary predictors, there are eight combinations.

4. Multiply and round the coefficients of the predictors to get a CPR scoring scheme, bearing in mind that we wanted predictor combinations with similar probabilities of ulcer to have the same score.

5. Repeat step 1 and step 2 using only the CPR score instead of monofilaments, pulses and history.

6. Calculate the probability of ulcer for each score using a population average method.

In the case of a patient who has already contributed to one of the four development data sets, the most accurate estimate of baseline risk will be the appropriate study-specific intercept. A common way to estimate baseline risk for patients not recruited to the development data sets is simply to use the average intercept; however, our preference is to use the population average intercept method described by Pavlou et al. 42 to get estimates of ulcer risk that are applicable to patients in new studies.

Validation of the clinical prediction rule

We assessed the internal validity of the CPR by examining its discrimination and calibration. Discrimination addresses how well the model’s predicted risks discriminate between those who will and those who will not develop an ulcer, and the calibration of how well the estimated risk matches the actual risk of ulceration. We used receiver operating characteristic (ROC) plots and the area under the ROC curve as a statistic of discrimination; the latter is also known as a c-statistic. We assessed calibration with calibration plots, estimation of the calibration slope, and calibration in the large. We assessed the external validity of the CPR in the Leese et al. 28 data set again by examining the discrimination and calibration in the same way. For the validity analyses, we used the probability of ulcer as estimated by the CPR score and compared it with actual ulcer outcome at 2 years.

Other methods of assessing model performance in terms of clinical benefit are available, such as net benefit and decision curves, but we also noted that the performance of the CPR would be addressed using a health economic model.

Using discrimination and calibration statistics in both the development data sets and the Leese et al. 28 validation data set aids comparison of the internal and external validity of the CPR. Exploratory analyses of all the data sets and investigation of heterogeneity was part of PODUS 2015. Hence we knew that the Leese et al. 28 data set was broadly similar to the other data sets. In fact, there was an overlap of patients recruited to the Crawford et al. 21 and Leese et al. 28 data sets, and so we had to remove some patients from the Leese data set to avoid duplication of data. Relevant tables are in Results.

We have also included a net benefit graph to assess potential clinical impact. 43 All analyses were conducted with SAS 9.4 [URL: www.sas.com (accessed 19 February 2019)] and R 3.4.2 [URL: https://cran.r-project.org/ (accessed 19 February 2019)]. The pROC,44 meta32 and rms45 packages in R statistical software (The R Foundation for Statistical Computing, Vienna, Austria) were used.

Description of the individual studies

The quality of the cohort studies used to create the PODUS CPR is detailed in Table 1. 31

The flow of patients in the Abbott et al. 20 data set is shown in Figure 1. The 38 patients with a history of amputation or ulcer and the 23 patients with a history of neither were coded accordingly. Thus, the number of complete cases was 6417 (97%) when recoded ulcer and amputation history was excluded and 6478 (98%) when it was included. One death was recorded in the study, but this patient was also missing pulses and so could not have been included in the development data set.

The number of complete cases in the Crawford et al. 21 data set was 1175 (98.5%), as 18 patients were dropped from the analysis because information on monofilament sensitivity was absent and a further five were dropped because no follow-up time was provided and so ulcer occurrence by 2 years could not be calculated. There were 59 deaths in total in the Crawford et al. 21 data set.

All of the variables required by the CPR were fully recorded in the Monteiro-Soares and Dinis-Ribeiro24 (n = 360) study and so we did not create a flow diagram. As the study setting was secondary care, these data are likely to be accurate. Some other data were missing in the Monteiro-Soares and Dinis-Ribeiro24 study, for example 189 (53%) patients were missing vibration perception threshold (VPT) data, but these were not required for the CPR. Deaths were not recorded.

In the Pham et al. 25 study, the number of complete cases was 242 (97.6%). Three patients were missing a monofilament measurement and three had no time to ulcer/end of follow-up. One patient with a negative amputation history but no ulcer history was coded as negative for history. There were 13 deaths in the Pham et al. 25 study.

The total number of patients in the development data sets was 8404 and the total number who contributed to the analyses was 8255, an overall rate of complete data of 98%.

Among the Leese et al. 28 data set, 295 patients were removed from the analysis as they were included in the Crawford et al. 21 data set. The Crawford et al. 21 and Leese et al. 28 studies recruited in a similar time period in overlapping geographical areas; however, we used the Scottish NHS patient identifier, the Community Health Index number46 (URL: www.ndc.scot.nhs.uk/Dictionary-A-Z/Definitions/index.asp?ID=128%26Title=CHI%20Number), to remove Crawford et al. 21 patients from the Leese et al. 28 data set. This reduced the size of the Leese et al. 28 data set from 3707 to 3412 patients.

The percentage of complete cases in the Leese et al. 28 data set was 97.4%; again, we considered this high enough not to require multiple imputation. During the follow-up period, 95 patients died.

We calculated summary statistics for all the predictors considered for the primary analysis of PODUS 2015, while noting that there is an extensive description of all the data sets in the PODUS 2015 publication. 15

Summary statistics for age, duration of diabetes mellitus, sex, length of follow-up, sensitivity to monofilaments, absent pulses, history of amputation or ulceration and the results of outcomes (ulcer) are in Tables 2–9.

Although the Leese et al. 28 study recorded patients’ test dates, in the case of occurrence only the year was recorded. Therefore, for this data set, we recorded an ulcer as having occurred within 2 years if one was recorded within 2 years of the year that the patient was first seen. This is not a precise way of coding ulcer outcome by 2 years, but it allowed us to use the data set. Ulcer outcomes were recorded from 2001 to 2007; the median year of occurrence was 2005.

From the coding detailed above, the total number of patients from the development data sets used in the logistic regression model underlying the CPR was 8255 (98%), of whom 430 had ulcer-positive outcomes and 7825 had ulcer-negative outcomes at 2 years. In the Leese et al. 28 validation data set 3324 patients had suitable data, of whom 128 had an ulcer by 2 years and 3196 did not. We did not compute unadjusted ORs for the predictor and outcome, as this work had already been done as part of PODUS 2015. 15

Calculating probability of ulcer for each predictor combination

We used Equation 2 to carry out step 3 of the CPR building by first calculating the log-odds of ulcer for each prediction combination and then converting that log-odds to a probability.

Generating a scoring scheme

Part of step 4 was examining ulcer risk probabilities (Table 10). This showed that some different predictor combinations had similar risk. For example, we wanted the (0,0,1) predictor combination with a probability of 0.134 to have the same score as the (1,1,0) combination with a probability of 0.118. Using the probabilities and the method of multiplying and rounding the predictor coefficients described by Steyerberg,41 the CPR scoring method is:

• score 1 if patient is insensitive to monofilaments

• score 1 if patient is missing any pulse

• score 2 if patient has a history of ulcer or amputation.

This results in a CPR that gives scores from 0 to 4. We calculated this score for each patient and refitted the logistic regression model using CPR score as the only predictor.

Refitting the logistic regression model with clinical prediction rule score as the only predictor

The resulting logistic regression model from steps 4 and 5 in Statistical analysis methods: transformation of the logistic regression model into a clinical prediction rule using CPR score is:

The intercept again was taken from a random-effects meta-analysis of the intercepts of the three studies with 2-year follow-up data. We did not use this formula to calculate the probability of an ulcer, but, if we had decided to, the corresponding formula for probability would be:

Using Equation 4 would be perfectly acceptable, but we could calculate population-averaged probabilities of ulcer, which should be generalisable to new studies. The formula for doing so is complex, and not something that can be done easily without statistical software. 42 We therefore calculated the probabilities for our end-users, as one of our aims is that our CPR be easy to use. This is the sixth and final step outlined in Statistical analysis methods: transformation of the logistic regression model into a clinical prediction rule.

Aim

The aim was to produce an overview of the effects of interventions to prevent foot ulceration in diabetes mellitus.

Objectives

The objectives were to identify SRs of RCTs and to obtain data about their effect on the incidence of foot ulcers to calculate measures of effect from individual RCTs, or pool estimates from several trials with which to populate an economic model.

Searches

We searched for SRs using electronic search strategies created for Ovid^® (Wolters Kluwer, Alphen aan den Rijn, the Netherlands) MEDLINE, Ovid EMBASE and the Cochrane Library without language restrictions from inception until February 2019 (see Appendix 4). Our approach to searching was informed by a search string created by staff at the Centre for Reviews and Dissemination at the University of York to identify SRs. SRs in progress were identified via PROSPERO [URL: www.crd.york.ac.uk/prospero/ (last accessed 1 May 2020)].

Eligibility criteria

We assessed the scope of each review to ensure that it matched our objectives.

Review selection and data extraction

One reviewer screened all titles and abstracts to identify potentially relevant SRs. A second reviewer screened a 10% random sample of the yield. Two reviewers (DN and AA) working independently screened the full texts of titles considered potentially relevant to determine whether or not the objectives of each review matched our own with regard to the population, interventions, comparisons, outcomes and study design. Disagreements were resolved by discussion with a third reviewer (FC). Data were extracted into a review-specific data extraction tool (see Appendix 5, Data extraction and quality assessment: systematic review of randomised controlled trials) by two reviewers working independently (DN and FC). Separate data extraction and quality assessment tools were designed and piloted for the SRs. The following data were extracted:

• review author and funder details

• review objectives

• eligibility criteria for trials to be included in the review

• populations (including risk of ulceration), interventions, comparisons, outcomes and method of synthesis (e.g. narrative synthesis or meta-analysis).

Risk-of-bias (quality) assessment

We undertook an assessment of the quality of reporting using the risk of bias in systematic reviews (ROBIS) tool. 57 This tool assesses bias in four domains that correspond to the main processes for conducting SRs: determination of study eligibility criteria, identification and selection of studies, data collection and study appraisal, and synthesis methods and findings (Tables 14 and 15).

We distinguished between SRs that included study designs other than RCTs and those that included RCTs alone, and we tabulated the two groups separately. This is because the ROBIS tool contains an assessment of the appropriateness of the synthesis relating to the nature and similarity in the research questions, study designs and outcomes across included studies (ROBIS domain 4; item 4.3), and we anticipated that the syntheses of data in reviews that included both RCTs and observational studies might be based on all included study designs (randomised and observational) and, if reviewers failed to separate the data from different types of study designs, might not reflect the findings from the RCT data alone.

Plan for data analysis

From each included SR we extracted absolute numbers for the primary and secondary outcomes and measures of effect with associated 95% CIs as reported in the reviews. We also noted the reviews’ overall conclusions about the effects of preventative interventions.

Interventions

Aim

The aim was to conduct a SR of the evidence of preventative effects of interventions for foot ulceration in diabetes mellitus that have been evaluated in RCTs.

Objective

The objective was to produce estimates of the effect of interventions with which to populate an economic model.

Searches

We searched for eligible RCTs of interventions using search strategies created for Ovid MEDLINE (from inception to February 2019), Ovid EMBASE and Central Register of Controlled Trials (CENTRAL) without restrictions (from inception until October 2018). Randomised controlled trials in progress were identified via the International Standard Randomised Controlled Trial Number (ISRCTN) registry (searched to February 2019).

Eligibility criteria

Risk-of-bias (quality) assessment

For RCTs, we carried out an assessment of the risk of bias using the recommended items in the Cochrane Handbook for Systematic Reviews of Interventions. 52

Risk of bias

We identified eight separate interventions:

1. antifungal treatment

2. elastic compression stockings

3. digital silicone device

4. education alone

5. podiatric care

6. digital thermometry

7. complex interventions

8. custom-made footwear and offloading.

Antifungal treatment

One trial evaluating the effect of antifungal treatment was identified by our searches. 102 Thirty-four participants in the intervention group received self-management advice (daily foot inspection) and antifungal nail lacquer (8% ciclopirax) for daily application, while 36 participants in the control group received only advice about foot self-inspection. Almost all patients (97%) were male; patients’ mean age was 70 years, and 57% had experienced previous foot ulcers. Their mean duration of diabetes mellitus was 12 years, but it was not reported how many had type 2 diabetes mellitus (T2DM). Standard care was reported to be a preventative care programme and telephone support, but the exact arrangements for this were unclear. At 12-month follow-up, there were two ulcerations in each group (RR 1.06, 95% CI 0.19 to 5.76). The concealment of the allocation and the blinding of the outcome assessor were unclear and the trial was rated as being at risk of selection and performance bias. No secondary outcomes were reported.

Elastic compression stockings

In one RCT evaluating the effect of elastic compression stockings,93 160 participants were randomised in equal numbers to the intervention or the control group for 48 months. Half of the trial participants were male; patients’ mean age was 53 years and none had a history of foot ulcers. Their mean duration of diabetes mellitus was 15 years, but the number with T2DM was not reported. The intervention group received knee-length elastic stockings with compression at the ankle of 25 mmHg, worn for at least 6 hours per day. There was a difference in the number of limbs that ulcerated in each group (three in the intervention group and 10 in the control group), but this did not reach statistical significance (RR 0.37, 95% CI 0.11 to 1.02). It was unclear how the random allocation was generated, and the nature of the intervention meant that it was not possible to blind patients to the allocation. The outcome assessment was not conducted by an investigator blinded to the random allocation, and the trial was judged to be at risk of selection bias and performance bias.

Secondary outcomes: elastic compression stockings

Thirteen amputations were reported during the 48-month trial: 3 out of 74 in the intervention arm and 10 out of 75 in the control arm.

Digital silicone device

One RCT evaluated the effect of digital silicone devices. 97 One hundred and sixty-seven participants with forefoot deformities who were considered to be at high risk of ulceration, as defined by a VPT of ≥ 25 V, were recruited into the trial. Their mean age was 56.5 years but gender was not reported. Eighty-eight per cent had T2DM and their mean duration of diabetes mellitus was 16 years. Participants in the intervention arm (n = 89) received a bespoke silicone digital orthotic with a variety of therapeutic intents and densities depending on the characteristics of the deformity. The patients in the intervention group received instructions about maintaining the orthotic and were advised to wear it until the end of the follow-up period. The 78 people in the control arm received standard therapy, which comprised the same examinations and procedures as the intervention group, but silicone orthotics were not provided. Both groups also received an accommodating soft insole and an extra-deep shoe. Outcomes collected at 3 months showed a statistically significant difference in the numbers of foot ulcers, with the digital orthotic group experiencing a beneficial effect (RR 0.07, 95% CI 0.01 to 0.55). Concealment of the allocation was not possible because of the nature of the intervention, and the trial was at risk of selection bias. No secondary outcomes were reported.

Podiatric care

Plank et al. 94 compared free chiropody care (n = 47) with no recommendation for chiropody care (n = 44) for 12 months. Fifty-six per cent of the 91 participants were male and their mean age was 65 years. All participants had a history of foot ulceration and were at high risk of another ulcer. Their mean duration of diabetes mellitus was 16 years, and 93% had T2DM. Those receiving free chiropody were recommended to seek care at least once per month. No recommendation was given to patients in the control group, but they could seek chiropody care if they were willing to pay for it. Standard care consisted of instructions on the possible benefits of regular chiropody care. It was unclear if both groups received standard care. There was no statistically significant difference in effect of number of foot ulcers (RR 0.67, 95% CI 0.43 to 1.05). In addition, there were fewer ulcerations in the intervention group (n =18) than in the control group (n = 25) but there were more amputations in the intervention group than in the control group (two vs. one, respectively). Fewer people died in the intervention group (n = 2) than in the control group (n = 4). It was not reported whether or not the outcome was assessed by an investigator who was blind to the allocation and, therefore, the trial data are at risk of performance bias.

Secondary outcomes: podiatric care

Two amputations occurred among 47 participants in the intervention arm and one amputation occurred among 44 participants in the control arm. Two people in the intervention arm died and four people in the control arm died. 33

Data on other secondary outcomes of interest, such as gangrene, self-care, hospital admissions, timing of screening and adverse events or harms, were absent from the trial reports.

Education alone

Three RCTs78–80 evaluated education alone.

One hundred and twenty-one people were followed up for 6 months. 78 Most of the individuals were male (60%); participants’ mean age was 71 years, and almost 11% had a history of foot ulceration. All had T2DM and their mean duration of diabetes mellitus was 15 years. The 60 people in the intervention group received a 2-hour group education programme (in groups of 5–7 patients). This comprised 30-minute face-to-face lessons on risk factors for foot ulcers, instructions on how to check their feet regularly, and information on ulcers and other foot conditions. In an additional 90-minute interactive session, health-care professionals demonstrated practical actions that would reduce the risk of foot ulcers. The 60 people in the control arm received a leaflet with recommendations for preventing foot ulcers. People in both arms received a standard multidisciplinary general education about diabetes mellitus at diagnosis, but they did not receive foot-specific education. At the 6-month follow-up, no statistically significant beneficial effect was observed in the intervention group (RR 0.08, 95% CI 0.00 to 1.31). The trial quality was compromised by a lack of concealment of the random allocation schedule, lack of collection of outcomes by an independent investigator and incomplete outcome data.

The RCT by Gershater79 comprised 131 people who had previously received treatment for a foot ulcer in a diabetes specialist foot clinic. They were recruited after the ulcers had healed and most of them were men (73%); their mean age was 64 years. The majority had T2DM, but the overall mean duration of diabetes mellitus was not reported. Those in the intervention group (n = 61) received foot care education led by a diabetes specialist nurse, designed to improve participants’ confidence in managing their foot health. This consisted of one 60-minute group session with 2–5 participants of the same gender in each group. The group discussions took place in the diabetes foot clinic conference room and were facilitated by a diabetes specialist nurse. Participants could choose to adopt a set of predefined actions/goals.

The 70 people in the control arm received standard, oral and written instructions on self-care based on the International Consensus on the Diabetic Foot. 118 Both arms received standard care, which comprised adjusted shoes and individually fitted insoles, and the recommendation for regular chiropody. All participants also received standard oral and written instructions on self-care provided by a diabetes specialist nurse. The trial quality was compromised by a lack of concealment of the random allocation schedule (selection bias) and lack of the collection of outcomes by an independent investigator (performance bias).

Lincoln et al. 80 randomised 172 people (two-thirds of whom were male) who had previously had ulcers to an intervention or a control group. Eighty-seven people in the intervention arm received a single 1-hour structured foot care education session from a researcher at home. The session included an explanation of the main causes of foot ulcers and a foot examination to identify risk factors (deformity, ischaemia or neuropathy), and advice was reinforced with written information. Shoes and insoles were examined for suitability and participants were advised to contact the clinic immediately if any new or recurrent foot problem emerged. The control group (n = 85) received standard care, consisting of unstructured and opportunistic foot care and the same written information as given to the intervention group. Standard care included podiatry and suitable orthoses when appropriate. At the 12-month follow-up, there was no statistically significant difference in the number of patients with foot ulceration between the two groups (RR 1.00, 95% CI 0.70 to 1.44). All quality assessment items were judged to be at a low risk of bias.

Education alone: meta-analysis

A pooled estimate of the effect of education on the incidence of foot ulceration at 6 months found that foot ulceration was not statistically significantly reduced in three trials of 423 people with diabetes mellitus (RR 1.04, 95% CI 0.55 to 1.97) (Figure 12). 78–80 The heterogeneity (I²) was 54%.

FIGURE 12.

Meta-analysis: education alone.

Secondary outcomes: education alone

Two trials of education interventions reported data on amputation,78,80 mortality,78 knowledge,78 behaviour80 and/or QoL. 80 One trial78 reported no amputations in either of its arms after 6 months’ follow-up. The other trial80 reported three amputations among 85 participants in the intervention arm, compared with no amputations among the 85 participants in the control arm at 6 months; by 12 months there was no difference between the arms (nine amputations in both).

One trial78 reported that two participants, one in each arm, had died by 6 months. In the same trial, a statistically significant difference in knowledge (as measured with the Patient Interpretation of Neuropathy knowledge score) was observed in the intervention arm. 78

One trial80 reported on QoL and found no differences between the two arms on the Diabetic Foot Scale, but scores on the Nottingham Assessment of Functional Footcare questionnaire, which assesses behaviour, were higher in the education arm than in the control arm.

Digital thermometry

Four RCTs87–89,120 involving 468 patients were identified by our searches.

Armstrong et al. 89 followed up 225 people (approximately 69 years of age) for 18 months. Their mean duration of diabetes mellitus was 13 years, but the authors did not report how many had T2DM. The number of participants randomised to the intervention group or the control group is not known. The intervention was an infrared skin thermometer to measure temperatures on six sites on the foot twice per day. Patients were told that, if they recorded a temperature difference of > 4 °F between feet at the same site, they should contact the study co-ordinator and reduce activity until their temperature normalised. Patients also received standard care consisting of footwear, education and professional foot care. The control group also received standard care and were advised to contact the study co-ordinator if they had any foot abnormalities. Participants’ feet were also checked for any signs of irritation from shoes or signs of impending ulcer. The number of ulcers in the intervention and control groups was reported to be 5 and 14, respectively, but the absence of the denominators for each group prevents the calculation of an effect. The quality of the trial is compromised because there was no allocation concealment, and because of the incompleteness of data reporting (selection and attrition bias).

A second RCT (i.e Lavery et al. 87) recruited 85 people (49% were male) with a mean age of 55 years. The mean duration of diabetes mellitus was 14 years, but it was not reported how many had T2DM. The participants were randomised to digital thermometry or standard care and followed up for 6 months. Forty-one participants received a hand-held infrared skin thermometer to measure the temperature at six predetermined sites on the sole of each foot in the morning and the evening. They received additional standard footwear, education and professional foot care, and were advised to contact a nurse case manager and to significantly reduce walking if they recorded a temperature difference of > 4 °F between feet at the same site. The control group received standard care footwear, education and professional foot care. No statistically significant difference was observed between the two groups (RR 0.15, 95% CI 0.02 to 1.19). There was a risk of bias from the generation and concealment of allocation being unclearly reported and there were incomplete outcome data. The method of generating the random sequence and the concealment of the allocation were both unclear and the trial data were at risk of selection bias.

The same authors (i.e. Lavery et al. 88) conducted a three-arm trial over 15 months. Of the 173 participants, 54% were male; participants’ mean age was 65 years and all had a history of foot ulceration. Approximately 95% of participants had T2DM, and their mean duration of diabetes mellitus was 13 years. The 59 participants in the first intervention arm received standard care plus a digital infrared skin thermometer and were asked to measure the temperature on the sole of their foot in the morning and the evening and record this information in a logbook. The participants were advised to contact a nurse manager if the difference in temperature between feet was > 4 °F and to reduce their activity until their temperature normalised. A video was used to teach participants how to use the infrared thermometer. In the second intervention arm, 56 participants received standard therapy plus training to conduct a structured foot examination twice daily using a mirror (to identify redness, swelling, inflammation, etc.). These participants could also could contact a nurse if they detected any abnormalities.

The 58 participants in the control received standard care (including lower extremity evaluation by a physician every 8 weeks, an education programme focused on foot complications and inspection, and therapeutic insoles and footwear). A podiatrist replaced or repaired insoles or footwear if needed. There was also an education component provided by video, and the participants were told to inspect their feet daily and to contact a nurse if necessary. There was statistically a significant difference between the digital thermometry group and the standard care group (RR 0.29, 95% CI 0.11 to 0.73). This trial was at a low risk of bias in all four quality assessment items.

Skafjeld et al. 120 followed up 41 participants (56% male; mean age of 58 years) for 12 months. All participants had previously had ulcers (71% had T2DM), and their mean duration of diabetes mellitus was 18 years. The 21 people randomised to the intervention group received a hand-held device with an infrared heat sensor to monitor their foot temperature. Participants recorded their daily physical activity using a step counter during the first week of the study, and throughout the study their recorded temperature at the same six places on the sole of each foot. Patients were advised that if the recorded skin temperature at the same spot differed by > 4 °F between their feet on 2 consecutive days they should contact the study nurse and reduce their physical activity until their temperature normalised. The control group of 20 participants received standard care, which involved inspecting their feet under the toes, below the toes and between the toes. Participants could contact the study nurse if they noticed changes in their feet, including a new ulcer. They were advised to always wear their customised footwear and consult their general practitioner if necessary. No statistically significant difference was observed in the number of ulcers between the intervention group and the control group (RR 0.67, 95% CI 0.32 to 1.41). Whether or not the allocation was concealed was not clear; in addition, the outcome data were incomplete and the trial data were at risk of selection and performance bias.

Digital thermometry: meta-analysis

A pooled analysis of data from three RCTs87,88,120 found that the use of digital infrared skin thermometry reduced the number of foot ulcers among 243 people with a history of foot ulceration (RR 0.41, 95% CI 0.19 to 0.86) (Figure 13). An I² of 33% was observed.

FIGURE 13.

Meta-analysis: digital thermometry.

Secondary outcomes: digital thermometry

Trials of dermal thermometry variously reported on amputation following infection,89 QoL (assessed using the SF-36),37 adherence to therapy87,88 and time to ulceration. 39,40

In one trial, amputations following infections were required in 0 out of 41 participants in the intervention group and in 2 out of 44 in the comparator group. 38 In the same trial, there was no statistically significant difference in QoL measured using SF-36 in any category or in the overall score. 87

Two trials88,120 found no statistically significant difference between the dermal thermometry group and the comparator group in the time for which prescribed footwear and insoles were worn, as measured using a self-report questionnaire with an ordinal scale of < 4 hours to > 12 hours per day. The time to ulceration was statistically significantly longer in the dermal thermometry treatment group than in the standard care group in one trial88 but not in another. 120

Complex interventions

Five RCTs evaluated the effects of complex interventions on the development of foot ulcer. 84,90–92,119

Cisneros90 recruited 53 participants (62% were male) with a mean age of 62 years, of whom 28% had previously had ulcers and 96% had T2DM. They had a mean duration of diabetes mellitus of 14.5 years. The 24-month intervention (n = 30) was a researcher-led preventative programme comprising therapeutic education (weekly group meetings) and the provision of two pairs of special protective shoes. The education component was delivered over four meetings, each lasting 90 minutes, in groups of up to eight participants; this addressed, with discussion, issues around the prevention of foot ulcers, inspection and foot hygiene, and the choice and use of footwear. Bespoke games were used as teaching aids to prompt individual reflection about lifestyle and diabetes mellitus, and there was a discussion at the end of each meeting between health professionals and individual patients. The pedagogical technique was based on communications and learning. Footwear was provided after completion of the educational programme, at the beginning of the study and after the fourth re-evaluation.

The control group (n = 23) received information on regular foot care and footwear use in response to demand during the individual consultations with the researcher. Both groups received standard care monitoring of their feet to survey the incidence and recurrence of neuropathic injury. Monofilament testing was performed using the monofilament Semmes–Weinstein 5.07 (10 g) to identify risk at three sites on the foot the (digital pulp of the hallux and the first and fifth metatarsal heads). All items in the quality assessment were unclear and the trial was possibly at risk from selection, performance, attrition and selective reporting bias.

A second RCT119 followed 79 participants (51% were male) with a mean age of 66 years for 12 months. It was reported that 42% of participants had previously had ulcers. The majority (94%) had T2DM, and their mean duration of diabetes mellitus was 11 years. Forty-one patients in the intervention received care in two parts. For the first 3 months, they worked with a physical therapist in exercise sessions to strengthen their lower extremity muscles and promote balance. In the second part, over 4–12 months, they increased their exercise activity by 50%. In addition, they received training in self-care and foot examination skills. The 38 participants in the control group were taught foot-related self-care skills, including daily foot examination, and could access their usual medical care from their own health-care providers. Every participant was referred to a local orthotist or podiatrist at enrolment and received therapeutic footwear to wear when weight bearing inside and outside the home. There was no statistically significant difference in the number of foot ulcers between the two groups (RR 0.19, 95% CI 0.02 to 1.52). All quality assessment items were rated as being at a low risk of bias.

A third RCT91 evaluating a complex intervention comprised 62 participants (56% were male) with a mean age of 56 years. Twenty-five per cent of participants had previously experienced an ulcer, and 87% were classified as having T2DM. On average, they had had diabetes mellitus for 11 years. Thirty-one participants in the intervention group received a foot care kit that contained nail clippers, foot care cream, a monofilament with 10-g pressure, a thermometer to measure the temperature of water for foot washing, alcohol cotton pieces and a mirror. They were shown how to use the kit and asked to carry out daily foot care, checking their feet every day with a mirror. Participants could attend diabetes education classes every 3–6 months and were followed up every month for 2 years by a nurse and an endocrinologist performing a foot examination. Both the control group and the intervention group received standard care consisting of medication adjustment, foot assessment and 2 hours of diabetes education. No statistically significant effect in the number of foot ulcers was evident (RR 0.07, 95% CI 0.00 to 1.12). All items in the quality assessment were unclear and the trial may be at risk from selection, performance, attrition and selective reporting bias.

In a fourth RCT92 evaluating a complex intervention, 396 participants were followed up for 12 months. Most of the participants were women (81%) and they had a mean age of 60 years; it was unclear whether or not they had previously experienced foot ulcers. Every patient had T2DM and had received a diagnosis at least 10 years before. The 191 patients in the intervention group received nurse-led patient education sessions involving one to four patients and covering appropriate foot care behaviours and footwear. Patients entered into a behavioural contract for desired self-foot care. A systems intervention was designed to prevent patient-specific risks using information flow sheets on foot-related risk factors for amputation in diabetic patients. Health-care providers used prompts to ask patients to remove their footwear and perform foot examinations, and patients were also given foot care education. Both groups received standard care, which consisted of a physical examination performed by nurse clinicians who were blind to patients’ randomised treatment. There was no statistically significant difference in the number of foot ulcers between the two groups (RR 0.47, 95% CI 0.20 to 1.12). All items in the quality assessment were judged to be unclear and the trial was possibly at risk from selection, performance, attrition and selective reporting bias.

In a RCT evaluating a complex intervention,84 the majority of the 2001 participants were male (53%) and the participants’ mean age was 60 years. It was not reported how many had previously had foot ulcers. Eighty per cent had T2DM, but their mean duration of diabetes mellitus was not reported. The 1001 participants in the intervention group received a foot risk assessment comprising testing of sensitivity to Semmes–Weinstein monofilaments, measurement of VPT using a biothesiometer and the palpation of pedal pulses. Those deemed to be at high risk of foot ulceration were invited to attend a weekly diabetic foot clinic, where they were provided with self-care advice, chiropody care and support hosiery and/or protective shoes. The 1000 participants in the control group continued to attend the general outpatient clinic and had ulcer and amputation outcomes collected but received no special care. The diabetes outpatient service advised those in the control group about the importance of daily foot inspection and washing, appropriate hosiery and footwear and making contact with the clinic whenever they deemed it necessary. No statistically significantly different effect in the number of foot ulcers was observed between the two groups (RR 0.69, 95% CI 0.41 to 1.15). Generation and concealment of allocation and blinding were unclear. Three items in the quality assessment were rated as unclear (apart from incomplete outcome data, which was rated as high risk) and the trial was rated as being at possible risk of selection, performance, attrition and selective reporting bias.

Complex interventions: meta-analysis

A pooled analysis of data from the 2587 people with diabetes mellitus in five RCTs showed that complex interventions statistically significantly reduced the number of foot ulcers in the intervention groups (pooled RR 0.59, 95% CI 0.38 to 0.90) (Figure 14). The ulcer risk category of those who took part was not reported in three trials,84,92,119 but two trials included 44 patients who had not previously experienced a foot ulcer. 90,91 Heterogeneity was found to be I² 10%; however, with the exception of one trial,119 the validity of these data may be affected by bias.

FIGURE 14.

Meta-analysis: complex intervention.

Secondary outcomes: complex interventions

Amputation,84,91 time to ulceration90 and/or knowledge of foot care91 were reported in three trials. In one trial,91 amputations occurred only in the control arm (two among 31 participants, compared with none among the 31 participants in the intervention arm) and in a second trial84 there were fewer amputations in the intervention group (one major and six minor) than in the control group (12 major and 13 minor). 84 The time to ulceration was shorter in the control group than in the intervention group in one trial, but this finding did not reach statistical significance. 90

In one trial, participants’ knowledge of foot care (as measured using a diabetes knowledge questionnaire) was statistically significantly better in the intervention group than in the control group. 91

Custom-made footwear and offloading

We identified six RCTs evaluating custom-made footwear and offloading devices.

Bus et al. 99 recruited, and followed up for 18 months, 171 participants, most of whom were men (82.5%), with a mean age of 62 years. All had previously had ulcers. Nearly three-quarters (71%) of participants had T2DM, and their mean duration of diabetes mellitus was 17 years. Eighty-five people in the intervention arm received either custom-made footwear or semi-customised footwear with offloading properties improved, which was prescribed by a specialist in physical and rehabilitation medicine and manufactured by a local orthopaedic shoe technician. The custom-made shoes were created from a plaster-cast mould of the foot or from three-dimensional digital scans. Participants received orthoses made from either a cork base with added microcork and a mid-layer of ethylene vinyl acetate or a Plastazote leather. Participants were permitted to wear any additional custom-made footwear they owned at study entry or that was prescribed during follow-up. The 86 people in the control group wore footwear that was not improved based on in-shoe pressure measurement. Both groups received written and verbal instructions on foot care and on proper use of footwear as standard practice. No statistically significantly different effect was detected (RR 0.88, 95% CI 0.61 to 1.26). All four quality assessment items were judged to be at a low risk of bias.

A three-arm trial of footwear and orthoses86 recruited, and followed up for 24 months, 400 participants (77% were male) with a mean age of 62 years, all of whom had experienced a foot ulcer. The majority of participants (93%) had T2DM and the duration of diabetes mellitus was > 25 years in 56% of the trial population. Participants were randomly allocated to a group receiving three pairs of therapeutic shoes with medium-density cork inserts and a neoprene closed-cell cover or to a group receiving three pairs of therapeutic shoes with prefabricated, tapered polyurethane inserts and a brushed nylon cover or to the control group, who received usual footwear. All groups received standard professional foot care and a lightweight terry-cloth house slipper. There was no statistically significant difference in number of foot ulcers between those who received therapeutic shoes with either neoprene inserts or polyurethane inserts and the control group (RR 1.00, 95% CI 0.70 to 1.43). The reporting of concealment of allocation was unclear.

A third RCT, of footwear and pressure-relieving devices,96 included 298 participants, with a mean age of 67 years, of whom 20% had had a previous ulcer. The majority (84%) had T2DM, for a mean duration of 18 years. One hundred and forty-eight participants were randomised to the intervention group and received custom-made orthoses and shoes. Shoes were mostly semi-orthopaedic footwear, available on the open market, or craft-made orthopaedic shoes. Each patient also received additional education session about the need to wear the shoes and to inspect their feet daily. Patients could receive an urgent consultation within 24 hours if they developed a new foot ulcer. Follow-up took place every 3 months to assess the feet and the condition of the footwear. Standard care for 150 people in the control group comprised education to prevent foot ulcers and advice about footwear. A statistically significant difference between groups was observed (RR 0.30, 95% CI 0.18 to 0.49), with custom-made orthoses and shoes producing a beneficial effect. The concealment of allocation and outcome assessor blinding were unclear and the trial data are at risk of selection and performance bias.

A fourth RCT, of custom-made footwear and insoles,95 comprised 299 participants (67% were male and the mean age was 70.5 years). Twenty-five per cent had experienced a foot ulcer and, on average, they had had diabetes mellitus for 12.5 years, but the number with T2DM was not reported. The 149 participants in the intervention group received therapeutic shoes in which the standard insole had been replaced with an insole designed to reduce shear. The control group (n = 150) received the same brand of therapeutic shoes containing the standard insole. In both groups insoles were replaced every 4 months and the shoes were replaced once per year. Both groups also received standard care comprising a foot and lower extremity evaluation by a physician every 10–12 weeks and an educational video that focused on foot complications and self-care practices. The video addressed the aetiology of DFUs, risk factors, self-care practices and early warning signs of diabetic foot disease. Patients were to contact the study nurse if they identified an area of concern on their feet. After 18 months’ follow-up, no statistically significant difference in effect was detected (RR 0.30, 95% CI 0.08 to 1.08). The generation and concealment of allocation were unclearly reported, as was the completeness of the outcome data, and the validity of the trial data may be compromised by selection and reporting bias.

Uccioli et al. 85 randomised 69 participants (62% were male and the mean age was 60 years). All participants had experienced a foot ulcer, 75% had T2DM, and the mean duration of diabetes mellitus of those recruited was 17 years. The period of follow-up was 12 months. The intervention arm comprised 33 people who received therapeutic shoes made from soft thermoformable leather with semirocker soles and custom-moulded insoles (which were of extra deep fit customised insoles and accommodate toe deformities) plus standard care. The 36 people in the control group were free to wear ordinary shoes, unless it was clearly dangerous for them to do so, and received only standard care, which comprised educational guidelines on foot care and general information about the importance of appropriate footwear. A statistically significant effect was observed (RR 0.47, 95% CI 0.25 to 0.87). Generation of the random allocation was judged to be at a high risk of bias, and concealment of allocation and blinding and completeness of the outcome data were unclear; consequently, the trial data may be at risk of selection, performance, attrition and reporting bias.

The sixth RCT98 randomised 150 participants (68% were male) with foot ulcers who had a mean age of 59.5 years. The number of participants with T2DM and the duration of diabetes mellitus were not reported. All participants were advised to remain in their healing devices or removable cast walkers until their foot ulcers healed and before shoes were dispensed at ≈7 weeks after randomisation. Seventy-nine people in the intervention group were randomised to receive bespoke orthoses with offloading properties to reduce pressure on the metatarsal heads. Modified using a computer-aided design process, the orthoses were milled from a block of ethylene vinyl acetate foam and covered with a polyurethane foam top cover. Seventy-one people in the control group received standard orthoses from three different manufacturers. Both groups were provided with extra-depth shoes from PW Minor (Batavia, NY, USA) but a different specification was required to accommodate the bespoke orthoses (DX2 as opposed to the extra-depth specification that was received by the control group). Study co-ordinators discussed self-care behaviour with all patients, focusing on wearing the study shoes for all steps taken and examining the feet daily to note and report problems to the study team. There was a statistically significant difference in effect at 15 months (RR 0.34, 95% CI 0.14 to 0.81). The risk of bias was low for all four quality assessment items.

Custom-made footwear and offloading: meta-analysis

A pooled estimate of data collected from trials evaluating custom-made footwear and offloading insoles showed a reduction in the number of foot ulcers (pooled RR 0.53, 95% CI 0.33 to 0.86) in 1387 people, of whom 464 had no history of foot ulceration; however, there was a high level of heterogeneity (I² = 78%) possibly arising from variation in the construction of the shoes and insoles as well as the difference in risk (Figure 15).

FIGURE 15.

Meta-analysis: custom-made footwear and offloading.

Custom-made footwear and offloading: subgroup analyses

A subgroup analysis of data collected only from patients with a history of foot ulceration (n = 424) did not find a statistically significantly different effect in these patients (RR 0.71, 95% 0.47 to 1.06), and the degree of heterogeneity was I² of 61% (Figures 16 and 17). Conversely, the RR of pooled data from two RCTs that included patients with no history of foot ulceration (n = 297) was 0.30 (95% 0.19 to 0.47) with I² of 0%. It should be noted that many of the data in these analyses may be affected by biases emanating from the conduct of the trials, as only two trials were judged to be completely free of bias. 98,99

FIGURE 16.

Subgroup analysis: custom-made footwear and offloading in patients with a history of foot ulceration.

FIGURE 17.

Subgroup analysis: custom-made footwear and offloading in patients with no history of foot ulceration.

Table 20 shows the outcomes from the quality assessment process. Only 5 of the 23 trials were judged to be free of bias.

Ongoing randomised controlled trials

The searches for ongoing trials to prevent foot ulceration in diabetes mellitus from the ClinicalTrials.gov website found 24 studies being conducted worldwide, the details of which are presented in Appendix 4.

Aim

The aim was to undertake a review of published cost–utility analyses of the prevention of DFUs.

Methods

We undertook a review to identify (1) costs and outcomes of relevant interventions for the prevention of diabetic foot ulceration and (2) published cost–utility analyses of such interventions. The review methods are provided in Appendix 5.

Results

Our search returned 11 relevant papers to be reviewed. Four other relevant papers were identified by ‘hand-searching’ the references of the 11 papers identified. Of these 15 papers, five were cost–utility analyses of the prevention of DFUs. 121–125 The remaining papers either were concerned with the cost–utility of treatments for patients who had already experienced DFUs or were papers relating to the cost burden of DFUs. 126–134 Further hand-searching did identify other studies in which an economic analysis was undertaken (i.e. some assessment of costs and outcomes); however, as these papers were not cost-effectiveness papers and were beyond the scope of this review, they were excluded.

We review here the five papers that were cost–utility analyses of the prevention of DFUs. Full details of the papers reviewed are given in Appendix 6, Table 52.

Quality of economic evaluations

All five cost–utility papers were quality assessed using the Drummond checklist135 for economic evaluations. All studies were deemed to be of high quality.

Review of papers

Eastman et al. 121 used a Markov model to simulate the natural history of diabetic patients and the rate of complications (including DFUs and LEA) and to evaluate the impact of preventative treatments. Information on age, sex, ethnicity and incidence and prevalence rates were based on US community and population studies. The authors then estimated the cost-effectiveness of glycaemic control for reducing diabetes-related complications. Costs and outcomes were estimated for patients aged between 25 and 74 years. The authors found that treatment that maintains an HbA_1c value of 7.2% is associated with a reduction in LEA of 67%. They found the cost per additional quality-adjusted life-year (QALY) of treatment was £16,002.

Ragnarson Tennvall and Apelqvist122 used a Markov model to assess the cost–utility of an ‘optimal’ prevention programme for patients at risk of DFUs and amputation. The prevention programme included foot inspection, appropriate footwear and education. Patients were stratified in the model according to their risk of ulceration. The treatment effect of the preventative programme was based on RCTs and observational studies reported in the literature (see Table 21). Costs and outcomes were assessed over a 5-year time horizon. The study found that optimal preventative treatment of high-risk patients was cost saving based on the achievement of a 25% lower incidence of DFUs and extremity amputation than in baseline prevention scenarios. Similarly, Ortegon et al. 123 used a Markov model to assess the cost–utility of optimal prevention and treatment of diabetic foot. The preventative programme included protective foot care, education, regular inspection, risk identification and the involvement of a multidisciplinary team. Treatment effect was based on a study that reported a reduction in LEA of between 49% and 85%. 135 Patients were stratified in the model according to risk. Costs and outcomes were measured over the lifetime of a patient. The study found that, based on the achievement of a reduction in ulceration incidence of 10%, preventative treatment was cost-effective, with a cost per QALY gained of < US$25,000 (2003 prices).

Rauner et al. 124 used a Markov model to assess the cost–utility of preventative education programmes to reduce DFUs. Treatment effect was based on the assumption of a 25–50% reduction in DFUs, based on estimates obtained from the literature. 82,136 The authors’ model built on previous work by stratifying patients according to not only risk but also age, allowing for the potential for more targeted decision-making. Costs and outcomes were assessed over a 10-year period. Their results suggest that preventative programmes are highly cost-effective when targeted at patients at high risk of DFUs and LEAs.

Barshes et al. 125 used a Markov model to assess the cost–utility of primary prevention programmes aimed at reducing the incidence of DFUs. In contrast to the other papers under review, which took the cost and effectiveness of treatment as given and estimated the cost-effectiveness, Barshes et al. 125 sought to estimate the effectiveness target necessary for a treatment to be considered cost saving. Costs and outcomes were estimated over a 5-year period. The study found that preventative programmes had a > 90% probability of being cost-saving when the annual prevention costs were < US$50 per person and/or when DFUs are reduced by at least 25%. If patients were deemed to be at high risk, programmes were cost-saving when incidence of DFU reduced by 10%.

All studies, with the exception of Eastman et al. ,121 used a Markov model to simulate the clinically relevant states experienced by patients at risk of developing DFUs. All of these studies included clinical states representing pre-ulceration, ulceration and post-ulceration outcomes; however, they varied in terms of which other relevant clinical factors they included in the modelling. The studies by Ortegon et al. 123 and Rauner et al. 124 were the only ones to stratify patients according to risk state. Three studies123–125 included additional clinical states associated with diabetic foot, such as ischaemia (e.g. neuropathy, peripheral vascular disease). Eastman et al. 121 used a Monte Carlo simulation to estimate the rate of complications, including DFUs and LEAs, associated with all patients with T2DM. The disparity in modelling strategies among these studies is a natural consequence of the complexity of the disease and the range of possible complications. Furthermore, the choice of modelling strategy, and the clinical events included, is a function of the specific decision problem addressed in each study; however, the result is that uncertainty about the most appropriate choice of model for estimating the cost-effectiveness of strategies to prevent DFUs.

All of the papers reviewed found that preventative treatment for DFUs was cost-effective or, in some cases, cost saving. However, the evidence on effectiveness in these studies comes from research conducted some time ago and/or studies that have small sample sizes; therefore, significant uncertainty remains about the treatment efficacy of potential interventions.

Only the model of Ortegon et al. 123 estimated the lifetime costs and outcomes relating to DFUs. As the costs (in terms of the ongoing care required) and outcomes (in terms of fewer DFU and amputations), and the significant morbidity associated with these, will have an impact over a patient’s full lifespan, it is necessary to take a lifetime perspective when estimating cost-effectiveness.

All the studies reviewed suggest that the effectiveness (and cost-effectiveness) of treatments is likely to vary according to patient risk, and that treatment is most likely to be considered cost-effective in the subgroup of patients at high risk. However, none of the papers reviewed have modelled how often a patient’s risk should be assessed (risk monitoring frequency).

Summary

Our review highlights considerable heterogeneity in how the clinical and cost consequences of treatments to prevent DFUs have been modelled in the literature. Furthermore, at the time these studies were published, there was little consensus about the efficacy of treatments available for preventing DFUs; therefore, existing models of cost-effectiveness rely on assumptions about treatment effect. Another important limitation of current models is that risk monitoring frequency has not been considered.

Overview

There is a lack of consensus among clinical guidelines about the monitoring frequency required for patients who are at risk of developing DFUs. NICE currently recommends annual monitoring of patients at low risk, and monitoring as frequently as once every 1 or 2 weeks for patients at high risk. SIGN recommends that the monitoring take place at least annually, but concedes that the optimal monitoring frequency is unknown; hence, there is a need for new economic evaluations to consider the impact of alternative monitoring frequencies on DFU outcomes and the implications for cost-effectiveness. As discussed in Review of papers, the choice of modelling strategy will depend on the specific decision problem at hand and, thus, clinical input into the model choice is crucial from an early stage. The literature review of economic models has shown that interventions that are able to reduce the incidence of DFUs in at-risk patients have the potential to be cost-effective. Based on our review of the literature, and on advice from our clinical colleagues, we developed a de novo health economic model to answer the questions stated in our objectives.

Cost perspective

This economic evaluation was undertaken from the perspective of the UK NHS and Personal Social Services137 that is, the costs relevant to the economic analysis were those incurred by the NHS and Personal Social Services.

Time horizon

This economic evaluation investigated the costs and health outcomes associated with each clinical pathway over a 20-year time horizon.

Discount rate

Future costs and health outcomes were discounted at 3.5%, in line with recommended practice. 137

Conceptual model of disease and treatment pathways

Our task was to undertake an economic evaluation of an ‘evidence-based pathway’ for the prevention of DFUs. To do so, we had to understand the important clinical and economic events in the care pathway. The conceptual model in Figure 18 is a visual representation of the events we sought to capture in our model and how these events relate to costs and QALY outcomes.

FIGURE 18.

Conceptual model of decision problem.

Conceptual model implemented as a Markov model

The model was implemented as a semi-Markov model, as the survival analyses used to estimate transitions indicated that transition probabilities varied according to time in state. Submodels were developed to estimate total discounted costs and QALYs for ulceration and amputation events to simplify the implementation of the model. Figure 19 illustrates graphically the transitions we require to estimate this conceptual model as a Markov model.

FIGURE 19.

Schematic of semi-Markov model and submodel.

NHS Fife Scottish Care Information – Diabetes Collaboration data set

NHS Fife SCI-Diabetes is an electronic patient record used in the routine management of NHS patients with diabetes mellitus. The data set we obtained contained information about individual patients who received foot care in NHS Fife foot monitoring clinics over a 12-year period. The data contained the results of risk assessments of the risk factors recommended by the SIGN guideline,12 only three of which are required in our validated CPR as reported in Chapter 4: monofilament sensitivity, pedal pulses and history of ulceration (all binary variables). Data were available from the beginning of 2005 until the end of 2017. A separate data set from NHS Fife SCI-Diabetes provided death data that were merged with the SCI-Diabetes foot monitoring data. No death data were recorded before 1 March 2009 in the data set; therefore, this date was used as the entry point into the defined cohort as we would not know whether or not patients with records that stopped before that date had died. The use of the SCI-Diabetes anonymised data set was approved by the information governance department of NHS Fife.

Clinical prediction rule definition of risk status

The economic model includes the use of the CPR, which defines a patient’s DFU risk status as follows:

• Low risk [CPR score of 0 or 1; probability of ulcer at 2 years ≤ 0.05 (see Table 10)] – negative for history and also negative for at least one of monofilament sensitivity and pedal pulses.

• Moderate risk (CPR score of 2; probability of ulcer at 2 years = 0.12) – positive for history but negative for both monofilament sensitivity and pedal pulses, or negative for history but positive for both monofilament sensitivity and pedal pulses.

• High risk (CPR score of 3 or 4; probability of ulcer at 2 years ≥ 0.25) – positive for history and also positive for at least one of monofilament sensitivity or pedal pulses.

This categorisation, although arbitrary, was chosen for ease of use in a clinical setting (just three categories of risk) and was decided on following group discussions at project team meetings.

NHS Fife Scottish Care Information – Diabetes Collaboration population

Patients who were recorded in the data before 1 March 2009 and were attending foot monitoring clinics on or after 1 March 2009 (prevalent patients) or who started to attend foot monitoring clinics after this date (incident patients) were defined as eligible for analysis. Having a mix of prevalent and incident patients is appropriate as any changes in the monitoring policy would apply to all patients. In the case of patients whose first recorded appointment was before 1 March 2009, the date of entry to the analysis cohort was considered to be 1 March 2009 and their CPR status was the last recorded CPR status before 1 March 2009. In the case of those patients whose first recorded appointment was on or after 1 March 2009, the date of entry to the cohort was taken as that date, alongside the patient’s CPR status at that date. Use of these criteria resulted in 26,154 patients being included in the cohort used to inform important parameters of the economic evaluation model. Table 22 summarises the key demographics of this cohort.

Figure 20 shows the total number of clinical visits per patient in the cohort over the period of approximately 8 years. The follow-up time varied considerably between each patient depending on whether they were a prevalent or an incident patient (and when they became an incident patient) and also if the patient died. As can be seen from Figure 20, almost 80% of patients in the data set had more than one visit to the foot assessment clinic, and it was not uncommon for a patient to have more than 10 clinical encounters.

FIGURE 20.

Histogram of total number of clinical visits per patient in the cohort.

Table 23 presents the number of key events of interest for the economic model that occurred during the follow-up of the identified cohort. Among the 26,154 patients, approximately 4% developed an ulcer, 1% required an amputation and almost 24% died.

If the only record for a patient in the data set was of their death (and hence there were no risk assessment data), we excluded that patient from the cohort used for analysis (analysis cohort, n = 26,086).

Missing data

There were significant numbers of missing data in the NHS Fife SCI-Diabetes data set. This was particularly an issue with regard to records for patients’ test results for the three CPR predictors (monofilament sensitivity, pedal pulses and history of ulceration) that were used to estimate risk status in the cohort. To determine the most likely mechanism for the occurrence of missing data, regression modelling was used to investigate the association between patient demographics and missingness. Patients’ characteristics were found to be associated with missingness, ruling out the possibility that the missing data mechanism was missing completely at random. 138 After consultations with clinicians and data controllers, we considered that MAR was also unlikely and, therefore, ruled out a multiple imputation approach to account for missing data. After consultating with NHS colleagues, we proceeded with the following rules.

When a patient had a missing record for one of their CPR predictor variables, the patient was recorded as not being at risk from this predictor (i.e. if the record for monofilaments was left blank, we assumed that the patient was sensitive to monofilaments). When a patient had a valid recording (positive or negative) followed by missing data in subsequent visits, we assumed that the patient did not change risk status for this variable unless indicated otherwise. Full details of the missing data approach are given in Appendix 6, Table 54.

Clinical prediction rule risk status over time

Table 24 shows that, at the first clinic appointment, 25,003 patients (96%) were classified as at low risk. This decreased to 23,867 patients (91%) by the final clinic appointment. Over time, there was an increase in patients classified as at moderate risk (first visit, 3%; final visit, 4%) and at high risk (first visit, 1%; final visit, 4%). Overall, 1397 (5%) of the cohort changed their risk status between their first visit and their final visit. The numbers in Table 24 that indicate a change in risk category are shown in italics.

Estimation of transition probabilities

Transition probabilities required for the model were estimated based on a set of parametric survival models built on the aforementioned analysis cohort (Table 25). We tested the following parametric models: exponential, Weibull, Gompertz, log-logistic, log-normal and generalised gamma. The choices of distribution were determined by a visual examination of Kaplan–Meier plots and log-cumulative hazard plots and Akaike information criterion (AIC)/Bayesian information criterion (BIC) tests of model fit, as recommended by NICE Decision Support Unit Technical Support Document on survival analysis methodology (see Appendix 6). 139 Kaplan–Meier survival plots for the key transitions in the economic model are presented in Appendix 6.

Estimating costs and utilities

Cost-effectiveness analysis

Costs and effects were measured for each potential treatment strategy. The cost-effectiveness of the alternative treatment pathways were evaluated based on their incremental cost-effectiveness ratio (ICER) and their net monetary benefit (NMB). ICERs are calculated as follows:

where ΔCosts is the difference in total costs between interventions and ΔQALYs is the difference in QALY gain between interventions.

If a new treatment is found to be more costly and less effective than the current treatment, the new treatment is said to be ‘dominated’. Conversely, if the new treatment is more effective and less costly than the current treatment, the new treatment is said to dominate the current treatment.

The NMB is a measure of the health benefit, expressed in monetary terms, which incorporates the cost of the new strategy, the health gain obtained and the societal willingness to pay for health gains. The NMB is expressed using the following formula:

where E is effectiveness, WTP is the willingness-to-pay threshold (£20,000 in the UK) and C is cost.

The NMB approach is recommended when comparing more than one intervention and provides a clear decision rule (i.e. if NMB > 0, the new strategy is cost-effective).

Comparators

The base-case treatment strategy in all scenarios is current practice. Current practice is defined as the natural history of the disease, with no CPR to monitor risk, and will involve whatever interventions are currently offered in practice and hence captured in the health outcomes of the current analysis cohort. Current practice is taken as the base case and is compared with the treatment strategies outlined in Table 28.

This modelling strategy allows us to incorporate the recommendations of clinical guideline bodies such as NICE and SIGN. Our model included the use of a CPR that is based on a patient’s response to monofilaments, presence or absence of pedal pulses and history of ulceration. These risk factors are recommended, although not exclusively, by NICE and SIGN. There is no evidence-based consensus on the most appropriate strategy for managing patients at risk of DFUs. Although both NICE and SIGN recommend that specialist footwear be considered, only SIGN recommends the use of multidisciplinary care teams. The use of digital thermometry is not recommended in clinical guidelines and is currently the focus of RCTs. SIGN notes that there is no evidence on the ideal monitoring frequency for patients at risk of DFUs; however, it advises annual monitoring. NICE, on the other hand, recommends monitoring annually for low-risk patients, every 3–6 months for patients at moderate risk, every 1–2 months for patients at high risk and every 1–2 weeks for patients for whom there is an immediate, pressing risk of DFUs. By investigating the use of the CPR at 2 years, 1 year and 6 months, our model allows the costs and outcomes associated with a range of preventative strategies recommended by clinical guideline bodies to be explored.

Probabilistic sensitivity analysis

Probabilistic sensitivity analysis was used to explore uncertainty in the model input parameters and to describe the impact that this uncertainty has on the model outcomes, namely costs and QALYs gained. A 1000-iteration Monte Carlo simulation was undertaken. Gamma distributions were used to represent uncertainty in the cost parameters; multivariate, normal distributions were used for (log-)survival regression parameters; beta distributions were used for health utility parameters; and a log-normal distribution was used to represent treatment effect parameters.

The distribution of ICERs produced by the Monte Carlo simulation was presented on the cost-effectiveness plane. Cost-effectiveness acceptability curves (CEACs) were used to present the uncertainty in the decision regarding the most cost-effective option, over a variety of monetary willingness-to-pay thresholds. 140

Value-of-information analysis

We conducted a value-of-information (VOI) analysis to determine the value to society of collecting further information on the effectiveness, costs and cost-effectiveness of alternative clinical pathway strategies.

To order to establish whether or not future research would be worthwhile, we calculated the expected value of perfect information (EVPI), which is the difference in net benefit of a decision based on our current information (evidence) and a decision with perfect information (i.e. no uncertainty). The situation of perfect information can be thought of as reducing the width of CIs around all of our model parameters to zero. We estimated the EVPI summed across the potential DFU population in NHS Fife and for a time period during which we expect this treatment strategy to remain the ‘gold standard’. In our analysis, we assumed that a reasonable time period for estimating the VOI would be 10 years. We also assumed that the number of patients in the NHS Fife SCI-Diabetes data set would be the number of eligible patients over a 10-year period (26,086 patients over 8 years in NHS Fife SCI-Diabetes, and hence 3261 patients per year). This equates to an ‘effective population’ (e.g. discounted population) of 15,239 patients, or 1524 patients per year, eligible in Fife. If the EVPI for the population is greater than the costs of carrying out the additional research, then carriyng out this research is potentially cost-effective. To determine what type of new evidence would be most valuable, we calculated the expected value of partial perfect information (EVPPI) for parameters of interest. Owing to the non-normal distribution of NMB, we estimated the VOI using a non-parametric simulation approach. 141

Custom-made footwear and offloading

Table 29 presents the results of the economic model in which patients are monitored with the CPR and treated with custom-made footwear and offloading. The results suggest that, in this scenario, treating all patients with special footwear is the most cost-effective strategy.

Digital infrared thermometry

Table 30 presents the results of the economic model in which patients are monitored with the CPR and treated with digital infrared thermometry. The results suggest that, in this scenario, treating all patients with digital thermometry is the most cost-effective strategy.

Complex interventions

Table 31 presents the results of the economic model in which patients are monitored with the CPR and treated with complex interventions. It can be seen that the expected costs of the different strategies with complex interventions are higher than for the others, but with no advantage in QALYs. As a result, at a willingness to pay of £20,000, current practice is the preferred option.

Cost-effectiveness planes

Figures 21–23 are graphical illustrations of the uncertainty surrounding the ICERs produced from a 1000-iteration of a Monte Carlo simulation. All three simulations suggest that there is considerable uncertainty surrounding the QALYs associated with each treatment strategy; however, there is comparatively less uncertainty around the total costs. Figure 23 shows that treating all with complex interventions is likely to have a significantly higher total cost than other treatment strategies.

FIGURE 21.

Distribution of incremental costs and QALYs associated with custom-made footwear and offloading.

FIGURE 22.

Distribution of incremental costs and QALYs associated with digital infrared thermometry.

FIGURE 23.

Distribution of incremental costs and QALYs associated with complex intervention.

Cost-effectiveness acceptability curves

Figures 24–26 show the probability that each treatment strategy is considered cost-effective, compared with the alternatives, for a range of thresholds of willingness to pay for a single QALY gain. Figures 24 and 26 show considerable uncertainty surrounding which intervention is most likely to be deemed cost-effective, with no clear strategy producing the greatest probability at a willingness-to-pay threshold of £20,000 per QALY gained. Only in the case of digital infrared thermometry (see Figure 25) does the treat-all strategy emerge as having the greatest probability of being cost-effective, although, even for this intervention, the CEAC suggests a probability of just over 30% that this strategy is the most cost-effective at a willingness-to-pay threshold of £20,000 per QALY.

FIGURE 24.

The CEAC for custom-made footwear and offloading.

FIGURE 25.

The CEAC for digital infrared thermometry.

FIGURE 26.

The CEAC for complex intervention.

One-way sensitivity analysis

One-way sensitivity analysis assessed the impact on cost-effectiveness of varying our chosen willingness-to-pay threshold from £20,000 per QALY gained (as recommended by NICE) to £13,000 per QALY gained (Claxton et al. 142). This had the effect of reducing the cost-effectiveness of custom-made footwear and offloading (from an incremental NMB of £766 to £250) and of infrared thermometry (from an incremental NMB of £1802 to £1139). The use of complex interventions is still considered not to be cost-effective. There was no change in the relative position of the cost-effectiveness of screening intervals.

Digital infrared thermometry

The EVPI per patient affected by the decision to recommend treatment based on a strategy of treating all with digital infrared thermometry is estimated to be £8533.

Based on the same assumptions as above with regard to eligible population, time horizon and willingness to pay for QALY gains, we estimate an EVPI of £10.4M per year and of £104M over 10 years. Figure 29 illustrates how this EVPI varies with willingness-to-pay threshold.

FIGURE 29.

The EVPI (population, £), based on infrared digital thermography.

The EVPPI suggests that the majority of the value of reducing parameter uncertainty in our model would be generated from reducing uncertainty around the parameters obtained from our survival analysis of NHS Fife SCI-Diabetes data (Figure 30).

FIGURE 30.

The EVPPI per patient (£), based on digital infrared thermometry.

Complex intervention

The EVPI per patient affected by the decision to recommend treatment based on a strategy of treating all with complex interventions is estimated to be £8968.

Based on the same assumptions as above with regard to eligible population, time horizon and willingness to pay for QALY gains, we estimate an EVPI of £10.9M per year and of £109M over 10 years. Figure 31 illustrates how this EVPI varies by willingness-to-pay threshold.

FIGURE 31.

The EVPI (population, £), based on complex intervention.

The EVPPI suggests that the majority of the value of reducing parameter uncertainty in our model would be generated from reducing uncertainty around the parameters obtained from our survival analysis of NHS Fife SCI-Diabetes data (Figure 32).

FIGURE 32.

The EVPPI per patient (£), based on complex interventions.

Across all three interventions available, the VOI analysis suggests that there is value in reducing the uncertainty related to the decision problem (i.e. whether or not it is cost-effective to use this treatment strategy). In terms of the specific groups of parameters we looked at, there was value in reducing the uncertainty relating to all of them, but reducing the uncertainty relating to the NHS Fife SCI-Diabetes data had the greatest value across all three analyses. This suggests that increasing our knowledge of the epidemiology of DFUs, in terms of how patients move between risk states and how often events occur will allow us to better capture the true cost and outcomes associated with our proposed treatment strategy, and allow us to minimise the health losses from choosing the wrong treatment strategy.

Summary of principal findings

• Our analysis of the NHS Fife SCI-Diabetes population that attend foot clinics suggests that patients’ DFU risk does not readily change over time.

• Despite the significant uncertainty, our health economic model suggests that preventative DFU interventions have the potential to be considered cost-effective.

• The CPR developed in this study can be used as a tool for identifying patients who are suitable for treatment.

In this chapter, we have (1) undertaken a review of published cost–utility models for the prevention of DFUs; (2) used this information, alongside input from clinical experts, to develop a health economic model capable of estimating the cost-effectiveness of DFU prevention treatments and alternative monitoring frequencies with a CPR; and (3) estimated the value of further research and suggested where this research should focus. Our results suggest that preventative treatments for DFUs have the potential to be considered cost-effective at a willingness-to-pay threshold £20,000 per QALY gained.

Our model included the use of annual monitoring with the CPR, as recommended by NICE and SIGN. We also looked at the impact of biannual monitoring and monitoring every 6 months; however, our model did not look at the strategy of monitoring only high-risk patients as often as every 1–2 months or monitoring patients of immediate concern every 1–2 weeks. As noted Chapter 3, Methods, our design of the economic model was influenced by the results of the survival analysis of the NHS Fife SCI-Diabetes data. Analysis of these data suggests that patients do not readily change risk status (i.e. move from low to moderate risk or from moderate to high risk). Indeed, over the 8 years for which we had data, just 5% of individuals changed risk status; therefore, it is unlikely that increasing the monitoring frequency from 6-monthly (which is included in the model) to, for example, every 1–2 months or every 1–2 weeks, will capture enough additional events to offset the additional monitoring costs.

Further research aimed at predicting which patients will change risk status is warranted, as these are the patients best placed to benefit from preventative treatment. Further research, for example in the form of developing a prognostic model capable of predicting the patients whose risk status is likely to change, rather than those who are likely to develop ulceration, would be welcome.

In conducting an economic evaluation of the introduction of the CPR, we needed to compare the effect of directing therapy with and without the CPR. In the model, we have assumed that if we do not have the CPR we have two options: give no-one the intervention(s) or give everyone the intervention(s). This is rather simplistic given that some kind of risk assessment currently takes place;28 however, given the multiple combinations of treatment strategies across both the NICE and the SIGN guidelines, in terms of risk factors, methods of assessment, and interventions and monitoring frequencies, it would have been challenging to define a specific strategy including a CPR that could constitute ‘current practice’. Hence, our comparison with ‘no CPR’ allows us to estimate the potential value of the new CPR and the value of further research into the new CPR.

In our analysis, we investigated the impact of introducing a CPR based on three risk factors. Current guidelines on how patients should be assessed for risk involve a greater range of potential risk factors (NICE: insensitivity to a 10-g monofilament, absence of pedal pulses, presence of significant structural abnormality, neuropathy disability score (> 5), history of ulceration, more formal assessments using a biothesiometer, Doppler ultrasound and/or an ABI test; SIGN: insensitivity to a 10-g monofilament, limb ischaemia, ulceration, callus, infection or inflammation, deformity, gangrene, amputation, Charcot arthropathy). Given the range of potential risk factors suggested by NICE and SIGN, and the lack of clarity about the adherence of clinicians to any particular strategy, it was not possible in our study to include the use of a current prognostic model to identify patients at risk of DFUs, with which we could compare the new CPR model developed in this study. Further research aimed at comparing different prognostic models would be welcome. Treating all patients attending a foot screening clinic is not currently recommended by the NICE or the SIGN guidelines; however, considering a ‘treat all patients attending foot screening’ strategy, and hence not using a CPR, allows us to distinguish between the value of introducing the CPR and the value of the interventions themselves.

The meta-analysis and economic modelling of potential preventative treatments suggests that custom-made footwear and offloading and digital thermometry have the potential to be cost-effective; however, given that patient adherence to both of these treatments is unknown and likely to be a critical factor, further research into patient acceptability of and adherence to these treatments would be welcome.

Our VOI analysis suggests that reducing the uncertainty about the epidemiology of DFUs is likely to produce the greatest value. This may be partly explained by the small number of ulceration and amputation events in this data set and, hence, the associated uncertainty. To reduce this uncertainty, a study similar to the one presented in this chapter could be run using diabetes mellitus register data for the whole of Scotland or the UK, and for longer follow-up periods, to reduce the uncertainty related to the number of events.

Our VOI analysis also suggested that there was little value in further research relating to evidence on the treatment effect of interventions; however, it should be stressed that only one form of uncertainty is captured by the VOI analysis, namely the uncertainty related to the parameters estimated and used in the health economic model (i.e. the CIs around the estimate). The analysis does not capture the uncertainty related to issues such as publication bias and heterogeneous populations in the original studies that informed the treatment effect estimates. Hence, the decision uncertainty relating to the treatment effect in our VOI analysis (and the value of reducing this) is likely to be an underestimate. It should also be noted that in our analysis the treatments were regarded as mutually exclusive; however, in reality a combination of these treatments could be offered to patients. Our analysis also assumed that treatment effect remained constant across level of patient risk, as the relative effectiveness of these treatments in patients with different levels of risk is unknown. In addition, the effectiveness of any of these treatments used at alternative monitoring frequencies is also unknown; hence, the effectiveness of combined treatments (for specific risk groups, and used at alternative frequencies) should be explored further. To address the uncertainty surrounding treatment efficacy, a RCT would be required; however, we found that, in an at-risk population of (approximately) 26,000, few patients changed risk status, or developed ulceration, or received amputations over an 8-year follow-up period. Given this, the size, duration and cost of a RCT in this area, and what it would add compared with a large observational study (based on routine data sources), would need to be considered carefully.

Strengths of this study

The NHS Fife SCI-Diabetes data set was obtained from monitoring patients in routine clinical practice. This has given us an insight into the treatment of patients in a ‘real-world’ setting. The data set contained > 26,000 patients followed up over an 8-year period. This allowed us to assess empirically how risk of ulceration changes over time.

As part of this study, we developed a de novo health economic model (guided by the advice of clinicians involved in the prevention of DFUs) capable of including both the use of a CPR and the costs and outcomes associated with alternative clinical pathways.

Limitations of this study

The NHS Fife SCI-Diabetes data set was obtained by monitoring patients in routine clinical practice (i.e. it was not designed to be used for research); therefore, considerable data cleaning and manipulation was required. Despite this, issues of missing data remained in the data set. In particular, patients’ results for the three predictive tests included in the CPR were frequently missing. Furthermore, the project team’s clinicians suggested that amputation events may be under-recorded. This is because patients who progress to the stage of requiring an amputation may no longer be seen by the foot care team and, as a result, their data are not captured by risk assessment monitoring. For each variable in our analysis, we have chosen what we believe to be the most appropriate form of imputation – that is, what we believe is most likely to produce a consistent and plausible record of patient monitoring, reflecting what is most likely to have happened in clinical practice.

Costs of ulceration were identified from the literature. 131 As patients may be treated either in an inpatient setting or in an outpatient setting, our cost of ulceration is a weighted average of the two costs, assuming that 90% of patients are treated in an outpatient setting (based on clinical expert opinion). Amputation costs were weighted according to which type of amputation was required. Based on the literature, we assumed that two-thirds of amputations that are required are minor and that one-third are major. 143 There was also some uncertainty about the costs of interventions. Digital infrared thermometry involves a relatively inexpensive device; however, because of the lack of data on its use in routine practice, it is not clear what other resource may be required to implement this treatment strategy (e.g. contacting nurses to report high temperatures and to seek further advice). The precise elements of a ‘complex intervention’ vary across the UK, with no single agreed-on approach or clarity about the best types of specialist care required; therefore, our estimate of resource use for complex intervention may differ from that delivered in practice across the UK.

The diabetes mellitus population used in our analysis was a mix of an incident and prevalent population. This was justified on the grounds that any new monitoring policy would probably be introduced for all patients, rather than only for new patients; however, it should be noted that we do not explicitly model the process of patient entry and we take the estimates from the survival analysis as representing a static cohort. Although this is not strictly technically correct, we believe that this is a reasonable compromise to simplify the modelling.

Our choice of low, moderate and high risk of DFUs was based on the predicted probabilities of DFUs from the CPR and was agreed in consultation with the wider project team; however, it should be noted that, based on our definitions of risk, the proportion of patients classified as high risk is lower than that found in other studies. This may have implications for our findings regarding the lack of movement between risk states over time. Further work on this model should include investigating the impact on cost-effectiveness of alternative definitions of risk according to the CPR; however, we do not believe that this is likely to significantly alter our overall results owing to the significant uncertainty in the epidemiology of the NHS Fife SCI-Diabetes data, the true costs of interventions and the effectiveness of potential interventions.

Implications of our findings

• The finding that patients’ DFU risk status does not change readily suggests that a move towards less frequent risk monitoring of at-risk patients would be acceptable.

• As the proportion of patients whose DFU risk status changes over time is low (5%), but the costs and health outcomes associated with this group are significant, further work to predict those patients whose risk score is most likely to change would be welcome.

• There is a need for further research into the effectiveness and acceptability of and adherence to potential preventative DFU interventions.

• There is a need to better understand what constitutes ‘current practice’ in DFU prevention across the UK, in terms of risk assessment methods (risk factors and how they are assessed), interventions offered and adherence to clinical guidelines.

Substantial amendment 1 (in italics and specific request underlined)

We suspect that the very small proportion of ulcers found in the Crawford dataset (n = 23) was due to the short length of follow-up, which was only 1 year, whereas in the other datasets most ulcers develop more than 1 year post baseline tests. We are therefore planning to follow up this cohort of patients via their hand held podiatry records to increase the length of follow-up in the Crawford (10) dataset and estimate that another 100 patients with ulcer will be identified.

Our favourable opinion from Scotland A REC (reference 16/SS/0213) approved the follow up of people with diabetes who gave consent to their participation in the cohort study by Crawford (2011) using the routinely collected data on SCI Diabetes and hand-held podiatry notes. The REC A specified that the project researchers must first of all check the survival of those who gave consent in 2006/2007 in order to ensure that no deceased patients relatives were contacted about the follow up study. Only then could the surviving participants be contacted to obtain consent to follow up. After confirmation from the podiatry department that patients hand held records are archived for at least 7 years after death we now seek approval to collected foot-related follow up data from the podiatry notes of those participants who have deceased since being recruited to the original cohort study. This amendment to our protocol has been recommended by our Study Steering Committee on 15^th March 2017 and is justified as a result of the known association between foot ulceration and death in those with diabetes. However, we have now had confirmation by the NHS Tayside MCN data facilitator that 45% of the original cohort of patients has died in the intervening period and this will greatly reduce the statistical power of our analysis. We now seek approval to collect foot ulcer data for those study patients who have deceased.

Substantial amendment 2 (in italics and specific request underlined)

In order to strengthen our analysis of the optimal risk assessment (monitoring) frequency we wish to obtain all NHS Fife patients’ foot screening data from SCI Diabetes over a 10 year period. These data will be anonymised by staff at the Health Informatics Centre at the University of Dundee (where NHS Fife routinely collected data is stored). We have discussed the need for Caldicott approval [Public Benefit Privacy Panel (PBPP)] with the NHS Fife Information Governance Advisor and attach her response to our enquiry for PBPP approval.

We can use these data to provide hazard ratios, estimates of the rates of ulceration, and estimates of sensitivity and specificity of the CPR and other risk assessment tools (e.g. NICE and SIGN guidelines) calculated for different time periods as required by the health economic model. The NHS Fife SCI Diabetes routinely collected data will be used by the project statisticians and health economists to calculate the transitional probability of a patient transitioning from one risk category to another and to a state of foot ulceration. We have been advised by NHS Fife data manager the numbers of people with diabetes who have received foot risk assessments are as follows:

1.  2007 n = 10,405

2.  2008 n = 10,829

3.  2009 n = 12,413

4.  2010 n = 10,949

5.  2011 n = 12,635

6.  2012 n = 13,003

7.  2013 n = 13,832

8.  2014 n = 14,496

9.  2015 n = 13,549

10.  2016 n = 4249

The identification of the optimal screening interval base case will require an estimate of the potential effectiveness and cost-effectiveness of the clinical risk score calculated during patients’ annual reviews based on the PODUS time-to-event data and the NHS Fife SCI-Diabetes data. This corresponds to the intent to develop a simple-to-use screen that can be used during routine assessments for people with diabetes mellitus. We will then estimate the rate at which the risk score varies and will try to estimate its effectiveness when applied more frequently at 6-monthly intervals and less frequently at biennial intervals using a hidden Markov chain analysis.

Approval granted 17 October 2017 (REC reference: 16/SS/0213/AM01).

Substantial amendment 2 (in italics and specific request underlined)

In accordance with our favourable opinion from Scotland A REC we have now contacted 649 participants by post and 243 gave written consent for the researchers to access their health records to obtain follow up data relating to foot ulcerations. We now seek another substantial amendment to send out a second and third invitation to participate in the follow up study for those who have not responded.

Approval granted 23 July 2018 (REC reference: 16/SS/0213/AM02).

Substantial amendment 3 (in italics and specific request underlined)

We also intend to conduct a search of electronic databases for individual randomised controlled trials which meet the following eligibility criteria.

Predictor replication in PODUS 2015

The Boyko et al. 29 data set was not supplied to the PODUS 2015 team, but a SAS^® software program (SAS Institute Inc., Cary, NC, USA) was sent to the Boyko et al. 29 team to be run in the data, and the analysis results were sent back. The SAS program refitted the logistic regression model using the same six predictors. This method meant that the Boyko et al. 29 team were kept blind to the results of the PODUS 2015 analyses and the Boyko et al. 29 analysis was completely prespecified, and the predictors were tested in a new data set.

Predictors were considered both predictive and replicated if the Boyko et al. 29 ORs achieved statistical significance and had the same direction as the PODUS 2015 ORs and if the CIs of the PODUS 2015 and Boyko et al. 29 ORs overlapped.

We compared the summary ORs from the meta-analysis of estimates from multivariable logistic regression using the data sets held by the PODUS 2015 researchers with the results of a multivariate logistic regression conducted in the external Boyko et al. 29 data set. The Boyko et al. 29 analysis did not replicate the results of the PODUS 2015 team for three predictors: age, sex and duration of diabetes mellitus. The effect of age was small and not significant in both the PODUS 2015 and the Boyko et al. 29 data sets, and the estimates had different directions (increasing age in PODUS 2015, but decreasing age in Boyko et al. 29) and were associated with increased ulcer risk (PODUS 2015: OR 1.005, 95% CI 0.994 to 1.016; Boyko et al. :29 OR 0.993, 95% CI 0.977 to 1.009). In addition, age was not a statistically significant predictor of ulceration in six of the nine studies held by the PODUS team. The effect of duration of diabetes mellitus was also not replicated, being predictive in the PODUS analyses and protective in the Boyko et al. 29 analyses (PODUS 2015: OR 1.024, 95% CI 1.011 to 1.036; Boyko et al. :29 OR 0.981, 95% CI 0.968 to 0.994). However, it should be noted that the recorded duration of diabetes mellitus is simply the known duration, and people may have had diabetes mellitus for years before a diagnosis was made; in addition, the health-care systems in the Boyko et al. 29 and other PODUS data sets differed. The Boyko et al. 29 data set was not suitable for testing sex as a predictor, as the study recruited veterans and 98% of these veterans were male, which is not reflective of the wider population of people with diabetes mellitus. 15 All ORs taken from the Boyko et al. 29 data set and all ORs taken from the PODUS 2015 data sets were adjusted for the predictors specified for the primary PODUS 2015 model.

The Boyko et al. 29 analyses did replicate the PODUS 2015 results for three predictors: monofilament sensitivity (PODUS 2015: OR 3.184, 95% CI 2.654 to 3.820; Boyko et al. :29 OR 3.489, 95% CI 2.486 to 4.896), pulses (PODUS 2015: OR 1.968, 95% CI 1.624 to 2.386; Boyko et al. :29 OR 2.557, 95% CI 1.220 to 5.361) and history (PODUS 2015: OR 6.589, 95% CI 2.488 to 17.45; Boyko et al. :29 OR 2.979, 95% CI 2.146 to 4.135). 15

Therefore, at the end of the PODUS 2015 project, we knew that we had evidence of an association between foot ulcer outcome and three binary variables in the internal data sets, and that the results of these three binary variables were similar in external data.

Comparison of the three-predictor with the six-predictor PODUS model

Table 32 shows, for each study, the baseline risk of ulcer at 2 years predicted by the two models in the case of a hypothetical 65-year-old female patient who has tested negative for monofilament sensitivity, pulses and history and who was diagnosed with diabetes mellitus 1 month previously. The three-predictor model comprises monofilaments, pulses and history, whereas the six-predictor model also includes age, sex and duration of diabetes mellitus.

Figure 34 shows the results for insensitivity to a 10-g monofilament and risk of developing an ulcer by 2 years, with the model fitted separately to each study. Estimates in the three-predictor group have been adjusted for pulses and history and estimates in the six-predictor group have been adjusted for pulses, history, sex, age and known duration of diabetes mellitus.

FIGURE 34.

Results for insensitivity to a 10-g monofilament.

Figure 35 shows the results for absence of any pedal pulse and risk of developing an ulcer by 2 years, with the model fitted separately to each study. Estimates in the three-predictor group has been adjusted for monofilaments and history and estimates in the six-predictor group have been adjusted for monofilaments, history, sex, age and known duration of diabetes mellitus.

FIGURE 35.

Results for absence of any pedal pulse.

Figure 36 shows the results for history of ulcer or amputation and association with ulcer by 2 years, with model fitted separately to each study. Estimates in the three-predictor group have been adjusted for monofilaments and pulses; estimates in the six-predictor group have been adjusted for monofilaments, pulses, sex, age and known duration of diabetes mellitus.

FIGURE 36.

Results for history of ulcer or amputation.

Figures 34–36 suggest that the estimates of risk associated with positive results for monofilament sensitivity, pulses and history vary little between the three-predictor and six-predictor model in each study. There is, perhaps, greater variation in the estimate of baseline risk (the risk of ulcer at 2 years when all predictors are test negative, sex is female, duration of diabetes mellitus is ≤ 1 month and age is 65 years); however, all estimates are comparable.

Figure 37 shows the forest plot for age (per year increase) and association with ulcer by 2 years, with the model fitted separately to each study. Estimates have been adjusted for monofilament sensitivity, pulses, history of ulcer or amputation, sex and known duration of diabetes mellitus.

FIGURE 37.

Forest plot for age (per year increase).

Figure 38 shows the forest plot for sex and association with ulcer by 2 years, with the model fitted separately to each study. Estimates have been adjusted for monofilament sensitivity, pulses, history of ulcer or amputation, age and known duration of diabetes mellitus.

FIGURE 38.

Forest plot for sex.

Figure 39 shows the forest plot for known duration of diabetes mellitus (per year increase) and association with ulcer by 2 years, with the model fitted separately to each study. Estimates have been adjusted for monofilament sensitivity, pulses, history of ulcer or amputation, age and sex.

FIGURE 39.

Forest plot for known duration of diabetes mellitus (per year increase).

Investigation of non-linearity

Non-linearity was investigated by using the Box–Tidwell method. 148 In brief, this method involves calculating a new variable from the variable thought to have a possibly non-linear effect. If this variable is x, the new variable is x × log(x), where the log is the natural logarithm. Both the new and the original variables are then included in the logistic regression model, and, if the new variable is statistically significant, then there is evidence of a curvilinear effect. Our two candidate predictors for curvilinear effects are age and duration of diabetes mellitus. Two models were fitted with all six predictors from the PODUS 2015 model (monofilament sensitivity, pulses, history, sex, age and duration of diabetes mellitus) and study as a fixed effect; one model had the Box–Tidwell age-transformed variable, and the other had the Box–Tidwell known duration of diabetes-transformed variable.

Our tentative conclusion from Tables 33 and 34 and forest plots and the results given Appendix 3 PODUS 2015 is that there is no strong evidence that age is a predictor of ulcer at 2 years. The evidence of an association with ulcer at 2 years is stronger for duration of diabetes mellitus, but, given the result in the Boyko et al. 29 study, that longer duration was protective against ulcer, this result may not be generalisable to different settings.

Meta-analyses using three or six predictors

We also present results of meta-analyses using the method described in Chapter 3, where study is fitted as a fixed effect in a logistic regression, using the four development data sets and either three or six predictors (Tables 35 and 36).

The three-predictor plus study meta-analysis had a c-statistic of 0.813 (95% CI 0.790 to 0.835). The six-predictor plus study meta-analysis had a c-statistic of 0.820 (95% CI 0.797 to 0.842). The performance of the six-predictor model is slightly better, but ease of use of the three-predictor model in a CPR is much greater. The results of the three- versus six-predictor analyses (see Figures 40–42) do not show a huge advantage of the six-predictor model; the discrimination and calibration results are similar.

FIGURE 40.

Calibration plot for the three-predictor plus study meta-analysis.

FIGURE 41.

Calibration plot for the six-predictor plus study meta-analysis.

FIGURE 42.

Discrimination plot for the three-predictor plus study meta-analysis vs. the six-predictor plus study meta-analysis.

Methods

In the time-to-event analyses undertaken, the outcome was the first foot ulcer that occurred during follow-up. Follow-up is defined as the time from the date of recruitment into a study until the date of the first foot ulcer. If no foot ulcer occurs, then follow-up is censored at the date of the last appointment or the end date of the study, whichever occurs first, or at the date of death of the patient.

Hazard ratios for the predictors were derived using two different statistical techniques for time-to-event analysis. Royston–Parmar flexible parametric survival analysis methods were used for the main analyses, because this statistical technique includes the baseline hazard and baseline cumulative hazard functions in the progression to ulceration during follow-up. 148,149 The more common Cox proportional hazards regression was used in sensitivity analyses; this modelling technique does not make any assumption about the shape of the underlying hazard function and, thus, leaves the baseline hazard rate unspecified. The proportional hazards assumption was checked for each predictor by using statistical tests and graphically examining log-minus-log plots and scaled Schoenfeld residuals on functions of time.

Baseline hazard functions were created for each study using splines with differing numbers of knots, and their shapes were examined. The three predictors (i.e. insensitivity to a 10-g monofilament, one or more absent pedal pulses, and history of ulceration or LEA) were included in the flexible parametric survival model, together with two dummy variables for each of two of the three studies, to take account of the different study populations. Similar models were developed with one, two and three internal knots to ascertain which model best fitted the data. The model with the lower AIC and lower log-likelihood value was preferred, unless the two AIC and log-likelihood values were very close in size, in which case the simpler model (with fewer degrees of freedom) was selected. Our final decision was to use the simplest one-knot model (Figure 45). For the Cox regression models, the three predictors were included first in a stratified model, to account for the differences between the studies, and then in a model that was adjusted for study. In each model, the Monteiro-Soares study was taken as the reference study, as the follow-up in this study was longer than those in the Crawford and Pham et al. 25 studies.

FIGURE 45.

Baseline hazard function for three studies with one internal knot (k = 1). The dashed lines represent the upper and lower confidence intervals for each of the baseline hazard functions for the individual studies.

All analyses were performed with IBM SPSS Statistics version 22 (IBM Corporation, Armonk, NY, USA) (URL: www.ibm.com/products/spss-statistics) and RStudio version 1.0.143 (RStudio Inc., Boston, MA, USA) (URL: www.rstudio.com/), which is an integrated development environment for R (The R Foundation for Statistical Computing, Vienna, Austria) (URL: https://cran.r-project.org/).

Results

For basic demographic and predictor descriptive statistics for each study, see Tables 2–9 and Figures 1–4.

As expected, Figure 43 shows that the baseline hazard for the Crawford data set is lower than that found in either the Monteiro-Soares or the Pham et al. 25 data set. Univariate results for each of the predictors are shown in Figures 46–48.

FIGURE 46.

Kaplan–Meier plot for progression to foot ulceration, comparing adults who are sensitive with adults who are insensitive to a 10-g monofilament.

FIGURE 47.

Kaplan–Meier plot for progression to foot ulceration, comparing adults with at least one absent pedal pulse with adults with no absent pedal pulses.

FIGURE 48.

Kaplan–Meier plot for progression to foot ulceration, comparing adults who have experienced a prior foot ulceration or LEA with adults who have not.

Checking the proportional hazards assumption

It is important to check the proportional hazards assumption when using Cox proportional hazards regression. Log-minus-log plots for the three pooled studies are in Figure 49. The results of additional statistical tests and graphs based on the scaled Schoenfeld residuals are also given in Figure 49.

As the Schoenfeld residuals are independent of time, the plots of residuals should be randomly distributed; if the plots of residuals are not randomly distributed, this would imply a violation of the proportional hazards assumption. In the final column of Table 39, the p-values for all three predictors and the global test are non-significant, which supports the proportional hazards assumption. We concluded from our checks that there is no strong evidence that the proportional hazard assumption has been violated (Table 39 and Figures 49 and 50).

FIGURE 49.

Log-minus-log (LML) plots for the three predictors to check that the proportional hazards assumption holds. (a) Sensitive vs. not sensitive; (b) pulses absent vs. pulses present; and (c) no ulcer vs. previous ulcer.

TABLE 39 - Results of statistical tests to check that the proportional hazards assumption holds

Predictor	P	χ2	p-value
Monofilament insensitivity	0.053	0.687	0.407
Absent pedal pulses	–0.056	0.622	0.430
History of ulcer/LEA	–0.066	1.087	0.297
Global	–	1.991	0.574

FIGURE 50.

Plots of the scaled Schoenfeld residuals against transformed time to check that the proportional hazards assumption holds. (a) Schoenfeld individual test: p = 4072; (b) Schoenfeld individual test: p = 0.4304; and (c) Schoenfeld individual test: p = 0.2972. Global Schoenfeld test: p = 0.5743.

The health economic model has considered three risk groups: low-risk people with CPR scores of 0 or 1, medium-risk people with CPR scores of 2, and high-risk people with CPR scores of 3 or 4. We present the rates of ulcer per risk group in Table 40.

TABLE 40 - Patterns of ulceration in the three risk groups according to the length of follow-up

Risk group	Length of follow-up (years)	New foot ulcer, n (%)	No new foot ulcer, n	Total, n (%)
Low	≤ 1	20 (45)	704	724 (56)
	1–2	15 (34)	415	430 (33)
	2–3	3 (7)	67	70 (5)
	≥ 3	6 (14)	78	84 (6)
	Total	44 (100)	1264	1308 (100)
Medium	≤ 1	15 (52)	76	91 (51)
	1–2	9 (31)	44	53 (29)
	2–3	1 (3)	20	21 (12)
	≥ 3	4 (14)	10	14 (8)
	Total	29 (100)	150	179 (100)
High	≤ 1	57 (50)	50	107 (36)
	1–2	25 (22)	53	78 (26)
	2–3	16 (14)	60	76 (26)
	≥ 3	17 (14)	20	37 (12)
	Total	115 (100)	183	298 (100)

The survival analyses suggest broad agreement with the logistic regression analyses in terms of the weighting of the coefficients of monofilament sensitivity, pulses and history. Across the entire follow-up period, the risk categories have an overall percentage of ulceration of 3.4%, 16.2% and 38.6% for low-, medium- and high-risk groups, respectively.

TABLE 41 - The TRIPOD checklist: prediction model development and validation

NA, not applicable; TRIPOD, Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis.

Items relevant only to the development of a prediction model are denoted by D, items relating solely to a validation of a prediction model are denoted by V, and items relating to both are denoted by D; V. We recommend using the TRIPOD checklist in conjunction with the TRIPOD explanation and elaboration document.

Section/topic	Item	Development/validation	Checklist item	Section
Title and abstract
Title	1	D; V	Identify the study as developing and/or validating a multivariable prediction model, the target population and the outcome to be predicted	Chapter 3, Introduction
Abstract	2	D; V	Provide a summary of objectives, study design, setting, participants, sample size, predictors, outcome, statistical analysis, results and conclusions	NA
Introduction
Background and objectives	3a	D; V	Explain the medical context (including whether diagnostic or prognostic) and rationale for developing or validating the multivariable prediction model, including references to existing models	Chapter 3, Introduction and Appendix 3
	3b	D; V	Specify the objectives, including whether the study describes the development or validation of the model or both	Chapter 3, Introduction
Methods
Source of data	4a	D; V	Describe the study design or source of data (e.g. randomised trial, cohort or registry data) separately for the development and validation data sets, if applicable	Chapter 3, Methods
	4b	D; V	Specify the key study dates, including start of accrual, end of accrual and, if applicable, end of follow-up	Chapter 3, Methods
Participants	5a	D; V	Specify key elements of the study setting (e.g. primary care, secondary care, general population) including number and location of centres	Chapter 3, Methods
	5b	D; V	Describe eligibility criteria for participants	Chapter 3, Methods
	5c	D; V	Give details of treatments received, if relevant	Chapter 3, Methods
Outcome	6a	D; V	Clearly define the outcome that is predicted by the prediction model, including how and when assessed	Chapter 3, Methods
	6b	D; V	Report any actions to blind assessment of the outcome to be predicted	Chapter 3, Methods
Predictors	7a	D; V	Clearly define all predictors used in developing or validating the multivariable prediction model, including how and when they were measured	Chapter 3, Methods
	7b	D; V	Report any actions to blind assessment of predictors for the outcome and other predictors	Chapter 3, Methods
Sample size	8	D; V	Explain how the study size was arrived at	Chapter 3, Methods
Missing data	9	D; V	Describe how missing data were handled (e.g. complete-case analysis, single imputation, multiple imputation) with details of any imputation method	Chapter 3, Methods
Statistical analysis methods	10a	D	Describe how predictors were handled in the analyses	Chapter 3, Methods
	10b	D	Specify type of model, all model-building procedures (including any predictor selection) and method for internal validation	Chapter 3, Methods and Appendix 3
	10c	V	For validation, describe how the predictions were calculated	Chapter 3, Methods
	10d	D; V	Specify all measures used to assess model performance and, if relevant, to compare multiple models	Chapter 3, Methods
	10e	V	Describe any model updating (e.g. recalibration) arising from the validation, if done	NA
Risk groups	11	D; V	Provide details on how risk groups were created, if done	Chapter 3, Methods
Development vs. validation	12	V	For validation, identify any differences from the development data in setting, eligibility criteria, outcome and predictors	Chapter 3, Methods
Results
Participants	13a	D; V	Describe the flow of participants through the study, including the number of participants with and without the outcome and, if applicable, a summary of the follow-up time. A diagram may be helpful	Chapter 3, Results
	13b	D; V	Describe the characteristics of the participants (basic demographics, clinical features, available predictors), including the number of participants with missing data for predictors and outcome	Chapter 3, Results
	13c	V	For validation, show a comparison with the development data of the distribution of important variables (demographics, predictors and outcome)	Chapter 3, Results
Model development	14a	D	Specify the number of participants and outcome events in each analysis	Chapter 3, Results
	14b	D	If done, report the unadjusted association between each candidate predictor and outcome	NA
Model specification	15a	D	Present the full prediction model to allow predictions for individuals (i.e. all regression coefficients, and model intercept or baseline survival at a given time point)	Chapter 3, Results
	15b	D	Explain how to the use the prediction model	Chapter 3, Results and Appendix 3
Model performance	16	D; V	Report performance measures (with CIs) for the prediction model	Chapter 3, Results
Model updating	17	V	If done, report the results from any model updating (i.e. model specification, model performance)	NA
Discussion
Limitations	18	D; V	Discuss any limitations of the study (such as non-representative sample, few events per predictor, missing data)	Chapter 3, Discussion
Interpretation	19a	V	For validation, discuss the results with reference to performance in the development data, and any other validation data	Chapter 3, Discussion
	19b	D; V	Give an overall interpretation of the results, considering objectives, limitations, results from similar studies and other relevant evidence	Chapter 3, Discussion
Implications	20	D; V	Discuss the potential clinical use of the model and implications for future research	Chapter 3, Discussion
Other information
Supplementary information	21	D; V	Provide information about the availability of supplementary resources, such as study protocol, web calculator and data sets	Chapter 8
Funding	22	D; V	Give the source of funding and the role of the funders for the present study	Chapter 8