A systematic review and individual patient data meta-analysis of prognostic factors for foot ulceration in people with diabetes: the international research collaboration for the prediction of diabetic foot ulcerations (PODUS)

Notes

Permissions

Chapter 1 Background

The ageing population, widespread obesity and improved survival all mean that the prevalence of diabetes will more than double between 2000 and 2030. 1 Consequently, the serious complications of the disease are also anticipated to escalate and thus place an increasing demand on health-care resources. These complications are observed in the lower limb as peripheral vascular disease, foot ulceration, osteomyelitis (infection), gangrene and lower-extremity amputations (LEAs), and all are more likely to be experienced by those with diabetes than by the general population. 2

Published studies have reported variation in the incidence of diabetes-related foot ulceration between < 2% in UK primary care and community settings and 18% in hospital-based populations globally. 3–5 Routinely collected data from Scotland indicate that 13,789 (5.2%) people with type 1 or type 2 diabetes experienced a foot ulcer in 2013. 6 These foot ulcers give rise to considerable morbidity and generate a high monetary cost for health- and social-care systems2,7 and, importantly, 80% of diabetes-related foot amputations are preceded by a foot ulcer. 8 For those who experience diabetes-related amputations, the 5-year survival is poor, with mortality estimates of between 25% and 50% in 1985 in UK populations. 9,10

Changes in diabetes-related LEAs have been reported in parts of the UK. In common with some European countries and the USA, major LEA rates in Scotland have been reported to fall. A statistically significant reduction of 40% in LEA rates occurred between 2004 and 2008. 11 However, the 2013 Scottish Diabetes Survey shows that the absolute numbers and percentages of diabetes-related foot ulcerations and LEA have increased, although this is attributed to better recording procedures. In England, an analysis of national hospital activity data from 1996 to 2005 found that, although LEAs in people with type 1 diabetes fell, type 2 LEAs increased. 12 High levels of variation in diabetes-related LEAs are known to exist between primary care trusts (PCTs) across England, which may be explained by variation in the delivery of care. 13

The optimal clinical management of people with diabetes includes annual foot risk assessment, as recommended in national and international clinical guidelines and the Quality and Outcomes Framework (QOF) of the General Medical Contract (GMC) in the UK. 14–17 Risk classifications of three or four levels (low, moderate, high and active) are increasingly being recommended. At present, the evidence underpinning these classifications is not from randomised trials and does not include the totality of evidence (i.e. data from all cohort studies), and the effect of such surveillance and the use of interventions thought to prevent the development of a foot ulcer in the at-risk population lack clear evidence of clinical effectiveness or cost-effectiveness. 18

Clinically effective and cost-effective health care requires the careful measurement of health outcomes, and the need for an evidence-based approach to foot care services in diabetes has been documented. 19,20 Two systematic reviews highlight the gaps in the knowledge about the best way to identify those at risk.

The first systematic review evaluated the independent contribution of predictive factors for foot ulceration based on meta-analyses of aggregate data. It found that the duration of diabetes, glycated haemoglobin (HbA_1c), peak plantar pressure (PPP) and vibration perception threshold (VPT) distinguish between those people who will develop a foot ulcer and those who will not. However, there was significant heterogeneity between studies, possibly owing to differences in lengths of follow-up, methods of ascertaining the presence of ulcers and the use of different cut-off points (thresholds) for some of the tests. Furthermore, some tests that are commonly believed to be predictive of risk, such as the absence of a pedal pulse, were not found to be so and data for other common tests such as monofilaments were not amenable to meta-analysis.

A second systematic review of clinical prediction rules (CPRs) used for risk assessment of developing diabetic foot ulceration (DFU) identified five different risk stratification tools derived from consensus among clinical experts, literature reviews and prospective studies using logistic regression methods. 21 The predictive factors in these five CPRs were foot deformity; peripheral neuropathy; peripheral vascular disease [absent pulses and/or positive ankle–brachial index (ABI) test] and previous amputation; the presence of callus; HbA_1c; tinea pedis; and onychomychosis. The review authors concluded that it was unclear which CPR possessed the greatest accuracy in the assessment of risk.

These two systematic reviews found marked variation between the incidences of foot ulcers across different study populations. The predictive factors and CPRs derived from high-risk populations may not be of value in predicting risk in the general ‘low-risk’ diabetes population. It is a matter of some concern that the accuracy of recommended foot risk assessment procedures has not been fully explored in different groups of people with diabetes and that little validation of derivative cohort studies has taken place. 22,23

These two systematic reviews of aggregate data represent the best attempts to integrate evidence about the independent contribution of risk factors and CPRs in the assessment of the foot in diabetes to date. These findings are compromised, however, because the authors of primary included studies approached their analyses in different ways: some present adjusted estimates, whereas others are unadjusted, and it is sometimes unclear which confounders or effect modifiers were used. Conventional meta-analytic techniques use aggregate data that are averaged across all individuals in a study and these do not permit adjustments for confounding to be performed. The best way to reliably analyse data from several cohort studies using a standard approach is to use individual patient data (IPD). 24,25

The success of IPD systematic reviews depends on a high level of collaboration, trust and commitment between multidisciplinary researchers and the authors of the primary studies. 25 The ownership of data from primary studies by the pharmaceutical industry can represent an obstacle to IPD analysis being accomplished. However, our background work found that none of the cohort studies included in the systematic reviews had industry sponsorship. The authors who possess the data from the cohort studies identified in the published systematic reviews agreed to take part in an IPD systematic review and to contribute anonymised data from their primary studies for reanalysis.

The purpose of this systematic review and meta-analysis of IPD is to clarify the best risk assessment procedures for foot ulcers in people with diabetes. The international nature of these data, which are from more than 16,000 patients worldwide, should ensure a balanced interpretation. Given the increased worldwide prevalence in diabetes, the identification of the most predictive risk factors could lead to reduced costs for health-care providers.

Chapter 2 Hypotheses

Our research focused on the questions outlined below.

Chapter 3 Methods

Systematic reviews and meta-analyses of IPD use ‘raw’ data obtained from the authors of individual studies instead of mean or aggregate data extracted from published reports. These complex reviews are more time-consuming and expensive than aggregate systematic reviews because obtaining study data and data dictionaries and undertaking data checking and cleaning takes more time than the extraction of data from a published report (Figure 1). 25

FIGURE 1.

Flow diagram: stages of an individual patient-based meta-analysis. Reproduced from the original25 with kind permission from John Wiley & Sons.

Individual patient data systematic reviews are useful for both randomised controlled trial (RCT) and observational data and enhance the main purpose of meta-analysis – the augmentation of statistical power – by permitting the conduct of complex statistical techniques, including multivariable analyses in which interactions between interventions and patient-level characteristics can be explored. 26 In the case of observational study designs, IPD is the best way to pool observational study data to allow adjustments and a standard statistical approach to be conducted.

This review method also confers an advantage on the process of quality assessment because the necessary communication between the review team and those contributing the data means that potential biases arising from the conduct, rather than the report, of the study can be investigated. However, although the opportunity to discuss the manner in which the study was conducted with the author means the reviewer is not required to interpret possible biases, IPD reviews do not avoid flaws in the original studies arising from conduct or design. 27

Chapter 4 Development of the model

The criteria for consideration for inclusion in the primary meta-analysis were:

1. variables had to have been collected in at least three data sets

2. variables had to have been consistently defined across data sets (or could be recoded so)

3. the extent of heterogeneity should not invalidate the results of meta-analysis.

The majority of the variables collected were not suitable for the primary meta-analysis, because often they had been collected in only one or two studies. Also, in some cases, there was some variation in the definition of the variable across all the data sets, and it was a matter of judgement to decide if the degree of inconsistency was acceptable or not. However, variables that met the first two criteria were used in univariate logistic regression to obtain ORs. The ORs were plotted in forest plots so that heterogeneity could be assessed.

As the assessment of heterogeneity includes both clinical and statistical aspects, it cannot be designed solely on methodological grounds. Therefore, to ensure that all important clinical considerations were fully addressed, all the variables considered to be potential candidates were presented to the clinical and methodological co-authors at our international meeting.

All authors of the original studies presented details of the study design and conduct, with particular emphasis on the characteristics of the patient cohort and the particular risk factors studied. Univariate analyses of common variables were presented on forest plots to display the degree of heterogeneity between studies. Because the co-authors were also involved in the original studies producing the data sets, there was ample opportunity to discuss reasons for heterogeneity, encompassing inconsistency in the definitions, and what variables were felt to be clinically important.

The variables that met the first two criteria (collected in at least three data sets and consistently defined) were:

• age

• sex

• body mass index (BMI)

• smoking

• height

• weight

• alcohol

• HbA_1c

• insulin regime

• duration of diabetes

• eye problems

• kidney problems

• monofilament

• pulses

• tuning fork

• biothesiometer

• ankle reflexes

• ABI

• PPP

• prior ulcer

• prior amputation

• foot deformity.

We chose not to present height and weight. BMI, height and weight are all obviously highly correlated. Using variables that are highly correlated in a statistical model can lead to collinearity problems, which are unstable estimates, as well as incorrect CIs and p-values. To avoid using a model with high collinearity, we decided to use BMI rather than height or weight. BMI is very commonly used as a measure of body size, and there were six studies with BMI data but only four each for height and weight.

The following variables, which did not meet the criteria, were also presented to demonstrate explicitly why we were unable to include them in any meta-analyses, despite their potential as predictors of foot ulceration: ethnicity, living alone, pinprick test, temperature test (i.e. possibly important demographic and foot sensation variables).

After a discussion of each variable in turn, the members of the group were asked to select a few variables to be examined for inclusion in the primary analysis to ensure that the final model was simple and easy to implement in a clinical environment; widely used CPRs tend not to have many predictors. In addition, for a study to be included, it must have collected data on all the relevant variables, which means that there is an inevitable trade-off between the number of variables and the number of studies to be included. For example, a model with just age and sex could use data from all the studies, but a model with age, sex and ABI could include only four studies. Therefore, limiting the number of predictors also maximises the number of studies that can be included.

The variables chosen at the international meeting for possible inclusion in the primary analysis were age, sex, duration of diabetes, prior ulceration or amputation and monofilament results. In addition, insulin use and kidney problems were to be added to the primary model to assess their impact on prediction of ulcer. E-mail was used to continue the discussion of the predictors after the meeting, resulting in some significant changes.

Patients with a known history of ulcer or amputation are already known to be at high risk of a further ulcer or amputation. These patients therefore have a different risk profile from patients with no history of ulceration or amputation, who may generally be at low risk. From a clinical view, it was regarded as important to be able to identify those patients without history who are nonetheless at high risk of ulceration to allow targeted treatment. Therefore, it was decided on clinical grounds to construct two models, one for all patients and the other for those patients with no history of ulceration or amputation, and, consequently, to drop history as a predictor from the model for patients with no previous ulceration or amputation.

Another predictor dropped from the primary analysis was insulin use. Discussion at the international meeting covered the difficulties of interpreting the use of insulin as a predictor. A patient may simply be using insulin because he or she has type 1 diabetes. However, it is also possible that a patient is using insulin because he or she has poorly controlled type 2 diabetes. A further complication is that some patients with type 2 diabetes achieve good control with insulin. Moreover, the definition of insulin use varied from insulin use at any time prior to the study to current insulin use at the time of recruitment to the study. This was given as a possible explanation for inconsistency and a high degree of heterogeneity among the ORs for insulin use. It was therefore decided to drop insulin use from the primary analysis and retain it for secondary analyses only.

A similar line of reasoning was followed for the predictor kidney problems. These had been defined as outright nephropathy in some data sets and were derived from estimated glomerular filtration rate (eGFR) in others, which may or may not always be an adequate proxy. 60 In multivariate models, the effect of kidney problems was not consistent, being apparently protective against ulceration in some cases5,61 but predictive in others, for example. 3,46,47,62 Again, it was decided to drop kidney problems from the primary model but retain it for secondary analyses.

Dropping three predictors from the primary model meant that other potential predictors could be considered. We added pulses to the primary model. The argument in favour of use of pulses is that it is a test of the vascular integrity of the foot. The model already included monofilaments, a neurological test. Therefore, by including both a vascular and a neurological test of the foot, the model would encompass the major mechanisms by which foot ulceration occurs in diabetes.

It was decided to include HbA_1c in a secondary analysis in order to include a predictor in the model that could be influenced by patients themselves. The only variables in the list of potential predictors that could be influenced by patients were BMI, smoking, alcohol and HbA_1c. However, discussion of BMI at the international meeting suggested that it would not be a good predictor, because a BMI in the low range could be indicative of two opposing states, either the patient does not gain weight through appropriate diet and exercise or the patient is unable to maintain weight owing to advanced diabetic illness. The univariate forest plot for BMI showed that a low BMI was protective of ulceration in some studies but predictive of ulceration in others. Furthermore, the data on the effect of smoking and alcohol were also not clear, with both smoking and drinking being protective against ulceration in some data sets and predictive of ulceration in others. These results were discussed, with some speculation on the vasodilation properties of nicotine and the possible benefits of moderate alcohol intake. Therefore, HbA_1c was chosen for use in a secondary analysis. It is also the only patient-modifiable predictor directly related to diabetes. Two further tests, namely VPT by either tuning fork or biothesiometer and the ABI, were also retained for secondary analyses, as these were of particular interest to the clinical co-authors.

In summary, the primary model has the following predictors: age, sex, duration of diabetes, monofilament and pulses. The secondary models are:

• age, sex and duration of diabetes

• age, sex, duration of diabetes and monofilament

• age, sex, duration of diabetes, monofilament and insulin

• age, sex, duration of diabetes, monofilament and kidney problems

• age, sex, duration of diabetes, monofilament, pulses and VPT

• age, sex, duration of diabetes, monofilament, pulses and HbA_1c

• age, sex, duration of diabetes, monofilament and ABI.

The primary model meets our aim of a parsimonious model of easily obtainable predictors. The purpose of the secondary models is to allow comparison with the primary model, for example to see what the value of HbA_1c is as a predictor in addition to the predictors of the primary model. Prediction models are often assessed in terms of their discrimination (how well they distinguish between groups of patients) and calibration (how well the model’s estimated probability matches the actual probability of outcome for each patient).

Given two patients, one with an ulcer at follow-up and the other with no ulcer at follow-up, the area under the curve (AUC) is the probability that the model calculates a higher risk for the ulcer patient; thus, the AUC can be used to assess the discrimination of the model. The AUC takes a value between 0.5 (no discrimination) and 1 (perfect discrimination). Values between 0 and 0.5 are theoretically possible but would only occur if a model was worse than using a coin toss to predict outcome. The Brier score is an indication of how well the model is calibrated and takes a value between 0 and 1. A perfect model would have a Brier score of 0, and, as a rule of thumb, a prediction model should have a Brier score of 0.25 or under. 63

Chapter 5 Validation of the model

Validation is an essential part of assessing prediction models. For most studies that have access to only one data set, the emphasis is on internal validation. Internal validation is an assessment of model performance based on the data set used for development. Prediction models tend to perform best in the data set in which they were developed, and, therefore, one of the purposes of internal validation is to try and assess to what degree the estimates reflect true relationships between variables rather than the idiosyncrasies of the development data set. Teasing apart true and spurious relationships can be a problem when there are too many predictors and/or predictors are selected using a data-driven method such as stepwise regression.

The aim of our methodology was to avoid this problem by choosing a priori few predictors based on the criteria described above; see Chapter 3, Choice of predictors. These criteria do not use any information based on p-values or CIs and so avoid the problems of data-driven methods.

Internal validation methods generally consider the concepts of model discrimination and model calibration. In this context, discrimination is a reflection of how well the model differentiates between patients who do and do not ulcerate. Calibration is a reflection of how well the model assigns the relative proportions of risk categories; a poorly calibrated model would place many patients in the wrong risk category. A statistic that encompasses both these concepts is the two-component Brier score, which takes values between 0 and 1, with a perfectly calibrated model having a score of 0 and model with no calibration having a score of 1.

Given that we had several data sets, we also addressed external validation. External validation is the assessment of model performance in data sets other than the development data set and was arguably more important than internal validation because it related directly to the generalisability of our results. Performing external validation required two decisions to be made: how should it be done and which data should be used? There are six different methods of external validation for logistic regression models described by Steyerberg et al. 64 We chose one method for ease of implementation and interpretation, which was simply to re-estimate the ORs of our final model in a new data set not previously used in any analysis. We then compared the ORs from the meta-analyses with those from the validation data set. The validation data set had to fulfil one mandatory criterion, that is, it had to have all the variables used in the primary model, and more than one data set fulfilled this criterion. However, one data set in particular seemed to be the natural choice as the validation data set. The reasons for this were:

• The validation data set was only available for analysis at a late stage. By using this data set for validation rather than model development, work on the meta-analyses could proceed in a timely manner.

• The validation data set was not accessible to those performing the meta-analyses. Analyses could be requested and aggregate results supplied from the validation data set. This ensured that the persons conducting the meta-analyses were not influenced by any validation results, which were requested only after the meta-analyses had been completed.

• The persons conducting the validation analyses were not informed of the results of any meta-analyses until after the validation analyses had been completed and supplied. This ensured that they were not influenced by any meta-analysis results.

Therefore, our methods allowed the validation process to be independent of the model development process and vice versa. It is also worth noting that the use of this particular data set, which was analysed by statisticians not otherwise involved in the project, meant that the method of external validation had to be simple, as the time available to conduct the validation by these statisticians was necessarily limited. To ensure that the time required was minimal, Statistical Analysis System [(SAS); SAS Institute Inc., Cary, NC, USA; see www.sas.com] programs (version 9.3) were supplied to produce all the required analyses.

Chapter 6 Results of the systematic review

The flow diagram below (Figure 2) depicts the flow of literature during the review process.

FIGURE 2.

Flow diagram of studies in the IPD systematic review, showing the studies included in the review and meta-analysis. 65 DTA, diagnostic test accuracy.

Chapter 7 Characteristics of included studies

There were 16 eligible studies identified from our search activities. We were unable to obtain IPD from six of these because either we could not make contact with the authors66–68 or the authors were no longer in possession of the data69–71 (Aristidis Veves, Harvard Medical School; Lawrence Lavery, UT Southwestern Medical Center, Texas, David Armstrong, Southern Arizona Limb Salvage Alliance, University of Arizona, personal communication) (see Figure 2).

The corresponding authors of eight studies made raw data available to the Data Management Committee. 3,5,61,62,72–75 Data from a ninth study46,47 were made available to the Data Management Committee via Safe Haven, a data-management system. Finally, a 10th corresponding author was not granted permission to share the data from a cohort study by the Institutional Review Board48 but was able to contribute to the meta-analysis by subjecting the data to the same analytical procedures as all other studies to provide estimates of effect, which could be incorporated into the final (meta) analysis. Of these 10 studies, nine were derivation studies, and all are summarised in Table 1. Below, we briefly describe each.

Chapter 8 Risk of bias

The tabulated results of the quality assessment process can be found in Appendix 5. Of the four items used to assess the quality of the conduct of the studies, three indicated a low risk of bias. Patients were recruited consecutively in all but one study. 74 Follow-ups were conducted at least 1 month after the data collection of risk factors, allowing enough time for an ulcer to develop, and all reports provided enough detail for the tests to be replicated.

The collection of outcomes in a ‘blind’ manner to protect the data from investigator bias was a feature of only 50% of the studies included in our review. 3,5,61,73,75

All study reports provided sufficient details about the conduct of the tests to permit their replication.

Chapter 9 Data cleaning and pattern of missingness

Chapter 10 Patients with diabetes: description

Demographics, anthropometrics and lifestyle

Appendix 6 shows the demographic, anthropometric and lifestyle profile of the diabetic population per study. The overall average age was 63 years, ranging from 54 and 55 years in the two earliest studies,74,75 to 71 years in the study by Crawford et al. 5 Figure 3 shows the similar distribution of age per study, with higher modes for the more recent studies by Monami72 and Crawford. 5 The overall percentage of men was 58% and varied from 44% to 57% for most studies, but was 98% in the study by Boyko et al. 49

FIGURE 3.

Distribution of age per study.

Average weight and height were recorded for only four studies, with BMI recorded more commonly. The overall mean weight and height were 89 kg and 171 cm, respectively. The distributions of weight and height were very similar across studies, although the patients in the study by Young et al. 75 were slightly lighter. Mean BMI ranged from 27 kg/m² to 31 kg/m², with an overall average of 30 kg/m² just at the threshold for obesity. Figure 4 shows the similar distribution of BMI per study. Most patients had a BMI between 20 kg/m² and 40 kg/m², with few cases of extreme and malignant obesity.

FIGURE 4.

Distribution of BMI per study.

The studies by Abbott et al. 3 and Crawford et al. 5 observed the proportion of diabetic patients living alone to be 22% and 29%, respectively. Smoking and alcohol consumption were heterogeneous across studies; 19–81% of patients had ever smoked and 17–55% were consuming alcohol. For most studies, around 50% or more of the patients had a history of smoking, and the trends were similar for alcohol consumption.

Diabetes and comorbidities

Appendix 7 shows the diabetic, eye and renal profile of the diabetic population per study. The majority of patients had type 2 diabetes. Three studies focused on patients with type 2 diabetes only; in the remaining studies the proportion of patients with type 2 diabetes was between 61% and 98% . Overall, type 1 diabetes accounted for about 9% of recorded types of diabetes. Insulin treatment accounted mostly for 20–40% of each diabetic population. Overall, the average diabetes duration was 9 years and was disparate across studies, average duration ranging from 7 to 16 years. Figure 5 shows that the distribution of diabetes duration per study was similar overall. HbA_1c was, on average, 8% in each study apart from the studies by Young et al. 75 (11%), Kästenbauer et al. 62 (10%) and Boyko et al. 49 (10%).

FIGURE 5.

Distribution of diabetes duration per study.

Four studies recorded visual impairment and/or blindness with heterogeneous results. Retinopathy was collected for four studies and recorded diagnoses ranged from 9% to 49% of the population. Renal problems were collected for most studies but in various ways. Nephropathy accounted for 2% and 17% of the population in two studies, stage 3–5 CKD accounted for 13–37% of the population in two studies, and end-stage renal failure for 2% and 4% of the population in another two studies.

Chapter 11 Common variables

The data dictionary relating to these tables can be found in Appendix 9. The variables common to the included studies can be found in Table 4.

All data were analysed with SAS 9.3 (www.sas.com) and R.2.13.1 (cran.r-project.org/). Logistic regression analyses were carried out using SAS PROC LOGISTIC; meta-analyses were performed using an edited version of the metagen function in the R meta package; and multiple imputation was carried out with the R MICE package.

Chapter 12 Univariate meta-analysis of the data sets: suitability of studies for meta-analysis

The purpose of the univariate meta-analyses was to explore potential differences between the data sets, assess heterogeneity and facilitate discussion at the international meeting of clinical and methodological co-authors. For each candidate predictor, a logistic regression analysis with ulceration as outcome was undertaken in each of the studies (see Table 4). The resulting ORs were then used in a generic inverse variance meta-analysis. ORs were displayed on forest plots and discussed at the meeting.

Forest plots with ORs for the following 24 variables were presented (see Appendix 10): age, sex, BMI, smoking, alcohol, HbA_1c, insulin regimen, duration of diabetes, eye problems, kidney problems, monofilament, pulses, tuning fork, biothesiometer, ankle reflexes, ABI, PPP, prior ulcer, prior amputation, foot deformity, ethnicity, living alone, pinprick test and temperature test.

The following variables were rejected for being included in fewer than three studies: ethnicity, living alone, pinprick test and temperature test. This left 20 candidate predictors. The following variables – eye problems, PPP and foot deformity – were rejected for being inconsistently defined across data sets, leaving 17 candidate predictors. Owing to high similarity the following variables were combined: tuning fork and biothesiometer, as both measure vibration perception; and prior ulcer and amputation, as both indicate a prior tendency to ulcerate. This left 15 candidate predictors.

We considered that the foot sensation tests would often be measuring the same underlying neurological impairment. From a statistical viewpoint, there are challenges to including highly correlated variables in one statistical model because it becomes difficult to assess the relationships between predictor variables and outcome. It was therefore deemed preferable to include fewer rather than more tests of foot sensation. It was also preferable to use tests that had been collected by more studies. Monofilaments, pulses and VPT (by either tuning fork or biothesiometer) had been collected in six studies, ABIs in four studies and ankle reflexes in three studies. Therefore, monofilaments, pulses and VPT were used in preference to the other tests, leaving 13 candidate predictors.

Some variables appeared to have complex relationships with ulceration outcome, namely BMI, smoking and alcohol. As discussed above (see Chapter 3, Choice of predictors), a high BMI is generally associated with worse health outcomes, but, for some diabetic patients, a low BMI can be an indication of weight loss as a result of a diabetes-related problem such as kidney disease. In addition, smoking and alcohol seemed to be protective against ulceration in some studies and predictive of ulceration in others. This may be another example of the so-called smoker’s paradox. 77 At our international collaborators’ meeting, there was some speculation about the biological effects of nicotine that could possibly help protect against diabetic foot disease, and it is also possible to speculate that both smoking and drinking alcohol could be associated with another variable that is genuinely protective against ulceration, for example younger age. However, given that the aim of these analyses is to produce a simple, parsimonious model that may be readily used in clinical contexts for screening, interaction terms that could be used to explore the relationships between BMI, smoking and drinking with ulceration were not utilised, although this could, of course, be an item for further research.

This left 10 variables: age, sex, duration of diabetes, monofilaments, pulses, insulin regime, kidney problems, VPT, HbA_1c and previous history of either amputation or ulceration. The univariate forest plots for these showed some degree of heterogeneity, as expected from clinical knowledge of the individual studies. Nonetheless, most of the forest plots have overlapping CIs for most of the studies, although there are some exceptions. We examined the forest plots together with the I²- and τ-statistics and concluded that the extent of heterogeneity, although present, did not preclude meta-analyses of multivariable estimates. We noted that further assessment of heterogeneity would be done for the multivariable meta-analyses, and where necessary, results would be interpreted cautiously.

Chapter 13 Multivariable meta-analysis: the final model

Primary meta-analysis

The primary meta-analysis included age, sex, duration of diabetes, monofilaments and pulses as predictors. The analysis was repeated twice, once for patients with no previous history of foot ulceration or LEA (Table 5) and again for all patients, including those with a previous history (Table 6).

TABLE 5 - Results of the primary meta-analyses for patients without history of ulceration or amputation

Predictor	OR	95% CI	I 2	τ
Age	1.008	0.995 to 1.021	29.8%	0
Duration of diabetes	1.029	1.017 to 1.04	4.9%	0
Monofilament	3.438	2.772 to 4.264	0%	0
Pulses	2.605	1.808 to 3.754	42.7%	0.054
Sex (female)	0.841	0.682 to 1.037	0%	0

TABLE 6 - Results of the primary meta-analyses for all patients

Predictor	OR	95% CI	I 2	τ
Age	1.005	0.994 to 1.016	37.4%	0
Duration of diabetes	1.024	1.011 to 1.036	38.1%	0
Monofilament	3.184	2.654 to 3.82	0%	0
Pulses	1.968	1.624 to 2.386	1.6%	0.001
Sex (female)	0.743	0.598 to 0.922	20.7%	0.013
Previous history	6.589	2.488 to 17.45	94.2%	1.134

In meta-analysis, it is desirable for valid estimates to exhibit low heterogeneity. For example, the OR for previous history in Table 6 below may not be generalisable, as the high estimates of heterogeneity (I² = 94.2%, τ = 1.134) and the forest plots (see Figure 17) suggest that the OR varies from study to study to an extent that rules out a valid meta-analysis.

The flow diagram below (Figure 6) shows the number of patients involved throughout the review process.

FIGURE 6.

Flow diagram of patients in the IPD meta-analysis.

The independent contribution of tests, symptoms and signs to the prediction of foot ulceration risk assessment procedures in people with no history of ulceration or lower-extremity amputation

The following graphs show pooled estimates for the prognostic utility of age (Figure 7), an increase of 1 year’s duration of diabetes (Figure 8), the inability to feel a 10-g monofilament (Figure 9), one absent pedal pulse (Figure 10) and sex (Figure 11).

FIGURE 7.

Pooled estimates for age in people with no history of ulceration or amputation (model adjusted for sex, duration of diabetes, inability to feel a 10-g monofilament and absent pedal pulses). The OR of 1.008 (95% CI 0.995 to 1.021) indicates that age is not predictive of diabetes-related foot ulceration. The observed heterogeneity was I² = 29.8%. External validation using Boyko et al. 200649 data: OR 0.984 (95% CI 0.965 to 1.003).

FIGURE 8.

Pooled estimates for an increase of 1 year’s duration of diabetes in people with no history of ulceration or amputation (model adjusted for age, sex, inability to feel a 10-g monofilament and absent pedal pulses). The OR of 1.029 (95% CI 1.017 to 1.04) indicates an increase of 1 year’s duration of diabetes is predictive of diabetes-related foot ulceration. The observed heterogeneity was I² = 4.9%. External validation using Boyko et al. 200649 data: OR 0.970 (95% CI 0.954 to 0.987).

FIGURE 9.

Pooled estimates for the inability to feel a 10-g monofilament in people with no history of ulceration or amputation (model adjusted for age, sex, duration of diabetes and absent pedal pulse). The OR of 3.438 (95% CI 2.772 to 4.264) indicates that the inability to feel a 10-g monofilament is predictive of diabetes-related foot ulceration. The observed heterogeneity was I² = 0%. External validation using Boyko et al. 200649 data: OR 3.913 (95% CI 2.581 to 5.933).

FIGURE 10.

Pooled estimates of one absent pedal pulse in people with no history of ulceration or amputation (model adjusted for age, sex, duration of diabetes and inability to feel a 10-g monofilament). The OR of 2.605 (95% CI 1.808 to 3.754) indicates that the absence of at least one pedal pulse is predictive of diabetes-related foot ulceration. The observed heterogeneity was I² = 42.7%. External validation using Boyko et al. 200649 data: OR 1.416 (95% CI 0.466 to 4.301).

FIGURE 11.

Pooled estimates of sex in people with no history of ulceration or amputation (model adjusted for age, duration of diabetes, inability to feel a 10-g monofilament and absent pedal pulses). The OR of 0.841 (95% CI 0.682 to 1.037) does not indicate that sex is predictive of diabetes-related foot ulceration. The observed heterogeneity was I² = 0.% External validation using Boyko et al. 200649 data: OR 1.303 (95% CI 0.282 to 6.022).

The independent contribution of tests, symptoms and signs in the total individual patient data population

The following graphs show pooled estimates for the prognostic utility of age (Figure 12), an increase of a 1 year duration of diabetes (Figure 13), the inability to feel a 10-g monofilament (Figure 14), one absent pedal pulse (Figure 15), sex (Figure 16) and previous history of foot ulceration or LEA (Figure 17).

FIGURE 12.

Pooled estimates for age in the total IPD population (model adjusted for sex, duration of diabetes, inability to feel a 10-g monofilament, absent pedal pulses and previous history of foot ulceration or amputation). The OR of 1.005 (95% CI 0.994 to 1.016) indicates that age is not predictive of diabetes-related foot ulceration. The observed heterogeneity was I² = 37.4%. External validation using Boyko et al. 200649 data: OR 0.993 (95% CI 0.977 to 1.009).

FIGURE 13.

Pooled estimates for an increase of 1 year’s duration of diabetes in the total IPD population (model adjusted for age, sex, inability to feel a 10-g monofilament, absent pedal pulses and previous history of foot ulceration or amputation). The OR of 1.024 (95% CI 1.011 to 1.036) indicates an increased duration of diabetes is predictive of diabetes-related foot ulceration. The observed heterogeneity was I² = 38.1%. External validation using Boyko et al. 200649 data: OR 0.981 (95% CI 0.968 to 0.994).

FIGURE 14.

Pooled estimates for the inability to feel a 10-g monofilament in the total IPD population (model adjusted for age, sex, duration of diabetes, absent pedal pulses and previous history of foot ulceration or amputation). The OR of 3.184 (95% CI 2.654 to 3.82) indicates that an inability to feel a 10-g monofilament is predictive of diabetes-related foot ulceration. The observed heterogeneity was I² = 0%. External validation using Boyko et al. 200649 data: OR 3.489 (95% CI 2.486 to 4.896).

FIGURE 15.

Forest plot showing pooled estimates for one absent pedal pulses in the total IPD population (model adjusted for age, sex, duration of diabetes, inability to feel a 10-g monofilament and previous history of foot ulceration or amputation). The OR of 1.968 (95% CI 1.624 to 2.386) indicates that the absence of a pedal pulse is predictive of diabetes-related foot ulceration. The observed heterogeneity was I² = 1.6%. External validation using Boyko et al. 200649 data: OR 2.557 (95% CI 1.220 to 5.361).

FIGURE 16.

Pooled estimates for sex in the total IPD population (model adjusted for age, duration of diabetes, inability to feel a 10-g monofilament, absent pedal pulses and previous history of foot ulceration or amputation). The OR of 0.743 (95% CI 0.598 to 0.922) indicates male sex to be predictive of diabetes-related foot ulceration. The observed heterogeneity was I² = 20.7%. External validation using Boyko et al. 200649 data: OR 1.491 (95% CI 0.418 to 5.317).

FIGURE 17.

Pooled estimates for previous history of LEA in the total IPD population. (Model adjusted for age, sex, duration of diabetes, inability to feel a 10-g monofilament and absent pedal pulses.) The OR of 6.589 (95% CI 2.488 to 17.454) indicates that a previous history of foot ulceration or LEA is predictive of diabetes-related foot ulceration. The observed heterogeneity was I² = 94.2%. External validation using Boyko et al. 200649 data: OR 2.979 (95% CI 2.146 to 4.135).

The absence of heterogeneity and consistency of the estimates for the inability to feel a 10-g monofilament was noted across a number of models and between the two patient groups (see Appendix 11). Out of the 14 meta-analyses that included monofilament as a predictor, only two did not estimate the heterogeneity to be zero, and these two studies had low heterogeneity estimates (Table 7). Some of these ORs are based on more data than others – depending on the variables in the model and those available in the individual studies. The OR for insensitivity to a 10-g monofilament is around 3.5, despite the test being conducted in a number of different ways by the individual study investigators, different anatomical sites on the foot being used and the number of sites varying. We had also expected to observe heterogeneity owing to other methodological and patient cohort differences.

TABLE 7 - Monofilament results for all models

Other model predictors	Patient group	OR	95% CI	I 2	τ
Age, duration, sex	No history	3.823	3.106 to 4.705	0%	0
Age, duration, sex, previous history	All	3.444	2.891 to 4.103	0%	0
Age, duration, pulses, sex	No history	3.438	2.772 to 4.264	0%	0
Age, duration, pulses, previous history	All	3.184	2.654 to 3.82	0%	0
Age, duration, insulin, sex	No history	3.763	2.837 to 4.991	0%	0
Age, duration, insulin, sex, previous history	All	3.189	2.524 to 4.028	0%	0
Age, duration, kidney problems, sex	No history	4.008	3.17 to 5.069	0%	0
Age, duration, kidney problems, sex, previous history	All	3.435	2.821 to 4.183	0%	0
Age, duration, pulses, sex, VPT	No history	2.501	1.844 to 3.393	0%	0
Age, duration, pulses, sex, VPT, previous history	All	2.003	1.333 to 3.011	27.9%	0.055
Age, duration, pulses, sex, HbA1c	No history	3.350	2.488 to 4.512	0%	0
Age, duration, pulses, sex, HbA1c, previous history	All	2.770	1.938 to 3.960	28.9%	0.033
Age, duration, pulses, sex, ABI	No history	2.635	0.824 to 8.426	0%	0
Age, duration, pulses, sex, ABI, previous history	All	2.657	1.127 to 6.261	0%	0

This consistency of results across individual studies and meta-analyses for the 10-g monofilament test is not observed for any other predictive variable, and this makes it harder to reach conclusions about their true value in risk assessment. All predictive factors were subject to some heterogeneity, which affects the generalisability of their estimates. All forest plots, with ORs for the individual studies and meta-analyses, together with I² and τ estimates of heterogeneity, can be found in Appendix 11.

We calculated an AUC and Brier score for the studies in model 4 for the total population and for patients with no previous history of foot ulceration or LEAs. The tables in Appendix 12 show AUC values of between 0.71832 and 0.8654 for the total population, and between 0.70436 and 0.77636 in the total population minus those without a history of a foot ulcer or LEA, thus showing that the model possesses good discrimination.

The low Brier scores indicate that the model is also well calibrated (total population = 0.03704 to 0.18342; total population minus those without a history of a foot ulcer or LEA = 0.01214 to 0.14659).

Imputation analysis: final model

The missing data patterns of the common harmonised variables have been assessed to be MAR. We could therefore apply the MICE method. In this section, we focus on imputing the set of variables selected in the final model and these are: age, sex, duration of diabetes, monofilament, pulses and previous history of ulcer or amputation. The data set to be imputed also includes the following outcome: ulcer. The set of variables for the final model was collected in five studies. 3,5,47,61,73

An initial step prior to any imputation for each data set is to look at the missing data patterns of the specified set of variables. Table 8 provides such patterns by study. Group 1 represents the ‘complete case’ where a patient has no missing data in any of the variables specified. Groups 2 to 9 represent patients with one or two missing variables. Apart from the study by Monteiro-Soares and Dinis-Ribeiro,61 which had no case of missing data for the specified set of variables, all studies had between 1% and 3% of overall missing data.

TABLE 8 - Missing data patterns for final model variables by study

								Study
								Abbott et al., 20023		Crawford et al., 20115		Monteiro-Soares and Dinis-Ribeiro, 201061		Pham et al., 200073		Leese et al., 201147
Group	Ulcer	Age	Sex	Diabetes duration	Monofilament	Pulses	Previous history	n	%	n	%	n	%	n	%	n	%
1	✗	✗	✗	✗	✗	✗	✗	6415	97.15	1178	98.74	360	100	243	97.98	3326	97.48
2	✗	✗	✗	✗		✗	✗	121	1.83	13	1.09			1	0.4
3	✗	✗	✗	✗			✗	3	0.05					2	0.81	2	0.06
4	✗	✗	✗	✗	✗		✗									74	2.17
5	✗	✗	✗		✗	✗	✗	31	0.47	2	0.17			1	0.4	10	0.29
6	✗	✗	✗			✗	✗	1	0.02
7	✗	✗		✗	✗	✗	✗	1	0.02
8	✗		✗	✗	✗	✗	✗	30	0.45					1	0.4
9	✗		✗		✗	✗	✗	1	0.02
Total 2–9								188	2.86	15	1.26	0	0	5	2.01	86	2.52

Because of the absence of missing data for the Monteiro-Soares and Dinis-Ribeiro61 study, there was no need for the use of multiple imputation. We applied MICE in the remaining four studies, where there were missing data, although the percentage of missing data was very low. We created 20 imputed data sets, recalculated the logistic regression estimates and pooled the estimates for each study. ORs and 95% CIs were calculated from the logistic regression estimates and standard errors and results were compared before and after imputation.

There was little difference in point estimates between the ORs of the ‘complete case’ final model and the ORs of the final model with imputation (ORs not shown but available on request). Most of the differences between ORs were quasi null, which can be explained by the low percentage of overall missing data in each study. The wider differences were seen in the Abbott et al. 3 study for the previous ulcer or amputation estimates (OR = 8.21 before, OR = 8.37 after, difference –0.152) and in the Pham et al. 73 study for the monofilament estimates (OR = 3.15 before, OR = 3.06 after, difference 0.083). Table 9 summarises and quantifies the differences between the ORs of the final model with multiple imputation and the ORs of the ‘complete case’ final model.

TABLE 9 - Difference in estimates (ORs) between the final model with multiple imputation and the ‘complete case’ final model

Predictors in final model	Study
	Abbott et al., 20023	Crawford et al., 20115	Pham et al., 200073	Leese et al., 201147
Age	0.000	0.000	0.000	0.000
Sex	0.009	–0.001	–0.005	0.001
Duration of diabetes	0.000	0.000	0.000	0.000
Previous ulcer or amputation	0.013	–0.012	0.083	–0.008
Monofilament	0.013	0.012	–0.019	–0.003
Pulses	–0.152	–0.011	–0.067	–0.054

The small differences observed in ORs enabled us to conclude that there was little bias attributable to missing data in our model, which is very likely to be related to a very small proportion of missing data in the original data sets.

Chapter 14 Secondary analyses

As part of the process of disseminating the findings of the systematic review and meta-analyses, we presented the preliminary analyses at two scientific seminars in the UK during 2014. 78,79 In response to questions raised by seminar participants about the value of using less or more tests (or signs), two additional (secondary) analyses have been performed.

What is the value of other commonly used tests not included in the models, particularly tests that permit patients to influence outcome?

Vibration perception threshold

Vibration perception threshold is often used in foot risk assessments for people with diabetes. A range of equipment can be used, including biothesiometers, neurothesiometers and tuning forks. These give continuous data (biothesiometers, neurothesiometers and calibrated tuning forks) or binary data (standard tuning forks).

We used data from four studies3,5,61,73 to calculate the predictiveness of VPT measured with any one of these types of tests. In the model, VPT is adjusted for age, sex, duration of diabetes, monofilament and pulses from study-level multivariable logistic regressions. The predictiveness of VPT in all patients – including those with a history of ulceration or amputation (n = 8003) – is shown in Figure 18 (OR 3.026, 95% CI 1.353 to 6.765). A high level of heterogeneity is observed with an I² of 73.3%.

FIGURE 18.

Predictiveness of VPT in all patients.

The predictiveness of VPT in 7370 people with no history of ulceration or amputation is shown in Figure 19 (OR 2.294, 95% CI 1.189 to 4.426). A low level of heterogeneity is observed in this smaller population (I² = 24.9%).

FIGURE 19.

Predictiveness of VPT in 7370 people with no history of ulceration or amputation.

Glycohaemoglobin or glycated haemoglobin

The glycated haemoglobin test

Glycated haemoglobin is the most common blood test used to assess the amount of glucose carried on the red blood cells and its control is thought to improve patient outcomes such as neuropathy and retinopathy. HbA_1c was traditionally expressed as a percentage but the unit has changed to mmol/mol in accordance with the International Federation of Clinical Chemistry reference measurement procedure. Because glucose attaches to the haemoglobin molecule in the red blood cell, which has a life cycle of 100–120 days, the plasma HbA_1c represents a record of the plasma glucose level for approximately the previous 3 months. Normal levels of HbA_1c are 6.5–7% or 48–53 mmol/mol. Conversion tools are available to convert percentages into mmol/mol. 80

We used data from three studies5,46,47,61 to calculate the predictiveness of an increase of 1% HbA_1c. HbA_1c is adjusted for age, sex, duration of diabetes, monofilament and pulses in the meta-analyses of estimates from study-level multivariable logistic regressions.

The predictiveness of a 1% increase in HbA_1c in all patients – including those with a history of ulceration or amputation (n = 4979) – is shown in Figure 20 (OR 1.218, 95% CI 0.969 to 1.532). A high level of heterogeneity is observed (I² = 79.8).

FIGURE 20.

Predictiveness of a 1% increase in HbA_1c in all patients.

The predictiveness of a 1% increase in HbA_1c in 4595 people with no history of ulceration or amputation is shown in Figure 21 (OR 1.201, 95% CI 0.971 to 1.178). A high level of heterogeneity is observed (I² = 79.8%).

FIGURE 21.

Predictiveness of a 1% increase in HbA_1c in 4595 people with no history of ulceration or amputation.

Monofilaments plus or minus absent pulses

Does the failure to feel a 10-g monofilament test plus absent pedal pulses identify those at risk of foot ulceration better than the monofilament test alone?

A comparison of model 2 and model 4

Models 2 and 4 included the same predictors, namely age, sex, duration of diabetes and insensitivity to monofilaments, although model 4 also included presence/absence of pulses. Our comparison of these two models is restricted to their performance in patients without a history of previous ulceration or amputation.

Various statistics are available for the comparison of regression models to assess different aspects of model performance, such as discrimination or calibration. However, in this clinical context, it is important to consider the consequences to the patient of a wrong prediction. A patient wrongfully predicted to be ulcer free who goes on to ulcerate will bear a much greater cost, which may include pain, loss of mobility, infection and amputation, than a patient wrongfully predicted to ulcerate who does not, for whom the consequences would be higher levels of foot and general diabetes health care. A full cost-effectiveness analysis is beyond the scope of this project, but below we examine predictions of ulceration in patients who do and do not develop ulcers in two models. The natural statistical framework for such an exploration is sensitivity, specificity and receiver operating characteristic (ROC) curves.

Sensitivity is the proportion of times the model will correctly predict an ulcer outcome out of all patients who go on to develop an ulcer. Specificity is the proportion of times the model will correctly predict an ulcer-free outcome out of all patients who remain ulcer free. However, the logistic regression models used for the main analyses do not provide predictions of ulcer versus ulcer-free outcomes for the individual patients. Instead, they provide an individual probability of ulceration for each patient based on his or her age, sex, duration of diabetes, insensitivity to monofilaments, and, in the case of model 4, presence/absence of pulses. These probabilities can then be used to make predictions about individual patients; predictions of an ulcer-free outcome could be applied to those with a small estimated probability of ulceration and predictions of ulceration could be applied to those with a high estimated probability of ulceration. This is a reasonable approach but requires a decision as to when the estimated probability, which may take any value between 0 and 1, becomes large enough that the prediction changes from ulcer free to ulceration. This point at which the prediction changes from one outcome to the other is known as the threshold. It is not possible to calculate a model’s sensitivity or specificity unless a threshold is used.

Choosing the value of this threshold is not a trivial task. Sensitivity and specificity have an inverse relationship as the threshold varies, so choosing a threshold to raise sensitivity will lower specificity and vice versa. For the prediction of foot ulcers in this clinical context, it would be preferable to favour sensitivity over specificity, but the choice of threshold is still somewhat arbitrary, as it is hard to judge to what extent sensitivity should be favoured. Given that the choice of any particular estimated probability as the threshold is hard to justify, we decided to use ROC curves. ROC curves are a way of comparing the sensitivity and specificity of a model without having to choose a threshold. The choice of threshold is avoided by using all possible thresholds. Each possible threshold is used to calculate the corresponding sensitivity and specificity. These sensitivity–specificity pairs are then plotted on a square graph. Traditionally, 1 minus specificity is plotted on the horizontal (x-)axis and sensitivity is plotted on the vertical (y-)axis, resulting in a characteristic curve known as a ROC curve. ROC curves go from the bottom left-hand corner to the top right-hand corner. The ROC curve for a perfect model would go vertically from the bottom left corner to the top left corner, and then horizontally to the top right corner. The ROC for a model with no predictive value would go straight from the bottom left to top left corner in a line at 45 degrees. Most ROC curves are somewhere in between, bending towards but not reaching the top left-hand corner. Because ROC curves use all possible thresholds, they allow comparison of models at all levels of predicted probability of ulceration.

We used empirical ROC curves, where each sensitivity–specificity pair plotted on the graph is calculated directly from the data, with straight lines connecting the pairs. In these particular ROC curves, the bottom left-hand area shows the performance of the models in higher-risk patients, while the top right area shows the performance for lower-risk patients (Figures 22–26 and Table 12).

FIGURE 22.

Receiver operating characteristic curves for models 2 and 4 when applied to the Abbott et al. 3 data set in patients without previous history of ulceration or amputation.

FIGURE 23.

Receiver operating characteristic curves for models 2 and 4 when applied to the Crawford et al. 5 data set.

FIGURE 24.

Receiver operating characteristic curves for models 2 and 4 when applied to the Monteiro-Soares and Dinis-Ribeiro61 data set in patients without previous history of ulceration or amputation.

FIGURE 25.

Receiver operating characteristic curves for models 2 and 4 when applied to the Pham et al. 73 data set in patients without previous history of ulceration or amputation.

FIGURE 26.

Receiver operating characteristic curves for models 2 and 4 when applied to the Leese et al. 47 data set in patients without previous history of ulceration or amputation.

TABLE 12 - Area under the ROC curve for models 2 and 4

Data set	Model 2	Model 4
Abbott et al., 20023	0.684	0.692
Crawford et al., 20115	0.681	0.767
Leese et al., 201147	0.759	0.759
Monteiro-Soares and Dinis-Ribeiro, 201061	0.760	0.772
Pham et al., 200073	0.623	0.573

A model with perfect discrimination would have an AUC of 1; a model that discriminates no better than chance would have an AUC of 0.5. The AUC may be interpreted as the probability that a patient who goes on to ulcerate will have a higher predicted probability than a patient who does not.

The only data set with which model 4 convincingly outperforms model 2 judging from the ROC curve and the AUC is taken from the Crawford et al. 5 study. However, in this data set, only 14 patients without a previous history of ulceration or amputation went on to develop an ulcer. This means that all the sensitivity estimates are based on only 14 patients and are not highly reliable estimates. The data in the study by Pham et al. 73 also come from only 14 patients with no history who developed ulcers. The three larger data sets, with more patients who developed ulcers, suggest that the differences between models 2 and 4 are minimal, with ROC curves that largely overlap and similar AUCs. The closeness of the ROC curves for models 2 and 4 for the Abbott et al. ,3 Monteiro-Soares and Dinis-Ribeiro,61 and Leese et al. 47 data sets suggests that the discrimination of the two models differs very little at all levels of risk. Consequently, these data, based on a large sample of patients (n = 10,375) recruited to three studies,3,47,61 do not indicate an advantage in using both the monofilament test and the ‘absence of pulses’ sign in assessing patients’ risk of developing a foot ulcer in diabetes.

Predictiveness of UK national and International Working group on the Diabetic Foot guidelines for foot ulceration in diabetes

There are two national clinical guidelines in use in the UK and one other issued by the International Working Group on the Diabetic Foot (IWGDF). 15,16,81 These all recommend the classification of people with diabetes into low, increased (or moderate) and high risk as part of annual foot risk assessments. The Scottish Intercollegiate Guidelines Network (SIGN) and IWGDF guidelines also differ from those of the National Institute for Health and Care Excellence (NICE) in respect of the number of risk factors. In addition, the Scottish Clinical Information – Diabetes (SCI-Diabetes) foot risk stratification tool, which underpins the SIGN guidelines, also contains a ‘traffic light’ grading system (see Appendix 13).

Scottish Clinical Information – Diabetes algorithm

The SCI-Diabetes electronic decision support tool algorithm (Figure 27) underpinning recommendations in the SIGN 116 guideline15 differs from that in the traffic light depiction of the diabetic foot risk stratification and triage system in SIGN 116, and the latter is not intended to determine patients’ risk score per se (Professor Graham Leese, Ninewells Hospital and Medical School, 2014, personal communication).

FIGURE 27.

Scottish Clinical Information – Diabetes foot risk algorithm.

Diabetic foot risk stratification overall distribution in the individual patient data diabetic foot ulceration data sets

Data for these combinations of variables were available for four studies. 3,5,47,61 A total of 11,568 diabetic patients had the necessary variables available at baseline to allocate their risk categories. Five studies62,72–75 did not collect these variables. Table 13 shows the number and percentage of patients allocated to each category per study.

TABLE 13 - Diabetic foot risk categories by IPD–DFU studies

SIGN category	Statistics	Study				Total
		Abbott et al., 20023	Crawford et al., 20115	Leese et al., 201147	Monteiro-Soares and Dinis-Ribeiro, 201061
High	n (%)	2348 (35.6)	344 (28.8)	464 (13.6)	205 (56.9)	3361 (29.1)
Moderate	n (%)	3081 (46.7)	643 (53.9)	806 (23.6)	121 (33.6)	4651 (40.2)
Low	n (%)	1174 (17.8)	206 (17.3)	2142 (62.8)	34 (9.4)	3556 (30.7)
Total	N	6603	1193	3412	360	11,568

Table 14 shows the number and percentage of patients with no previous ulcer or amputation allocated to each category per study. Because all patients with a history of ulcer or amputation are categorised in the high category, the number of patients is naturally reduced in this category when those patients are excluded.

TABLE 14 - Diabetic foot risk categories by IPD–DFU studies for those with no previous ulcer or amputation

SIGN category	Statistics	Study				Total
		Abbott et al., 20023	Crawford et al., 20115	Leese et al., 201147	Monteiro-Soares and Dinis-Ribeiro, 201061
High	n (%)	2036 (32.4)	258 (23.3)	268 (8.3)	68 (30.5)	2630 (24.3)
Moderate	n (%)	3081 (49.0)	643 (58.1)	806 (25.1)	121 (54.3)	4651 (42.9)
Low	n (%)	1174 (18.7)	206 (18.6)	2142 (66.6)	34 (15.3)	3556 (32.8)
Total	N	6291	1107	3216	223	10,837

Foot ulcer and the diabetic foot risk stratification by study

Table 15 shows the total number of foot ulcers (outcome) per foot risk category pooled from each of the four studies. Within the high-, moderate- and low-risk categories, 15.5%, 3.0% and 1.9%, respectively, of the total population developed a foot ulcer. Of those 730 patients who developed a foot ulcer, 71.4% were in the high-risk category, 19.3% were in the moderate-risk category and 9.3% were in the low-risk category.

TABLE 15 - Diabetic foot risk categories in the total population

SIGN category	Statistics	Foot ulcer		Total
		No	Yes
High	n (% row) (% column)	2840 (84.5) (26.2)	521 (15.5) (71.4)	3361 (29.1)
Moderate	n (% row) (% column)	4510 (97.0) (41.6)	141 (3.0) (19.3)	4651 (40.2)
Low	n (%row) (% column)	3488 (98.1) (32.2)	68 (1.9) (9.3)	3556 (30.7)
Total	N (%)	10,838 (93.7)	730 (6.3)	11,568 (100)

Table 16 shows the categories for 402 patients with no history of foot ulcer or amputation who developed a foot ulcer during the study period. The risk categories distribution of those who developed a foot ulcer but had no previous history is as follows: 48.0%, 35.1% and 16.9% in the high-, moderate- and low-risk categories, respectively.

TABLE 16 - Diabetic foot risk categories for those with no previous ulcer or amputation

SIGN category	Statistics	Foot ulcer		Total
		No	Yes
High	n (% row) (% column)	2437 (92.7) (23.4)	193 (7.3) (48.0)	2630 (24.3)
Moderate	n (% row) (% column)	4510 (97.0) (43.2)	141 (3.0) (35.1)	4651 (42.9)
Low	n (% row) (% column)	3488 (98.1) (33.4)	68 (1.9) (16.9)	3556 (32.8)
Total	N (% row)	10,435 96.3	402 (3.7)	10,837 (100)

Tables 17 and 18 detail the risk categories of those who developed a foot ulcer for each study. In two studies (Crawford et al. 5 and Monteiro-Soares and Dinis-Ribeiro61), there were no patients in the ‘low’-risk category.

TABLE 17 - Diabetic foot risk categories and number of foot ulcers by study for the whole population

SIGN category	Statistics	Study				Total
		Abbott et al., 20023	Crawford et al., 20115	Leese et al., 201147	Monteiro-Soares and Dinis-Ribeiro, 201061
High	n (%)	220 (75.6)	18 (78.3)	195 (60.6)	88 (93.6)	521 (71.4)
Moderate	n (%)	59 (20.3)	5 (21.7)	71 (22.1)	6 (6.4)	141 (19.3)
Low	n (%)	12 (4.1)	0 (0.0)	56 (17.4)	0 (0.0)	68 (9.3)
Total	N	291	23	322	94	730

TABLE 18 - Diabetic foot risk categories and number of foot ulcers by study for those with no history of ulcer or amputation

SIGN category	Statistics	Study				Total
		Abbott et al., 20023	Crawford et al., 20115	Leese et al., 201147	Monteiro-Soares and Dinis-Ribeiro, 201061
High	n (%)	119 (62.6)	10 (66.7)	45 (26.2)	19 (76.0)	193 (48.0)
Moderate	n (%)	59 (31.1)	5 (33.3)	71 (41.3)	6 (24.0)	141 (35.1)
Low	n (%)	12 (6.3)	0 (0.0)	56 (32.6)	0 (0.0)	68 (16.9)
Total	N	190	15	172	25	402

Meta-analyses of the prognostic utility of the SCI-Diabetes foot risk stratification tool

Below we present meta-analyses of the prognostic utility of the high and moderate SCI-Diabetes risk classification in two populations of patients. In the first we include all patients from each of the four studies with corresponding variables (n = 11,568) and in the second we include the data from only those patients without a history of foot ulceration or amputation (n = 10,837).

The calculated estimates of effect (ORs) show the SCI-Diabetes high-risk category to be predictive of a foot ulcer in the total population (OR 11.2, 95% CI 5.7 to 21.8). The level of heterogeneity is high (I² = 89.2%) (Figure 28).

FIGURE 28.

Forest plot with ORs of the new foot ulcer predicted by the SCI-Diabetes foot risk categories (high vs. moderate + low).

Being classified in the moderate rather than the low SCI-Diabetes category was predictive of the development of a foot ulcer (pooled OR 2.7, 95% CI 1.5 to 5.1). A high level of heterogeneity is observed (I² = 67.4%) (Figure 29).

FIGURE 29.

Forest plot with ORs of the new foot ulcer predicted by the diabetic foot risk categories (moderate vs. low).

Figure 30 shows a meta-analysis of data from people with no previous ulcer or amputation predicted by the SCI-Diabetes foot risk categories high versus moderate plus low. The OR of 4.5 (95% CI 3.3 to 6.2) shows that the risk classification tool is predictive in this population too, but it is much lower than that obtained in the meta-analysis including patients with a history of ulceration or amputation (see Figure 28). The heterogeneity is also much lower. (I² = 33.2%).

FIGURE 30.

Forest plot with ORs of the new foot ulcer (with no previous ulcer or amputation) predicted by the diabetic foot risk categories (high vs. moderate + low).

Figure 31 shows the meta-analysis of data from people with no previous ulcer or amputation predicted by the SCI-Diabetes foot risk categories; moderate versus low. The OR exactly matches that calculated in the meta-analysis of moderate versus low using data collected from the whole IPD population (i.e. including those with a history of ulceration or amputation) (OR 2.74, 95% CI 1.47 to 5.12) (see Figure 29). The same high level of heterogeneity is also observed.

FIGURE 31.

Forest plot with ORs of the new foot ulcer (with no previous ulcer or amputation) predicted by the diabetic foot risk categories (moderate vs. low).

Scottish Intercollegiate Guidelines Network 116: management of diabetic foot disease traffic light system

Categories are defined as follows:

• High: previous ulceration or amputation or more than one risk factor present (e.g. loss of sensation or signs of peripheral vascular disease with callus or deformity).

• Moderate: one risk factor present (e.g. loss of sensation or signs of peripheral vascular disease without callus or deformity).

• Low: no risk factor present (e.g. no loss of sensation, no signs of peripheral vascular disease and no other risk factors).

For a total of 11,755 diabetic patients from five studies, the necessary data were available to allocate them to the defined risk categories.

Foot ulcer

Table 19 shows the total number of new foot ulcers per diabetic foot risk categories for five studies together. Within the high-, moderate- and low-risk categories, 13.6%, 5.6% and 2.0% of patients, respectively, developed a foot ulcer. The analysis shows that 3496 patients were not categorised into any of the active/high, moderate or low definitions of risk, and use of this classification would mean that 14% of foot ulcers would be missed.

TABLE 19 - Diabetic foot risk categories stated in the SIGN 11615 traffic light system for patients in five studies

SIGN risk category	New ulcer, n (%)
Frequency	No	Yes	Total
Not classified	3391 (30.79)	105 (14.19)	3496 (29.74)
High	3153 (28.62)	497 (67.16)	3650 (31.05)
Moderate	1217 (11.05)	72 (9.73)	1289 (10.97)
Low	3254 (29.54)	66 (8.92)	3320 (28.24)
Total	1101 (93.70)	740 (6.30)	11,755 (100.00)

Meta-analyses of the predictive value of the clinical guideline recommendations from NICE CG10 and the Quality and Outcomes Framework

Our meta-analyses found that the high- and increased-risk categories in the NICE guideline are predictive of foot ulceration.

Figure 32 shows the results of a meta-analysis comparing the predictiveness of categories (high vs. increased + low). The pooled estimates (OR 13.5, 95% CI 3.6 to 51.3) show the high-risk category to be predictive with a high level of heterogeneity (I² = 96.7%).

FIGURE 32.

Pooled estimate of new foot ulcer predicted by the NICE (QOF) diabetic foot risk categories (high vs. increased + low).

Figure 33 shows a meta-analysis of increased- versus low-risk categories and the pooled meta-analysis estimates based on data from three studies.

Being in the increased-risk category, rather than the low-risk category, was predictive of the development of a foot ulcer (OR 3.3, 95% CI 1.6 to 7.0) with a high level of heterogeneity (I² = 73.6).

FIGURE 33.

Pooled estimate of new foot ulcer predicted by the NICE (QOF) diabetic foot risk categories (increased vs. low).

Figure 34 shows the results of a meta-analysis populated with data from those patients with no history of ulcer or amputation from four studies. This meta-analysis compared high versus increased + low categories, and the pooled estimates (OR 5.1, 95% CI 3.0 to 8.6) are smaller than those in the total population but still show the high-risk category to be predictive with less heterogeneity (I² = 53.2%).

FIGURE 34.

Pooled estimate of new foot ulcer in people with no history of ulceration or amputation predicted by the NICE (QOF) diabetic foot risk categories (high vs. increased + low).

Figure 35 shows the results of a meta-analysis based on data from those with no history of ulcer or amputation in three studies. The comparison is between the increased- and low-risk categories. The pooled estimate (OR 3.4, 95% CI 1.6 to 7.4) and levels of heterogeneity are similar to those in the meta-analysis of data from the total population and are highly predictive of foot ulceration (see Figure 18).

FIGURE 35.

Pooled estimate of new foot ulcer in people with no history of ulceration or amputation predicted by the NICE (QOF) diabetic foot risk categories (increased vs. low).

Meta-analyses of the predictive value of the clinical guideline recommendations from the International Working Group on the Diabetic Foot

All the ORs calculated presented the diabetic foot risk category high as being predictive of the development of a new foot ulcer. The estimates of the Crawford et al. 5 and Kästenbauer et al. 62 studies had to be removed from the forest plots owing to a complete separation case. The Pham et al. 73 estimates were removed from the forest plot (see Figure 38) for the same reason.

Figure 36 shows the forest plot with ORs of the new foot ulcer predicted by the diabetic foot risk categories (high vs. medium + low) by study and the pooled meta-analysis estimates. The estimates across studies and the pooled estimates (OR 6.7, 95% CI 3.4 to 13.1) show the high-risk category to be predictive of the development of a new foot ulcer, although with a high heterogeneity (I² = 90.6%).

FIGURE 36.

Pooled estimate of new foot ulcer predicted by the IWGDF diabetic foot risk categories (high vs. medium + low).

Figure 37 shows the forest plot with ORs of the new foot ulcer predicted by the diabetic foot risk categories (medium vs. low) by study and the pooled meta-analysis estimates. The pooled estimates (OR 2.3, 95% CI 1.5 to 3.3) show the medium-risk category rather than the low-risk category to be predictive of the development of a new foot ulcer although estimates across studies are not consistently showing this association.

FIGURE 37.

Pooled estimates of new foot ulcer predicted by the IWGDF diabetic foot risk categories (medium vs. low).

Figure 38 shows the forest plot with ORs of the new foot ulcer (with no previous ulcer or amputation) predicted by the diabetic foot risk categories (high vs. medium + low) by study and the pooled meta-analysis estimates. Apart from the estimates from the Monteiro-Soares and Dinis-Ribeiro study,61 the estimates across studies and the pooled estimates (OR 5.3, 95% CI 3.5 to 8.1) are lower than in Figure 22 but still show the high-risk category to be predictive of the development of a new foot ulcer, with less heterogeneity (I² = 47.2%).

FIGURE 38.

Pooled estimates of new foot ulcer in people with no history of ulceration or amputation by the IWGDF diabetic foot risk categories (high vs. medium + low).

Figure 39 shows the forest plot with ORs of the new foot ulcer (with no previous ulcer or amputation) predicted by the diabetic foot risk categories (medium vs. low) by study and the pooled meta-analysis estimates. The estimates are higher than those of Figure 22, although significant for only two studies out of five. The pooled estimates (OR 3.4, 95% CI 2.0 to 5.8) show the medium-risk category rather than the low-risk category to be predictive of the development of a new foot ulcer with heterogeneity (I² = 41.6%).

FIGURE 39.

Pooled estimates of new foot ulcer in people with no history of ulceration or amputation predicted by the IWGDF diabetic foot risk categories (medium vs. low).

Chapter 15 Discussion

This systematic review and meta-analysis of IPD includes patients recruited to cohort studies conducted worldwide and is the first of its kind to evaluate the predictive factors of foot ulceration in diabetes. The resultant increased statistical power has permitted analyses that have previously not been possible in individual studies and has resulted in new insights into the independent contribution of symptoms signs and diagnostic tests used for foot risk assessment in different patient populations. The increased statistical power has also allowed us to compare fully the performance of individual tests against the risk categories contained in national and international diabetes guidelines.

In our meta-analyses, a previous ulceration or diabetes-related LEA produced large ORs and there is no doubt about the correctly categorised high-risk status of individuals with this history. It was not possible to explore the predictive value of LEA (either minor or major) as an independent explanatory variable because there were relatively few events of this nature in the data sets (n = 146). The requirement for patients to be ambulant to meet the inclusion criteria in some of the included studies may explain the small number of patients with LEA who were recruited.

However, it is important to distinguish between meta-analyses based on data collected from patients who have experienced a foot ulcer or LEA and meta-analyses based on data from those who have not, because it is preferable to identify those at risk of ulceration earlier in the disease pathway so that attempts can be made to alter the clinical course of the disease and beneficially influence patient outcomes. More clinically useful are the estimates observed in the never-ulcerated population, which show that being insensate to a 10-g monofilament, having one absent pedal pulse or having a longer duration of a diabetes diagnosis are independently predictive of risk in ulcer-naive patients.

The most consistent results were obtained from the 10-g monofilament test and clearly show this quick, simple and relatively cheap test to be predictive of foot ulceration for everyone with a diagnosis of diabetes. The almost complete absence of heterogeneity in the meta-analyses is remarkable given that the pooled estimates are based on data from five different studies and 11,522 people from three different countries. Additionally, the 10-g monofilament was applied to different sites on the feet in each of the five studies and this indicates that the number of sites and the anatomical position of the sites matters little. The estimates for absent pedal pulses are also consistent in the two meta-analyses and show that the absence of at least one pedal pulse is independently predictive of risk. However, adding the palpation of pedal pulses to the risk assessment examination appears to confer no additional predictive value than using a 10-g monofilament alone. This is clear from the ROC analyses of five individual studies – the data from the largest studies show almost identical estimates for these two tests, but the consistency of the results for the 10-g monofilament favour its use. This observed effect may be due to the underlying pathophysiology of the majority of foot ulcers in these cohorts being neurological rather than vascular. 83

Perhaps not surprisingly, the number of years that a person has had a diagnosis of diabetes was found to be a risk factor, although there is a high level of heterogeneity in both meta-analyses. Because the OR is close to 1, this aspect of patients’ history is much less predictive than an inability to feel a 10-g monofilament, the absence of one pedal pulse or a history of ulceration and LEA.

When data from the never-ulcerated population are separated from the total population, male sex is no longer observed to be predictive of risk of ulceration. However, the ORs for the data in the largest studies are hardly altered in the never-ulcerated and total population group analyses, and most of the variation occurs in the smaller data sets where the difference in OR could be explained by the play of chance. Comparison of these independent predictive factors with the risk stratification categories in national and international diabetes guidelines does indicate that using the various groups of tests recommended therein does not produce additional predictive value. The meta-analyses of data from the SIGN, NICE and IWGDF guidelines allow a direct comparison of the effects from recommended risk categories. The ORs and CIs are not statistically significantly different from those estimates obtained from a failure to feel a 10-g monofilament in populations at both high and moderate risk and history of ulceration and LEA in high-risk populations.

The large number of patients in the derivation data sets and the use of two different approaches to validate the model underpin its reliability and suggest that the guidance in clinical guidelines and the QOF should be simplified to include only the inability to feel a 10-g monofilament or an absent pedal pulse to identify those at moderate (or increased) risk of ulceration. Using patient history of ulceration and/or a previous LEA will accurately classify those at high risk. The implementation of this greatly simplified approach to annual diabetes foot checks would reduce the amount of clinical time spent testing, avoid the cost associated with acquiring more expensive tests and reduce the ambiguity currently surrounding some components of risk assessment procedures such as ‘unable to see or reach foot’ and avoid unclassified/missed cases from the use of the traffic light system contained in the SIGN guideline. 15 CPRs are usually defined as containing three or more variables22 and the fewer tests and elements from the patient history that health-care professionals are required to consider, the more likely it is that risk assessment procedures will be performed.

The accurate assessment of risk is, however, only the first step in an overall improvement in health outcomes, and there is little randomised evidence about the effect of annual foot screening in existence. 84 One RCT found that those screened demonstrated statistically significantly fewer amputations than a group of patients whose risk was not assessed, but a statistically significant reduction in incident foot ulcerations was not observed in the study population and the true value of specialist foot care services remains uncertain. 18,85 There also remain gaps in the knowledge about any benefit of potentially preventative interventions, such as patient education, routine podiatry, foot orthoses and specialist footwear. 84,86–88 NICE recommends further research to identify the appropriate level and combination of risk factors used to categorise patients as being at high risk of ulceration and that these individuals should then be offered attendance on a protection programme. A UK-wide RCT to evaluate the clinical effectiveness and cost-effectiveness of such protection programmes is needed to evaluate the delivery of this type of health care.

The quality of the conduct of the 10 studies included in the systematic review was assessed as high. Only one item was found to risk the validity of the included studies: the blinding of the individuals who ascertained the outcome variable (ulceration) to the status of the exposure variables was not maintained in 50% of the included studies. This is widely believed to be an important quality factor in prognostic studies and CPRs. 23 However, the meta-analyses on which our conclusions are based included only one study in which the investigators knew the status of the index test results in some, but not all, cases46,47 and the estimates these data contribute statistically differ from pooled estimates for only one prognostic factor – previous history of ulceration or amputation. Data from the study by Leese et al. 47 were found to be statistically significantly more predictive than the pooled estimate. However, rather than this effect arising from an absence of blinding, it may result from the inherent difference in study design, this study being the only one to use routinely collected data.

Of the 10 studies included in this systematic review and meta-analysis, few contained data for variables associated with patients’ systemic health such as history of stroke, myocardial infarctions or kidney failure. 46,47 Consequently, we are unable to make suggestions about the independent contribution of elements from patients’ general medical history.

Chapter 16 Conclusions

The consistent results for the inability to feel a 10-g monofilament test in all five different regression models, together with the total absence of heterogeneity, has produced robust evidence for the high predictive value of this cheap and simple-to-use diagnostic test. An inability to feel a 10-g monofilament appears to be at least as predictive as the groups of tests currently recommended in national and international clinical guidelines.

Acknowledgements

The idea for this research came from FC (Senior Health Services Researcher) and the mentors of her CSO/DH NHS Research and Development Postdoctoral fellowship (2004–7): Professors Tom Fahey (General Practice), Jos Kleijnen (Systematic Reviews) and Aziz Sheikh (General Practice R&D).

We thank Dr Chantelle Anandan (Research Fellow, Epidemiology) for her work on preliminary aspects of the review, Mr Martin Maxwell (Public Partner) for his enduring enthusiasm about our research and Mrs Elsbeth Hamilton (project secretary) for working in the most efficient way possible to help keep the project on-track. Ms Nikki Coates (Specialist Diabetes Podiatrist), Dr Nicola Leech (Consultant Diabetologist), Professor William Jeffcoate (Professor of Diabetic Medicine), Professor Tom Fahey, Dr Jayne Tierney (Senior Research Scientist) and Ms Marshall Dozier (Librarian and Information Specialist) all helped at important times during the process.

We are also grateful to Professors Martin Bland (Professor of Medical Statistics) and Benjamin Lipsky (Professor of Medicine) for their insightful questions about aspects of our analyses after seminar and conference presentations. These prompted additional analyses and we hope their questions are finally answered in the secondary analysis section.

The collegiate spirit of the authors of primary cohort studies who donated, or provided access to, anonymised IPD is commendable: Dr Caroline Abbott (Research Fellow), Professor Andrew Boulton (Professor of Medicine), Professor Edward Boyko (Professor General Internal Medicine and Epidemiology), Dr Thomas Kästenbauer (Biologist), Professor Graham Leese (Consultant and Honorary Professor in Diabetes/Endocrinology), Dr Matteo Monami (Director of the Diabetes Section), Dr Matilde Monteiro-Soares (Podiatrist and Research Fellow), Dr Stephen Rith-Najarian (Family Practice Physician), Dr Amber Seelig (Medical Statistician), Professor Aris Veves (Assistant Professor of Surgery) and Dr Matthew Young.

We are conscious that this research would not have been possible without the consent of 16,385 people with diabetes who allowed their data to be used for research purposes and we are all indebted to them.

References

1. Wild S, Roglic G, Green A, Sicree R, King H. Global prevalence of diabetes: estimates for the year 2000 and projections for 2030. Diabetes Care 2004;27:1047-53. http://dx.doi.org/10.2337/diacare.27.5.1047.
2. Boulton AJ, Vileikyte L, Ragnarson-Tennvall G, Apelqvist J. The global burden of diabetic foot disease. Lancet 2005;366:1719-24. http://dx.doi.org/10.1016/S0140-6736(05)67698-2.
3. Abbott CA, Carrington AL, Ashe H, Bath S, Every LC, Griffiths J, et al. The North-West Diabetes Foot Care Study: incidence of, and risk factors for, new diabetic foot ulceration in a community-based patient cohort. Diabet Med 2002;19:377-84. http://dx.doi.org/10.1046/j.1464-5491.2002.00698.x.
4. Crawford F, Inkster M, Kleijnen J, Fahey T. Predicting foot ulcers in patients with diabetes: a systematic review and meta-analysis. QJM 2007;100:65-86. http://dx.doi.org/10.1093/qjmed/hcl140.
5. Crawford F, McCowan C, Dimitrov BD, Woodburn J, Wylie GH, Booth E, et al. The risk of foot ulceration in people with diabetes screened in community settings: findings from a cohort study. QJM 2011;104:403-10. http://dx.doi.org/10.1093/qjmed/hcq227.
6. Scottish Diabetes Monitoring Group . Scottish Diabetes Survey 2013 n.d. www.diabetesinscotland.org.uk/Publications/SDS2013.pdf (accessed 14 October 2014).
7. Mason J, O’Keeffe C, McIntosh A, Hutchinson A, Booth A, Young RJ. A systematic review of foot ulcer in patients with Type 2 diabetes mellitus. I: prevention. Diabet Med 1999;16:801-12. http://dx.doi.org/10.1046/j.1464-5491.1999.00133.x.
8. Pecoraro RE, Reiber GE, Burgess EM. Pathways to diabetic limb amputation. Basis for prevention. Diabetes Care 1990;13:513-21. http://dx.doi.org/10.2337/diacare.13.5.513.
9. Spencer F, Sage R, Graner J. The incidence of foot pathology in a diabetic population. J Am Podiatr Med Assoc 1985;75:590-2. http://dx.doi.org/10.7547/87507315-75-11-590.
10. Young MJ, McCardle JE, Randall LE, Barclay JI. Improved survival of diabetic foot ulcer patients 1995–2008: possible impact of aggressive cardiovascular risk management. Diabetes Care 2008;31:2143-7. http://dx.doi.org/10.2337/dc08-1242.
11. Kennon B, Leese GP, Cochrane L, Colhoun H, Wild S, Stang D, et al. Reduced incidence of lower-extremity amputations in people with diabetes in Scotland: a nationwide study. Diabetes Care 2012;35:2588-90. http://dx.doi.org/10.2337/dc12-0511.
12. Vamos EP, Bottle A, Majeed A, Millett C. Trends in lower extremity amputations in people with and without diabetes in England, 1996–2005. Diabetes Res Clin Pract 2010;87:275-82. http://dx.doi.org/10.1016/j.diabres.2009.11.016.
13. Holman N, Young RJ, Jeffcoate WJ. Variation in the recorded incidence of amputation of the lower limb in England. Diabetologia 2012;55:1919-25. http://dx.doi.org/10.1007/s00125-012-2468-6.
14. McIntosh A, Peters J, Young R, Hutchinson A, Chiverton R, Clarkson S, et al. Prevention and Management of Foot Problems in Type 2 diabetes: Clinical Guidelines and Evidence. Sheffield: University of Sheffield; 2003.
15. 116 Management of Diabetes: A National Clinical Guideline. Edinburgh: SIGN; 2010.
16. International Working Group on the Diabetic Foot. Amsterdam: International Diabetes Federation; 2011.
17. British Medical Association . General Medical Services Contract 2011 n.d. www.bma.org.uk/employmentandcontracts/independent_contractors/quality_outcomes_framework/qofguidance2011.jsp (accessed 24 April 2011).
18. Crawford F. How can we best prevent new foot ulcers in people with diabetes?. BMJ 2008;337. http://dx.doi.org/10.1136/bmj.a1234.
19. Macran S, Kind P, Collingwood J, Hull R, McDonald I, Parkinson L. Evaluating podiatry services: testing a treatment specific measure of health status. Qual Life Res 2003;12:177-88. http://dx.doi.org/10.1023/A:1022257005017.
20. Jeffcoate WJ. Stratification of foot risk predicts the incidence of new foot disease, but do we yet know that the adoption of routine screening reduces it?. Diabetologia 2011;54:991-3. http://dx.doi.org/10.1007/s00125-011-2075-y.
21. Monteiro-Soares M, Boyko EJ, Ribeiro J, Ribeiro I, Dinis-Ribeiro M. Risk stratification systems for diabetic foot ulcers: a systematic review. Diabetologia 2011;54:1190-9. http://dx.doi.org/10.1007/s00125-010-2030-3.
22. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. JAMA 1997;277:488-94. http://dx.doi.org/10.1001/jama.1997.03540300056034.
23. McGinn TG, Guyatt GH, Wyer PC, Naylor CD, Stiell IG, Richardson WS. Users’ guides to the medical literature: XXII: how to use articles about clinical decision rules. Evidence-Based Medicine Working Group. JAMA 2000;284:79-84. http://dx.doi.org/10.1001/jama.284.1.79.
24. Clark MJ, Stewart LA, Egger M, Davey Smith G, Altman DG. Systematic Reviews in Health Care: Meta-Analysis in Context. London: BMJ Books; 2001.
25. Stewart LA, Clarke MJ. Practical methodology of meta-analyses (overviews) using updated individual patient data. Cochrane Working Group. Stat Med 1995;14:2057-79. http://dx.doi.org/10.1002/sim.4780141902.
26. Stewart LA, Tierney JF, Clark MJ, Higgins JPT, Green S. Reviews of Individual Patient Data. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 2011. http://handbook.cochrane.org/ (accessed 19 July 2015).
27. Systematic Reviews: CRD’s Guidance on Undertaking Reviews in Health Care. Centre for Reviews and Dissemination, University of York: York; 2009.
28. Medical Research Council (MRC) Ethics Series . Personal Information in Medical Research 2000 n.d. www.mrc.ac.uk/Utilities/Documentrecord/index.htm?d=MRC002452 (accessed 14 May 2012).
29. Crawford F, Anandan C, Chappell FM, Murray GD, Price JF, Sheikh A, et al. Protocol for a systematic review and individual patient data meta-analysis of prognostic factors of foot ulceration in people with diabetes: the international research collaboration for the prediction of diabetic foot ulcerations (PODUS). BMC Med Res Methodol 2013;13. http://dx.doi.org/10.1186/1471-2288-13-22.
30. Lefebvre C, Manheimer E, Glanville J, Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. Cochrane; 2011.
31. Hayden JA, Cote P, Bombardier C. Evaluation of the quality of prognosis studies in systematic reviews. Ann Intern Med 2006;144:427-37. http://dx.doi.org/10.7326/0003-4819-144-6-200603210-00010.
32. Crombie IK. The Pocket Guide to Critical Appraisal. London: BMJ Publishing Group; 1996.
33. McShane LM, Altman DG, Sauerbrei W, Taube SE, Gion M, Clark GM. Reporting recommendations for tumour marker prognostic studies (REMARK). J Natl Canc Inst 2005;97:1180-4. http://dx.doi.org/10.1093/jnci/dji237.
34. Altman DG, Lyman GH. Methodological challenges in the evaluation of prognostic factors in breast cancer. Breast Cancer Res Treat 1998;52:289-303. http://dx.doi.org/10.1023/A:1006193704132.
35. Rector TS, Taylor BC, Wilt TJ. Systematic review of prognostic tests. J Gen Intern Med 2012;27:S94-101. http://dx.doi.org/10.1007/s11606-011-1899-y.
36. Altman DG. Systematic reviews of evaluations of prognostic variables. BMJ 2001;323:224-8. http://dx.doi.org/10.1136/bmj.323.7306.224.
37. Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules. Applications and methodological standards. New Engl J Med 1985;313:793-9. http://dx.doi.org/10.1056/NEJM198509263131306.
38. Fowkes FG, Fulton PM. Critical appraisal of published research: introductory guidelines. BMJ 1991;302:1136-40. http://dx.doi.org/10.1136/bmj.302.6785.1136.
39. Vandenbroucke JP, von Elm E, Altman DG, Gotzsche PC, Mulrow CD, Pocock SJ, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. PLOS Med 2007;4. http://dx.doi.org/10.1371/journal.pmed.0040297.
40. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol 2008;61:344-9. http://dx.doi.org/10.1016/j.jclinepi.2007.11.008.
41. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011;155:529-36. http://dx.doi.org/10.7326/0003-4819-155-8-201110180-00009.
42. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA 2000;283:2008-12. http://dx.doi.org/10.1001/jama.283.15.2008.
43. Debray TP, Moons KG, Abo-Zaid GM, Koffijberg H, Riley RD. Individual participant data meta-analysis for a binary outcome: one-stage or two-stage?. PLOS ONE 2013;8. http://dx.doi.org/10.1371/journal.pone.0060650.
44. Stewart GB, Altman DG, Askie LM, Duley L, Simmonds MC, Stewart LC. Statistical analysis of individual participant data meta-analyses: a comparison of methods and recommendations for practice. Res Synth Methods 2010;1:97-111. http://dx.doi.org/10.1371/journal.pone.0046042.
45. Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. Cochrane; 2011.
46. Leese GP, Reid F, Green V, McAlpine R, Cunningham S, Emslie-Smith AM, et al. Stratification of foot ulcer risk in patients with diabetes: a population-based study. Int J Clin Pract 2006;60:541-5. http://dx.doi.org/10.1111/j.1368-5031.2006.00899.x.
47. Leese GP, Cochrane L, Mackie AD, Stang D, Brown K, Green V. Measuring the accuracy of different ways to identify the ‘at-risk’ foot in routine clinical practice. Diabet Med 2011;28:747-54. http://dx.doi.org/10.1111/j.1464-5491.2011.03297.x.
48. Boyko EJ, Ahroni JH, Stensel V, Forsberg RC, Davignon DR, Smith DG. A prospective study of risk factors for diabetic foot ulcer. The Seattle Diabetic Foot Study. Diabetes Care 1999;22:1036-42. http://dx.doi.org/10.2337/diacare.22.7.1036.
49. Boyko EJ, Ahroni JH, Cohen V, Nelson KM, Heagerty PJ. Prediction of diabetic foot ulcer occurrence using commonly available clinical information: the Seattle Diabetic Foot Study. Diabetes Care 2006;29:1202-7. http://dx.doi.org/10.2337/dc05-2031.
50. Riley RD, Steyerberg EW. Meta-analysis of a binary outcome using individual participant data and aggregate data. Res Synth Methods 2010;1:2-19. http://dx.doi.org/10.1002/jrsm.4.
51. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Methods 2010;1:97-111. http://dx.doi.org/10.1002/jrsm.12.
52. Deeks JJ, Higgins JPT, Altman DG, Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration; 2011.
53. van Buuren S, Groothuis-Oudshoorn K. MICE: Multivariate imputation by chained equations in R. J Stat Software 2011;45:1-67.
54. White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med 2011;30:377-99. http://dx.doi.org/10.1002/sim.4067.
55. Sterne JAC, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ 2009;338. http://dx.doi.org/10.1136/bmj.b2393.
56. Resche-Rigon M, White IR, Bartlett JW, Peters SA, Thompson SG. PROG-IMT Study Group . Multiple imputation for handling systematically missing confounders in meta-analysis of individual participant data. Stat Med 2013;32:4890-905. http://dx.doi.org/10.1002/sim.5894.
57. Little RJA. Regression with missing X’s: a review. J Am Stat Assoc 1992;87:1227-37. http://dx.doi.org/10.2307/2290664.
58. Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York, NY: Springer; 2009.
59. Harrell FE. Regression Modelling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York, NY: Springer-Verlag; 2001.
60. O’Riordan P, Stevens PE, Lamb EJ. Estimated glomerular filtration rate. BMJ 2014;348. http://dx.doi.org/10.1136/bmj.g264.
61. Monteiro-Soares M, Dinis-Ribeiro M. External validation and optimisation of a model for predicting foot ulcers in patients with diabetes. Diabetologia 2010;53:1525-33. http://dx.doi.org/10.1007/s00125-010-1731-y.
62. Kästenbauer T, Sauseng S, Sokol G, Auinger M, Irsigler K. A prospective study of predictors for foot ulceration in type 2 diabetes. J Am Podiatr Med Assoc 2001;91:343-50. http://dx.doi.org/10.7547/87507315-91-7-343.
63. Gerds TA, Cai T, Schumacher M. The performance of risk prediction models. Biometrical J 2008;50:457-79. http://dx.doi.org/10.1002/bimj.200810443.
64. Steyerberg EW, Borsboom GJ, van Houwelingen HC, Eijkemans MJ, Habbema JD. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med 2004;23:2567-86. http://dx.doi.org/10.1002/sim.1844.
65. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol 2009;62:1006-12. http://dx.doi.org/10.1016/j.jclinepi.2009.06.005 (accessed 4 April 2015).
66. Litzelman DK, Marriott DJ, Vinicor F. Independent physiological predictors of foot lesions in patients with NIDDM. Diabetes Care 1997;20:1273-8. http://dx.doi.org/10.2337/diacare.20.8.1273.
67. Murray HJ, Young MJ, Hollis S, Boulton AJ. The association between callus formation, high pressures and neuropathy in diabetic foot ulceration. Diabet Med 1996;13:979-82. http://dx.doi.org/10.1002/(SICI)1096-9136(199611)13:11&lt;979::AID-DIA267&gt;3.0.CO;2-A.
68. Peters EJ, Lavery LA. Effectiveness of the diabetic foot risk classification system of the International Working Group on the Diabetic Foot. Diabetes Care 2001;24:1442-7. http://dx.doi.org/10.2337/diacare.24.8.1442.
69. Veves A, Murray HJ, Young MJ, Boulton AJ. The risk of foot ulceration in diabetic patients with high foot pressure: a prospective study. Diabetologia 1992;35:660-3. http://dx.doi.org/10.1007/BF00400259.
70. Lavery LA, Armstrong DG, Wunderlich RP, Tredwell J, Boulton AJ. Predictive value of foot pressure assessment as part of a population-based diabetes disease management program. Diabetes Care 2003;26:1069-73. http://dx.doi.org/10.2337/diacare.26.4.1069.
71. Armstrong DG, Lavery LA, Holtz-Neiderer K, Mohler MJ, Wendel CS, Nixon BP, et al. Variability in activity may precede diabetic foot ulceration. Diabetes Care 2004;27:1980-4. http://dx.doi.org/10.2337/diacare.27.8.1980.
72. Monami M, Vivarelli M, Desideri CM, Colombi C, Marchionni N, Mannucci E. Pulse pressure and prediction of incident foot ulcers in type 2 diabetes. Diabetes Care 2009;32:897-9. http://dx.doi.org/10.2337/dc08-1679.
73. Pham H, Armstrong DG, Harvey C, Harkless LB, Giurini JM, Veves A. Screening techniques to identify people at high risk for diabetic foot ulceration: a prospective multicenter trial. Diabetes Care 2000;23:606-11. http://dx.doi.org/10.2337/diacare.23.5.606.
74. Rith-Najarian SJ, Stolusky T, Gohdes DM. Identifying diabetic patients at high risk for lower-extremity amputation in a primary health care setting. A prospective evaluation of simple screening criteria. Diabetes Care 1992;15:1386-9. http://dx.doi.org/10.2337/diacare.15.10.1386.
75. Young MJ, Breddy JL, Veves A, Boulton AJ. The prediction of diabetic neuropathic foot ulceration using vibration perception thresholds. A prospective study. Diabetes Care 1994;17:557-60. http://dx.doi.org/10.2337/diacare.17.6.557.
76. Abbott CA, Garrow AP, Carrington AL, Morris J, Van Ross ER, Boulton AJ. Foot ulcer risk is lower in South-Asian and African-Caribbean compared with European diabetic patients in the U.K.: the North-West diabetes foot care study. Diabetes Care 2005;28:1869-75. http://dx.doi.org/10.2337/diacare.28.8.1869.
77. Cohen DJ, Doucet M, Cutlip DE, Ho KK, Popma JJ, Kuntz RE. Impact of smoking on clinical and angiographic restenosis after percutaneous coronary intervention: another smoker’s paradox?. Circulation 2001;104:773-8. http://dx.doi.org/10.1161/hc3201.094225.
78. Crawford F, Cezard G, Chappell FM, Murray GD. behalf of the international PODUS group . Predictive Factors for Foot Ulceration in Diabetes: An Individual Patient Data Systematic Review and Meta-Analysis n.d.
79. PODUS group . Predictive Factors for Diabetic Foot Ulcerations: An Individual Patient Data Systematic Review and Meta-Analysis n.d.
80. Diabetes UK. Testing 2014. www.diabetes.org.uk/Guide-to-diabetes/Monitoring/Testing/#HbA1c (accessed 30 May 2014).
81. National Institute for Health and Care Excellence . Clinical Guidelines for Type 2 Diabetes. Prevention and Management of Foot Problems 2014. http://publications.nice.org.uk/type-2-diabetes-foot-problems-cg10 (accessed 29 May 2014).
82. Bakker K, Schaper NC. International Working Group on the Diabetic Foot Editorial Board . The development of global consensus guidelines on the management and prevention of the diabetic foot 2011. Diabetes Metab Res Rev 2012;28:116-18. http://dx.doi.org/10.1002/dmrr.2254.
83. Singh N, Armstrong DG, Lipsky BA. Preventing foot ulcers in patients with diabetes. JAMA 2005;293:217-28. http://dx.doi.org/10.1001/jama.293.2.217.
84. Dorresteijn JAN, Kriegsman DMW, Valk GD. Complex interventions for preventing diabetic foot ulceration. Cochrane Database Syst Rev 2010;1. http://dx.doi.org/10.1002/14651858.CD007610.pub2.
85. McCabe CJ, Stevenson RC, Dolan AM. Evaluation of a diabetic foot screening and protection programme. Diabet Med 1998;15:80-4. http://dx.doi.org/10.1002/(SICI)1096-9136(199801)15:1&lt;80::AID-DIA517&gt;3.0.CO;2-K.
86. Dorresteijn JAN, Kriegsman DMW, Assendelft WJJ, Valk GD. Patient education for preventing diabetic foot ulceration. Cochrane Database Syst Rev 2012;10. http://dx.doi.org/10.1002/dmrr.2237.
87. Paton J, Bruce G, Jones R, Stenhouse E. Effectiveness of insoles used for the prevention of ulceration in the neuropathic diabetic foot: a systematic review. J Diabetes Complications 2011;25:52-6. http://dx.doi.org/10.1016/j.jdiacomp.2009.09.002.
88. Bus SA, Valk GD, van Deursen RW, Armstrong DG, Caravaggi C, Hlavacek P, et al. The effectiveness of footwear and offloading interventions to prevent and heal foot ulcers and reduce plantar pressure in diabetes: a systematic review. Diabetes Metab Res Rev 2008;24:162-80. http://dx.doi.org/10.1002/dmrr.850.
89. Burgers JS, Bailey JV, Klazinga NS, Van Der Bij AK, Grol R, Feder G. Inside guidelines: comparative analysis of recommendations and evidence in diabetes guidelines from 13 countries. Diabetes Care 2002;25:1933-9. http://dx.doi.org/10.2337/diacare.25.11.1933.

Appendix 1 Committee structure

Appendix 2 Data confidentiality agreement

(PDF download)

Appendix 3 EMBASE and MEDLINE searches

Appendix 4 Data extraction and quality assessment checklist

Appendix 5 Risk of bias

Appendix 6 Demographic, anthropometric and lifestyle profile of the diabetic population by study

Appendix 7 Diabetes and comorbidities by study

Appendix 8 Foot measurements by study

Appendix 9 Full data variable dictionary

The completed list of variables available in the data set has been produced automatically from each data set with SAS.

The list of variables for Boyko et al. 49 has been provided from the author in a different format because the data set is not available to use externally.