Prognostic models for identifying risk of poor outcome in people with acute ankle sprains: the SPRAINED development and external validation study

Incidence and costs

Ankle sprains are one of the most common musculoskeletal injuries. Between 3% and 5% of people who attend an emergency department (ED) in the UK do so as a result of sustaining a sprained ankle. 1 The vast majority of sprains are of the lateral (outside) ligaments, and vary from minor stretching (grade 1) to a complete tear (grade 3). 2 Recent systematic reviews3,4 conclude that ≈30% of people still have problems with their injury 1 year after an ankle sprain, depending on the outcome measured and, perhaps more importantly, the sampling frame. Many studies are restrictive in their sampling frame, either concentrating on elite athletes or excluding younger and older people. Studies also have variable inception and follow-up points, which further complicates interpretation. A large multicentre randomised controlled trial (RCT) conducted in EDs in the UK reported an estimated prevalence of poor outcome of 30% at 9 months. 5 Other studies agree that recovery plateaus at around 9 months, and that residual disability after this point is likely to be persistent. 6 One potential consequence of ankle sprain, chronic ankle instability (CAI), is implicated in the development of ankle osteoarthritis, even without an acute osteochondral lesion. 7

Usual clinical pathway

Assessment of the injury in the acute phase is challenging as the ankle is often so swollen and painful that it cannot easily be examined. Most people are advised to rest, to elevate the ankle and to apply ice and compression; crutches are often issued if bearing weight is difficult. The Ottawa guidance8 can be used to reduce the requirement for imaging without missing significant fractures. If clinicians are concerned about the degree of injury, most health-care providers operate a system of review within 1 week in a trauma clinic or equivalent injury service. This time frame allows some resolution of swelling, and greater certainty in ascertainment of injury severity and presence of other significant mechanical derangement. 9 Treatment options at this stage include further watchful waiting, diagnostics, intensive physiotherapy and immobilisation. Surgery may be considered at this stage, although most centres would initiate a test of conservative management first. We have previously published a survey of practice,1 which remains a reasonable reflection of current management in the UK.

Study Steering Committee

The Study Steering Committee (SSC) provided overall supervision of the study on the behalf of the funder and was chaired by an independent member. The SSC abided by the OCTRU Standard Operating Procedure (accredited by the UK Clinical Research Collaboration Clinical Trials Unit registration process) and SSC charter. The SSC monitored study progress and advised on scientific credibility.

Study Management Group

The SMG was made up of SPRAINED study investigators and staff working on the project within OCTRU and the Critical Care, Trauma and Rehabilitation Trials Group. This group oversaw the day-to-day running of the trial and met regularly.

Search strategy

Searches of the following electronic databases were conducted from inception to September 2016: Allied and Complementary Database (AMED), EMBASE, PsycINFO (via Ovid), Cumulative Index to Nursing and Allied Health Literature (CINAHL) and SPORTDiscus (via EBSCOhost), PubMed and Cochrane Register of Clinical Trials. Relevant medical subject heading (MeSH) terms were used when appropriate in these databases. Search strings containing terms for the health condition or body region were used in Physiotherapy Evidence Database (PEDro), International Foot and Ankle Biomechanics, International Ankle Symposium and OpenGrey. No language restrictions were applied and the reference lists of included studies were screened for potentially relevant studies. The search strategy is available in Appendix 2.

Eligibility criteria

Studies were eligible for inclusion if they had all of the following factors:

• a sample, or a separately analysed subgroup, with a clinical diagnosis of acute (≤ 7 days from injury to assessment) lateral ankle ligament sprain

• a longitudinal design, with at least one follow-up time point

• statistical assessment of at least one baseline prognostic factor on recovery outcomes.

Excluded studies were those that included participants with ankle fracture (excluding flake fracture of < 2 mm) and other recent (< 3 months since injury) lower limb injuries.

Data extraction

Titles and abstract were screened by two members of the review team (JT, CB or MAW). The Ouzzani et al. 28 systematic review web application was used to manage screening. Full-text articles for potentially eligible records were independently reviewed by two of three reviewers (JT, CB or MAW). Data extraction and risk-of-bias assessments were completed independently by two reviewers (JT and CB). Discrepancies between reviewers decisions were resolved by discussion, or in consultation with a third reviewer (MMS or DJK).

Risk-of-bias assessment

Study quality was assessed using the Quality In Prognosis Studies (QUIPS) tool,29 which considers the six following domains of validity and risk of bias in prognostic factor studies:

1. study participation

2. study attrition

3. prognostic factor measurement

4. confounding measurement and account

5. outcome measurement

6. analysis and reporting.

Data synthesis and reporting

A narrative synthesis was conducted, meta-analysis being considered inappropriate because of heterogeneity in the prognostic factors, outcome measures and follow-up durations and limited number of studies. Follow-up time points from injury were grouped as short term (≤ 8 weeks), medium term (≤ 4 months) and long term (> 4 months).

Study characteristics

Table 2 illustrates the characteristics of the nine included studies. Six studies30–33,35,37 were prospective cohorts and three15,34,36 were retrospective analyses of RCTs. Three studies were based in the Netherlands,30,34,35 three in the USA,31,32,37 and one each in England,15 Northern Ireland36 and Germany. 33 The median participant sample size was 33 (range 20–553 participants), and follow-up data ranged from 1 day to 12 months after injury. Three studies31,32,37 recruited high school or university athletes; the remainder were based in primary or secondary care.

Risk-of-bias assessment

Table 3 shows the outcome of the risk-of-bias assessments. One study was judged as being at a low risk of bias,15 five were judged as being at a moderate risk of bias,30,32,34,36,37 and three studies were judged as being at a high risk of bias. 31,33,35 Incomplete and/or inadequate reporting standards were common issues; for example, it was difficult to identify whether prognostic factors were eliminated because of statistical reasons or poor clinical utility. No studies reported on the performance of the prognostic models by using methods to assess internal or external validation.

Prognostic factors identified

Prognostic factors included in the final models for each included study are shown in Tables 4 (short term), 5 (medium term) and 6 (long term).

Limitations of ankle sprain prognostic factor studies

In the majority of the included studies, follow-up was only short term, and was discontinued at a time when symptoms were still prominent and resolving, and hence recovery was quite variable. Methodological shortcomings were evident across the studies, for example, none reported an assessment of interval validity or attempted an external validation of its models. Adjustments for confounding factors such as time since injury, were not employed. Regression analyses were often not reported in sufficient detail to identify whether prognostic factors were eliminated because of small sample size or poor clinical utility. Two studies30,35 dichotomised a continuous outcome measure. The cut-off points that were used were not well justified or prespecified.

The study15 judged as being of high quality tended to report conservative estimates of associations between predictors and outcome. However, a limited range of prognostic factors was investigated.

Although a wide range of prognostic factors have been investigated, the limitations of previous studies highlight the need for large-scale studies that employ robust prognostic research methods10 and adhere to recognised reporting guidelines. 25 The systematic review that we conducted did provide some evidence to inform the decision making processes within the consensus exercise.

Participants

We aimed to recruit a range of key stakeholders, including patient and public representatives, health-care professionals and clinical researchers, to represent a range of parties involved in ankle sprain care and research in the UK NHS. We invited 30 individuals to participate, including a variety of health-care professionals from across the UK who worked in ambulance services, general practice, radiology, emergency and trauma surgery departments, as well as clinical researchers. We also aimed to recruit patient and public representatives from the south central area of the UK who had experience of an ankle sprain or were able to represent an individual or group that had such experience. We placed adverts for patient and public representatives in local supermarkets, in the John Radcliffe Hospital, on the People in Research21 website and the NIHR Research Design Service South Central’s mailing list.

Facilitators

The SPRAINED study team facilitators were guided by a lead facilitator (KH) with experience in conducting mNGT processes in musculoskeletal research. 46 Additional facilitators were provided with a standardised brief to follow during the meeting and supervised by the lead facilitator.

Consensus process

We conducted the consensus process in three main stages, outlined in the following sections.

Participants

Of the 30 individuals invited, 25 clinicians and clinical researchers agreed to participate comprising: paramedics (n = 6), physiotherapists (n = 6), ED nurses (n = 4), ED consultants (n = 5), radiology consultant (n = 1), trauma and orthopaedic consultant (n = 1) and clinical researchers (n = 2). Three patient and public representatives responded to the advertisements, but only one was able to attend the consensus meeting. The pre-meeting electronic questionnaire was returned by 17 individuals; and 18 individuals attended the meeting and participated in the first two rounds of group discussions. Two participants were unable to complete the final round of individual voting. Hence, only 16 participants voted for the factors that had been prioritised throughout the day.

Pre-meeting questionnaire results

The results of the electronic pre-meeting questionnaire are shown in Table 8. Three baseline factors were rated as critically important (scoring between 7 and 9) and the remainder as important but not critical (scoring between 4 and 6). The respondents nominated 20 additional factors that were deemed critically important. There was a varied response to when and how delayed prognostic factors should be collected. The most frequent preferences were 4 weeks post injury and by telephone.

Consensus meeting results

Eighteen participants, divided into three groups, participated in the two rounds of facilitated discussions and prioritisation exercises. Some groups were unable to agree on or did not use the maximum number of ranks. Priority rankings of the prognostic factors rated by the three groups of key stakeholders are shown in Table 9. The prognostic factors of the highest priority included repeatedly spraining ankle previously, older age and mechanism of injury. Only 6 of the 20 additional factors nominated in the pre-meeting questionnaire were deemed high priority for inclusion in the prognostic model: (1) occult fracture/diagnostic imaging result, (2) history of chronic pain/problems, (3) desire to get better, (4) psychosocial factors about recovery, (5) weight-bearing status immediately post injury and (6) self-efficacy. Following the facilitated discussions, 16 participants completed the final vote for which factors to include in the prognostic model. Participants agreed to include 8 of the 14 originally proposed prognostic factors in the prognostic model: (1) pain intensity, (2) pain intensity on weight bearing, (3) weight-bearing status in the ED, (4) age, (5) body mass index (BMI), (6) working status, (7) mechanism of injury and (8) repeatedly sprained ankle previously. Only one additional factor nominated from the pre-meeting questionnaire was agreed on for inclusion in the prognostic model – psychosocial recovery factors (see Table 9). No delayed factors were agreed on for inclusion in the prognostic model.

Individual participant data used to develop the models (study population)

CAST was a pragmatic, multicentre RCT, with blinded assessment of the outcome, designed to estimate the clinical effectiveness and cost-effectiveness of three different types of mechanical ankle support [Aircast^® ankle brace (DJO Incorporated, Vista, CA, USA), Bledsoe^® boot (Bledsoe Boot Systems, Grand Prairie, TX, USA) or 10-day below-knee cast] in the treatment of severe ankle sprain (defined as an injury of grade 2 or 3, without fracture) compared with a double-layer tubular compression bandage.

The trial population comprised 584 individuals aged ≥ 16 years attending EDs in the UK with an ankle sprain and an inability to fully bear weight on the injured ankle at the time of presentation to the ED and their review clinic appointment (the trial’s baseline assessment). People were excluded if they presented with an ankle fracture (apart from flake fractures of ≤ 2 mm), any other recent fracture, any contraindication to any of the four arms of the trial, poor skin viability preventing splinting or casting, or if their injury occurred > 7 days before the first presentation at the recruiting ED.

The different time points in CAST and a summary of the data collected at each point are defined in Table 10.

Definition of the primary outcomes

Ankle function at 9 months after ankle sprain was the primary outcome for CAST. For the SPRAINED study, our primary outcome was ‘poor outcome’. We used two definitions of poor outcome that were based on key indicators of poor function and instability of the joint, which is typified by recurrent sprains or a significant lack of confidence in the ankle (a persistent feeling of giving way), with or without chronic pain. The selection of these outcome indicators is supported by evidence from van Rijn et al. ,52 who reported that recovery was most closely associated with improvements in pain and giving way, and Wikstrom et al. ,7 according to whom pain and instability are of greatest concern to patients. The definitions were considered and agreed by the patient and public involvement group convened for the SPRAINED study.

Data to classify these outcomes were collected in the CAST data set as outlined in the following sections.

Available candidate predictors and initial selection of variables for modelling

A complete list of the 16 available variables in the ED pro forma is provided in Box 1, and a complete list of the 154 available variables in the CAST baseline data set is provided in Box 2. Variables available in the CAST data set included sociodemographic indicators (age, sex, BMI, education, employment status); pre-injury quality of life, mobility and lifestyle (e.g. engagement in sports activities); clinical data on injury presentation; and indicators of current mobility levels, pain and weight-bearing status. From these lists, 32 variables were preselected to form the group of candidate predictors considered to be plausibly predictive of either of the two outcomes. This initial selection was made internally by the research team, taking into account the results from the systematic review (see Chapter 3) and the conclusions from the consensus group meeting (see Chapter 4). The preselected candidate predictor variables and their details (type, name, categories or units, questionnaire in which the data were originally recorded and number of missing data) are listed in Table 12.

In addition to the baseline predictors, a few variables from the CAST 4-week follow-up questionnaire were selected to be investigated as potential predictors that could add some incremental value to the developed prognostic models. The list of these variables and their characteristics are listed in Table 13.

Development data set preparation

The CAST data set contains individual participant information at baseline and at follow-up assessments (time points 2 and 3, as described in Table 10) for 584 participants recruited to take part in the study. To include the data collected at the time of ED presentation (time point 1, as described in Table 10) in the analysis, it was necessary to merge the CAST main data set with a separate data set that included information on 1487 people screened during the recruitment period of the trial. The information collected at this time point was anonymised; consequently, the data set has no information on the participants’ identification number. Information from these two data sets was merged by matching the cases using the individuals’ information on date of birth and sex; any duplicates in each data set were disregarded to avoid mismatching.

This process added information on five of the candidate predictors collected during the first contact with the participants to 289 cases in the CAST main data set (see Box 1 for details on the predictors collected at the time of ED presentation). As the results of the trial have been published and all documentation archived, there was no need for further data cleaning and the only data manipulation performed with the resulting data set was generating new variables from existing variables or recategorisation of existing variables (see Available candidate predictors and initial selection of variables for modelling and Exploratory analysis and data transformation) and missing data imputation (see Handling missing data).

Exploratory analysis and data transformation

Baseline and 4-week follow-up characteristics of the participants in CAST were summarised using means, standard deviations (SDs) and ranges for continuous variables, or counts and percentages for categorical variables. After merging the data sets, three variables from the ED data set had > 60% missing information (anterior drawer test, talar tilt test and proximal fibular tender ligament test) and were excluded from the list of candidate predictors (see Table 12 for detailed information on the number of missing data for each candidate predictor). It was also discussed and agreed during the consensus group meeting (see Chapter 4) that it would be reasonable to exclude these variables from the pool of candidate predictors because of the variability in technique between assessors when performing the tests.

Each binary or categorical predictor was tabulated against the outcomes to check for empty or low cell counts. When this was the case, categorical variables were recategorised by collapsing some of their categories, providing it made clinical sense to do so. The manipulated variables with details on the changes performed are presented in Table 14. When collapsing categories would not solve the problem (or make clinical sense), the predictor variable was omitted from any further analysis. This was the case for the following candidate predictors: ‘ankle/foot swelling’ (from baseline assessment) and ‘returned to ED because of repeat injury’ (from 4-week follow-up questionnaire).

The distribution of continuous predictors was also assessed: first, by considering the predictors’ empirical distributions by producing histograms, and then by assessing these for normality by means of normal probability plots. The presence of any outliers was assessed based on visual examination of box plots. Extreme values were inspected to confirm whether or not they were clinically plausible. No individual participant information was deleted from the data set and data transformation (normalisation) was performed as appropriate.

The correlations between candidate predictors were also examined using Spearman’s rank-correlation coefficient to identify any highly correlated predictors (r ≥ 0.8). It causes unnecessary complication to include highly correlated predictors together in multivariable models. Highly correlated predictors explain the same variation in outcome, and this was found for two groups of variables: (1) ‘difficulty with running’, ‘difficulty with jumping’ and ‘difficulty with twisting/pivoting’ (from both baseline and 4-week follow-up) and (2) ‘previous instability’ and ‘previous instability frequency’ (from baseline).

To deal with the first group of correlated variables, a new binary variable (yes or no) was created to indicate whether or not a participant presented a positive answer to any of the original variables; these individuals were characterised as presenting difficulty with running, jumping or twisting. This new composite variable was then used instead of the three highly correlated variables in the remaining analyses. The decision about which predictor should be taken to the modelling stage, between previous instability and previous instability frequency, took into account the individual predictive ability of each variable. The predictor with lower face validity for outcomes 1 and 2 (‘previous instability’ in both cases) was then omitted from subsequent analyses.

Initial individual associations between each candidate predictor and poor recovery at 9 months after ankle sprain were performed by fitting unadjusted logistic regression models for outcomes 1 and 2.

Handling missing data

Some missing data in the development data set occurred as a result of missed appointments and losses to follow-up during the conduct of CAST, but also because of the lack of a unique patient identification in the trial’s screening (ED presentation) data set, which did not allow all the information collected at this point to be merged with the main CAST data set (see Development data set preparation for further details). The percentage of missing data in the final merged data set is presented for each candidate prognostic variable in Tables 12 and 13. To conform to current guidelines, multiple imputation for all participants with at least one missing value was performed. 53 Since there were several predictor variables of different types (i.e. binary, categorical and continuous) with missing data, multiple imputation by chained equations (MICE) was carried out using the mi impute chained function in Stata^® version 14.2 (StataCorp, College Station, TX, USA) with the options logit (for imputation of binary variables), mlogit (for imputation of categorical variables) and truncreg (for imputation of continuous variables, setting the lower and upper limits for imputed values as 0 and 100, respectively).

In MICE, all missing values are filled in by simple random sampling with replacement from the observed values to allow the regression models to be fitted on all values. Then, the variable with the lowest number of missing observations, for example x₁, is regressed on all other variables. Missing values are then replaced by drawing from the estimated corresponding posterior predictive distribution of x₁. Then, the next variable with the lowest number of missing observations is regressed on all other variables including (and using the imputed values of) x₁. This process is repeated until all variables with missing values are imputed, forming one cycle. Cycles are repeated to stabilise the results and the whole procedure is repeated m times to give m imputed data sets. An important characteristic of MICE is the capacity of handling different variable types (continuous, binary, unordered and ordered categorical) because each variable is imputed using its own imputation model using different types of regression analysis.

Multiple imputation was performed under the assumption that all missing data were missing at random (MAR). In other words, the probability of data being missing does not depend on the unobserved data, conditional on the observed data. Therefore, imputation models included all available observed characteristics for the predictors of interest (both at baseline and at the 4-week follow-up), predictors of predictors (e.g. weight and height for BMI) and the outcomes, as recommended by White et al. 53 The models were independently estimated for outcomes 1 and 2, and imputations were therefore performed in separate procedures, producing two different sets of 50 complete data sets. This number of imputed data sets was chosen based on the number of missing data for the variable with the highest rate of missing observations (312/584 for ‘days from injury to assessment’). No data transformation was performed on continuous predictor variables before imputing missing observations.

Despite using the augmented-regression approach,54 some predictors were also excluded during this process because of the issue of ‘perfect prediction’ when imputing categorical variables. 55 Perfect prediction occurs whenever there is a level of a categorical explanatory variable for which the observed values of the outcome are all 1 (or all 0). Perfect prediction then leads to infinite coefficients with infinite standard errors and causes instability during estimation, which prevents the imputation model from achieving convergence. We resolved this issue by dropping the predictors causing the perfect prediction from the multiple imputation model: two from baseline [(1) ‘difficulty with running, jumping or twisting’ and (2) ‘previous instability frequency’] and one from 4-week follow-up [‘able to perform ankle range of motion plantarflexion’]. A complete list of predictors that were excluded before the modelling process, with reasons for exclusion, is provided in Table 15.

Sample size considerations

Sample size requirements for logistic regression are based on the concept of events per variable (EPV). It is widely recommended that, to develop a prediction model, the data set should contain a minimum of 5–10 EPV. 56–61 Based on a number of at least five EPV, the outcome rates (see Table 11) observed in the CAST data set allowed the inclusion of 23 (116/5) and 28 (140/5) candidate predictor variables in the models for outcomes 1 and 2, respectively. After the exclusion of nine preselected candidate predictors for the reasons described in Table 15, 23 variables from baseline remained as candidate predictors. However, some of these predictors were categorical variables with more than two levels, which affects the EPV as these predictors require the generation of indicator variables for each category (e.g. employment status coded as ‘no’, ‘part time’ or ‘full time’ will require three parameters to be estimated). Therefore, we ended with 35 candidate parameters, which means that the EPV ratio was approximately 3 and 4 for outcomes 1 and 2, respectively. It is also important to note that some of the candidate predictors were continuous variables, which could require non-linear modelling and therefore increase even more the number of regression coefficients to be estimated and affect the EPV (e.g. if using fractional polynomials and the best transformation for age was found to be age + age2, then age would relate to two predictors instead of one). However, this was not the case.

To the best of our knowledge, this was the first project aiming to develop prediction models to assess the risk of poor recovery after an acute ankle sprain. Therefore, we have opted for relaxing the EPV rule in favour of including more potentially important predictors in the analyses. However, we have adopted several strategies to minimise bias and overfitting, including the estimation of heuristic shrinkage factors to account for possible extreme predictions resulting from overestimated associations (see Data modelling, Model update, Assessment of model performance and Shrinkage).

Data modelling

Since both outcomes were binary (poor outcome after ankle sprain – yes/no), the prognostic models were developed using a logistic regression modelling framework with the logit probability of poor outcome as the response variable. The 23 remaining candidate predictors were included together in full logistic regression models as independent variables, and further selection of predictors was based on the statistical significance of their adjusted relationship with the outcomes. At this point, continuous variables were kept as continuous to avoid loss of prognostic information. 62 Therefore, the shape of the relationship between continuous predictors and the outcome should be studied and modelling performed with non-linear functions, such as fractional polynomials, when appropriate. 63

Non-linear relationships were investigated using fractional polynomials and the ‘best transformation’ for each continuous predictor was used when fitting the models. As more than one continuous variable was included in the full models, the multivariable fractional polynomial (MFP) algorithm was used. 64,65 The MFP algorithm selects predictors and their transformations that best predict the outcome variable using a backward selection process. A nominal alpha of 0.15 was used to warrant exclusion from the model to reduce the risk of overfitting. Another advantage of the MFP algorithm is that selection of predictors and transformations is done simultaneously, preserving the nominal type 1 statistical error probability.

As the analyses were performed in sets of 50 multiply imputed data sets; the MFP algorithm was applied using the Stata command mfpmi together with logit. The mfpmi allows binary, ordinal and non-ordinal categorical variables to be included alongside continuous variables in the same model, and simultaneously select the appropriate fractional polynomial transformation of continuous predictors combining the estimates of multiply imputed data sets. The multivariable models were fitted in each of the 50 complete data sets and the estimated regression parameters (coefficients and variances) were combined using Rubin’s rule. 66,67

Ideally, prognostic models should be flexible, easy to understand and parsimonious, so that they are simple and quick to apply in clinical practice. Therefore, after identifying the best transformation terms for continuous variables in the full multivariable models with all candidate predictors, the statistically significant predictors (and the corresponding transformations of continuous variables, when applicable) were selected using the AIC as the decision rule and kept in the final model. 68 Therefore, a p-value of < 0.157 (equivalent to AIC) was conservatively taken to warrant inclusion of predictors in the final model and to reduce the risk of overfitting.

Model update

After developing the prognostic models for outcomes 1 and 2 including only predictors collected at baseline (baseline variables), the additional incremental value of candidate predictors collected at the 4-week follow-up point were investigated. First, all additional candidate predictors were included together in the final baseline models and only those predictors achieving p < 0.157 (AIC) were considered for inclusion in the updated models (i.e. prognostic models including baseline + 4-week predictors). Finally, these updated models were compared with the original baseline models by DCA plots69,70 to investigate whether or not the inclusion of additional predictors was reflected in an increased net benefit. The DCA was performed in Stata, using the command dca.

Assessment of model performance

After developing a prognostic model, it is important to evaluate its performance. Table 16 provides an overview of the main ways in which model performance can be assessed from Thangaratinam et al. 71

The performance of the prognostic models was characterised by evaluating calibration and discrimination.

Shrinkage

Newly developed prognostic models are often optimistic as a result of overfitting, which leads to worse prediction in independent data. Reasons for overfitting include small EPV, the selection of predictors based on p-values and modelling non–linear relationships between predictors and the outcome. To estimate the amount of overfitting likely to be present in the developed prognostic models, heuristic shrinkage factors were calculated independently for each model as:

where model χ² is the model likelihood ratio, or –2log-likelihood of a model with only an intercept and the fitted model, and df is the number of degrees of freedom in the fitted model. The number of degrees of freedom in the fitted model is defined by the number of degrees of freedom considered for all explored candidate predictors, plus all corresponding transformations, when applicable.

A shrinkage factor of 1 implies no shrinkage. The regression coefficients from the prognostic models were multiplied by the shrinkage factor to adjust the models for optimism. The shrinkage of the intercept was estimated by fitting a logistic regression model for each studied outcome, including the linear predictor (log-odds) calculated using the shrunk coefficients as the only independent variable, and constraining its coefficient to one (offset variable). 75

Baseline characteristics

The baseline characteristics of the participants in the CAST data set are summarised in Table 17. Participants were aged 29.88 years, on average, with the age range varying from 16 to 72 years. Participants had a mean BMI of 26.34 kg/m² and lower pain sores when resting (mean 37.75/100 points) than when bearing weight on the injured ankle (mean 75.42/100 points); < 25% of participants reported not being able to bear any weight on their ankles at the time of baseline assessment. Most participants reported not feeling pain in the ankle before the injury (86.56%) and not seeking treatment for a recurrent sprain (90.38%). Most participants were in full-time employment (61.64%), had an education level higher than General Certificate of Secondary Education (GCSE) (84.98%) and fewer than one-quarter engaged in any leisure-time physical activity more than once a week (24.09%). Among the CAST participants, injuries occurred mostly during the practice of sports (36.91%).

All continuous variables presented at least a minimal departure from a normal distribution, as evidenced in Figures 4–10. Some outliers were observed for participants’ age, weight, BMI and pain score when bearing weight. However, all extreme values were clinically plausible, so no observations were dismissed.

FIGURE 4.

Participants’ age. (a) Histogram; and (b) box plot.

FIGURE 5.

Participants’ height. (a) Histogram; and (b) box plot.

FIGURE 6.

Participants’ weight. (a) Histogram; and (b) box plot.

FIGURE 7.

Participants’ BMI. (a) Histogram; and (b) box plot.

FIGURE 8.

Pain score when resting at ED presentation. (a) Histogram; and (b) box plot.

FIGURE 9.

Pain score when bearing weight at baseline assessment. (a) Histogram; and (b) box plot. VAS, visual analogue scale.

FIGURE 10.

Short Form questionnaire-12 items (SF-12) mental component score. (a) Histogram; and (b) box plot.

Spearman’s rank-correlation coefficients of the baseline predictors are presented in Table 18. Highly correlated candidate predictors included ‘difficulty with running’ and ‘difficulty with jumping’ (r = 1.000); ‘difficulty with running’ and ‘difficulty with twisting/pivoting’ (r = 0.859); ‘difficulty with jumping’ and ‘difficulty with twisting/pivoting’ (r = 0.859); and ‘previous instability’ and ‘previous instability frequency’ (r = 0.997). As these variables should not be included together in the regression models, the first set of highly correlated variables was combined into a single composite variable to identify those participants with difficulties in running, jumping or twisting/pivoting. For the second pair of highly correlated variables, previous instability frequency only was included in the subsequent analysis.

Multivariable models

The summary of the full multivariable model estimates (predictor coefficients, 95% CIs and p-values) is presented in Table 19. For outcome 1, 7 of the 23 candidate predictors were selected for inclusion in the final model, based on the AIC (p < 0.157): (1) age, (2) BMI, (3) pain when resting, (4) pain when bearing weight, (5) number of days from injury to assessment, (6) ability to bear weight and (7) whether or not the injury was a recurrent sprain. For the outcome 2, almost the same set of candidate predictors were selected for inclusion in the final model, except for age and BMI. For outcome 2 educational level was found to be a statistically important candidate predictor. However, education was identified as a low-priority variable by the consensus committee. There were particular difficulties with this variable, as the criteria used in the CAST study to identify different education achievements have been superseded, and in the interim, a number of new additional categories of study have become more popular (for example University of the Third Age). Given the marginal statistical significance, inability to replicate the categories in an external validation, low priority given in the consensus, and the reluctance of clinicians to probe this information, we did not include this variable in the final model for outcome 2.

The best fit for all continuous predictors was found to be linear transformations (mean subtractions), which were incorporated into the model by updating the intercepts accordingly. A summary of the estimates from the final multivariable models (predictor coefficients, 95% CIs and p-values) is presented in Table 20. For outcome 1, BMI was not statistically significant according to AIC in the final model. Nevertheless, it was decided not to exclude this variable from the model, given its clinical importance, and to reduce the risk of overfitting. Both models were fairly simple, composed of just a few predictors that are routinely collected in the clinical setting.

Only pain when bearing weight at 4 weeks after the injury was included in the updated models (baseline + week 4 predictors) for both outcomes 1 and 2 (Table 21). By inspecting the DCA plots shown in Figures 11 and 12, it is possible to see a clear net benefit gain over the entire range of thresholds when using any of the developed prognostic models in comparison to considering all patients (or no patient) at risk of having poor outcome after an acute ankle sprain. Furthermore, the inclusion of the week 4 predictor (pain when bearing weight) consistently improved the performance of the models for both outcomes 1 and 2.

FIGURE 11.

Decision curve analysis plot for outcome 1. Adapted with permission from Schlüssel et al. 26 © Author(s) (or their employer(s)) 2018. Re-use permitted under CC BY. Published by BMJ. This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

Adapted with permission from Schlüssel et al. 26 © Author(s) (or their employer(s)) 2018. Re-use permitted under CC BY. Published by BMJ. This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

FIGURE 12.

Decision curve analysis plot for outcome 2.

Model performance

Model performance was assessed in terms of calibration and discrimination. The overall discriminatory ability (apparent performance) was 0.82 (95% CI 0.75 to 0.89) for the model developed to predict outcome 1 and 0.73 (95% CI 0.66 to 0.81) for the model developed to predict outcome 2, as measured by the c-statistic estimated after regressing the predictors selected for the final model against the outcomes using the original CAST data set (complete-case analysis, n = 194 and n = 200 for outcomes 1 and 2, respectively). The combined results from the analysis of the 50 imputed data sets provided a less optimistic measure of the discriminatory ability for the two models. For the model developed to predict outcome 1, the combined c-statistic was 0.74 (95% CI 0.70 to 0.79). For the model developed to predict outcome 2, the combined c-statistic was 0.70 (95% CI 0.65 to 0.74). The addition of one variable with information on pain when bearing weight on the ankle at 4 weeks after the injury improved the discriminatory ability and apparent calibration of both models. For the updated model to predict outcome 1, the c-statistic was 0.77 (95% CI 0.73 to 0.82). For the updated model to predict outcome 2, the c-statistic was 0.75 (95% CI 0.71 to 0.80).

Calibration plots overlying the results of the analysis on the 50 imputed data sets are presented in Figures 13 and 14. Perfect predictions should lie on the 45° line in the calibration plot for agreement with the outcome. As anticipated, on average, the calibration across all models was consistently strong, with close agreement between the observed and predicted risks of developing outcomes 1 (Figure 13) and 2 (Figure 14). Shrinkage suggested both prognostic models to be unstable, with a considerable amount of optimism. The heuristic shrinkage factor for the coefficients of the predictors in the baseline prognostic model for outcome 1 was 0.71, suggesting that 29% of the model fit was non-replicable noise. For the updated versions (baseline and week 4 predictors) of both prognostic models, the estimated heuristic shrinkage factor was 0.84. The shrunk coefficients and intercepts are presented in Table 22.

FIGURE 13.

Calibration plot of the prognostic model for outcome 1. (a) The baseline model; and (b) the updated (baseline plus 4 weeks) model.

FIGURE 14.

Calibration plot of the prognostic model for outcome 2. (a) The baseline model; and (b) the updated (baseline plus 4 weeks) model.

Application of the SPRAINED study model

The following section will provide an example of how the internally validated SPRAINED study model can be applied in practice. To make predictions with the SPRAINED prognostic models, the following equations are required (please note that all linear terms selected by the MFP for continuous predictors were incorporated into the models’ intercepts).

Cohort design and study population

People with acute ankle sprain attending 10 NHS EDs across England were recruited for the SPRAINED cohort study (see Acknowledgements for details on recruiting centres) over a period of 9 months (July 2015–March 2016). This was an observational cohort study; therefore, participants were not randomised, nor did they receive any interventions other than usual care at each site. Data collection took place at the time of a participant’s presentation to any of the study recruiting sites (baseline) and subsequently at 4 weeks and 4 and 9 months after the initial injury.

People were invited to take part in the study if they met the following inclusion criteria and none of the exclusion criteria.

Sample size

The recommended sample size estimation for an external validation of a prognostic model is that 100 outcome events are required, this being the minimum number needed to ensure accurate estimation of the calibration of the model. 72,76 The event rates for the outcomes of interest in CAST were between 26% and 32%, depending on the definition of the outcomes (three symptoms and four symptoms/clinical events, respectively), this would require an overall sample size of between 313 and 385 participants. Assuming a rate of 25% for loss to follow-up and a lower event rate (20%) when recruiting all grades of ankles sprains, a minimum of 675 participants were targeted for recruitment to increase the chances of achieving the required event rates. We anticipated recruiting people with a range of sprains, including grades I to III.

Screening and eligibility assessment

People were screened by clinicians on admission to EDs and assessed for eligibility to take part in the SPRAINED cohort study. A member of the research team at the study centres administered the study clinical data set form (CDF) and recorded responses and findings from the clinical examination (see Appendix 4). The short CDF served three purposes:

1. collection of routine core clinical data set in a tick-box format (reflecting the data that would be normally recorded in the course of routine clinical practice)

2. to record, via a tick box, that clinicians had provided potential participants with the trial information pack and a brief explanation of the trial

3. to record, via a tick box, whether or not the individual had given permission for a member of the research team to make contact with him/her to discuss the study further and complete the informed consent process.

One copy of the CDF was filed in the person’s medical notes as a treatment record and a second copy, when agreement was given, was passed to the local research team. The team member then contacted the individual and continued the informed consent process. Only once consent was obtained was the clinical data set sent to the central study office. The clinical data set of any person who did not agree to study participation remained at the site in his/her medical notes.

Informed consent and recruitment

The initial approach was made by a member of the ED clinical team. A verbal explanation of the study, along with a study information leaflet, was given to all potentially eligible people. Posters were displayed in all participating departments to inform participants that the study was occurring.

The informed consent process was carried out by a registered health-care professional with delegated authority from the principal investigator at the recruiting site. Before consenting to participate in the study, the person was asked by a member of the local clinical team for permission to allow the local research team to speak to them, either in person or by telephone, to take forward the informed consent process. Formal consent to participation was provided either in person, by post or by telephone. Before any data were provided to the study team, the participant personally signed and dated the latest approved version of the informed consent form (ICF), or verbal consent was recorded by a member of the local team on a form during the informed consent telephone call. The participant had the opportunity to question the clinical/research team, and to consult their general practitioner (GP) or other independent parties to decide whether or not they would participate in the study.

Written informed consent was obtained by means of participant-dated signature and dated signature of the person who presented and obtained the informed consent. Verbal informed consent was obtained by means of the dated signature of the local team member taking consent over the telephone. A copy of the completed written or verbal ICF was retained by the participant (or posted to the participant in the case of oral consent). One copy was sent to the study co-ordinating team in Oxford. The original signed consent form was retained in the medical notes, and a copy was held in the investigator site file.

Participants consented to allow the study team to use the CDF completed during the ED attendance and an additional questionnaire 4 weeks after this (SPRAINED prognostic model and any additional important information), as well as follow-up questionnaires at 4 and 9 months, which aimed to map the recovery trajectory and final recovery status at 9 months. A questionnaire at 4 months served as a reminder of the study and, as loss to follow-up was likely to increase over time, helped to ensure that responses on the core components of the outcomes of interest were available for as many participants as possible.

Data collection and management

Baseline data were collected from participants and recorded on a paper CDF. Data for the three study follow-up points [4 weeks (prognostic variables) and 4 and 9 months (outcome data) after baseline assessment] were collected by using paper case report forms (CRFs) sent to participants via post, or completed by telephone call when necessary. The telephone calls enabled collection of at least the core data on the outcome measures for participants that did not return the questionnaire to the trial office. When preferred by the participant, secure online data collection took place for the 4-week time frame.

Baseline CDFs were sent by a member of the local research team to the study co-ordinating office in Oxford by post. Follow-up CRFs were sent by the participant to the study co-ordinating office in Oxford by post, using a Freepost return envelope. When telephone follow-up was used, a member of the central study team recorded data directly onto the relevant forms.

On receipt of data forms (CDFs and CRFs), appropriate data quality and validation checks were carried out and the data were entered into a study-dedicated database, which was developed and maintained by OCTRU, a UK Clinical Research Network (UKCRN)-registered clinical trials unit. OpenClinica software (OpenClinica LLC, Waltham, MA, USA) was used to develop and maintain the study database. To identify manual entry errors, a 10% double-entry check was carried out at regular intervals during the data collection phase of the study.

Details relating to ethics approvals and monitoring are outlined in Ethics approval and monitoring.

Study assessments

Outcome measures

For the external validation data set (SPRAINED observational cohort study), poor outcome at 9 months after ankle sprain was defined in the same way as it was in the development study (see Chapter 5). The same questions were asked to SPRAINED study participants, so the same two outcomes could be constituted. Therefore, the definition of poor outcome was the presence of any, or a combination, of the following symptoms or clinical events (for further details see Definition of the primary outcomes).

Predictors of poor outcome at 9 months after ankle sprain

All variables included in the prognostic models developed to predict the occurrence of poor outcome at 9 months after ankle sprain (the SPRAINED prognostic models, see Chapter 5) were included in the baseline CRFs. Data collection on a few additional candidate predictors that were not included in the final models was also conducted to allow some room for model updating, if necessary. However, the data collected at baseline were kept to a minimum, prioritising the predictors included in the two developed models and those candidate predictors that the consensus group considered to have the most clinical importance and relevance to patients. Except for pain scores (collected as discrete variables in the SPRAINED cohort study), data collection on all variables was performed respecting their original format in the CAST data set. A complete list of the variables collected at baseline in both CAST and the SPRAINED cohort study, with formats and number of missing data, is given in Table 24.

Statistical methods

Exploratory analysis

The study recruited a cohort of 682 participants across 10 EDs between 20 July 2015 and 17 March 2016. The flow of participants through the cohort study is detailed in Figure 15. Baseline characteristics of the SPRAINED observational cohort study participants are summarised in Table 25. On average, participants were slightly older in the SPRAINED cohort study than in CAST (33.62 years vs. 29.88 years, respectively). Participants in the SPRAINED cohort study had an average BMI in the overweight category (27.08 kg/m²), similar to the CAST participants (26.34 kg/m²). The mean pain scores when resting (38.5 points) or bearing weight on the ankle (71.3 points) of the SPRAINED cohort study participants were also very similar to those observed for the CAST participants (37.75 points when resting and 75.42 points when bearing weight). In contrast to CAST, in the SPRAINED cohort study about half of participants were female (52.05%), presented to an ED for assessment within 2 days of injury (90.03%) and were able to bear some weight on their injured ankles (73.56%).

FIGURE 15.

Flow of participants through the SPRAINED observational cohort study.

Continuous predictor variables presented at least a minimal departure from a normal distribution, as evidenced in Figures 16 and 17. Some outliers were observed for participant age and BMI. However, all extreme values were clinically plausible, so no observations were dismissed. Correlations between predictors are presented in Table 26, ranging from very low values (r = 0.011 for BMI and ability to bear weight on the injured ankle) to moderate values (r = 0.549 for pain when resting and pain when bearing weight), which did not raise concerns about including them together in a multivariable model.

FIGURE 16.

Distribution of age values in the SPRAINED cohort study data set. (a) Histogram; (b) normal plot; (c) box plot; and (d) dot plot.

FIGURE 17.

Distribution of BMI values in the SPRAINED cohort study data set. (a) Histogram; (b) normal plot; (c) box plot; and (d) dot plot.

Events rates in the SPRAINED cohort study and CAST data sets for both outcomes, and the number of symptoms, at 9 months after injury, are described in Table 27. There was a lower rate of poor outcome for the SPRAINED cohort than for the CAST cohort.

Model performance

The performance of the prediction models in the external validation data set (SPRAINED cohort study) was assessed in terms of calibration and discrimination. Calibration was graphically assessed with a calibration plot that showed calibration lines for each of the 50 imputed data sets, which was supplemented with the calibration slope and intercept. These parameters were first estimated with the original prognostic model, with poor outcome 9 months after ankle sprain (yes/no) as the outcome variable, and the linear predictor (log-odds) of the original prediction model (see Chapter 5, Application of the SPRAINED study model for the equation to calculate the linear predictor) as the only covariate.

Combined performance measures (by using Rubin’s rules) are presented in Table 28 and calibration plots overlaying the calibration lines from the 50 individual calibration plots are presented in Figures 18 and 19.

FIGURE 18.

Calibration plots for the baseline and updated prognostic models to predict outcome 1, overlaying the 50 calibration lines derived from the individual imputed SPRAINED data sets. (a) Baseline; and (b) updated (baseline + 4-week predictors). Adapted with permission from Schlüssel et al. 26 © Author(s) (or their employer(s)) 2018. Re-use permitted under CC BY. Published by BMJ. This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

Adapted with permission from Schlüssel et al. 26 © Author(s) (or their employer(s)) 2018. Re-use permitted under CC BY. Published by BMJ. This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

FIGURE 19.

Calibration plots for the baseline and updated prognostic models to predict outcome 2, overlaying the 50 calibration lines derived from the individual imputed SPRAINED data sets. (a) Baseline; and (b) updated (baseline + 4-week predictors).

Overall, discrimination of the models for outcome 1 stayed fairly stable when compared with the performance of the model in the development data set: combined c-statistic 0.72 (95% CI 0.66 to 0.79). For outcome 2, a decrease in the discriminatory ability was noted: c-statistic 0.63 (95% CI 0.58 to 0.69).

Calibration of the prognostic model in the external validation data set was poor for outcome 1, as can be evidenced by inspecting Figure 19 (a calibration plot with overlaid calibration lines from the 50 imputed data sets). Well-calibrated models should produce calibration lines lying on (or at least close to) the 45° dashed line of perfect prediction (observed proportion and predicted probability matching perfectly). In this scenario, the calibration slope would be equal (or very close) to 1 and the calibration intercept equal (or very close) to 0. The combined calibration slope was > 1 (1.13, 95% CI 0.76 to 1.50) and the calibration intercept was smaller than zero (–0.91, 95% CI –1.18 to –0.65).

A calibration slope of > 1 indicates that the regression coefficients of the original model were too close to zero, which was the case after the correction for optimism (shrinkage) of the model. A calibration intercept different from zero indicates that the model’s predicted probabilities in the validation data set are systematically too high (intercept < 0) or too low (intercept > 0).

For the prognostic model developed to predict outcome 2, calibration was better than for the model to predict outcome 1 in terms of the calibration intercept (–0.25, 95% CI –0.44 to –0.06), and slope (1.03, 95% CI 0.65 to 1.42) (see Table 28). The updated model (baseline + 4-week predictors) for outcome 1 presented a better discriminatory ability in the SPRAINED data set than the baseline model (c-statistic = 0.78, 95% CI 0.72 to 0.84), but not better calibration in terms of intercept (–0.62, 95% CI –0.89 to –0.34). The same was observed for the updated model for outcome 2 (better discrimination but worse calibration) (see Table 28).

Table 29 shows how many of 1000 people would be identified as being at high risk of developing the outcome (based on thresholds of 5%, 10%, 15% and 20%), using the developed prognostic models, and how many of these would actually present poor outcome 9 months after an acute ankle sprain. There seems to be little difference between the baseline and updated models for outcome 1, with both models identifying a similar number of patients who experience a poor outcome after ankle sprain. However, fewer patients are deemed as being at high risk by using the updated model for outcome 1 (fewer false positives) across all thresholds of predicted probability, as estimated by the prognostic models. For outcome 2, the updated model misses more patients who actually develop the outcome (false negatives) when compared with the baseline model. Using either of the models seems to be beneficial when compared with not using any model (or considering all patients as being at high risk of developing poor outcome).

Subgroup analyses

As the prognostic models were developed using a data set from a clinical trial that included only participants with moderate or severe injuries (grades 2 or 3), it was decided that separate results on the models’ performance would also be presented for a subgroup of participants classified according to their injury severity degree (grades 2 and 3).

Overall, both the calibration (intercepts and slopes) and discrimination (c-statistics) did not show any substantial improvement in the subgroup analysis for the baseline prognostic models to predict either outcome 1 or 2 (see Table 28). For the updated models (baseline + 4-week predictors), the intercept of the prognostic model to predict outcome 2 presented some improvement in terms of the calibration intercept, but not for the calibration slope (see Table 28).

Model recalibration

Before recalibrating the models, we considered investigating the predictive ability of two additional candidate predictors not included in the development phase (no data were available in the CAST data set), but for which information was collected at baseline in the SPRAINED cohort study: sprain severity and recovery expectancy (time to recover from injury, as reported by the participants). Neither of the two variables showed statistically significant crude associations with the outcomes and presented very low predictive ability. For sprain severity, c-statistics were 0.48 (95% CI 0.40 to 0.57) and 0.50 (95% CI 0.44 to 0.56) for outcomes 1 and 2, respectively. For recovery expectancy, c-statistics were 0.56 (95% CI 0.48 to 0.64) and 0.50 (95% CI 0.44 to 0.55) for outcomes 1 and 2, respectively.

Results from the model update are presented in Tables 30 and 31. Predictor transformations were very similar to those observed for the original prognostic models developed with CAST data, apart from the fact that measures of pain were measured on a scale ranging from 0 to 10, and therefore an index was added, indicating that values derived from assessments conducted with the visual analogue scale (which ranges from 0 to 100) should be divided by 10 before any transformation is performed when applying the model to estimate individual risks (see Table 30). Coefficients obtained from the logistic regression models employed to update the models are presented in Table 31. Shrunk coefficients after applying the heuristic shrinkage factor to reduce optimism in the re-estimated model are also presented (see Table 31).

The results of the prognostic development (see Chapter 5) and validation are summarised and discussed together in Chapter 7.

Summary

The first prognostic model was developed to predict a composite outcome representing the presence of at least one of the following symptoms at 9 months after injury: persistent pain, functional difficulty or lack of confidence (outcome 1).

The second model was developed to predict a composite outcome representing the presence of at least one of the following symptoms or clinical events at 9 months after injury: persistent pain, functional difficulty, lack of confidence or recurrence of injury (outcome 2).

The models for outcome 1 and outcome 2 provided reasonable predictions of poor outcome for people with acute ankle sprain on the population used in their derivation (see Chapter 5).

There was a slight decrease in model discrimination for both models when evaluated in a prospectively collected external validation cohort study (see Chapter 6). The model for outcome 1 had better discrimination than the model for outcome 2. The variables for poor outcome used in model 1 (persistent pain, functional difficulty or lack of confidence) were, therefore, easier and more reliable to predict, and appear to have good clinical utility. Hence this would be the model of choice.

The model predicting presence of either persistent pain, functional difficulty, lack of confidence or recurrence of injury (outcome 2) showed good calibration, whereas there was miscalibration of the model predicting persistent pain, functional difficulty or lack of confidence (outcome 1).

Updating these models, which used baseline data collected at the ED, with an additional variable at 4 weeks after the injury (pain when bearing weight on the ankle) improved the discriminatory ability and apparent calibration. However, improvements in model performance were modest. Balancing the practical challenges and resource implications of obtaining additional data at 4 weeks after presentation at the ED with the improvements in prediction is likely to be an important consideration when selecting a model for use in clinical practice.

Despite some miscalibration of the models, the external validation study (see Chapter 6) found that the model performance was reasonable for identifying patients at increased risk of poor outcome after acute ankle sprain, and showed benefit when compared with not using any model. To the best of our knowledge, there are no other prognostic models that have been developed and externally validated using robust methods for this patient group (see Chapter 3). The SPRAINED prognostic models may assist clinical decision-making when assessing and advising people with ankle sprains in the ED setting and when deciding on ongoing management. The models benefit from using predictors that are simple to obtain during routine clinical assessment. Recalibration of the models may be required to improve the accuracy of the predicted risks in other populations (both in and outside the UK).