Diagnostic management strategies for adults and children with minor head injury: a systematic review and an economic evaluation

Adults

The design and patient characteristics of the 19 studies (representing 22 articles)26,27,29,46,54–71 that evaluated the diagnostic performance of clinical decision rules for identifying ICI or need for neurosurgery in adults with MHI are summarised in Table 2. Eight studies were from the USA,27,29,55,58,59,61,62,64 two each from Italy,54,57,71 Canada26,46 and the Islamic Republic of Iran,66,67 and one each from the Netherlands,68–70 Australia,65 Japan,63 Spain60 and Denmark. 56 Six were multicentre studies. 26,46,62,66–70 Cohorts ranged in size from 16863 to 13,728. 62 Fourteen studies derived a new rule. 26,27,29,54–56,60,61–64,66,67,69 Four studies46,57,60,68–71 reported validation results for more than one rule in the same cohort. Data were collected prospectively in 15 studies,26,27,29,46,56–63,66–71 of which participants were recruited consecutively in 13,26,27,29,56–60,62,63,66–71 as a convenience sample in one,46 and one did not report the method of participant recruitment. 61 The remaining four studies were retrospective. 54,55,64,65 Of the 19 studies, three reported both a derivation and a validation cohort,27,61,63 making a total of 22 different cohorts.

Median prevalence of neurosurgical injury was 0.95% [interquartile range (IQR) 0.3% to 1.5%]. Median prevalence of ICI was 7.2% (IQR 6.3% to 8.5%). Variations in prevalence may be owing to differences in inclusion criteria, reference standards and outcome definitions. Participant inclusion ages ranged from > 3 years27 to adults aged ≥ 17 years,55 with five studies including all ages or not reporting an age limit. 29,56,59,61,62 In seven studies,29,55,59,61,62,64,65 patients were enrolled only if they had a CT scan and in nine studies26,27,29,46,55,58,59,65,68–70 patients were selected on the basis of clinical characteristics, such as amnesia or LOC at presentation, which, in some studies, were used as criteria for having a CT scan. Five studies defined MHI as GCS 14–1554,57,58,60,63,71 and included only patients presenting within this range. Four studies collected data only on those with GCS 15,27,29,65,67 one study collected data on GCS 14 only,59 two studies61,62 included data from all GCS categories and two did not report GCS status. 56,64 The remaining five studies26,46,55,66,68–70 included patients with GCS 13–15. Ten studies26,27,29,46,57,59,63,66–71 stated that they enrolled people who presented within 48 hours of injury, although the more usual figure was within 24 hours of injury.

Definitions of outcomes and the reference standards used varied across studies (Table 3). If CT was not an inclusion criterion and was not performed on all then the reference standard used telephone follow-up and/or review of hospital records to identify clinically significant lesions. This method would not be expected to accurately identify all intracranial injuries and would potentially affect estimates of sensitivity and specificity. 80 Eight studies reported neurosurgery as an outcome. 26,29,46,54,57,59,65,68–71 The length of follow-up for neurosurgery varied from being not reported to up to 30 days after injury. The main difference in outcome definition for ICI involved the perception of clinical significance, with five cohorts defining this and 16 identifying any common acute lesion (listed in Table 3). Definitions of surgical lesions also varied, but most definitions included haematoma evacuation, elevation of depressed skull fracture and intracranial pressure monitoring.

Children and infants

The design and patient characteristics of the 14 studies (representing 16 papers)30,81–95 that evaluated the diagnostic performance of clinical decision rules for identifying ICI or need for neurosurgery in children and/or infants with MHI are summarised in Table 4. Six studies82,84–86,90,91,93,95 recruited only infants or reported a subset of infants-only data. Eight studies were from the USA,82,84–6,88,90,91,93–95 one from the USA and Canada,81 and one each from Italy,83 the UK,30 Turkey,87 Finland89 and Canada. 92 Nine studies30,81–87,89,90,92–94 derived a new rule or rules and five validated existing rules. 30,88–91,95 Three studies both derived and validated rules. 30,89,90 Six studies30,81,83,90–92,95 were multicentre studies. Eleven studies30,81,83–86,88,90–95 were prospective, one of which used a convenience sample,81 seven83–86,88,91,92,94,95 of which recruited consecutive patients, and three30,90,93 did not report how the sample was recruited. Three further studies82,87,89 used retrospective data. Two studies30,90 were very large with cohorts over 20,000. The smallest study was 97 patients. 82

The median value for the prevalence of neurosurgery was 1.2% (IQR 0.2% to 1.4%). The median value for the prevalence of ICI was 6.5% (IQR 1.0% to 9.8%). Cohorts were not similar in terms of inclusion and exclusion criteria. For studies of children, the upper age limit ranged between 1630,83,87,89,92 and 21 years,81 and the lower limit between 081 and 5 years. 88 For infants, the upper age limit was usually 2 years, but in one case was 3 years82 of age. Eight studies30,83–85,89,91,93–95 included all severities of head injury; six81,82,87,88,90,92 recruited those with MHI. Two of these studies reported results for a MHI subset of the larger cohort. 86,93 Five studies excluded those with trivial head injury and/or recruited only those with clinical characteristics consistent with head trauma. 88,90,92,93,94 Six studies81,84,87,88,91,94 included only those who had a CT scan and two reported a subset, all of whom underwent CT. 86,93 Selection of patients on the basis of having had a CT scan and exclusion on the basis of trivial injury or not presenting with clinical characteristics is likely to recruit a patient spectrum with greater risk of ICI.

Definitions of outcomes and the reference standards used varied across studies (Table 5). The predominant differences in outcome definition for ICI involve the perception of clinical significance, with four cohorts30,89–91,95 having this defined and the remaining ten studies81–88,92–94 failing to define a positive outcome or just identifying any common acute lesion. The reference standards used where CT was not possible for all, and was not an inclusion criterion, usually comprised telephone follow-up, review of hospital records or both. The length of follow-up for neurosurgery varied from being not reported to following up until discharge, which may not capture all neurosurgical procedures leading to inaccurate estimations of diagnostic accuracy. Definitions of surgical lesions also varied or were not reported, but most definitions included haematoma evacuation and intracranial pressure monitoring; only one mentioned elevation of skull fracture explicitly.

Adults

The methodological quality assessment of each included study is summarised in Figures 2 and 3. Overall, most of the included studies were well reported and generally satisfied the majority of the quality assessment items of the QUADAS tool, but with notable exceptions. 54–57,71 Despite poor reporting of the reference standards in most studies, the main source of variation was for patient spectrum, which will affect comparability across cohorts and application of conclusions to practice.

FIGURE 2.

Decision rules for adults with MHI – methodological quality graph. Review authors’ judgements about each methodological quality item presented as percentages across all included studies.

FIGURE 3.

Decision rules for adults with MHI – methodological quality summary. Review authors’ judgements about each methodological quality item for each included study. Minus sign, negative score; plus sign, positive score; question mark, unclear whether item scores negatively or positively; blank space, not applicable. a, Data from Fabbri et al. 57 and Stein et al. 71 b, Ono et al. 63 recruited two separate cohorts for validation and derivation. The derivation cohort was treated differently for the ICI outcome: the reference standard was not adequate, it was unclear whether partial verification bias was avoided and differential verification bias was not avoided. c, Smits et al. 68–70

The spectrum of patients was appropriate in only one study,66 was unclear in three studies54,56,60 and did not completely match the desired patient spectrum in the remaining 15 studies, often because patients were selected on the basis of having a clinical characteristic at presentation (Table 2). Although 11 studies carried out CT in all participants,27,55,58–65,67 they did not state whether CT was performed within 24 hours and were therefore rated as unclear for the ICI reference standard quality assessment item. A further three cohorts performed CT on all participants within 24 hours and so scored well. 29,66,68–70 The remaining five studies did not perform CT in all participants and so scored negatively for this item. 26,46,54,56,57 The reference standard for neurosurgery was not reported for two studies54,68–70 and not considered adequate in the remaining six. 26,29,46,57,59,65,71 This was usually because not all patients were followed up.

Partial verification bias was largely avoided, with only two cohorts scoring unclear54 or negatively. 57,71 However, these two cohorts were large, and one reported results for a number of rules. 71 Partial verification bias may be more of an issue for the neurosurgery data as no cohort scored well. Differential verification bias for ICI may have affected results in the same large cohort reporting several rules. 57,71 Here participants received different reference standards according to clinical characteristics at presentation or the judgement of the treating physician. Criteria for CT were identical to the rule being tested in the case of the Neurotraumatology Committee of the World Federation of Neurosurgical Societies (NCWFNS)72 rule. In four cases26,46,54,56 it was unclear, although the majority avoided differential verification bias. For neurosurgery, it was unclear if differential verification bias was avoided in six cohorts26,46,54,57,65,68–71 and was scored negatively in two cohorts. 29,59

The execution of the index test was well described in all studies. The execution of the reference standards (either one or both) was not reported well in nine studies54–59,63,65,68 and scored negatively for this item. Diagnostic and test review biases may affect results as less than half of the studies scored well for blinding; the index test was interpreted blind in eight cases,26,27,29,46,58,62,64,66, but blinding status was unclear in 11. 54–57,59,60,61,63,65,67–70 The reference standard was interpreted blind in seven cases,26,46,60–62,66,67 and was not interpreted blind in two cases;64,68–70 blinding status was unclear in ten cases. 27,29,54–59,63,65 Studies were of mixed quality for clinical review bias, with almost equal numbers scoring in each quality category. Information about uninterpretable results was only given in one study,64 with all other studies scoring unclear for this item. Studies scored well for withdrawals, with only four studies55,57,59,65,71 scoring unclear because it was not apparent whether all patients were accounted for at the end of the study.

Children and infants

The methodological quality assessment of each included study is summarised in Figures 4 and 5. Overall, most of the included studies were poorly reported and did not satisfy the majority of the quality assessment items of the QUADAS tool. The study30 that scored the most negatives and fewest positives was also one of the two large cohorts (> 20,000), and consequently has the potential to influence the results. This study scored poorly mainly owing to the use of pragmatic reference standards.

FIGURE 4.

Decision rules for children and infants with MHI – methodological quality graph. Review authors’ judgements about each methodological quality item presented as percentages across all included studies.

FIGURE 5.

Decision rules for children and infants with MHI: methodological quality summary. Review authors’ judgements about each methodological quality item for each included study. Minus sign, negative score; plus sign, positive score; question mark, unclear whether item scores negatively or positively; blank space, not applicable.

The patient spectrum item scored worst overall, with only one study (which was one of the large studies) scoring positively. 90 Studies failed this quality item for a range of reasons and sometimes for multiple reasons. Problems included selecting only patients who had had a CT scan or those who presented with clinical characteristics, including patients with all severities of head injury, recruiting patients regardless of time since injury and using a retrospective design.

The reference standard for ICI was of a mixed standard: only three scored positively. 84–86,94 Although, a further four81,87,88,91,95 did undertake CT in all participants, they failed to state whether this was within 24 hours and so scored unclear. The remaining seven studies scored negatively30,82,83,89,90,93 or unclear. 92 This represents a potential source of bias. Equally, the reference standards for neurosurgery scored negatively or unclear in all but one study. 93

Studies were well reported in terms of description of selection criteria and test execution details, with 1230,81–88,90,92–94 and 1130,81–88,90,91,93 studies, respectively, reporting these criteria adequately. Descriptions of the execution of the reference standard were mixed, with just over half scoring well. 30,81,82,84–86,88,89,91,95 Uninterpretable results were not reported in 10 studies81–83,85,86,88,92,93 and so scored unclear for this item.

Partial verification bias was generally avoided (11 studies scored well)81–91,94,95 for ICI where a reference standard was applied to all participants, but not for neurosurgical outcomes, for which only one study scored well. 81 The picture was less clear for differential verification bias of ICI, with three scoring negatively30,83,88 and almost equal numbers scoring well81,84–87,91,94,95 and unclear82,89,90,92,93 where, for example, it was not clear whether or not clinical characteristics (index test) may have contributed to the decision to give CT rather than follow-up as a reference standard. There is some potential for this bias to affect the results, especially as neither large cohort30,90 scored well. For neurosurgical outcomes only one study scored well. 81 Blinding was generally poorly reported, with seven studies scoring unclear82–84,87,89,92,94 for the test review bias and 11 studies30,81–89,92,93 scoring unclear for the diagnostic review bias. However, six studies scored well for test review bias. 81,85,86,88,90,91,93,95 Clinical review bias was avoided in retrospective studies by definition, but for most it was unclear81–83,85,86,88,92,93 or negative. 90,91,95 There is potential for these biases to affect the results. Few studies reported withdrawals and so most scored well for this item. 81,82,85–89,91–95

Adults

From the 19 studies reporting diagnostic data for decision rules for adults with MHI, a total of 25 decision rules1,19,20,26,27,29,54–56,58,60–64,66,67,69,72–77,79 were identified and are outlined in Tables 6 and 7a and b. Eleven rules1,19,26,27,54,61–63,72,73,78,79 were evaluated in more than one data set and one further rule29 was evaluated in two cohorts: one of GCS 15 (derivation cohort)29 and one of GCS 14. 59 Nine of the decision rules1,19,26,70,72–75,78,79 existed in two forms: one to identify those most at risk (termed variously as high risk, mandatory, emergency, moderate and strict) and a second more inclusive version to identify those at medium risk (termed variously as medium risk, recommended, urgent, mild and lenient). These two risk categories were often intended to identify those at risk of needing neurosurgery (high risk) and those at risk of ICI (medium risk).

Figures 6 and 7 show the sensitivities and specificities for any ICI and neurosurgical injury, respectively, for rules that have been evaluated in multiple cohorts. Figures 8 and 9 show the corresponding parameters for rules that have been evaluated in only one cohort.

FIGURE 6.

Decision rules for adults with MHI – sensitivity and specificity of decision rules for which more than one data set is available for the outcome ICI. NEXUS II, National Emergency X-Radiography Utilization Study II.

FIGURE 7.

Decision rules for adults with MHI – sensitivity and specificity of decision rules for which more than one data set is available for the outcome need for neurosurgery.

FIGURE 8.

Decision rules for adults with MHI – sensitivity and specificity of decision rules for which only one data set is available for the outcome ICI.

FIGURE 9.

Decision rules for adults with MHI – sensitivity and specificity of decision rules for which only one data set is available for the outcome need for neurosurgery. NEXUS II, National Emergency X-Radiography Utilization Study II.

The CCHR26 has been validated in a number of studies26,46,60,65,68,71 and tested using either the high-risk criteria as a threshold or high- and medium-risk criteria. Using the high-risk criteria alone it has a sensitivity of 99–100% and a specificity of 48–77% for neurosurgical injury. 26,46,65,71 The high-risk criteria were derived to identify neurosurgical injury and were not tested to identify any intracranial injuries by the original researchers. Two other studies,65,71 however, used the high-risk rule to identify intracranial injuries. Results varied dramatically between the two studies, so no useful conclusions can be drawn.

Using the high- or medium-risk criteria, the CCHR has a sensitivity of 99–100%65,68,71 and 80–100%26,46,60,65,68,71 for neurosurgical and any ICI, respectively, and corresponding specificities of 37–48% and 39–50%, respectively. The variation in sensitivity for any ICI is probably due to variation in the reference standard: sensitivity was 98–100% in studies in which clinically low-risk patients received telephone follow-up rather than CT,26,46,71 but was 80–86% in studies in which all patients had CT. 60,65,68 This likely reflects differential identification of false-negative patients who were at low clinical risk and suffered no complication, but had ICI on CT.

Overall, it therefore appears that the CCHR has high sensitivity for detecting neurosurgical injuries, whether high-risk or high- and medium-risk criteria are used. This was consistent when some of the original patient exclusion criteria were included as risk factors (see Figure 6, CCHR high and medium risk adapted to cohort). 68 Sensitivity for any ICI is probably more modest, but the missed cases are unlikely to be clinically significant. Specificity is adequate to allow a meaningful proportion of patients to avoid CT scanning.

The NOC27 rule has been validated in several studies and shown to have excellent sensitivity for neurosurgical lesion (99–100%)46,65,68,71 and any intracranial lesion (95–100%). 27,46,60,65,68,71 However, specificity for neurosurgical lesions (3–31%) and any intracranial lesion (3–33%) was generally poor. In most cohorts, application of the NOC rule would have resulted in all patients having a CT scan.

The NICE guidelines1,19 were developed using the CCHR high- and medium-risk criteria. Sensitivity for neurosurgical injury and any injury varied from 88% to 98%57,70,71 and from 67% to 99%57,70,71 respectively, while corresponding specificities varied from 29% to 66% and from 31% to 70%, depending upon whether the 200319 or 20071 guidelines were tested and whether strict or lenient criteria were used. Amendment of the guidelines in 20071 entailed new recommendations for children and a change to management of patients over 65 years with LOC or amnesia. The revised NICE guidelines1 appeared to improve sensitivity at the cost of specificity, although the latter was still acceptable at 31%. The two versions of the rule were tested in the same cohort,57,71 which included adolescents over 10 years of age. It is possible that the improvement in performance of the rule is driven by the changes to the management of children rather than the relatively minor change in the management of adults.

Both the NCWFNS guidelines72 and the SIGN guidelines20 have sensitivities in a similar range to the CCHR when lenient criteria are used, but results for specificity are very variable and generally much lower. The Scandinavian lenient criteria73 have diagnostic parameters in the same range, but with more variation in sensitivity for neurosurgical injury (94–99%)69,74 and specificity for neurosurgical (20–50%) or any injury (21–60%). 60,69,74 The NEXUS II (National Emergency X-Radiography Utilization Study II) rule62 appears to have high sensitivity for both neurosurgical and any injury, but variable specificity and very limited validation. Other rules have not been validated in sufficient cohorts and settings to draw meaningful conclusions.

Children

From the 14 studies reporting diagnostic data for decision rules for children with MHI, a total of 15 decision rules30,81–94,96 were identified and are outlined in Table 8 a and b. Four studies presented more than one version of a rule: Greenes and Schutzman derived a decision rule85 for any severity of injury and a scoring system86 for asymptomatic patients from the same cohort; Kupperman et al. 90 reported a second rule for those aged < 2 years; the Canadian Assessment of Tomography for Childhood Injury (CATCH)92 rule had a high and a medium- and high-risk format; the University of California–Davis rule (UCD)93 had three versions, each designed to identify a different outcome (need for neurosurgery, brain injury and intervention or brain injury). Four of the rules or their versions were specifically for infants. 82,85,86,90

Of studies reporting prediction of ICI, only four rules30,90,91,93 were tested in more than one cohort (Figure 10). Of these four rules, the UCD rule93 for identifying patients with TBI or who needed acute intervention (which equates to ‘any ICI’) had the highest sensitivity (99% and 100%)89,93 with variable values for specificity (12% and 43%). A modified version of the UCD rule reported in Sun et al. ,95 in which ‘headache’ and ‘vomiting’ were redefined as ‘severe headache’ and ‘severe vomiting’, produced lower sensitivity (91%) but similar specificity (43%).

FIGURE 10.

Decision rules for children with MHI – sensitivity and specificity of decision rules for which more than one data set is available for the outcome ICI. a, Sun et al. 95 give a stricter definition of headache and vomiting than the original rule; b, derivation and validation cohorts reported in the same article.

The CHALICE rule30 had the next best sensitivity (98% and 98%), but very variable specificity (87% and 5%). The derivation cohort30 used a poor reference standard (3% given CT) and the other cohort89 had different patient inclusion criteria (selecting only those admitted), both of which may contribute to the difference in specificity.

The Paediatric Emergency Care Applied Research Network (PECARN) rule for children ≥ 2 years to < 18 years was tested in two cohorts, a derivation and a validation cohort, reported in the same paper. 90 Sensitivity (97% and 97%) and specificity (58% and 60%) were very consistent. The rule appears to sacrifice a small degree of sensitivity for a higher specificity when compared with other rules.

The NEXUS II rule was tested in two studies. 89,91 These reported similar sensitivity (96% and 99%) and specificity (15% and 21%), despite differences in the adequacy of the reference standard in one study, and differences in cohort selection and outcome definitions. Although these results seem less promising than the rules discussed earlier, further validation work in a different setting is warranted before conclusions can be drawn.

Nine further rules81,83,84,87–89,92,94,96 were tested in only one cohort (Figure 11) against the outcome of ICI. Of these, one rule (that of Da Dalt et al. 83) had excellent sensitivity (100%) and specificity (87%). Further validation studies are needed before conclusions can be drawn regarding this rule. The Royal College of Surgeons (RCS) guidelines96 appeared to have excellent diagnostic accuracy. 30 However, the reference standard used was management according to the RCS guidelines with only some patients followed up. This is likely to significantly increase the estimates of sensitivity and specificity.

FIGURE 11.

Decision rules for children with MHI – sensitivity and specificity of decision rules for which only one data set is available for the outcome ICI.

Six rules30,8188,90,92,93 were tested for prediction of the need for neurosurgery (Figure 12) and all in only one cohort. All had very good sensitivity (98–100%), but variable specificity (24–86%). The CHALICE rule30 had the highest specificity, but the lowest sensitivity. As observed with the PECARN criteria90 for children ≥ 2 years, the CHALICE rule30 appeared to sacrifice a degree of sensitivity for an improved specificity. All of these rules need further investigation and validation testing in other settings before firm conclusions can be drawn.

FIGURE 12.

Decision rules for children with MHI – sensitivity and specificity for the outcome neurosurgery.

For infants (Figure 13), only the PECARN rule was tested in two cohorts against the outcome ICI. 90 This rule gave the most promising results out of the seven rules identified for ICI. 82,84–86,90,91,93,95 Only the PECARN90 rule (Figure 14) was tested against the outcome of need for neurosurgery in infants. All of these rules require further investigation and validation testing in other settings before firm conclusions can be drawn.

FIGURE 13.

Decision rules for infants with MHI – sensitivity and specificity for the outcome ICI. a, derivation and validation cohorts reported in the same article; b, Sun et al. 95 give a stricter definition of headache and vomiting than the original rule.

FIGURE 14.

Decision rules for infants with MHI – sensitivity and specificity for the outcome neurosurgery.

Adults

The design and patient characteristics of the 42 studies (representing 44 papers)26,27,29,46,54,55,57–63,67,77,98–126 that evaluated the diagnostic accuracy of individual characteristics for identifying ICI or need for neurosurgery in adults with MHI are summarised in Table 9. Twenty-three studies were from the USA,27,29,55,58,59,61,62,100–102,104,107–112,116,118–123,126 two each from Italy,54,57 Germany,98,115 Spain,60,103 Japan,63,106 Canada,26,46 and India,120,124 and one from each of Saudi Arabia,77,114 Malaysia,99 Hong Kong,105 Islamic Republic of Iran,67 Denmark117 and Taiwan. 125 One further study was an international collaboration. 113 Ten studies were multicentre. 26,46,67,98,106,108,62,113,118,119 Of the 42 studies, 22 were prospective; 16 recruited consecutive patients,26,27,29,57–60,62,67,98,100,101,105,108,111,115,124 whereas two selected a convenience sample46,126 and four did not report the method of selection. 61,109,113,118 Sixteen studies were retrospective54,55,67,102–106,110,112,114,116,117,119,121–123,125 and four63,107,120,127 did not report the mode of data collection.

The cohort sizes of the included studies ranged from 39120 to 13,728. 62 The mean age of the cohorts ranged from 17114 to 4760 years, with two cohorts108,110 reporting older patients separately, with mean ages in excess of 70 years. The variation in mean age range appeared to be influenced by the minimum age for inclusion in the study; some studies included all ages29,62,77,111,112,114,118,119,123,124 or did not report an age restriction,59,61,104,106,115,117,120–122,125 whereas others set a lower age limit. 26,27,46,54,55, 57,58,60,63,67,98–103,105,107–110,113,116,126 The median prevalence of need for neurosurgery was 1.7% (IQR 1.2% to 3.8%). The prevalence of ICI ranged from 0.48%117 to 78.1%99 with a median prevalence of 9.4% (IQR 6.8% to 18%). This wide variation is likely to be owing to differences in patient selection criteria, adequacy of reference standards and definitions of ICI, and neurosurgery. There was no study that clearly selected the whole population of interest. As detailed in Table 9, patients were excluded based on GCS score, absence or presence of clinical characteristics at presentation or because they had not had a CT scan; alternatively, selection criteria were rendered unclear by phrases such as ‘those admitted’. Twenty studies selected only patients with GCS 13–15,26,46,55,98,99,101–103,105,110,112,113,115,119–125 six only patients with GCS 14 or 15,57,58,60,63,108,109 another five only patients with GCS 15,27,29,67,107,111,126 one only those with GCS 14,59 three studies54,104,118 selected all severities of injury (with data available for a GCS 13–15 subgroup in two studies)54,104 and six did not report GCS scores. 61,62,77,106,114,116,117 One further study included GCS 13–15 or GCS < 13 if intoxicated. 100 In 26 studies26,27,46,54,57,58,60,63,67,98,100–103,105,107–109,112,114,115,117,119,121,123,124 patients were not selected on the basis of having had a CT scan, whereas in 14 studies29,55,59,61,62,99,104,110,111,113,116,118,122,125,126 patients were only enrolled if they had a CT scan. The remaining two studies106,120 did not state whether this was used as an inclusion criterion. Selection of patients based on clinical characteristics at presentation varied widely.

Definitions of outcomes and the reference standards used varied across the 42 studies (Table 10). For ICI, 21 studies54,55,58,63,67,77,99,101,102,107–110,112,113,116,118,120,122,125,126 gave only a very general description of the outcome, such as ICI or positive CT findings, with no definition. The remainder varied in the level of detail provided and the type of injuries included, with some including all common acute lesions including skull fractures (e.g. Biberthaler et al. ),98 and others defining injury in terms of severity and clinical significance (e.g. Mower et al. ;62 Stiell et al. 26,46). 62 For the outcome, ‘need for neurosurgery’ there was again a variety of definitions across the 16 studies, including narrow definitions which, for example, only included ‘urgent surgery’122 or specified a timescale,105,121 and definitions that included any neurosurgical procedure, including fitting an intracranial pressure monitor. 119

In the 10 studies26,46,54,57,67,77,101,105,112,117 in which CT was not possible for all and was not an inclusion criterion, the reference standard varied, with four studies26,46,54,67 using telephone follow-up and five57,77,101,105,112 not reporting how ICI was identified in those not undergoing CT. One study admitted those not undergoing CT. 117 Telephone follow-up and no follow-up are both likely to miss some intracranial injuries, affecting estimates of diagnostic accuracy. The length of follow-up for neurosurgery varied from being not reported to following until discharge, which may not capture all neurosurgical procedures, again leading to inaccurate estimations of diagnostic accuracy.

Children and infants

The design and patient characteristics of the 29 studies (representing 30 papers)30,81–84,86–91,93–95,127–142 that evaluated the diagnostic accuracy of individual clinical characteristics for identifying ICI (including the need for neurosurgery) in children and/or infants with MHI are summarised in Table 11. Three studies84,90,135 provided separate data for children and infants, whereas two studies82,86 provided data for infants only, and one study84 provided data for infants as a subset of data from a cohort of children up to age 18 years. In one study only adolescents were selected. 127 Eighteen studies were from the USA,82,84,86,88,90,91,93,94,129–131,135,137–142 three from Turkey,87,128,132 two from the UK,30,136 one a USA–Canadian collaboration81 and one each from Italy,83 Finland,89 Poland,133 Australia134 and Hong Kong. 127 Eight studies30,81,83,90,91,95,129,135,140 were multicentre. Cohorts ranged in size from 39136 to 31,69490 patients with two cohorts30,90 providing a large data set of over 20,000 participants. Seventeen30,81,83,84,86,88,90,91,93–95,127,128,130,131,135,141,142 studies were prospective, seven of which were consecutive,83,84,86,88,91,94,95,127 one convenience81 and the remaining nine30,90,93,128,130,–131,135,141,142 did not report the method of patient recruitment. Twelve82,89,129,132–134,136–140 studies were retrospective.

For studies of children, the upper age limit ranged between 12141 and 21 years,81 and the lower limit between 081 and 5 years. 88 For infants, the upper age limit was 284,86,90,135 or 3 years. 82,91 Mean age was not reported in the majority of cases; where it was reported it ranged from 4 years 10 months94 to 12 years 10 months. 88 Prevalence of neurosurgery ranged from 1.0%137 to 8.5%131 (median 3.3%, IQR 1.55% to 7.23%) and prevalence of ICI ranged from 0.58%83 to 54.6%134 (median 12.1%, IQR 4.1% to 21.0%). It was clear in only one study30 that only the whole population of interest had been selected. Variations in selection criteria include selection of patients on the basis of having had a CT scan,81,84,87,88,91,93–95,129,130,134–138,140,142 selecting only patients presenting with some clinical characteristics,83,88,90,93,94,127–129,131,132,138,139,141,142 selecting only those admitted89,127,139 and selecting a spectrum of patients with a wider or narrower range of GCS scores. Five studies selected only those with GCS 15,86,88,128,129,137 five only those with GCS 14 or 15,82,90,93,132,138 one only those with GCS 13 or 14140 and 1283,84,89,91,94,95,127,130,134–136,141 either did not report selection on the basis of GCS or selected all severities. The remaining six81,87,131,133,139,142 selected or reported a subset of patients with GCS 13–15.

Definitions of outcomes and the reference standards used varied across the 29 studies (Table 12). 30,81–84,86–91,93–95,127–142 The outcome definition for ICI differed across the 28 studies30,81–84,86–91,93–95,127–130,132–142 that reported this outcome. Four studies30,89–91,95 defined this as injuries of clinical significance, 13 studies81,82,84,86,88,93,128,130,132–135,138 had more general definitions including common acute lesions (listed in Table 12) and 11 studies83,87,94,127,129,136,137,139–142 did not give a definition. The reference standards used where CT was not possible for all and was not an inclusion criterion was unclear in five cases. 30,89,127,132,140 Other reference standards comprised telephone follow-up, review of hospital records or both. Neurosurgery was poorly defined in most cases; one study included other medical interventions88 and one study excluded skull fracture surgery,137 but it was unclear if these were included or excluded in other studies. The length of follow-up for neurosurgery varied from being not reported to following until discharge88 or at an outpatients clinic. 131

Adults

The methodological quality assessment of each included study is summarised in Figures 15 and 16. Overall, most of the included studies were poorly reported and did not satisfy the majority of the quality assessment items of the QUADAS tool.

FIGURE 15.

Individual clinical characteristics in adults with MHI – methodological quality graph. Review authors’ judgements about each methodological quality item presented as percentages across all included studies.

FIGURE 16.

Individual clinical characteristics in adults with MHI – methodological quality summary. Review authors’ judgements about each methodological quality item for each included study. Minus sign, negative score; plus sign, positive score; question mark, unclear whether item scores negatively or positively; blank space, not applicable.

The main source of variation was patient spectrum, for which no study scored positively (further details are provided in Description of included studies). Fewer than one-quarter of the studies used an adequate reference standard for ICI,29,67,108,111,113,115,122 with the majority scoring unclear or negatively. Although 21 studies27,55,58–63,99,100,102,104,107,110,116,118,120,123–126 carried out CT in all participants, they failed to state whether this was done within 24 hours and were therefore scored unclear. Of the 14 studies26,29,46,54,57,59,102–105,111,121,122,124,126 that reported the outcome neurosurgery, all either reported an inadequate reference standard or were unclear on this point. Poor scores were usually given because length of follow-up was not adequate.

Partial verification bias was largely avoided. Similarly, studies scored well generally for differential verification bias, with reference standards being applied to the whole cohort in 29 cases. 26,27,29,46,55,58,60,62,63,67,69,98–100,102–104,107,108,110,111,113,115,116,118,120–126 However, it should be noted that three54,57,106 of the four largest cohorts scored negatively or unclear across the reference standard and verification items, and two54,57 of these report data for a large number of clinical characteristics. There is the potential for bias in these studies to influence results.

The execution of the index test was reported more often than the reference standard. This probably reflects the routine nature of CT scanning, whereas the index tests required more explanation. Test review and diagnostic review biases were largely unreported. This may have been considered an unnecessary detail to report as it is likely that clinical characteristics will have been assessed prior to CT scanning and, therefore, blinded by default. However, where it is not clear that this is the case, studies have been scored unclear. Blinding of the index test results when reading the reference standard may have been thought unethical, although no study examined this issue. It was difficult to assess to what extent a lack of blinding has influenced results. Clinical review bias scored a little better as retrospective studies by definition reflect real-life practice, but overall scored poorly or unclear. Uninterpretable results were rarely discussed. In the one case where they were,108 it was unclear how these results were treated for analysis. Withdrawals were generally not reported and, as there was no evidence to suggest that there were any withdrawals to report, all but one study59 scored well for this item.

Children

The methodological quality assessment of each included study is summarised in Figures 17 and 18. Overall, most of the included studies were poorly reported and did not satisfy the majority of the quality assessment items of the QUADAS tool.

FIGURE 17.

Individual clinical characteristics in children and infants with MHI – methodological quality graph. Review authors’ judgements about each methodological quality item presented as percentages across all included studies.

FIGURE 18.

Individual clinical characteristics in children and infants with MHI – methodological quality summary. Review authors’ judgements about each methodological quality item for each included study. Minus sign, negative score; plus sign, positive score; question mark, unclear whether item scores negatively or positively; blank space, not applicable.

The main source of variation was patient spectrum, for which only one study scored positively90 (further details are in Description of included studies). Only five studies used an adequate reference standard for ICI,84,94,133,134,137 with the majority scoring unclear or negatively, including the two very large cohorts. 30,90 Although 13 studies81,87,88,91,95,128–130,135,136,138,139,141,142 did carry out CT in all participants, they failed to state whether this was within 24 hours and were therefore scored unclear. Of the four88,131,137,139 studies that reported the outcome data for neurosurgery, none reported an adequate reference standard: two reported an inadequate reference standard and two were unclear on this point.

Partial verification bias was largely avoided, with 24 studies81–84,86–91,94,128–131,133–139,141,142 scoring well for this item, although one of the large cohorts scored negatively. 30 Similarly, studies scored well generally for differential verification bias, with reference standards being applied to the whole cohort in 19 cases. 81,84,86,87,91,94,128–131,133–139,141,142 For the two very large cohorts, one study scored negatively for this item,30 whereas for the other study90 the reference standard was determined at the physician’s discretion so the item scored unclear.

The execution of the index test was reported more often than the reference standard. Test review and diagnostic review biases were largely unreported. Blinding of the index test results when reading the reference standard may have been thought unethical, though no study examined this issue. Clinical review bias scored a little better as retrospective studies by definition reflect real-life practice, but over half scored poorly or unclear. Uninterpretable results were discussed in only one study,81 with reference to a single uninterpretable CT scan that was treated as a positive. Withdrawals were generally not reported and, as there was no evidence to suggest that there were any withdrawals to report, most studies scored well for this item.

Adults

Tables 13 and 14 show the sensitivity, specificity, PLR and NLR for each individual clinical characteristic for predicting ICI or need for neurosurgery in adults. Further details are provided in Appendix 5. Only individual clinical characteristics that were defined consistently and in a clinically meaningful way were included in the meta-analysis. Two studies108,126 were excluded from the meta-analysis because they did not define the characteristics they reported (neurological examination) in a way similar enough to other studies to be meaningfully meta-analysed.